Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Refine Noir.js API #2732

Merged
merged 42 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ea66d2f
add base64 decode method
kevaundray Sep 16, 2023
723028f
add wip version of input validation
kevaundray Sep 16, 2023
56dc4c1
add initial witness generation method - does not handle return values
kevaundray Sep 16, 2023
581758f
add serialize method to abstract away the discrepancies with serializ…
kevaundray Sep 16, 2023
752535d
export new methods
kevaundray Sep 16, 2023
e94f00b
add mocha, prettier and add `type: module` for node
kevaundray Sep 16, 2023
c45fc8f
add compiled_examples folder for noir tests
kevaundray Sep 16, 2023
bcbc0ce
do not ignore the target folders
kevaundray Sep 16, 2023
6664b72
commit .json files for noir_compiled examples
kevaundray Sep 16, 2023
8bd188c
add bb test backend code
kevaundray Sep 16, 2023
f431c72
add comment to explain why we are doing type assertions
kevaundray Sep 16, 2023
ec3cc28
add some smoke tests
kevaundray Sep 16, 2023
153a72f
add bb.js test to show how it may interact with noir.js
kevaundray Sep 16, 2023
98bfa9f
increase timeout
kevaundray Sep 16, 2023
6386400
add another instance of null function
kevaundray Sep 16, 2023
e479c9c
update yarn.lock
kevaundray Sep 16, 2023
c7d48fd
Merge remote-tracking branch 'origin/master' into kw/more-noir-js
kevaundray Sep 16, 2023
4cdeeae
add `yarn test`
kevaundray Sep 16, 2023
dd6b837
remove .only
kevaundray Sep 16, 2023
de1f7ef
no test is using 1_mul so we can remove it
kevaundray Sep 16, 2023
1fb56d0
add separate inner proof -- this quickly verifies that inner proof cr…
kevaundray Sep 16, 2023
3666b5a
add bug-doc test that creates and verifies an inner and outer proof u…
kevaundray Sep 16, 2023
23f9bfa
Merge remote-tracking branch 'origin/master' into kw/more-noir-js
kevaundray Sep 17, 2023
f406033
update yarn.lock
kevaundray Sep 17, 2023
c7146dd
add correct eslint and prettier
kevaundray Sep 18, 2023
97ad1fc
change file to .cjs since we have instructed typescript to treat node…
kevaundray Sep 18, 2023
b01a215
update yarn.lock
kevaundray Sep 18, 2023
f2fd224
commit recursive
kevaundray Sep 18, 2023
13304e7
remove recursive
kevaundray Sep 18, 2023
5562e21
add array case to type_to_string
kevaundray Sep 18, 2023
5b3f24d
lint
kevaundray Sep 18, 2023
edcb749
explicit any
kevaundray Sep 18, 2023
ed6369c
hack around linting -> I think any should be explicit
kevaundray Sep 18, 2023
ae9a6fa
cannot remove ts-ignore unfortunately, or it will not compile
kevaundray Sep 18, 2023
be7e09b
update dependencies
kevaundray Sep 18, 2023
403d9b5
linter
kevaundray Sep 18, 2023
460a8a9
Merge branch 'master' into kw/more-noir-js
TomAFrench Sep 19, 2023
75c226d
chore: update integration test glob
TomAFrench Sep 19, 2023
7af1dc6
Merge branch 'master' into kw/more-noir-js
kevaundray Sep 20, 2023
f8ad376
fix package.json
kevaundray Sep 21, 2023
0a52241
Merge remote-tracking branch 'origin/master' into kw/more-noir-js
kevaundray Sep 21, 2023
6d5d960
cargo fmt
kevaundray Sep 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tooling/noir_js/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!test/noir_compiled_examples/*/target
8 changes: 8 additions & 0 deletions tooling/noir_js/.mocharc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"require": "ts-node/register",
"loader": "ts-node/esm",
"extensions": ["ts"],
"spec": [
"test/node/**/*.test.ts*"
]
}
6 changes: 6 additions & 0 deletions tooling/noir_js/.prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"parser": "typescript",
"printWidth": 120,
"singleQuote": true,
"trailingComma": "all"
}
18 changes: 17 additions & 1 deletion tooling/noir_js/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@
"version": "0.12.0",
"packageManager": "[email protected]",
"license": "(MIT OR Apache-2.0)",
"type": "module",
"dependencies": {
"@noir-lang/acvm_js": "0.26.1",
"@noir-lang/noirc_abi": "workspace:*"
"@noir-lang/noirc_abi": "workspace:*",
"fflate": "^0.8.0"
},
"files": [
"lib",
Expand All @@ -17,10 +19,24 @@
"main": "lib/index.js",
"types": "lib/index.d.ts",
"scripts": {
"dev": "tsc --watch",
"build": "tsc",
"test": "yarn test:node",
"test:node": "mocha --timeout 10000",
"prettier": "prettier 'src/**/*.ts'",
"prettier:fix": "prettier --write 'src/**/*.ts' 'test/**/*.ts'",
"lint": "NODE_NO_WARNINGS=1 eslint . --ext .ts --ignore-path ./.eslintignore --max-warnings 0"
},
"devDependencies": {
"@aztec/bb.js": "0.7.2",
"@types/chai": "^4",
"@types/mocha": "^10.0.1",
"@types/node": "^20.6.2",
"@types/prettier": "^3",
"chai": "^4.3.8",
"mocha": "^10.2.0",
"prettier": "3.0.3",
"ts-node": "^10.9.1",
"typescript": "^5.2.2"
}
}
13 changes: 13 additions & 0 deletions tooling/noir_js/src/base64_decode.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// Since this is a simple function, we can use feature detection to
// see if we are in the nodeJs environment or the browser environment.
export function base64Decode(input: string): Uint8Array {
if (typeof Buffer !== 'undefined') {
// Node.js environment
return Buffer.from(input, 'base64');
} else if (typeof atob === 'function') {
// Browser environment
return Uint8Array.from(atob(input), (c) => c.charCodeAt(0));
} else {
throw new Error('No implementation found for base64 decoding.');
}
}
11 changes: 8 additions & 3 deletions tooling/noir_js/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
import * as acvm from "@noir-lang/acvm_js";
import * as noirc from "@noir-lang/noirc_abi";
export { acvm, noirc };
import * as acvm from '@noir-lang/acvm_js';
import * as abi from '@noir-lang/noirc_abi';

export { acvm, abi };

import { generateWitness } from './witness_generation.js';
import { acirToUint8Array, witnessMapToUint8Array } from './serialize.js';
export { acirToUint8Array, witnessMapToUint8Array, generateWitness };
55 changes: 55 additions & 0 deletions tooling/noir_js/src/input_validation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Check if all of the input values are correct according to the ABI
export function validateInputs(inputs: any, abi: any) {
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
for (const param of abi.parameters) {
const inputValue = inputs[param.name];
if (inputValue === undefined) {
// This is checked by noirc_abi, so we could likely remove this check
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
return { isValid: false, error: `Input for ${param.name} is missing` };
}
if (!checkType(inputValue, param.type)) {
return {
isValid: false,
error: `Input for ${param.name} is the wrong type, expected ${type_to_string(
param.type,
)}, got \"${inputValue}\"`,
};
}
}
return { isValid: true, error: null };
}

// Checks that value is of type "type"
// Where type is taken from the abi
function checkType(value: any, type: any) {
switch (type.kind) {
case 'integer':
if (type.sign === 'unsigned') {
return isUnsignedInteger(value, type.width);
}
// Other integer sign checks can be added here
break;
// Other type.kind checks can be added here
}
return false;
}

function type_to_string(type: any) {
switch (type.kind) {
case 'integer':
if (type.sign === 'unsigned') {
return `uint${type.width}`;
}
break;
}
return false;
}

// Returns true if `value` is an unsigned integer that is less than 2^{width}
function isUnsignedInteger(value: any, width: any) {
try {
const bigIntValue = BigInt(value);
return bigIntValue >= 0 && bigIntValue <= BigInt(2) ** BigInt(width) - 1n;
} catch (e) {
return false; // Not a valid integer
}
}
17 changes: 17 additions & 0 deletions tooling/noir_js/src/serialize.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { WitnessMap, compressWitness } from '@noir-lang/acvm_js';
import { decompressSync as gunzip } from 'fflate';
import { base64Decode } from './base64_decode.js';

// After solving the witness, to pass it a backend, we need to serialize it to a Uint8Array
export function witnessMapToUint8Array(solvedWitness: WitnessMap): Uint8Array {
// TODO: We just want to serialize, but this will zip up the witness
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
// TODO so its not ideal
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
const compressedWitness = compressWitness(solvedWitness);
return gunzip(compressedWitness);
}

// Converts an bytecode to a Uint8Array
export function acirToUint8Array(base64EncodedBytecode: any): Uint8Array {
const compressedByteCode = base64Decode(base64EncodedBytecode);
return gunzip(compressedByteCode);
}
24 changes: 24 additions & 0 deletions tooling/noir_js/src/witness_generation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { abiEncode } from '@noir-lang/noirc_abi';
import { validateInputs } from './input_validation.js';
import { base64Decode } from './base64_decode.js';
import { WitnessMap, executeCircuit } from '@noir-lang/acvm_js';

// Generates the witnesses needed to feed into the chosen proving system
export async function generateWitness(compiledProgram: any, inputs: any): Promise<WitnessMap> {
// Validate inputs
const { isValid, error } = validateInputs(inputs, compiledProgram.abi);
if (!isValid) {
throw new Error(error?.toString());
}
const witnessMap = abiEncode(compiledProgram.abi, inputs, null);

// Execute the circuit to generate the rest of the witnesses
try {
const solvedWitness = await executeCircuit(base64Decode(compiledProgram.bytecode), witnessMap, () => {
throw Error('unexpected oracle during execution');
});
return solvedWitness;
} catch (err) {
throw new Error(`Circuit execution failed: ${err}`);
}
}
122 changes: 122 additions & 0 deletions tooling/noir_js/test/backend/barretenberg.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
// @ts-ignore
import { Barretenberg, Crs, RawBuffer } from '@aztec/bb.js';
// TODO: This should be re-exported from @aztec/bb-js
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
import { Ptr } from '@aztec/bb.js/dest/browser/types';
import { acirToUint8Array } from '../../src/index.js';

export class Backend {
// These type assertions are used so that we don't
// have to initialize `api` and `acirComposer` in the constructor.
// These are initialized asynchronously in the `init` function,
// constructors cannot be asynchronous which is why we do this.
api = {} as Barretenberg;
acirComposer = {} as Ptr;
acirUncompressedBytecode: Uint8Array;

constructor(acirBytecodeBase64: string) {
this.acirUncompressedBytecode = acirToUint8Array(acirBytecodeBase64);
}
async init() {
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
const numThreads = 4;

const { api, composer } = await this.initBarretenberg(numThreads, this.acirUncompressedBytecode);

this.api = api;
this.acirComposer = composer;
}

async initBarretenberg(numThreads: number, acirUncompressedBytecode: Uint8Array) {
const api = await Barretenberg.new(numThreads);

const [exact, total, subgroupSize] = await api.acirGetCircuitSizes(acirUncompressedBytecode);
const crs = await Crs.new(subgroupSize + 1);
await api.commonInitSlabAllocator(subgroupSize);
await api.srsInitSrs(new RawBuffer(crs.getG1Data()), crs.numPoints, new RawBuffer(crs.getG2Data()));

const acirComposer = await api.acirNewAcirComposer(subgroupSize);
return { api: api, composer: acirComposer };
}

// Generate an outer proof. This is the proof for the circuit which will verify
// inner proofs and or can be seen as the proof created for regular circuits.
//
// The settings for this proof are the same as the settings for a "normal" proof
// ie one that is not in the recursive setting.
async generateOuterProof(decompressedWitness: Uint8Array) {
const makeEasyToVerifyInCircuit = false;
return this.generateProof(decompressedWitness, makeEasyToVerifyInCircuit);
}

// Generates an inner proof. This is the proof that will be verified
// in another circuit.
//
// This is sometimes referred to as a recursive proof.
// We avoid this terminology as the only property of this proof
// that matters, is the fact that it is easy to verify in another
// circuit. We _could_ choose to verify this proof in the CLI.
//
// We set `makeEasyToVerifyInCircuit` to true, which will tell the backend to
// generate the proof using components that will make the proof
// easier to verify in a circuit.
async generateInnerProof(witness: Uint8Array) {
const makeEasyToVerifyInCircuit = true;
return this.generateProof(witness, makeEasyToVerifyInCircuit);
}

async generateProof(decompressedWitness: Uint8Array, makeEasyToVerifyInCircuit: boolean) {
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
const proof = await this.api.acirCreateProof(
this.acirComposer,
this.acirUncompressedBytecode,
decompressedWitness,
makeEasyToVerifyInCircuit,
);

return proof;
}

// Generates artifacts that will be passed to a circuit that will verify this proof.
//
// Instead of passing the proof and verification key as a byte array, we pass them
// as fields which makes it cheaper to verify in a circuit.
//
// The proof that is passed here will have been created using the `generateInnerProof`
// method.
//
// The number of public inputs denotes how many public inputs are in the inner proof.
async generateInnerProofArtifacts(proof: Uint8Array, numOfPublicInputs: number = 0) {
const proofAsFields = await this.api.acirSerializeProofIntoFields(this.acirComposer, proof, numOfPublicInputs);

// TODO: perhaps we should put this in the init function. Need to benchmark
// TODO how long it takes.
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
await this.api.acirInitVerificationKey(this.acirComposer);

// Note: If you don't init verification key, `acirSerializeVerificationKeyIntoFields`` will just hang on serialization
kevaundray marked this conversation as resolved.
Show resolved Hide resolved
const vk = await this.api.acirSerializeVerificationKeyIntoFields(this.acirComposer);

return {
proofAsFields: proofAsFields.map((p: any) => p.toString()),
vkAsFields: vk[0].map((vk: any) => vk.toString()),
vkHash: vk[1].toString(),
};
}

async verifyOuterProof(proof: Uint8Array) {
const makeEasyToVerifyInCircuit = false;
const verified = await this.verifyProof(proof, makeEasyToVerifyInCircuit);
return verified;
}

async verifyInnerProof(proof: Uint8Array) {
const makeEasyToVerifyInCircuit = true;
return this.verifyProof(proof, makeEasyToVerifyInCircuit);
}

async verifyProof(proof: Uint8Array, makeEasyToVerifyInCircuit: boolean) {
await this.api.acirInitVerificationKey(this.acirComposer);
return await this.api.acirVerifyProof(this.acirComposer, proof, makeEasyToVerifyInCircuit);
}

async destroy() {
await this.api.destroy();
}
}
Loading
Loading