Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Provide cross compile to cjs. #2566

Merged
merged 6 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .ignore

This file was deleted.

6 changes: 3 additions & 3 deletions barretenberg/cpp/.rebuild_patterns
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
^barretenberg/.*\\.(cpp|cc|cxx|c\\+\\+|h|hpp|hxx|h\\+\\+|c|h|inl|inc|ipp|tpp|cmake)$
^barretenberg/.*CMakeLists\\.txt$
^barretenberg/.*Dockerfile.*$
^barretenberg/cpp/.*\\.(cpp|cc|cxx|c\\+\\+|h|hpp|hxx|h\\+\\+|c|h|inl|inc|ipp|tpp|cmake)$
^barretenberg/cpp/.*CMakeLists\\.txt$
^barretenberg/cpp/.*Dockerfile.*$
^barretenberg/cpp/scripts/
2 changes: 1 addition & 1 deletion barretenberg/ts/.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ node_modules
Dockerfile
.yarn
!.yarn/releases
.tsbuildinfo
.tsbuildinfo*
2 changes: 1 addition & 1 deletion barretenberg/ts/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ COPY yarn.lock yarn.lock
COPY .yarnrc.yml .yarnrc.yml
RUN yarn --immutable
COPY . .
RUN yarn formatting && yarn build:ts
RUN yarn formatting && SKIP_CPP_BUILD=1 yarn build
CMD ["yarn", "test"]
9 changes: 0 additions & 9 deletions barretenberg/ts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,6 @@ in size) is loaded and keeps page load times responsive.
const { Barretenberg, RawBuffer, Crs } = await import('@aztec/bb.js');
```

### CommonJS Usage

The import syntax is slightly different in a CommonJS environment.

```typescript
const { loadModule } = await import('@aztec/bb.js');
const { Barretenberg, RawBuffer, Crs } = await loadModule();
```

## Development

Create a symlink to the root script `bb.js-dev` in your path. You can now run the current state of the code from
Expand Down
10 changes: 0 additions & 10 deletions barretenberg/ts/cjs-entry/index.cjs

This file was deleted.

1 change: 0 additions & 1 deletion barretenberg/ts/cjs-entry/index.d.ts

This file was deleted.

15 changes: 8 additions & 7 deletions barretenberg/ts/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
"homepage": "https://github.com/AztecProtocol/aztec-packages/tree/master/barretenberg/ts",
"license": "MIT",
"type": "module",
"types": "./cjs-entry/index.d.ts",
"types": "./dest/node-cjs/index.d.ts",
"exports": {
".": {
"require": "./cjs-entry/index.cjs",
"require": "./dest/node-cjs/index.js",
"browser": "./dest/browser/index.js",
"default": "./dest/node/index.js"
}
Expand All @@ -22,11 +22,12 @@
"README.md"
],
"scripts": {
"clean": "rm -rf ./dest .tsbuildinfo",
"build": "yarn clean && yarn build:wasm && yarn build:ts",
"build:dev": "tsc -b --watch",
"build:wasm": "cd ../cpp && cmake --preset wasm-threads && cmake --build --preset wasm-threads && cmake --preset wasm && cmake --build --preset wasm && ./scripts/strip-wasm.sh",
"build:ts": "tsc -b && cp ./src/barretenberg-threads.wasm ./dest/node && webpack && chmod +x ./dest/node/main.js",
"clean": "rm -rf ./dest .tsbuildinfo .tsbuildinfo.cjs",
"build": "yarn clean && yarn build:wasm && yarn build:esm && yarn build:cjs && yarn build:browser",
"build:wasm": "./scripts/build_wasm.sh",
"build:esm": "tsc -b && chmod +x ./dest/node/main.js",
"build:cjs": "tsc -b tsconfig.cjs.json && ./scripts/cjs_postprocess.sh",
"build:browser": "webpack",
"build:bindings": "cd .. && ./scripts/bindgen.sh",
"formatting": "prettier --check ./src && eslint --max-warnings 0 ./src",
"formatting:fix": "prettier -w ./src",
Expand Down
21 changes: 21 additions & 0 deletions barretenberg/ts/scripts/build_wasm.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/sh
set -eu

if [ -z "$SKIP_CPP_BUILD" ]; then
# Build the wasms and strip debug symbols.
cd ../cpp
cmake --preset wasm-threads && cmake --build --preset wasm-threads
cmake --preset wasm && cmake --build --preset wasm
./scripts/strip-wasm.sh
cd ../ts
fi

# Copy the wasm to its home in the bb.js dest folder.
# We only need the threads wasm, as node always uses threads.
# We need to take two copies for both esm and cjs builds. You can't use symlinks when publishing.
# This probably isn't a big deal however due to compression.
# When building the the browser bundle, both wasms are inlined directly.
mkdir -p ./dest/node/barretenberg_wasm
mkdir -p ./dest/node-cjs/barretenberg_wasm
cp ../cpp/build-wasm-threads/bin/barretenberg.wasm ./dest/node/barretenberg_wasm/barretenberg-threads.wasm
cp ../cpp/build-wasm-threads/bin/barretenberg.wasm ./dest/node-cjs/barretenberg_wasm/barretenberg-threads.wasm
14 changes: 14 additions & 0 deletions barretenberg/ts/scripts/cjs_postprocess.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/sh
cat >dest/node-cjs/package.json <<!EOF
{
"type": "commonjs"
}
!EOF

DIR="./dest/node-cjs"

# Iterate over all .js files in the directory
for FILE in $(find "$DIR" -name "*.js"); do
# Use sed to replace 'import.meta.url' with '""'
sed -i 's/import\.meta\.url/""/g' "$FILE"
done
1 change: 0 additions & 1 deletion barretenberg/ts/src/barretenberg-threads.wasm

This file was deleted.

1 change: 0 additions & 1 deletion barretenberg/ts/src/barretenberg.wasm

This file was deleted.

1 change: 1 addition & 0 deletions barretenberg/ts/src/barretenberg_wasm/barretenberg.wasm
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,17 @@ import { Worker } from 'worker_threads';
import { dirname } from 'path';
import { fileURLToPath } from 'url';

function getCurrentDir() {
if (typeof __dirname !== 'undefined') {
return __dirname;
} else {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
return dirname(fileURLToPath(import.meta.url));
}
}

export function createMainWorker() {
const __dirname = dirname(fileURLToPath(import.meta.url));
const __dirname = getCurrentDir();
return new Worker(__dirname + `/main.worker.js`);
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,17 @@ import { Worker } from 'worker_threads';
import { dirname } from 'path';
import { fileURLToPath } from 'url';

function getCurrentDir() {
if (typeof __dirname !== 'undefined') {
return __dirname;
} else {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
return dirname(fileURLToPath(import.meta.url));
}
}

export function createThreadWorker() {
const __dirname = dirname(fileURLToPath(import.meta.url));
const __dirname = getCurrentDir();
return new Worker(__dirname + `/thread.worker.js`);
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import barretenbergModule from '../../../barretenberg.wasm';
import barretenbergThreadsModule from '../../../barretenberg-threads.wasm';
import barretenbergModule from '../../barretenberg.wasm';
import barretenbergThreadsModule from '../../barretenberg-threads.wasm';

// Annoyingly the wasm declares if it's memory is shared or not. So now we need two wasms if we want to be
// able to fallback on "non shared memory" situations.
Expand Down
12 changes: 11 additions & 1 deletion barretenberg/ts/src/barretenberg_wasm/fetch_code/node/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,18 @@ import { readFile } from 'fs/promises';
import { dirname } from 'path';
import { fileURLToPath } from 'url';

function getCurrentDir() {
if (typeof __dirname !== 'undefined') {
return __dirname;
} else {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
return dirname(fileURLToPath(import.meta.url));
}
}

// eslint-disable-next-line @typescript-eslint/no-unused-vars
export async function fetchCode(multithreaded: boolean) {
const path = dirname(fileURLToPath(import.meta.url)) + '/../../../barretenberg-threads.wasm';
const path = getCurrentDir() + '/../../barretenberg-threads.wasm';
return await readFile(path);
}
4 changes: 2 additions & 2 deletions barretenberg/ts/src/bindgen/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { generateRustCode } from './rust.js';
import { generateTypeScriptCode } from './typescript.js';

const [, , exports = '../exports.json', lang = 'ts'] = process.argv;
const [, , exp = '../exports.json', lang = 'ts'] = process.argv;

function generateCode(exports: string, lang: string) {
switch (lang) {
Expand All @@ -14,4 +14,4 @@ function generateCode(exports: string, lang: string) {
}
}

console.log(generateCode(exports, lang));
console.log(generateCode(exp, lang));
12 changes: 11 additions & 1 deletion barretenberg/ts/src/crs/node/ignition_files_crs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,20 @@ import { readFile } from 'fs/promises';
import { dirname } from 'path';
import { fileURLToPath } from 'url';

function getCurrentDir() {
if (typeof __dirname !== 'undefined') {
return __dirname;
} else {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
return dirname(fileURLToPath(import.meta.url));
}
}

/**
* The path to our SRS object, assuming that we are in barretenberg/ts folder.
*/
export const SRS_DEV_PATH = dirname(fileURLToPath(import.meta.url)) + '/../../../cpp/srs_db/ignition/monomial';
export const SRS_DEV_PATH = getCurrentDir() + '/../../../cpp/srs_db/ignition/monomial';

/**
* Downloader for CRS from a local file (for Node).
Expand Down
22 changes: 0 additions & 22 deletions barretenberg/ts/src/info.json

This file was deleted.

23 changes: 23 additions & 0 deletions barretenberg/ts/src/info.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/* eslint-disable camelcase */
export const acvmInfoJson = {
language: {
name: 'PLONK-CSAT',
width: 3,
},
opcodes_supported: ['arithmetic', 'directive', 'brillig', 'memory_init', 'memory_op'],
black_box_functions_supported: [
'and',
'xor',
'range',
'sha256',
'blake2s',
'keccak256',
'schnorr_verify',
'pedersen',
'hash_to_field_128_security',
'ecdsa_secp256k1',
'ecdsa_secp256r1',
'fixed_base_scalar_mul',
'recursive_aggregation',
],
};
2 changes: 1 addition & 1 deletion barretenberg/ts/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import createDebug from 'debug';
import { readFileSync, writeFileSync } from 'fs';
import { gunzipSync } from 'zlib';
import { Command } from 'commander';
import acvmInfoJson from './info.json' assert { type: 'json' };
import { acvmInfoJson } from './info.js';
createDebug.log = console.error.bind(console);
const debug = createDebug('bb.js');

Expand Down
8 changes: 8 additions & 0 deletions barretenberg/ts/tsconfig.cjs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"module": "CommonJS",
"outDir": "dest/node-cjs",
"tsBuildInfoFile": ".tsbuildinfo.cjs"
}
}
2 changes: 1 addition & 1 deletion barretenberg/ts/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
"rootDir": "src",
"tsBuildInfoFile": ".tsbuildinfo"
},
"include": ["src", "src/info.json"]
"include": ["src"]
}