Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[o1js-main] Add o1js-stub to Mina repo in place of SnarkyJS #14461

Merged
merged 55 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
5dfb734
run only fast checks in rosetta int tests
dkijania Sep 27, 2023
6118445
add fast script for rosetta
dkijania Sep 27, 2023
1d47673
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
dkijania Sep 27, 2023
1d89857
correct path
dkijania Sep 28, 2023
287fdc4
Update rosetta-integration-tests-fast.sh
dkijania Sep 28, 2023
2109210
more validations
dkijania Sep 30, 2023
591b5bf
update counte
dkijania Sep 30, 2023
536d242
Revert "Auxiliary commit to revert individual files from 591b5bfb040d…
dkijania Oct 2, 2023
4066dc1
add quick test with minimal checks and no zkapps use for pull request…
dkijania Oct 2, 2023
32e266f
compilation fixes
dkijania Oct 2, 2023
356d7da
remove leftovers
dkijania Oct 2, 2023
e4ba43a
remove comment
dkijania Oct 2, 2023
6b55520
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
dkijania Oct 2, 2023
528e0bf
remove zkapps completely
dkijania Oct 3, 2023
439ec8f
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
dkijania Oct 3, 2023
daeaf73
Use zkapp_test_transaction for Rosetta integration tests (debug attem…
shimkiv Oct 5, 2023
43dec12
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
shimkiv Oct 5, 2023
d850951
Typo.
shimkiv Oct 5, 2023
16716c9
Typo.
shimkiv Oct 5, 2023
47e96fd
Sending zkapp txns agaisnt Daemon's GraphQL endpoint.
shimkiv Oct 5, 2023
f07fa96
Let's try with Python.
shimkiv Oct 5, 2023
e59e805
python3 -> python
shimkiv Oct 5, 2023
3403057
Install python.
shimkiv Oct 5, 2023
1ba5689
Install python.
shimkiv Oct 5, 2023
645a768
Install python and deps when needed (debugging attempt).
shimkiv Oct 5, 2023
b6a842a
Escaping quotes with jq.
shimkiv Oct 6, 2023
2d7a523
Tested network comms locally.
shimkiv Oct 6, 2023
b3f59d4
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
shimkiv Oct 6, 2023
e0286bd
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
dkijania Oct 9, 2023
d45ab3e
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
dkijania Oct 9, 2023
535358f
test full in PR
dkijania Oct 9, 2023
b829895
Merge remote-tracking branch 'origin/dkijania/tailor_rosetta_int_test…
dkijania Oct 9, 2023
0ebb8a6
log statements in rosetta
dkijania Oct 10, 2023
5b1cfc4
Merge branch 'berkeley' into dkijania/tailor_rosetta_int_tests
dkijania Oct 10, 2023
51347a5
feat(o1js-stubs): add new dune and o1js_stub.ml files for js_of_ocaml…
MartinMinkov Oct 29, 2023
ea04bcb
feat(kimchi): add kimchi lib from o1js-bindings
MartinMinkov Nov 13, 2023
7ae9412
refactor(nix): remove snarky_js and update paths for kimchi and plonk…
MartinMinkov Nov 13, 2023
587a888
feat(dune-project): add bindings_js package to support JavaScript bin…
MartinMinkov Nov 14, 2023
13b8d9f
chore(CODEOWNERS): remove snarkyjs directory from product-eng-reviewe…
MartinMinkov Nov 15, 2023
94bc10b
refactor(Makefile): remove snarkyjs and snarkyjs_no_types build targe…
MartinMinkov Nov 15, 2023
ad4d4d0
Merge branch 'o1js-main' into feat/add-o1js-stubs
mitschabaude Nov 29, 2023
e4249f5
remove(SnarkyJSTest.dhall): delete SnarkyJSTest.dhall file as it's no…
MartinMinkov Nov 30, 2023
7fa9a22
feat(buildkite/scripts): remove test-snarkyjs-bindings-minimal.sh and…
MartinMinkov Nov 30, 2023
2012e2d
chore: remove redundant SnarkyJS build scripts
MartinMinkov Nov 30, 2023
6287942
refactor(Makefile): remove snarkyjs and snarkyjs_no_types build targe…
MartinMinkov Nov 30, 2023
892a941
chore(CODEOWNERS): remove snarkyjs directory from product-eng-reviewe…
MartinMinkov Nov 30, 2023
ae94ff0
Merge branch 'feat/add-o1js-stubs' of github.com:MinaProtocol/mina in…
MartinMinkov Nov 30, 2023
c633177
Merge remote-tracking branch 'origin/dkijania/tailor_rosetta_int_test…
MartinMinkov Dec 11, 2023
8438840
Merge pull request #14683 from MinaProtocol/feat/port-rosetta-zkapp-c…
MartinMinkov Dec 11, 2023
07ba705
delete js_backend from crypto/kimchi
MartinMinkov Dec 11, 2023
3180007
[rosetta] Fix broken SQL quesry for zkapp account updates.
Sventimir Oct 18, 2023
d4314b0
Merge branch 'feat/add-o1js-stubs' of github.com:MinaProtocol/mina in…
MartinMinkov Dec 13, 2023
b6a14e0
move /kimchi/{wasm,js} to kimchi_bindings
mitschabaude Dec 18, 2023
f7a0abc
fix jsoo complaint about free variable tsBindings
mitschabaude Dec 18, 2023
cb151f6
add back newline
mitschabaude Dec 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,4 @@
branch = master
[submodule "src/external/c-reference-signer"]
path = src/external/c-reference-signer
url = https://github.com/MinaProtocol/c-reference-signer.git
[submodule "src/lib/snarky_js_bindings/snarkyjs"]
path = src/lib/snarkyjs
url = https://github.com/o1-labs/snarkyjs.git
url = https://github.com/MinaProtocol/c-reference-signer.git
1 change: 0 additions & 1 deletion CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@
/src/lib/snarky_curves/ @MinaProtocol/crypto-eng-reviewers
/src/lib/snarky_field_extensions/ @MinaProtocol/crypto-eng-reviewers
/src/lib/snarky_group_map/ @MinaProtocol/crypto-eng-reviewers
/src/lib/snarkyjs @MinaProtocol/product-eng-reviewers
/src/lib/snarky_log/ @MinaProtocol/crypto-eng-reviewers
/src/lib/unsigned_extended/ @MinaProtocol/crypto-eng-reviewers

Expand Down
12 changes: 0 additions & 12 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -115,18 +115,6 @@ build_intgtest: ocaml_checks
dune build --profile=$(DUNE_PROFILE) src/app/test_executive/test_executive.exe src/app/logproc/logproc.exe
$(info Build complete)

snarkyjs: ocaml_checks
$(info Starting Build)
((ulimit -s 65532) || true) && (ulimit -n 10240 || true) \
&& bash ./src/lib/snarkyjs/src/bindings/scripts/build-snarkyjs-node.sh
$(info Build complete)

snarkyjs_no_types: ocaml_checks
$(info Starting Build)
((ulimit -s 65532) || true) && (ulimit -n 10240 || true) \
&& bash ./src/lib/snarkyjs/src/bindings/scripts/build-snarkyjs-node-artifacts.sh
$(info Build complete)

rosetta_lib_encodings: ocaml_checks
$(info Starting Build)
(ulimit -s 65532 || true) && (ulimit -n 10240 || true) && dune build src/lib/rosetta_lib/test/test_encodings.exe --profile=mainnet
Expand Down
17 changes: 0 additions & 17 deletions buildkite/scripts/build-js-tests.sh

This file was deleted.

12 changes: 0 additions & 12 deletions buildkite/scripts/build-snarkyjs-bindings.sh

This file was deleted.

16 changes: 0 additions & 16 deletions buildkite/scripts/test-snarkyjs-bindings-minimal.sh

This file was deleted.

19 changes: 0 additions & 19 deletions buildkite/scripts/test-snarkyjs-bindings.sh

This file was deleted.

50 changes: 0 additions & 50 deletions buildkite/src/Jobs/Test/SnarkyJSTest.dhall

This file was deleted.

19 changes: 0 additions & 19 deletions nix/javascript.nix
Original file line number Diff line number Diff line change
Expand Up @@ -23,25 +23,6 @@ in {
mv src/*.js $out/share/client_sdk
'';
};
snarky_js = nix-npm-buildPackage.buildNpmPackage {
src = ../src/lib/snarkyjs;
preBuild = ''
BINDINGS_PATH=./src/bindings/compiled/node_bindings
mkdir -p "$BINDINGS_PATH"
cp ${plonk_wasm}/nodejs/plonk_wasm* "$BINDINGS_PATH"
cp ${ocamlPackages_mina.mina_client_sdk}/share/snarkyjs_bindings/snarky_js_node*.js "$BINDINGS_PATH"
chmod -R 777 "$BINDINGS_PATH"

# TODO: deduplicate from ./scripts/build-snarkyjs-node.sh
# better error messages
# TODO: find a less hacky way to make adjustments to jsoo compiler output
# `s` is the jsoo representation of the error message string, and `s.c` is the actual JS string
sed -i 's/function failwith(s){throw \[0,Failure,s\]/function failwith(s){throw globalThis.Error(s.c)/' "$BINDINGS_PATH"/snarky_js_node.bc.js
sed -i 's/function invalid_arg(s){throw \[0,Invalid_argument,s\]/function invalid_arg(s){throw globalThis.Error(s.c)/' "$BINDINGS_PATH"/snarky_js_node.bc.js
sed -i 's/return \[0,Exn,t\]/return globalThis.Error(t.c)/' "$BINDINGS_PATH"/snarky_js_node.bc.js
'';
npmBuild = "npm run build";
};

# Jobs/Release/LeaderboardArtifact
leaderboard = nix-npm-buildPackage.buildYarnPackage {
Expand Down
40 changes: 0 additions & 40 deletions nix/ocaml.nix
Original file line number Diff line number Diff line change
Expand Up @@ -297,46 +297,6 @@ let
dune exec --profile=dev src/app/reformat/reformat.exe -- -path . -check
'';

# Javascript Client SDK
mina_client_sdk = self.mina-dev.overrideAttrs (_: {
pname = "mina_client_sdk";
version = "dev";
src = filtered-src;

outputs = [ "out" ];

checkInputs = [ pkgs.nodejs-16_x ];

MINA_VERSION_IMPLEMENTATION = "mina_version.dummy";

buildPhase = ''
dune build --display=short \
src/lib/crypto/kimchi_bindings/js/node_js \
src/app/client_sdk/client_sdk.bc.js \
src/lib/snarky_js_bindings/snarky_js_node.bc.js \
src/lib/snarky_js_bindings/snarky_js_web.bc.js
'';

doCheck = true;
checkPhase = ''
node src/app/client_sdk/tests/run_unit_tests.js

dune build src/app/client_sdk/tests/test_signatures.exe
./_build/default/src/app/client_sdk/tests/test_signatures.exe > nat.consensus.json
node src/app/client_sdk/tests/test_signatures.js > js.nonconsensus.json
if ! diff -q nat.consensus.json js.nonconsensus.json; then
echo "Consensus and JS code generate different signatures";
exit 1
fi
'';

installPhase = ''
mkdir -p $out/share/client_sdk $out/share/snarkyjs_bindings
mv _build/default/src/app/client_sdk/client_sdk.bc.js $out/share/client_sdk
mv _build/default/src/lib/snarky_js_bindings/snarky_js_*.js $out/share/snarkyjs_bindings
'';
});

# Integration test executive
test_executive-dev = self.mina-dev.overrideAttrs (oa: {
pname = "mina-test_executive";
Expand Down
8 changes: 4 additions & 4 deletions nix/rust.nix
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ in
};

kimchi-rust = rustChannelFromToolchainFileOf
../src/lib/snarkyjs/src/bindings/kimchi/wasm/rust-toolchain.toml;
../src/lib/crypto/kimchi/wasm/rust-toolchain.toml;

# TODO: raise issue on nixpkgs and remove workaround when fix is applied
kimchi-rust-wasm = (final.kimchi-rust.rust.override {
Expand All @@ -124,7 +124,7 @@ in
plonk_wasm =
let

lock = ../src/lib/snarkyjs/src/bindings/kimchi/wasm/Cargo.lock;
lock = ../src/lib/crypto/kimchi/wasm/Cargo.lock;

deps = builtins.listToAttrs (map
(pkg: {
Expand Down Expand Up @@ -165,10 +165,10 @@ in
pname = "plonk_wasm";
version = "0.1.0";
src = final.lib.sourceByRegex ../src [
"^lib(/snarkyjs(/src(/bindings(/kimchi(/wasm(/.*)?)?)?)?)?)?$"
"^lib(/crypto(/kimchi(/wasm(/.*)?)?)?)?$"
"^lib(/crypto(/proof-systems(/.*)?)?)?$"
];
sourceRoot = "source/lib/snarkyjs/src/bindings/kimchi/wasm";
sourceRoot = "source/lib/crypto/kimchi/wasm";
nativeBuildInputs = [ final.wasm-pack wasm-bindgen-cli ];
buildInputs = with final; lib.optional stdenv.isDarwin libiconv;
cargoLock.lockFile = lock;
Expand Down
1 change: 0 additions & 1 deletion scripts/build-snarkyjs-node.sh

This file was deleted.

1 change: 0 additions & 1 deletion scripts/update-snarkyjs-bindings.sh

This file was deleted.

1 change: 1 addition & 0 deletions src/dune-project
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
(package (name best_tip_merger))
(package (name best_tip_prover))
(package (name bignum_bigint))
(package (name bindings_js))
(package (name blake2))
(package (name blockchain_snark))
(package (name block_producer))
Expand Down
28 changes: 28 additions & 0 deletions src/lib/crypto/kimchi/js/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
This library provides a wrapper around the WebAssembly prover code, which
allows `js_of_ocaml` to compile the mina project against the WebAssembly
backend.

The different versions of the backend are generated in subdirectories; e.g. the
NodeJS backend is generated in `node_js/` and the Web backend is generated
in `web/`. To use a backend, run `dune build backend/plonk_wasm.js` and copy
`backend/plonk_wasm*` to the project directory.

Note that the backend code is not automatically compiled while linking against
the backend library. You should always manually issue a build command for the
`plonk_wasm.js` for the desired backend to ensure that it has been generated.
For example, to run the nodejs tests in the `test/nodejs` directory you will
need to run

```
dune build src/lib/marlin_plonk_bindings/js/test/nodejs/nodejs_test.bc.js
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be changed to the right repo.

src/lib/marlin_plonk_bindings/js/test/nodejs/copy_over.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

```

Similarly, to run the web tests in `test/web`, you can run

```
dune build src/lib/marlin_plonk_bindings/js/test/web/web_test.bc.js
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

src/lib/marlin_plonk_bindings/js/test/web/copy_over.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

```

and then visit `http://localhost:8000` from a browser.
50 changes: 50 additions & 0 deletions src/lib/crypto/kimchi/js/bindings-bigint256.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/* global tsBindings
*/

// Provides: caml_bigint_256_of_numeral
// Requires: tsBindings
var caml_bigint_256_of_numeral = tsBindings.caml_bigint_256_of_numeral;

// Provides: caml_bigint_256_of_decimal_string
// Requires: tsBindings
var caml_bigint_256_of_decimal_string = tsBindings.caml_bigint_256_of_decimal_string;

// Provides: caml_bigint_256_num_limbs
// Requires: tsBindings
var caml_bigint_256_num_limbs = tsBindings.caml_bigint_256_num_limbs;

// Provides: caml_bigint_256_bytes_per_limb
// Requires: tsBindings
var caml_bigint_256_bytes_per_limb = tsBindings.caml_bigint_256_bytes_per_limb;

// Provides: caml_bigint_256_div
// Requires: tsBindings
var caml_bigint_256_div = tsBindings.caml_bigint_256_div;

// Provides: caml_bigint_256_compare
// Requires: tsBindings
var caml_bigint_256_compare = tsBindings.caml_bigint_256_compare;

// Provides: caml_bigint_256_print
// Requires: tsBindings
var caml_bigint_256_print = tsBindings.caml_bigint_256_print;

// Provides: caml_bigint_256_to_string
// Requires: tsBindings
var caml_bigint_256_to_string = tsBindings.caml_bigint_256_to_string;

// Provides: caml_bigint_256_test_bit
// Requires: tsBindings
var caml_bigint_256_test_bit = tsBindings.caml_bigint_256_test_bit;

// Provides: caml_bigint_256_to_bytes
// Requires: tsBindings
var caml_bigint_256_to_bytes = tsBindings.caml_bigint_256_to_bytes;

// Provides: caml_bigint_256_of_bytes
// Requires: tsBindings
var caml_bigint_256_of_bytes = tsBindings.caml_bigint_256_of_bytes;

// Provides: caml_bigint_256_deep_copy
// Requires: tsBindings
var caml_bigint_256_deep_copy = tsBindings.caml_bigint_256_deep_copy
Loading
Loading