Add CI test that checks that attempt publishing crates and compile them to Wasm

[MaksymZavershynskyi]

Some of the crates in nearcore are made publishable to crates.io. This is done so that external tools like near-sdk-sim can use them without pulling cross-repo github dependency which frequently break.

Also, some of the crates need to be compilable to Wasm because they are used for test in near-sd-as.

Unfortunately, there are no CI checks that make sure this is correct, and PRs like this one d336b3f#r52341955 can break the above invariants. We need to add the following checks into our CI:

• A dry-run attempt to publish publishable crates to crates.io;
• A Wasm compilation of certain crates.

[mina86]

Adding a dry-run publish test won’t work at this stage. Before such
check is added we need to:

• publish a new near-primitives release version 0.1.0 (alternatively
  update dependencies to in Cargo.toml to 0.1.0-pre.1),

• publish a new ‘near-vm-errors’ release which includes
  protocol_feature_alt_bn128 feature and

• publish a new near-vm-runner release version 3.0.0,

• create a new near-evm-runner crate.

To be honest I’m sceptical that such test will actually be beneficial.
For example, it will force us to publish crates as soon as a new
feature is added to them. Similarly, make new crates even if they’re
at a very early experimental stages.

[matklad]

Also, some of the crates need to be compilable to Wasm because they are used for test in near-sd-as.

Oh wow, I didn't realize that assembly script tests work by compiling our vmlogic to wasm to implement mocked blockchain in the javascript environment. That makes sense, but also indeed places a rather big constraint on what we are allowed to do in the relevant parts.

Publishing to crates.io and wasm compatability are rather big asks, because the establish a boundary in the code. It's best if this boundary is physically reflected in code layout, rather than it being an informal agreement that "we publish this, this, and this crate, and we don't publish that crate". As such, I think the best long-term solution is to introduce a single umbrella crate for publishing & wasm compat.

So, something like nearcore-sim-api crate in this repository would re-export vm-logic, vm-errors and everything else which is needed by use-cases outside of nearcore. This crate will have a readme explaining that this is a semi-public API of nearcore used by the simulators, it will have additional CI checks, and a reminder to not needlessly break its APIs. Long-term, we can even work on future-proofing this crate's API, so as to minimize catch up work by near-sdk-as / -sim.

That's a long-term solution though, because it requires changing existing consumers of already-published crates. Not sure what's the best near-term solution would be. I suspect it actually might be working towards the long-term solution? I bet the current APIs have already diverged enough to make upgrade of nearcore crates in near-sdk-as non-trivial, so we might as well try to introduce a proper boundary here?

cc @austinabell, @willemneal

[bowenwang1996]

@matklad given what you said, should we close this issue and create a new one on the long term solution?

[mina86]

@matklad, @nearmax, any decision here?

[matklad]

I believe we are still in the state of confusion regarding this issue, no decision has been made, and the status-quo is self-contradictory.

To reach a decision, we need input from devx team (cc @mikedotexe) on the following questions:

• Has the decision on sandbox have been reached (cc (@DiscRiskandBisque as product vision owner)? Are we going to maintain&develop our current testing infra, or are we freezing it as is, and focus all efforts on the sandbox?

• will near-sdk-rs update its dependency on near-vm-logic and near-primitives-core?

• will near-sdk-sim update its dependencies on a bunch of nearcore crates?

• will near-sdk-as update its dependencies on near-vm-logic and near-vm-errors

• will near-sdk-as migrate to using near-sdk-sim compiled to wasm or host?

• Additionally, I've heard from @frol that there's a desire to share some nearcore code with projects that want to interact with nearcore RPC endpoints. Is there a write up somewhere describing what do we want to share exactly?

I suggest the following course of action:

• If we are unsure if we'd want to upgrade sdk deps on nearcore, let's do nothing and wait. If the need comes to publish the crates, we can fix publishing at that point.
• if we think that we won't be using nearcore in sdks, then we can close this issue, and reconsider primitives/primitives-core split
• if we think that we will continue developing our current testing infra, then yes, we need to continue checking that vm-logic is publishable and compatible with wasm

Note that I believe that the issue is non-urgent -- there's no active work which is blocked on reaching the decision here.

[matklad]

I suggest closing the issue (feel free to re-open if you strongly disagree).

• we still don't have specific understanding of what we want to publish.
• we no longer need to compile stuff to wasm, sdk now doesn't depend on nearcore on wasm

It seems like it's less long-term work to setup publishing later, when we either know what exactly do we want to publish, or when we need to publish something ASAP.