Move to Polkadot SDK

[kayabaNerve]

Running cargo +nightly clippy --all-features --all-targets on remove-subxt (which this builds off) compiles 1215 crates. This compiles 1195 and includes a lot of version increases.

This prematurely merges paritytech/polkadot-sdk#1631 and paritytech/polkadot-sdk#1313 into an experimental branch of serai-dex/polkadot-sdk. The former is needed, as cargo can't resolve the crate graph if we pull in the old libp2p with its older crypto dependencies. The latter is solely a nice to have.

This PR exists to evaluate if the CI passes or if further edits are needed.

[kayabaNerve]

2023-11-28 08:28:12 Error running offchain workers at 0x3514ed54f2dbfa21208dfa1799a87e3b9aa5f2d1e7d4f5055a3e60d59b6914d9: Execution failed: Other error happened while constructing the runtime: failed to instantiate a new WASM module instance: maximum concurrent memory limit of 1 reached for stripe 0

Seems to be a wasmtime misconfig somewhere.

---

====================

Version: 0.1.0-unknown

   0: sp_panic_handler::set::{{closure}}
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2021:9
      std::panicking::rust_panic_with_hook
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:735:13
   2: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:601:13
   3: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/sys_common/backtrace.rs:170:18
   4: rust_begin_unwind
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5
   5: core::panicking::panic_fmt
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14
   6: core::panicking::panic
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:127:5
   7: <sc_service::client::client::Client<B,E,Block,RA> as sc_client_api::backend::LockImportRun<Block,B>>::lock_import_and_run::{{closure}}
   8: sc_consensus_grandpa::environment::finalize_block
   9: <sc_consensus_grandpa::environment::Environment<B,Block,C,N,S,SC,VR> as finality_grandpa::voter::Environment<<Block as sp_runtime::traits::Block>::Hash,<<Block as sp_runtime::traits::Block>::Header as sp_runtime::traits::Header>::Number>>::finalize_block
  10: <finality_grandpa::voter::Voter<H,N,E,GlobalIn,GlobalOut> as core::future::future::Future>::poll
  11: <finality_grandpa::voter::Voter<H,N,E,GlobalIn,GlobalOut> as core::future::future::Future>::poll
  12: <sc_consensus_grandpa::VoterWork<B,Block,C,N,S,SC,VR> as core::future::future::Future>::poll
  13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  14: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll
  15: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  16: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  17: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll
  18: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll
  19: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
  20: tokio::runtime::park::CachedParkThread::block_on
  21: tokio::runtime::context::runtime::enter_runtime
  22: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
  23: tokio::runtime::task::core::Core<T,S>::poll
  24: tokio::runtime::task::harness::Harness<T,S>::poll
  25: std::sys_common::backtrace::__rust_begin_short_backtrace
  26: core::ops::function::FnOnce::call_once{{vtable.shim}}
  27: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2007:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2007:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/sys/unix/thread.rs:108:17
  28: <unknown>
  29: clone


Thread 'tokio-runtime-worker' panicked at 'attempt to divide by zero', /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/ops/arith.rs:484

[kayabaNerve]

This force push doesn't fix any of the prior observed issues and is just misc improvements. It's most notable for updating to polkadot-sdk/experimental rebased on top of the current polkadot-sdk/serai and effectively removing directories-next/option-ext via #467.

This brings the entire Serai tree to 1193 crates when running cargo +nightly clippy --all-features --all-targets (develop is 1215).

[kayabaNerve]

Now the issue is InvalidTransaction. Some minor distinction with TX encoding, I guess? Or I updated the runtime but not serai-client?

[kayabaNerve]

Due to paritytech/polkadot-sdk#2944, we should probably target 1.10 which predates its merge. That conflicts with how libp2p 0.52 still hasn't been merged into polkadot-sdk (so it'll inherently be a later commit).

We could patch libp2p 0.51 to use dalek 4.0 internally? That'd be the other valid solution.

[kayabaNerve]

Polkadot 1.14 has all the dependency updates we want. 1.15 fixes some bugs I have tracked.

As I said above. litep2p is now in tree and both more performant for Polkadot and unstable (yet present and causing various effects, such as re-arranged internals, accordingly). Each new update will presumably make the internals closer to where they should be and make litep2p more and more stable.

I think, given our timeline and the timeline of releases, 1.15/1.16 would be a fair target.