[feat] Add the functionality for custom RISC-V intrinsics, support hints in RISC-V

[Golovanov399]

This should resolve INT-2498, INT-2474, INT-2409.
Also towards INT-2489

What this does:

• Create auxiliary macros custom_insn_i! and custom_insn_r! for creating our own new instructions in Rust-V (maybe in the future we will need other modifications, as to which registers we care about and which we don't),
• Create some io functions like "read a byte", "read a vector" and so on (basically "read a byte" is "run this asm instruction and then return the contents of this register"),
• Add another variation of PhantomInstruction::HintInput that differs only in prepending with 4-byte length decomposition instead of 1-usize.

---

Copilot:
This pull request includes several changes to the toolchain and vm modules to enhance the handling of hint streams and custom instructions. The most important changes include the addition of new hint stream utilities, the introduction of a new instruction variant, and updates to existing instruction handling logic.

Enhancements to Hint Stream Handling:

• New Hint Stream Utilities: Added various functions and macros in toolchain/riscv/axvm/src/intrinsics/io.rs for reading from and writing to the hint stream, including read_u32, read_vec, read_size_and_vec, and hint_input.
• Module Export: Exported the new hint stream utilities in toolchain/riscv/axvm/src/intrinsics/mod.rs.

Introduction of New Instruction Variant:

• New Instruction Variant: Added HintInputRv32 to the PhantomInstruction enum in toolchain/instructions/src/phantom.rs, which prepares the next input vector for hinting with a 4-byte decomposition of its length.

Updates to Instruction Handling Logic:

• Custom Instruction Macros: Added macros in toolchain/riscv/platform/src/intrinsics/mod.rs for defining custom instructions, such as custom_insn_i and custom_insn_r.
• Instruction Handling: Updated the logic in toolchain/riscv/transpiler/src/rrs.rs and vm/src/system/phantom/mod.rs to handle the new HintInputRv32 instruction variant. [1] [2]

Additional Changes:

• Feature Flag: Enabled the asm_const feature in toolchain/riscv/axvm/src/lib.rs.
• Example Program: Added a new example program in toolchain/riscv/examples/hint/program to demonstrate the usage of the new hint stream utilities and instructions. [1] [2] [3]

These changes collectively improve the handling of hint streams and custom instructions, providing more flexibility and functionality for developers working with the toolchain and vm modules.

[linear]

INT-2498 need a different `hintinput_rv32` phantom instruction when `len` is a word

hintinput loads len into input stream as F

but for hintstorew_rv32 use, we need len loaded as 4 bytes.

INT-2474 Macros for custom_insn in axvm-platform

Make macros custom_insn_r! and custom_insn_i! to wrap the asm! macro for inserting custom RISC-V insn directives for intrinsics.

This should go in some src/platform/intrinsics/mod.rs file.

INT-2409 Intrinsics for `hintstorew` and `hintinput`

Make intrinsic opcode for HINTSTORE_RV32 and update transpiler.

The actual hint function in rust will be more complicated since it will need to hint in a byte slice &[u8].

End flow: you have a struct that is input you want hinted into program.
Provide the struct to VirtualMachine constructor.
There is some converter function that serializes struct into &[u8] and then adds it to hint_stream.
In the program when you call something like axvm_hintit will read in the entire &[u8] by repeatedly calling HINTSTORE_RV32, and then deserialize back into the struct.

Need to study how risc0 does this; I believe they just use bincode(no_std) for serialize and deserialize.

• update: basically they use bincode (no_std crate) but implement a custom Reader that impl Read trait: https://github.com/risc0/risc0/blob/main/risc0/zkvm/src/guest/env/read.rs

For this ticket, just implement in axvm/src/intrinsics/io.rs the inline function wrapping the custom riscv instructions for hintstorew and hintinput.

[jonathanpwang]

is this a nightly feature?

[Golovanov399]

the imm = const $imm in asm! macro doesn't work without this. I don't know if it's nightly or not, here is its origin: rust-lang/rust#93332

[jonathanpwang]

what is this magic

[jonathanpwang]

ok it seems it's because we hint to memory and register is stack. please add some comments because you're too clever

[Golovanov399]

Sorry, I was just doing what compiler suggested

[jonathanpwang]

LGTM

[linear]

INT-2489 Rust hint input support

Firstly we just want a rust function that will hint in a Vec<u8> using combination of hintinput and hintstorew intrinsics.

This is something like:

• https://github.com/risc0/risc0/blob/main/risc0/zkvm/src/guest/env/read.rs
• or https://github.com/succinctlabs/sp1/blob/dev/crates/zkvm/lib/src/io.rs read_vec
  • I think this is simpler.

Next is that we don't want to just hint in a raw Vec<u8> but an arbitrary type T: Deserialize. Here it seems there are two different approaches:

• risc0: write your own deserializer https://github.com/risc0/risc0/blob/main/risc0/zkvm/src/serde/deserializer.rs
• sp1: just use bincode https://github.com/succinctlabs/sp1/blob/626b8de877366b389e9983362c7e73506b4a5d6d/crates/zkvm/lib/src/io.rs#L86

I think we can just use bincode to start, although rkyv is worth investigating (https://www.reddit.com/r/rust/comments/jss6h4/rkyv_a_zerocopy_deserialization_framework_for_rust/)