trustfall_core: Optimize serde_type_serializer

[nvzqz]

Rather than collect a new BTreeMap with Strings for Types, this instead will:

• Serialize over the existing BTreeMap
• Create shorter-lived Strings

[obi1kenobi]

Awesome!

[nvzqz]

Is there a benchmark that exercises this code path? Just curious to know what the numbers might be.

[obi1kenobi]

Serializing IR is a rare use case, so this code is primarily used in the test suite. It does the usual (and clever!) compilers trick that allows it to run end-to-end tests but with all stages in parallel: the repo contains input queries and snapshots of the behavior/output produced by every stage (parser, frontend, interpretation, query outputs).

Each snapshot is the output of one stage and the input to the next, so we have tests for "each stage when fed proper input produces proper output" and the composition of those tests is an end-to-end correctness guarantee as well.

This is the primary place where we serialize and deserialize IR right now. The serialization path is used when generating a new IR file either for a new test input or to update an existing one.

[obi1kenobi]

The best thing to benchmark is the contents of ./scripts/recreate_all_ir.sh but the variables portion of that is relatively small: there are 0-4 variables per query and a lot more other IR content. So even that isn't a great benchmark target because it's noisy and large compared to the effect being measured.

But I'm sure there's more low-hanging fruit to be picked.

For example, DataContext in the interpreter gets cloned a lot, and I suspect that making its vertices / folded_contexts / folded_values / imported_tags maps be copy-on-write and structurally shared instead of deep-cloned would produce a serious perf improvement. It's been on my todo list for a while but I've never gotten around to it.

I wouldn't touch the other fields, the rest are either about to be changed to a different representation altogether or removed entirely or are almost-always empty so cloning is cheap.