Remove protobufjs in favor of protobuf-es

[talentlessguy]

This closes #31

Switches to a more correct, maintained and up to date library - protobuf-es. It's more strict so a lot of things are failing. And I also switched from v2 to v3 protobufs because some tooling didn't work properly with it.

WIP

Progress:

yarn test --test-reporter=tap | grep -E '^# tests|^# pass|^# fail'
# tests 258
# pass 246
# fail 12

things that are failing

symlinks:

 actual: Uint8Array(10) [
      10,   8,   8,  4, 18,
       2, 104, 105, 24,  2
    ],
    expected: Uint8Array(8) [
      10, 6,   8,   4,
      18, 2, 104, 105
    ],

raw files with no content:

    actual: Uint8Array(6) [ 10, 4, 40, 0, 56, 0 ],
    expected: Uint8Array(6) [ 10, 4, 8, 0, 24, 0 ],

there's some missing default value in all of those but i'm not sure where

[alanshaw]

Does changing the test framework need to happen in this PR? This looks like quite a large PR to review anyway. It would be easier to review if it didn't include changes not related to the goal.

[talentlessguy]

mocha breaks with ESM for whatever reason, I didn't want to waste time figure it out so I switched to built-in node test framework while it's a draft

[alanshaw]

@talentlessguy you might gain more savings switching to protons and protons-runtime instead? It would dudupe with protons-runtime in ipfs-unixfs-exporter.

[alanshaw]

Note: I think in general I'd prefer to switch to protobuf-es as it seems to have a big community and usage behind it.

[talentlessguy]

@talentlessguy you might gain more savings switching to protons and protons-runtime instead? It would dudupe with protons-runtime in ipfs-unixfs-exporter.

thanks for the suggestion

[achingbrain]

FWIW, last time I ran the benchmark suite, protons was just over 20% faster than protobuf-es when serializing/deserializing - https://github.com/ipfs/protons/tree/main/packages/protons-benchmark#usage

These kind of performance characteristics are very important to high traffic deployments like Lodestar so it's unlikely to be switched away from elsewhere.

[talentlessguy]

FWIW, last time I ran the benchmark suite, protons was just over 20% faster than protobuf-es when serializing/deserializing - https://github.com/ipfs/protons/tree/main/packages/protons-benchmark#usage

do you mean protobufjs, and not protons?

[achingbrain]

No, I mean protons:

protobuf.js is also quite a bit faster than protobuf-es but it's CJS and the stack is ESM-only so that rules it out.