-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BCF-991 optimize JSON unmarshal'ing for EVM block #8644
Conversation
I see that you haven't updated any CHANGELOG files. Would it make sense to do so? |
6a624e1
to
b733cea
Compare
Use go/codec code generation to optimatize JSON unmarshal'ing of EVM block. Benchmark comparison: TL;DR ~33% runtime reduction, at the cost of 1.5x memory increase Benchmark obtained with /usr/local/go/bin/go test -benchmem -run=^$ -bench="^(BenchmarkBlock_Small_JSONUnmarshal|BenchmarkBlock_Medium_JSONUnmarshal|BenchmarkBlock_Large_JSONUnmarshal|BenchmarkBlock_XL_JSONUnmarshal)$" -count=10 github.com/smartcontractkit/chainlink/core/chains/evm/types > ~/work/jira/BCF-991-json/<tag>.txt Result: benchstat ~/work/jira/BCF-991-json/std.txt ~/work/jira/BCF-991-json/codec.txt goos: darwin goarch: arm64 pkg: github.com/smartcontractkit/chainlink/core/chains/evm/types │ /Users/kreherma/work/jira/BCF-991-json/std.txt │ /Users/kreherma/work/jira/BCF-991-json/codec.txt │ │ sec/op │ sec/op vs base │ Block_Small_JSONUnmarshal-10 10.158µ ± 0% 7.058µ ± 1% -30.51% (p=0.000 n=10) Block_Medium_JSONUnmarshal-10 222.0µ ± 2% 147.6µ ± 1% -33.52% (p=0.000 n=10) Block_Large_JSONUnmarshal-10 1.741m ± 3% 1.159m ± 0% -33.42% (p=0.000 n=10) Block_XL_JSONUnmarshal-10 13.905m ± 0% 9.226m ± 2% -33.65% (p=0.000 n=10) geomean 483.4µ 324.9µ -32.79% │ /Users/kreherma/work/jira/BCF-991-json/std.txt │ /Users/kreherma/work/jira/BCF-991-json/codec.txt │ │ B/op │ B/op vs base │ Block_Small_JSONUnmarshal-10 1.820Ki ± 0% 5.750Ki ± 0% +215.88% (p=0.000 n=10) Block_Medium_JSONUnmarshal-10 48.04Ki ± 0% 117.05Ki ± 0% +143.65% (p=0.000 n=10) Block_Large_JSONUnmarshal-10 383.2Ki ± 0% 932.0Ki ± 0% +143.26% (p=0.000 n=10) Block_XL_JSONUnmarshal-10 2.843Mi ± 0% 7.766Mi ± 0% +173.15% (p=0.000 n=10) geomean 99.38Ki 265.8Ki +167.42% │ /Users/kreherma/work/jira/BCF-991-json/std.txt │ /Users/kreherma/work/jira/BCF-991-json/codec.txt │ │ allocs/op │ allocs/op vs base │ Block_Small_JSONUnmarshal-10 45.00 ± 0% 59.00 ± 0% +31.11% (p=0.000 n=10) Block_Medium_JSONUnmarshal-10 867.0 ± 0% 1240.0 ± 0% +43.02% (p=0.000 n=10) Block_Large_JSONUnmarshal-10 6.701k ± 0% 9.755k ± 0% +45.58% (p=0.000 n=10) Block_XL_JSONUnmarshal-10 53.30k ± 0% 77.86k ± 0% +46.06% (p=0.000 n=10) geomean 1.932k 2.730k +41.31%
b733cea
to
40adb87
Compare
//go:build go1.6 | ||
// +build go1.6 | ||
|
||
// Code generated by codecgen - DO NOT EDIT. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file looks pretty gnarly. Is it possible/would it make sense to have it in an internal/
subdirectory?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe?
if so, it will be a bigger change. the generated methods need to be in the same package as the original types.
it almost works by moving blockInternal and transactionInternal to /internal/foo BUT blockInternal depends on Transaction, not transactionInternal.
so i think we would need another translation between []transactionInternal and []Transaction
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nbd in that case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually, i already did it.
as a nice side effect, the code is even faster now. my guess is the my original code gen had a bunch of extra unneeded stuff.
i like the code layout more now, too
SonarQube Quality Gate 81.0% Coverage on New Code (is less than 90%) See analysis details on SonarQube Fix issues before they fail your Quality Gate with SonarLint in your IDE. |
Use go/codec code generation to optimize JSON unmarshal'ing
of EVM block.
Benchmark comparison:
TL;DR ~50% runtime reduction, and 30% memory reduction
Benchmark obtained with
/usr/local/go/bin/go test -benchmem -run=^$ -bench="^(BenchmarkBlock_Small_JSONUnmarshal|BenchmarkBlock_Medium_JSONUnmarshal|BenchmarkBlock_Large_JSONUnmarshal|BenchmarkBlock_XL_JSONUnmarshal)$" -count=10 github.com/smartcontractkit/chainlink/core/chains/evm/types > ~/work/jira/BCF-991-json/<tag>.txt
Result:
benchstat ~/work/jira/BCF-991-json/std.txt ~/work/jira/BCF-991-json/codec.txt