Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BCF-991 optimize JSON unmarshal'ing for EVM block #8644

Merged
merged 7 commits into from
Mar 8, 2023

Conversation

krehermann
Copy link
Contributor

@krehermann krehermann commented Mar 7, 2023

Use go/codec code generation to optimize JSON unmarshal'ing
of EVM block.

Benchmark comparison:

TL;DR ~50% runtime reduction, and 30% memory reduction
Benchmark obtained with

/usr/local/go/bin/go test -benchmem -run=^$ -bench="^(BenchmarkBlock_Small_JSONUnmarshal|BenchmarkBlock_Medium_JSONUnmarshal|BenchmarkBlock_Large_JSONUnmarshal|BenchmarkBlock_XL_JSONUnmarshal)$" -count=10 github.com/smartcontractkit/chainlink/core/chains/evm/types > ~/work/jira/BCF-991-json/<tag>.txt

Result:
benchstat ~/work/jira/BCF-991-json/std.txt ~/work/jira/BCF-991-json/codec.txt

benchstat ~/work/jira/BCF-991-json/std-refactor.txt ~/work/jira/BCF-991-json/codec-refactor.txt 
goos: darwin
goarch: arm64
pkg: github.com/smartcontractkit/chainlink/core/chains/evm/types
                              │ /Users/kreherma/work/jira/BCF-991-json/std-refactor.txt │ /Users/kreherma/work/jira/BCF-991-json/codec-refactor.txt │
                              │                         sec/op                          │              sec/op                vs base                │
Block_Small_JSONUnmarshal-10                                               10.177µ ± 1%                         5.740µ ± 2%  -43.60% (p=0.000 n=10)
Block_Medium_JSONUnmarshal-10                                               222.0µ ± 0%                         101.0µ ± 1%  -54.49% (p=0.000 n=10)
Block_Large_JSONUnmarshal-10                                               1741.2µ ± 0%                         786.5µ ± 1%  -54.83% (p=0.000 n=10)
Block_XL_JSONUnmarshal-10                                                  13.931m ± 1%                         6.342m ± 0%  -54.47% (p=0.000 n=10)
geomean                                                                     483.8µ                              231.9µ       -52.07%

                              │ /Users/kreherma/work/jira/BCF-991-json/std-refactor.txt │ /Users/kreherma/work/jira/BCF-991-json/codec-refactor.txt │
                              │                          B/op                           │               B/op                 vs base                │
Block_Small_JSONUnmarshal-10                                               1.820Ki ± 0%                        2.945Ki ± 0%  +61.80% (p=0.000 n=10)
Block_Medium_JSONUnmarshal-10                                              48.04Ki ± 0%                        23.91Ki ± 0%  -50.22% (p=0.000 n=10)
Block_Large_JSONUnmarshal-10                                               383.2Ki ± 0%                        188.2Ki ± 0%  -50.89% (p=0.000 n=10)
Block_XL_JSONUnmarshal-10                                                  2.843Mi ± 0%                        1.922Mi ± 0%  -32.40% (p=0.000 n=10)
geomean                                                                    99.38Ki                             71.47Ki       -28.09%

                              │ /Users/kreherma/work/jira/BCF-991-json/std-refactor.txt │ /Users/kreherma/work/jira/BCF-991-json/codec-refactor.txt │
                              │                        allocs/op                        │             allocs/op              vs base                │
Block_Small_JSONUnmarshal-10                                                 45.00 ± 0%                          41.00 ± 0%   -8.89% (p=0.000 n=10)
Block_Medium_JSONUnmarshal-10                                                867.0 ± 0%                          540.0 ± 0%  -37.72% (p=0.000 n=10)
Block_Large_JSONUnmarshal-10                                                6.701k ± 0%                         4.127k ± 0%  -38.41% (p=0.000 n=10)
Block_XL_JSONUnmarshal-10                                                   53.30k ± 0%                         32.80k ± 0%  -38.46% (p=0.000 n=10)
geomean                                                                     1.932k                              1.316k       -31.90%

@github-actions
Copy link
Contributor

github-actions bot commented Mar 7, 2023

I see that you haven't updated any CHANGELOG files. Would it make sense to do so?

@krehermann krehermann force-pushed the feat/BCF-991-faster-json branch 3 times, most recently from 6a624e1 to b733cea Compare March 7, 2023 20:56
Use go/codec code generation to optimatize JSON unmarshal'ing
of EVM block.

Benchmark comparison:

TL;DR ~33% runtime reduction, at the cost of 1.5x memory increase

Benchmark obtained with
/usr/local/go/bin/go test -benchmem -run=^$ -bench="^(BenchmarkBlock_Small_JSONUnmarshal|BenchmarkBlock_Medium_JSONUnmarshal|BenchmarkBlock_Large_JSONUnmarshal|BenchmarkBlock_XL_JSONUnmarshal)$" -count=10 github.com/smartcontractkit/chainlink/core/chains/evm/types > ~/work/jira/BCF-991-json/<tag>.txt

Result:
benchstat ~/work/jira/BCF-991-json/std.txt ~/work/jira/BCF-991-json/codec.txt
goos: darwin
goarch: arm64
pkg: github.com/smartcontractkit/chainlink/core/chains/evm/types
                              │ /Users/kreherma/work/jira/BCF-991-json/std.txt │ /Users/kreherma/work/jira/BCF-991-json/codec.txt │
                              │                     sec/op                     │          sec/op           vs base                │
Block_Small_JSONUnmarshal-10                                      10.158µ ± 0%                7.058µ ± 1%  -30.51% (p=0.000 n=10)
Block_Medium_JSONUnmarshal-10                                      222.0µ ± 2%                147.6µ ± 1%  -33.52% (p=0.000 n=10)
Block_Large_JSONUnmarshal-10                                       1.741m ± 3%                1.159m ± 0%  -33.42% (p=0.000 n=10)
Block_XL_JSONUnmarshal-10                                         13.905m ± 0%                9.226m ± 2%  -33.65% (p=0.000 n=10)
geomean                                                            483.4µ                     324.9µ       -32.79%

                              │ /Users/kreherma/work/jira/BCF-991-json/std.txt │ /Users/kreherma/work/jira/BCF-991-json/codec.txt │
                              │                      B/op                      │          B/op            vs base                 │
Block_Small_JSONUnmarshal-10                                      1.820Ki ± 0%              5.750Ki ± 0%  +215.88% (p=0.000 n=10)
Block_Medium_JSONUnmarshal-10                                     48.04Ki ± 0%             117.05Ki ± 0%  +143.65% (p=0.000 n=10)
Block_Large_JSONUnmarshal-10                                      383.2Ki ± 0%              932.0Ki ± 0%  +143.26% (p=0.000 n=10)
Block_XL_JSONUnmarshal-10                                         2.843Mi ± 0%              7.766Mi ± 0%  +173.15% (p=0.000 n=10)
geomean                                                           99.38Ki                   265.8Ki       +167.42%

                              │ /Users/kreherma/work/jira/BCF-991-json/std.txt │ /Users/kreherma/work/jira/BCF-991-json/codec.txt │
                              │                   allocs/op                    │        allocs/op          vs base                │
Block_Small_JSONUnmarshal-10                                        45.00 ± 0%                 59.00 ± 0%  +31.11% (p=0.000 n=10)
Block_Medium_JSONUnmarshal-10                                       867.0 ± 0%                1240.0 ± 0%  +43.02% (p=0.000 n=10)
Block_Large_JSONUnmarshal-10                                       6.701k ± 0%                9.755k ± 0%  +45.58% (p=0.000 n=10)
Block_XL_JSONUnmarshal-10                                          53.30k ± 0%                77.86k ± 0%  +46.06% (p=0.000 n=10)
geomean                                                            1.932k                     2.730k       +41.31%
@krehermann krehermann force-pushed the feat/BCF-991-faster-json branch from b733cea to 40adb87 Compare March 7, 2023 21:06
@krehermann krehermann marked this pull request as ready for review March 7, 2023 21:33
@krehermann krehermann requested a review from jmank88 March 7, 2023 21:33
//go:build go1.6
// +build go1.6

// Code generated by codecgen - DO NOT EDIT.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file looks pretty gnarly. Is it possible/would it make sense to have it in an internal/ subdirectory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe?

if so, it will be a bigger change. the generated methods need to be in the same package as the original types.

it almost works by moving blockInternal and transactionInternal to /internal/foo BUT blockInternal depends on Transaction, not transactionInternal.

so i think we would need another translation between []transactionInternal and []Transaction

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nbd in that case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, i already did it.

as a nice side effect, the code is even faster now. my guess is the my original code gen had a bunch of extra unneeded stuff.

i like the code layout more now, too

jmank88
jmank88 previously approved these changes Mar 7, 2023
@krehermann krehermann requested a review from jmank88 March 7, 2023 23:01
GNUmakefile Outdated Show resolved Hide resolved
@cl-sonarqube-production
Copy link

SonarQube Quality Gate

Quality Gate failed

Failed condition 81.0% 81.0% Coverage on New Code (is less than 90%)
Failed condition 1 New Critical Issues (is greater than 0)

See analysis details on SonarQube

Fix issues before they fail your Quality Gate with SonarLint SonarLint in your IDE.

@krehermann krehermann merged commit 372aa4c into develop Mar 8, 2023
@krehermann krehermann deleted the feat/BCF-991-faster-json branch March 8, 2023 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants