Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supply delta live tracer #29347

Merged
merged 30 commits into from
Jun 3, 2024
Merged

Conversation

ziogaschr
Copy link
Contributor

@ziogaschr ziogaschr commented Mar 26, 2024

This PR is based on #24723 by @karalabe / @holiman (and the successor #25662) and uses the live chain tracing with hooks implementation #29189 by @s1na.

This PR adds a live tracer which stores the supply delta of each block during import of a block.
For finding the supply for the whole chain, a fresh sync has to be started.

The tracer output for a single block is:

{
  "issuance": {
    "genesisAlloc": "0x0",
    "reward": "0x0",
    "withdrawals": "0x0"
  },
  "burn": {
    "eip1559": "0x0",
    "blob": "0x0",
    "misc": "0x0"
  },
  "blockNumber": 1,
  "hash": "0x54177b2adb754306d0d267f82537551c467201bf0c8509e60b77229b41c99e3a",
  "parentHash": "0xaf41e72f748de317965454508c749f7e14dc4fe444cd07bca4c981c7e952364d"
}

For using this tracer the CLI flag --vmtrace supply has to be set. Additional options for the tracer can be set as --vmtrace.config '{"path": "/tmp/supply_logs/"}'.

Currently the following options are supported:

path        Path to the directory where the tracer logs will be stored
maxSize     MaxSize is the maximum size in megabytes of the tracer log file before it gets rotated. It defaults to 100 megabytes.

The output of this tracer is stored in log rotated JSONL files in the configured directory, those exported files have to be analysed with another tool.

Example tracer output:

{"delta":320000001000000000000000000,"reward":0,"withdrawals":0,"burn":0,"blockNumber":0,"hash":"0x25a5cc106eea7138acab33231d7160d69cb777ee0c2c553fcddf5138993e6dd9","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000"}
{"delta":2000000000000000000,"reward":2000000000000000000,"withdrawals":0,"burn":0,"blockNumber":1,"hash":"0x696d95da6726a67afd5be2a37d3883e9be8008491b30d5bd1069ea5922fa2a41","parentHash":"0x25a5cc106eea7138acab33231d7160d69cb777ee0c2c553fcddf5138993e6dd9"}
{"delta":2000000000000000000,"reward":2000000000000000000,"withdrawals":0,"burn":0,"blockNumber":2,"hash":"0x89b977ffab1052f5f6f778e4680354a8a1a1e0f623db9b2f20beba9f36f68721","parentHash":"0x696d95da6726a67afd5be2a37d3883e9be8008491b30d5bd1069ea5922fa2a41"}
{"delta":2000000000000000000,"reward":2000000000000000000,"withdrawals":0,"burn":0,"blockNumber":3,"hash":"0x1970908e724ac37efbb0b3bf9f77ebdf04e1ccb3f9da1463bec00252cb3a701f","parentHash":"0x89b977ffab1052f5f6f778e4680354a8a1a1e0f623db9b2f20beba9f36f68721"}

@@ -214,6 +214,44 @@ const (
BalanceDecreaseSelfdestructBurn BalanceChangeReason = 14
)

// String returns a string representation of the reason.
func (r BalanceChangeReason) String() string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use https://pkg.go.dev/golang.org/x/tools/cmd/stringer to generate this method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fjl is this what you meant? 9e21e2d

eth/tracers/live/supply.go Outdated Show resolved Hide resolved
We try to solve the following error:

```
--- FAIL: TestSupplySelfdestruct (1.56s)
    testing.go:1231: TempDir RemoveAll cleanup: remove C:\Users\appveyor\AppData\Local\Temp\1\TestSupplySelfdestruct3803137642\001\supply.jsonl: The process cannot access the file because it is being used by another process.
```
Was expecting lumberjack.Rotate() to close the file
core/blockchain.go Outdated Show resolved Hide resolved
eth/tracers/live/supply.go Outdated Show resolved Hide resolved
ziogaschr and others added 2 commits March 31, 2024 16:55
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
@ziogaschr ziogaschr marked this pull request as ready for review March 31, 2024 20:06
Copy link
Contributor

@s1na s1na left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I was co-author :) I'll wait for others to chime in as well

OnSkippedBlock SkippedBlockHook
OnGenesisBlock GenesisBlockHook
OnBlockchainInit BlockchainInitHook
OnBlockchainTerminate BlockchainTerminateHook
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@s1na what do you think of this name over "OnBlockchainEnd"?

@s1na
Copy link
Contributor

s1na commented Apr 16, 2024

Linking the reader here for visibility: https://github.com/ziogaschr/supply-tracer-parser

core/blockchain.go Outdated Show resolved Hide resolved
Comment on lines 46 to 49
Delta *big.Int `json:"delta"`
Reward *big.Int `json:"reward"`
Withdrawals *big.Int `json:"withdrawals"`
Burn *big.Int `json:"burn"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

big.Int marshals as json numbers. Which means that we're kind of limited to 53 bits, because of javascript. So better if we marshal it as hex-strings (or dec-strings for that matter).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Delta is a signed number. The best thing I can think of is to add a new field for the sign.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your idea @s1na, as supporting Negative numbers for hex isn't the nicest thing and also the consumers of the JSON will have to find a library that supports negative numbers as well.

Another suggestion might be if we use two different fields like DeltaInflation and DeltaDeflation (we have to think of better names) (thanks to @meowsbits for this suggestion)

Copy link
Contributor Author

@ziogaschr ziogaschr May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of this? ziogaschr@3ac74bd#r142142290

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@holiman @s1na I pushed some commits which change the output to the following after our discussion with @s1na.
We dropped the delta, which could have negative numbers and we will let the consumer to do the calculations on their side.

{
  "issuance": {
    "genesisAlloc": "0x0", // omitempty
    "reward": "0x0",
    "withdrawals": "0x0"
  },
  "burn": {
    "1559": "0x0",
    "blob": "0x0",
    "misc": "0x0"
  },
  "blockNumber": 1,
  "hash": "0x54177b2adb754306d0d267f82537551c467201bf0c8509e60b77229b41c99e3a",
  "parentHash": "0xaf41e72f748de317965454508c749f7e14dc4fe444cd07bca4c981c7e952364d"
}

@holiman
Copy link
Contributor

holiman commented Apr 25, 2024

This needs a rebase

# Conflicts:
#	core/tracing/hooks.go
The new output becomes:

```
{
  "issuance": {
    "reward": 0,
    "withdrawals": 0
  },
  "burn": {
    "1559": 0,
    "blob": 0,
    "misc": 0
  },
  "blockNumber": 1,
  "hash": "0x",
  "parentHash": "0x"
}
```
eth/tracers/live/supply.go Outdated Show resolved Hide resolved
Copy link
Contributor

@s1na s1na left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! 🙌 🙌

@s1na s1na merged commit c9e0b31 into ethereum:master Jun 3, 2024
3 checks passed
@s1na s1na added this to the 1.14.4 milestone Jun 3, 2024
@Pazibad
Copy link

Pazibad commented Jun 7, 2024

0x34a9c05b638020a07bb153bf624c8763bf8b4a86

jorgemmsilva pushed a commit to iotaledger/go-ethereum that referenced this pull request Jun 17, 2024
Introduces the first built-in live tracer. The supply tracer tracks ETH supply changes across blocks
and writes the output to disk. This will need to be enabled through CLI using the `--vmtrace supply` flag.

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
stwiname pushed a commit to subquery/data-node-go-ethereum that referenced this pull request Sep 9, 2024
Introduces the first built-in live tracer. The supply tracer tracks ETH supply changes across blocks
and writes the output to disk. This will need to be enabled through CLI using the `--vmtrace supply` flag.

Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants