debug_traceCall returning inaccurate gasUsed when both blockOverrides and stateOverrides are used

[Bitwise0x]

System information
Geth 1.14.5
Go Version: go1.22.4

The gas returned from debug_traceCall with stateOverrides often resulting inaccuracy when trying to simulate for the next block. Over estimating in all the cases. Am I missing something?

``` ` // Adding 12 seconds to the current timestamp
nextBlockTime = header.Time + 12
tx := request{To: transaction.To().Hex(), From: from.Hex(), Value: fmt.Sprintf("0x%x", transaction.Value()), Gas: fmt.Sprintf("0x%x", transaction.Gas()), GasPrice: "0x" + transaction.GasFeeCap().Text(16),Data: hexutil.Encode(transaction.Data())}
	params := make(map[string]interface{})
	params[]"tracer"] = "callTracer"
	params["tracerConfig"] = tracerConfig{OnlyTopCall: true}
        params["blockOverrides"] = make(map[string]interface{})
	params["blockOverrides"].(map[string]interface{})["number"] = "0x" + nextBlock.Text(16)
	params["blockOverrides"].(map[string]interface{})["time"] = fmt.Sprintf("0x%x", nextBlockTime)
	if overrides != nil {
		params["stateOverrides"] = make(map[string]interface{})
		switch stateoverride := params["stateOverrides"].(type) {
		case map[string]interface{}:
			for key, value := range overrides {
				stateoveride[key] = make(map[string]interface{})
				switch statediff := stateoveride[key].(type) {
				case map[string]interface{}:
					statediff["stateDiff"] = make(map[string]string)
					statediff["stateDiff"] = value
				}
			}
		}
	}
gClient.Call(&result, "debug_traceCall", tx, "latest", params)``

[Bitwise0x]

@karalabe @rjl493456442 @holiman

[s1na]

The gas returned is how much gas was used during the simulation. It is not necessarily a good estimate for submitting the tx. For that you should rather use eth_estimateGas.

Otherwise if it is inaccurate it helps if you demonstrate what you expect and what is returned.

[Bitwise0x]

@s1na
Why is debug_traceCall not accurate when both stateOverrides and blockOverrides are used within the same call? I am trying to simulate my transaction beneath another transaction, and was expecting an accurate gasUsed.

debug_traceCall with just blockOverrides works perfectly. however, when combined with stateOverrides, it overestimates gas usage by approximately 5-15k.

[s1na]

@Bitwise0x This is a difficult question to answer. You are modifying the state, that can change the flow of execution and require a different amount of gas. When submitting on chain the state might not be the same as in your override though.

Ideally you give me something to reproduce.

[Otto-AA]

I'm not sure if this is the same issue the OP described, but here's a minimal example at Erigon: erigontech/erigon#11254

I've tested it with Quicknode (Geth/v1.14.5-stable-0dd173a7/linux-amd64/go1.22.4) and it reports the same issue.

[s1na]

debug_traceCall shows a lower gasCost for addresses, that were modified by the stateOverrides.

Based on this I thought maybe we consider the overridden storage as a dirty slot (in EIP-2200 terms). But that seems not to be the case.

I have this contract deployed:

common.BytesToAddress([]byte{32}): {Code: common.Hex2Bytes("600260055500"), Nonce: 1, Storage: map[common.Hash]common.Hash{common.BytesToHash([]byte{0x5}): common.BytesToHash([]byte{0x1})}}

Then do the following traceCall:

> debug.traceCall({ to: '0x0000000000000000000000000000000000000020', gas: '0x186a0' }, 'latest', { stateOverrides: { '0x0000000000000000000000000000000000000020': { stateDiff: { '0x0000000000000000000000000000000000000000000000000000000000000005': '0x0000000000000000000000000000000000000000000000000000000000000000' } } }})

I.e. the "original" value for the slot is 0x1. I change the "current" to 0x0. The code will write the value of 0x2 to the slot. So that should try to deduct some of the refund (and fail). But instead I get the "correct" gas cost for the SSTORE:

{
      depth: 1,
      gas: 78994,
      gasCost: 22100,
      op: "SSTORE",
      pc: 4,
      stack: ["0x2", "0x5"],
      storage: {
        0000000000000000000000000000000000000000000000000000000000000005: "0000000000000000000000000000000000000000000000000000000000000002"
      }
  }

[s1na]

This clause commits the overrides so they are not considered dirty. And I'm pretty sure we don't add them to the access list either. So I can't think why addresses modified by stateOverrides should have a different gas cost.

go-ethereum/internal/ethapi/api.go

Lines 1014 to 1017 in 733fcbb

	// Now finalize the changes. Finalize is normally performed between transactions.
	// By using finalize, the overrides are semantically behaving as
	// if they were created in a transaction just before the tracing occur.
	statedb.Finalise(false)

[fjl]

@Bitwise0x again, could you please provide an example so we can reproduce and see the expected gas usage ourself?

[Bitwise0x]

@fjl @s1na when I tried to simulate the backrun transaction with stateoverrides, gas usage was significantly higher

Signal: 0xfa64ed459ec08a373cb1555e4f11672da415d5e7562b768207b23bc225ae77f7
Backrun: 0x772bd92cf60fbcd76cffb27a04a141e8275a16bb483a8f7b6596552c70fad6e5

[Otto-AA]

@s1na After spending some more hours on this issue, I think the issue I reported occurs only with Erigon. It seems that my provider transparently switches the used node based on the block number, which made me think I also reproduced it on Geth :(

Did you test the python script from the linked Erigon issue (repro.py.txt)? If you run it against a Geth instance and there is no "Found first step with differences" printed to the console, then it should be an Erigon-only issue and you can ignore my comments above. I can't find a public Geth instance to test it against.

[gballet]

@Bitwise0x any update on this? We need your input, and will close this issue otherwise. To be clear, your last response does not help us, we would like to know of a way to reproduce this issue. There's not much we can do otherwise.