perf(tree): adjust cross-block cache config

[fgimenez]

Based on top of #13769

• Switched to size-aware eviction to have deterministic maximum cache sizes, done for all the cross-block caches.

• Added a test to estimate the sizes of the different elements of the hierarchical structure of the storage cache, applied the result values to the cache's weigher closure.

• Added time_to_live and time_to_idle to all cross-block caches to keep in memory only the most relevant data.

With these values we get similar performance compared with the previous ones, during an execution of reth-bench for a range of 1000 blocks:

• Execution gas/s:

• Storage cache hit rate:

• Storage cache miss rate:

• Storage cache size:

[mattsse]

this a lot for regular machines,
maybe we consider lowering the defaults

since this is the cache max, we will reach this eventually, right?

[fgimenez]

maybe we consider lowering the defaults

yeah, i believe we can get it much lower with the same perf, will give it a try

since this is the cache max, we will reach this eventually, right?

this is the most pessimistic scenario, with all the addresses stored having also 1000 slots used, probably most of them will use less

[fgimenez]

ok, reduced the limits without losing perf, I've updated the charts in the description with the results for these values. also added a note about the worst-case scenario, now it is 9.4Gb.

[mattsse]

let's try this

pending @Rjected

[fgimenez]

let's try this

pending @Rjected

@mattsse thx a lot! just pushed changes for implementing size-aware eviction, with this we have a much more predictable memory usage being at the same time more flexible and keeping very good perf metrics.

Now there is a capacity limit for each cache and, as long as it is not reached, the available space can be used to include entries dictated by requests, for instance a contract can have a lot more storage slots in the cache than the rest without increasing the overall memory footprint.

This works with a weigher closure that determines the memory usage per entry, I've added a test to estimate them accurately and set the right numbers on each weigher closure.

I've also included settings for time-based evictions with explicit values for ttl and idle evictions.

The limits currently set for each cache are: storage: 8Gb, accounts: 0.5Gb, bytecode: 0.5Gb, with this we get very good perf metrics, will update the charts in the description shortly. these are currently hardcoded values, but now it is much more clear what they mean (as opposed to a number of entries in the cache that need to be translated into actual memory usage) and can for instance be configured from the cli.

[Rjected]

great, this all makes sense, and should properly bound everything by size