-
Notifications
You must be signed in to change notification settings - Fork 20.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
eth/catalyst: evict old payloads, type PayloadID #24236
Conversation
2928711
to
1d4e02f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I've added a commit on top that enables some more tracing
3c1e5fb
to
0ab6a0d
Compare
Amended commit to fix logging inconsistency: |
@protolambda I've added another test on top |
1189cbe
to
8d4c6ea
Compare
8d4c6ea
to
8a04197
Compare
We discussed this PR in triage a bit and came to the conclusion that a cache is not a great structure for this. |
@MariusVanDerWijden Can you define the desired eviction rules? Simplicity is nice too. It's just temporary memory of the latest few payloads that produce/get functions need to exchange. I can help if you can describe what you need in more detail. |
eth/catalyst/api.go
Outdated
@@ -86,7 +88,8 @@ type ConsensusAPI struct { | |||
eth *eth.Ethereum | |||
les *les.LightEthereum | |||
engine consensus.Engine // engine is the post-merge consensus engine, only for block creation | |||
preparedBlocks map[uint64]*ExecutableDataV1 | |||
preparedBlocks *lru.Cache // preparedBlocks caches payloads (*ExecutableDataV1) by payload ID (PayloadID) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is no really a good one. The role of a "cache" is to keep some recent data available-ish, but it's at the sole discretion of the implementation as to how long and when to evict. Whilst it might seem like not a bad idea at first to use a cache to store recent payloads, losing control of storage determinism can make thing in the future a lot harder to debug.
E.g. Certain caches have limits on the maximum size an item can have and will simply refuse to store something larger. In this context it could happen that if the payload exceeds a certain size, the cache would not store it anymore, making it unmineable. If all nodes would create a similarly large block they can't store, we could end u[ with a stalled network. Now I don't know if this particular cache behaves like that or not - or what the limits are - but this is exactly why a cache should not be used, because it's a disk access "optimization", not a storage data structure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed about a cache not being the right fit here after a second look. But previously it would just keep growing indefinitely, and I discussed this change and eviction number with Marius before implementing it. Least-recently-used as eviction strategy + a fixed size limit seemed reasonable enough. Can you describe what you prefer instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depends how much work you want to put in and what the size of the set will be.
If we take the current cap of 10
, then you could use a simple slice and just shift the contents on every insert and do a linear search on every retrieval. We're expected do do these ops rarely when it's our turn to sign, and even then once to create the payload and once to retrieve it within a 14 second time fame. A 10 item iteration doesn't matter in that scope.
If we were to make the set larger, the best solution would probably be a map as it was originally and an eviction order queue. That could be a channel or a circular queue (we have one in the code). But I don't see a reason to complicate it like that.
My 2c, unless there's a reason no to, just go with slice shifting / iteration.
eth/catalyst/api.go
Outdated
) | ||
|
||
const preparedPayloadsCacheSize = 10 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pls delete this, it's dead code. Otherwise LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM, will merge on green
* eth/catalyst: evict old payloads, type PayloadID * eth/catalyst: added tracing info to engine api * eth/catalyst: add test for create payload timestamps * catalyst: better logs * eth/catalyst: computePayloadId return style * catalyst: add queue for payloads * eth/catalyst: nitpicks Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* eth/catalyst: evict old payloads, type PayloadID * eth/catalyst: added tracing info to engine api * eth/catalyst: add test for create payload timestamps * catalyst: better logs * eth/catalyst: computePayloadId return style * catalyst: add queue for payloads * eth/catalyst: nitpicks Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Changes:
PayloadResponse
: dead code since last spec update,ForkchoiceUpdatedV1
returns the ID with status insteadEvict prepared payloads with a LRU of size 10. Preparing payloads with 10 different inputs in parallel, and reverting to the inputs of the first request should be very rare. This prevents the node from running out of memory over time when producing blocks.Update: LRU cache has been replaced with one specific for this functionality.> 8 bytes
IDs were allowed)InvalidPayloadID
: non-standard errorForkchoiceUpdatedV1
from panicking on concurrent map writes to the previousmap
cc @MariusVanDerWijden
edit: updated, forgot goimports