Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change parameter estimations to work in presence of caches #7681

Open
Tracked by #5995
jakmeier opened this issue Sep 23, 2022 · 0 comments
Open
Tracked by #5995

Change parameter estimations to work in presence of caches #7681

jakmeier opened this issue Sep 23, 2022 · 0 comments
Assignees
Labels
A-params-estimator Area: runtime params estimator T-contract-runtime Team: issues relevant to the contract runtime team

Comments

@jakmeier
Copy link
Contributor

Estimations today disable nearcore internal caches on top of RocksDB. Specifically, trie caches and compiled code caches.

Disabling these caches overestimates the amount of IO we need, as we often access the same data multiple times in normal processing. So far this was "no problem" because it appears estimations were mostly reading data from RockDB's internal in-memory caches. But now with #7484 and #7502 removing that estimator bug, some estimation results have sky-rocketed. (see #7519)

I suggest we enable the caches from now in estimations. However, that only makes sense if we modify existing estimations.

For example the estimations for storage related host functions otherwise read all values from the cache after the first iteration. The iterations in that case are inside the estimator smart contract, so we have to fix it on that level.

Another example is code compilation. Maybe we can keep those caches disabled. Or we could compile different code in each iteration. Or even just clear the cache between iterations.

The goal here is that we can change the estimator setup to make use of caches where it makes sense. Ideally, all estimations should then be roughly what they were a few weeks ago, or there should be a clear correlation between the changed results and the fixed IO estimation bug.

@jakmeier jakmeier added T-contract-runtime Team: issues relevant to the contract runtime team A-params-estimator Area: runtime params estimator labels Sep 23, 2022
@jakmeier jakmeier self-assigned this Sep 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-params-estimator Area: runtime params estimator T-contract-runtime Team: issues relevant to the contract runtime team
Projects
None yet
Development

No branches or pull requests

1 participant