[Bug] Investigate in-memory caching of blocks

[lutter]

Bug report

We recently added a way to cache blocks in memory (#4215) though it looks like that did not have the desired effect. We also added metrics (#4440 ) to better understand how that cache works.

We need to figure out why this cache isn't effective. The underlying issue that needs to be addressed is that reading from the block cache in the database can require a lot of IO bandwidth, and we need to reduce that.

[github-actions]

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.

[azf20]

@lutter is this still valid?

[lutter]

Honestly, I am not sure where this ended up; the block cache shards still seem to experience high IO load though. Maybe @neysofu remembers the details of what still needed to be done?

[neysofu]

Not really, I'm afraid. The high IO load came from a different set of queries after the deployment, that much I remember – which probably means some problematic queries were optimized away but not nearly enough to reduce the IO load. We never started an actual follow-up investigation beyond that.

[azf20]

OK - closing for now, and we can revisit if this surfaces again