Primary caching 12: bare-bone range support #4784

teh-cmc · 2024-01-11T11:39:26Z

Prefer on a per-commit basis, stuff has moved around

Range queries are back!... in the most primitive form possible.

No invalidation, no bucketing, no optimization, no nothing. Just putting everything in place.

24-01-11_12.37.56.patched.mp4

Part of the primary caching series of PR (index search, joins, deserialization):

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)
I have tested the web demo (if applicable):
- Using newly built examples: app.rerun.io
- Using examples from latest main build: app.rerun.io
- Using full set of examples from nightly build: app.rerun.io
The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG

teh-cmc · 2024-01-11T11:40:53Z

crates/re_query_cache/src/latest_at.rs

All of this stuff was only moved around, no modifications whatsoever.

teh-cmc · 2024-01-11T11:41:52Z

crates/re_query_cache/src/query.rs

Barely any changes: the latest-at stuff has been moved to latest_at.rs and this file now simply redirects the caller to either latest_at.rs or range.rs appropriately.

teh-cmc · 2024-01-11T11:43:29Z

crates/re_query_cache/src/cache.rs

-
-// ---
-
-// NOTE: Because we're working with deserialized data, everything has to be done with metaprogramming,
-// which is notoriously painful in Rust (i.e., macros).
-// For this reason we move as much of the code as possible into the already existing macros in `query.rs`.
-
-/// Caches the results of `LatestAt` queries.
-#[derive(Default)]
-pub struct LatestAtCache {
-    /// Organized by _query_ time.
-    ///
-    /// If the data you're looking for isn't in here, try partially running the query and check
-    /// if there is any data available for the resulting _data_ time in [`Self::per_data_time`].
-    pub per_query_time: BTreeMap<TimeInt, Arc<RwLock<CacheBucket>>>,
-
-    /// Organized by _data_ time.
-    ///
-    /// Due to how our latest-at semantics work, any number of queries at time `T+n` where `n >= 0`
-    /// can result in a data time of `T`.
-    pub per_data_time: BTreeMap<TimeInt, Arc<RwLock<CacheBucket>>>,
-
-    /// Dedicated bucket for timeless data, if any.
-    ///
-    /// Query time and data time are one and the same in the timeless case, therefore we only need
-    /// this one bucket.
-    //
-    // NOTE: Lives separately so we don't pay the extra `Option` cost in the much more common
-    // timeful case.
-    pub timeless: Option<CacheBucket>,
-
-    /// Total size of the data stored in this cache in bytes.
-    pub total_size_bytes: u64,
-}


Moved to latest_at.rs

crates/re_query_cache/src/cache.rs

teh-cmc · 2024-01-11T16:40:13Z

crates/re_query_cache/tests/latest_at.rs

Nothing interesting here, just made it so it doesn't depend on production components.

teh-cmc · 2024-01-11T16:40:47Z

crates/re_query_cache/tests/range.rs

1:1 port of the tests in crates/re_query/tests/archetype_range_tests.rs, with caching enabled.

Of course they don't pass yet. We'll get there in the upcoming PRs.

teh-cmc · 2024-01-11T16:41:16Z

Update: ported the original range test suite over.

teh-cmc · 2024-01-11T17:09:21Z

Updated so that the app options now toggle latest-at and range queries as a whole:

latest-at defaults to enabled, while range stays disabled for now.

crates/re_query_cache/src/range.rs

Wumpf

thanks for the notes on what has moved, made reviewing it a lot easier.
Lgtm as well, but struggled a little bit to follow in a few spots, but not concerned :)

Wumpf · 2024-01-12T16:08:04Z

crates/re_query_cache/tests/range.rs

@@ -0,0 +1,388 @@
+//! Contains:
+//! - A 1:1 port of the tests in `crates/re_query/tests/archetype_range_tests.rs`, with caching enabled.


it would be nice to share some of store setup code from there

Eh, I do expect them to grow apart as time goes on; they don't worry about the same things exactly.

) _99% grunt work, the only somewhat interesting thing happens in `query_archetype`_ Our query model always operates with two distinct timestamps: the timestamp you're querying for (`query_time`) vs. the timestamp of the data you get back (`data_time`). This is the result of our latest-at semantics: a query for a point at time `10` can return a point at time `2`. This is important to know when caching the data: a query at time `4` and a query at time `8` that both return the data at time `2` must share the same single entry or the memory budget would explode. This PR just updates all existing latest-at APIs so they return the data time in their response. This was already the case for range APIs. Note that in the case of `query_archetype`, which is a compound API that emits multiple queries, the data time of the final result is the most recent data time among all of its components. A follow-up PR will use the data time to deduplicate entries in the latest-at cache. --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800

…ation (#4712) Introduces the notion of cache deduplication: given a query at time `4` and a query at time `8` that both returns data at time `2`, they must share a single cache entry. I.e. starting with this PR, scrubbing through the OPF example will not result if more cache memory being used. --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800

Introduces a dedicated cache bucket for timeless data and properly forwards the information through all APIs downstream. --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800

#4785) Title. https://github.com/rerun-io/rerun/assets/2910679/cf2c2748-a461-49fe-8124-c2a94164c956 --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800

… range queries (#4793) Our low-level range APIs used to bake the latest-at results at `range.min - 1` into the range results, which is a big problem in a multi tenant setting because `range(1, 10)` vs. `latestat(1) + range(2, 10)` are two completely different things. Side-effect: a plot with a window of len 1 now behaves as expected: https://github.com/rerun-io/rerun/assets/2910679/957ac367-35a6-4bea-9f40-59d51c556639 --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800

The most obvious and most important performance optimization when doing cached range queries: only upsert data at the edges of the bucket / ring-buffer. This works because our buckets (well, singular, at the moment) are always dense. - #4793 ![image](https://github.com/rerun-io/rerun/assets/2910679/7246827c-4977-4b3f-9ef9-f8e96b8a9bea) - #4800: ![image](https://github.com/rerun-io/rerun/assets/2910679/ab78643b-a98b-4568-b510-2b8827467095) --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800

Range queries used to A) return the frame a T-1, B) accumulate state starting at T-1 and then C) yield frames starting at T. A) was a huge issue for many reasons, which #4793 took care of by eliminating both A) and B). But we need B) for range queries to be context-free, i.e. to be guaranteed that `Range(5, 10)` and `Range(4, 10)` will return the exact same data for frame `5`. This is crucial for multi-tenant settings where those 2 example queries would share the same cache. It also is the nicer-nicer version of the range semantics that we wanted anyway, I just didn't realize back then that it would require so little changes, or I would've gone straight for that. --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800 - #4851 - #4852 - #4853 - #4856

Simply add a timeless path for the range cache, and actually only iterate over the range the user asked for (we were still blindly iterating over everything until now). Also some very minimal clean up related to #4832, but we have a long way to go... - #4832 --- - Fixes #4821 --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800 - #4851 - #4852 - #4853 - #4856

Implement range invalidation and do a quality pass over all the size tracking stuff in the cache. **Range caching is now enabled by default!** - Fixes #4809 - Fixes #374 --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800 - #4851 - #4852 - #4853 - #4856

- Quick sanity pass over all the intermediary locks and refcounts to make sure we don't hold anything for longer than we need. - Get rid of all static globals and let the caches live with their associated stores in `EntityDb`. - `CacheKey` no longer requires a `StoreId`. --- - Fixes #4815 --- Part of the primary caching series of PR (index search, joins, deserialization): - #4592 - #4593 - #4659 - #4680 - #4681 - #4698 - #4711 - #4712 - #4721 - #4726 - #4773 - #4784 - #4785 - #4793 - #4800 - #4851 - #4852 - #4853 - #4856

@rustbasic

## What * Closes #5315 ## egui changelog ### ⭐ Added * Add `Image::uri()` [#4720](emilk/egui#4720) by [@rustbasic](https://github.com/rustbasic) ### 🔧 Changed * Better documentation for `Event::Zoom` [#4778](emilk/egui#4778) by [@emilk](https://github.com/emilk) * Hide tooltips when scrolling [#4784](emilk/egui#4784) by [@emilk](https://github.com/emilk) * Smoother animations [#4787](emilk/egui#4787) by [@emilk](https://github.com/emilk) * Hide tooltip on click [#4789](emilk/egui#4789) by [@emilk](https://github.com/emilk) ### 🐛 Fixed * Fix default height of top/bottom panels [#4779](emilk/egui#4779) by [@emilk](https://github.com/emilk) * Show the innermost debug rectangle when pressing all modifier keys [#4782](emilk/egui#4782) by [@emilk](https://github.com/emilk) * Fix occasional flickering of pointer-tooltips [#4788](emilk/egui#4788) by [@emilk](https://github.com/emilk) ## eframe changelog * Web: only capture clicks/touches when actually over canvas [#4775](emilk/egui#4775) by [@lucasmerlin](https://github.com/lucasmerlin) ## Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6785?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6785?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! - [PR Build Summary](https://build.rerun.io/pr/6785) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.

@rustbasic

## What * Closes #5315 ## egui changelog ### ⭐ Added * Add `Image::uri()` [#4720](emilk/egui#4720) by [@rustbasic](https://github.com/rustbasic) ### 🔧 Changed * Better documentation for `Event::Zoom` [#4778](emilk/egui#4778) by [@emilk](https://github.com/emilk) * Hide tooltips when scrolling [#4784](emilk/egui#4784) by [@emilk](https://github.com/emilk) * Smoother animations [#4787](emilk/egui#4787) by [@emilk](https://github.com/emilk) * Hide tooltip on click [#4789](emilk/egui#4789) by [@emilk](https://github.com/emilk) ### 🐛 Fixed * Fix default height of top/bottom panels [#4779](emilk/egui#4779) by [@emilk](https://github.com/emilk) * Show the innermost debug rectangle when pressing all modifier keys [#4782](emilk/egui#4782) by [@emilk](https://github.com/emilk) * Fix occasional flickering of pointer-tooltips [#4788](emilk/egui#4788) by [@emilk](https://github.com/emilk) ## eframe changelog * Web: only capture clicks/touches when actually over canvas [#4775](emilk/egui#4775) by [@lucasmerlin](https://github.com/lucasmerlin) ## Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6785?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6785?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! - [PR Build Summary](https://build.rerun.io/pr/6785) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.

teh-cmc added 🔍 re_query affects re_query itself 📉 performance Optimization, memory use, etc do-not-merge Do not merge this PR include in changelog labels Jan 11, 2024

teh-cmc commented Jan 11, 2024

View reviewed changes

teh-cmc force-pushed the cmc/primcache_12_barebone_range branch from 2d39d88 to e547ef4 Compare January 11, 2024 16:39

teh-cmc commented Jan 11, 2024

View reviewed changes

teh-cmc commented Jan 12, 2024

View reviewed changes

crates/re_query_cache/src/range.rs Outdated Show resolved Hide resolved

This was referenced Jan 12, 2024

Primary caching 14: don't bake LatestAt(T-1) results into low-level range queries #4793

Merged

Primary caching 15: range read performance optimization #4800

Merged

Wumpf self-requested a review January 12, 2024 15:35

Wumpf approved these changes Jan 12, 2024

View reviewed changes

teh-cmc added 9 commits January 15, 2024 13:24

pure refactor: put everything latest-at in its own dedicated corner

8de6faa

set up all the routing infrastructure

25d91c6

implement barebone range caching

f245d46

lints

c74f2a7

self-review

8f9a3a5

fix broken example components

1081675

port re_query's range test suite 1:1

73baf8a

toggle latest-at and/or range queries as a whole

ac15cfe

remove lies

5fb6abd

teh-cmc force-pushed the cmc/primcache_12_barebone_range branch from 17b548b to 5fb6abd Compare January 15, 2024 12:26

post-merge craziness

a64f1c3

teh-cmc force-pushed the cmc/primcache_12_barebone_range branch from 5480cc3 to a64f1c3 Compare January 15, 2024 14:38

teh-cmc removed the do-not-merge Do not merge this PR label Jan 15, 2024

fix the arrow2 bench that hasnt been building for a year

339eed2

teh-cmc merged commit 886014a into main Jan 15, 2024
13 of 25 checks passed

teh-cmc deleted the cmc/primcache_12_barebone_range branch January 15, 2024 14:49

This was referenced Jan 18, 2024

Primary caching 16: context-free range semantics #4851

Merged

Primary caching 17: timeless range #4852

Merged

Primary caching 18: range invalidation (ENABLED BY DEFAULT 🎊) #4853

Merged

Primary caching 19 (final): de-staticify cache globals #4856

Merged

abey79 changed the title ~~Primary caching 12: barebone range support~~ Primary caching 12: bare-bone range support Feb 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Primary caching 12: bare-bone range support #4784

Primary caching 12: bare-bone range support #4784

teh-cmc commented Jan 11, 2024 •

edited by github-actions bot

Loading

teh-cmc Jan 11, 2024

teh-cmc Jan 11, 2024

teh-cmc Jan 11, 2024

teh-cmc Jan 11, 2024

teh-cmc Jan 11, 2024

teh-cmc commented Jan 11, 2024

teh-cmc commented Jan 11, 2024

Wumpf left a comment

Wumpf Jan 12, 2024

teh-cmc Jan 15, 2024

		@@ -0,0 +1,388 @@
		//! Contains:
		//! - A 1:1 port of the tests in `crates/re_query/tests/archetype_range_tests.rs`, with caching enabled.

Primary caching 12: bare-bone range support #4784

Primary caching 12: bare-bone range support #4784

Conversation

teh-cmc commented Jan 11, 2024 • edited by github-actions bot Loading

Checklist

teh-cmc Jan 11, 2024

Choose a reason for hiding this comment

teh-cmc Jan 11, 2024

Choose a reason for hiding this comment

teh-cmc Jan 11, 2024

Choose a reason for hiding this comment

teh-cmc Jan 11, 2024

Choose a reason for hiding this comment

teh-cmc Jan 11, 2024

Choose a reason for hiding this comment

teh-cmc commented Jan 11, 2024

teh-cmc commented Jan 11, 2024

Wumpf left a comment

Choose a reason for hiding this comment

Wumpf Jan 12, 2024

Choose a reason for hiding this comment

teh-cmc Jan 15, 2024

Choose a reason for hiding this comment

teh-cmc commented Jan 11, 2024 •

edited by github-actions bot

Loading