Allow archetypes to clear and/or update existing components

[emilk]

Backstory

• provide empty() method on Rust archetype implementations to log empty #3279
• Decide if archetypes logging APIs should log non-provided components as empty-lists  #3341

What should the following code do:

log("points", Points3D::new(positions).with_colors(…));
…
log("points", Points3D::new(new_positions));

Should the new positions inherit the colors of the previous log call?

That would be quite non-intuitive.

Related to this: how does one update just the colors of a point cloud?

Design

We've all converged on the following API:

// We do NOT implement `Default` for archetypes.
// users need to explcitly use `clear` or `update`
struct Points3D {
   positions: Optional<Vec<Vec3>>,
   color: Optional<Vec<Color>>,
   radii: Optional<Vec<f32>>,
}

impl Points3D {
    /// Clear all components of this archetype to the empty list.
    fn clear() -> Self {
        Self {
            positions: Some(vec![]),
            color: Some(vec![]),
            radii: Some(vec![]),
        }
    }
    
    /// Create a new `Point3d` point cloud with some positions,
    /// and with emtpy lists for all other archetypes.
    fn new(positions: Vec<Vec3>) -> Self {
        Self {
            positions: Some(positions),
            ..Self::clear()
        }
    }
    
    /// An empty `Point3d` point cloud, with `None` for each component.
    fn update() -> Self {
        Self {
            positions: None,
            color: None,
            radii: None,
        }
    }
}

fn main() {
    // Log a point cloud:
    log("points", Points3D::new(…).with_colors(…));
    
    log("points", Clear{}); // will clear all components of all archetypes
    log("points", Points3D::clear()); // will clear this archetype of the entity
    log("points", Points3D::new(positions).with_colors(…)); // overwrites everything
    log("points", Points3D::update().with_colors(…)); // overwrites only colors

    // This is also possible, though a somewhat weird way to write it:
    log("points", Points3D::clear().with_colors(…)); // overwrites everything
}

Python:

log("points", Clear()); # will clear all components of all archetypes
log("points", Points3D.clear()); # will clear this archetype of the entity
log("points", Points3D(positions, colors=[…], radii=None); # overwrites everything, even radii
log("points", Points3D.update(colors=[…], radii=None)); # overwrites only colors and radii

Necessary changes

• All components are Options, even the required ones
• When querying, an empty lists should be treated the same as a missing components
• In the UI we should probably hide empty components in the streams view

Component lists implications

A component list (e.g. radii) can have one of the four special values:

• null/missing
• 0 length ("empty")
• 1 length ("splat")
• N length (1<N)

null components aren't logged, but empty components are.
When querying the store, missing components are treated the same as empty components.

Related issues:

• Visualize density graph of empty batches differently #7117

[emilk]

If this proves to be more complex than anticipated, then we'll push it to 0.10

[emilk]

Needs a follow-up where we hide empty components from the streams view

[emilk]

We need to think through how num_instances would work for an update. We could take the max-length of all its components, but that won't work if you're only logging splats. Perhaps we don't need num_instances for updates; it's really only used for the primary (e.g. the indicator).

[jleibs]

This API should also handle specifying the InstanceKeys for the collection that's being updated.

[teh-cmc]

Instance keys and num_instances are not a problem anymore (or won't be, soon).

[jleibs]

@teh-cmc we were talking through this with respect to Transforms I realized this approach has the potential to cause some annoying performance issues for chunks.

In particular, every unused optional component will still require growing the offset array for each of the optional components by one element (in many cases, just a repeated 0). This is similar to the overhead of growing sparse unions (growing each child array), and strictly worse than the overhead of sparse unions (just growing one child array).

This suggests to me we're going to need some kind of chunk-specific optimization.

This also has an annoying ripple-effect on datafusion / dataframe UI of showing a bunch of empty / cleared (not nulled) columns all over the place.

[teh-cmc]

Some notes on why I think (parts of) this is needed in order to unblock tagged components.

Context: partial updates in a tagged environment

We need an ergonomic way to work with raw components (as opposed to archetypes) that still allows for proper tagging.

As of today, as soon as we re-enable tags, all user-level logging code (including our own code that does a bunch of runtime writes for all things re_blueprint_*!) that mixes logging archetypes and logging components will turn into undefined behavior (which is why we disabled tagging entirely for 0.21 in the first place).

E.g.:

rr.set_time_sequence("frame", 42)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]], colors=[255, 0, 0]))

rr.set_time_sequence("frame", 42)
rr.log("points", [rr.components.ColorBatch([0, 0, 255])])

rr.set_time_sequence("frame", 43)
rr.log("points", [rr.components.ColorBatch([0, 255, 0])])

# Undefined behaviour, there are three columns in the database now:
# * `points@Points3D:Position3D#positions`
# * `points@Points3D:Color#colors`
# * `points@Color`
#
# What colors should the visualizer use for frame #42? `points@Points3D:Color#colors`, `points@Color`, something else entirely?
# What colors should the visualizer use for frame #43? `points@Points3D:Color#colors`, `points@Color`, something else entirely?
# What colors should the visualizer use for frame #100? `points@Points3D:Color#colors`, `points@Color`, something else entirely?

The correct thing to do, as of today, is something like this:

rr.set_time_sequence("frame", 42)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]], colors=[255, 0, 0]))

rr.set_time_sequence("frame", 42)
rr.log("points", [rr.components.ColorBatch([0, 0, 255]).with_descriptor_overrides(
    archetype_name = "rerun.archetypes.Points3D",
    archetype_field_name = "colors"
)])

rr.set_time_sequence("frame", 43)
rr.log("points", [rr.components.ColorBatch([0, 255, 0]).with_descriptor_overrides(
    archetype_name = "rerun.archetypes.Points3D",
    archetype_field_name = "colors"
)])

# There are only two columns in the database:
# * `points@Points3D:Position3D#positions`
# * `points@Points3D:Color#colors`
#
# The visualizer knows what to pick.

which obviously is a pretty hard thing to sell to end users as-is 🫠.

Path forward 1: make it ergonomic

The first possible path forward is very related to what is already proposed in this issue: make these things actually ergonomic!

rr.set_time_sequence("frame", 42)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]], colors=[255, 0, 0]))

rr.set_time_sequence("frame", 42)
rr.log("points", [rr.Points3D.update(colors=[0, 0, 255])]) # gets properly tagged

rr.set_time_sequence("frame", 43)
rr.log("points", [rr.Points3D.update(colors=[0, 255, 0])]) # gets properly tagged

# There are only two columns in the database:
# * `points@Points3D:Position3D#positions`
# * `points@Points3D:Color#colors`
#
# The visualizer knows what to pick.

Upsides:

• Ergonomic.
• Fixes the issue.
• We will eventually need something like this in any case, so that users can work ergonomically with partial updates and/or tags.
• Everything else mentioned in this ticket.

Downsides:

• It's a big revamp of all logging APIs.
• Everything else mentioned in this ticket.

Also keep in mind that we don't necessarily need all the features of the original proposal, just enough to get tags rolling (whether that is at all possible, I don't know).

Path forward 2: new query semantics

The other path forward is to introduce new query semantics. One set of semantics that comes fairly naturally to mind is "most-specific at timestamp wins" (whether it's actually a good idea in practice, who knows).

I.e. with the code from earlier:

rr.set_time_sequence("frame", 42)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]], colors=[255, 0, 0]))

rr.set_time_sequence("frame", 42)
rr.log("points", [rr.components.ColorBatch([0, 0, 255])])

rr.set_time_sequence("frame", 43)
rr.log("points", [rr.components.ColorBatch([0, 255, 0])])

# There are three columns in the database now:
# * `points@Points3D:Position3D#positions`
# * `points@Points3D:Color#colors`
# * `points@Color`
#
# What colors should the visualizer use for frame #42? `points@Points3D:Color#colors`, because it is the most specific one available at this timestamp.
# What colors should the visualizer use for frame #43? `points@Color`, because it is the most specific one available at this timestamp.
# What colors should the visualizer use for frame #100? `points@Color`, because it is the most specific one available at the closest timestamp.

Upsides:

• Fixes the issue.
• Does not even require touching user code! Maybe even helps us maintain backwards compat for old untagged RRD files a bit longer?
• We will eventually need some set of new query semantics in any case, in order for tagged and untagged data to coexist in a peaceful and useful way.

Downsides:

• Getting these semantics right will likely not be an easy feat.
• Getting these semantics wrong can quickly turn into a rabbit hole of unforeseen issues and never-ending refactorings.

Conclusion

Ultimately, I do not think this is an either/or situation: we will need both better logging APIs and improved query semantics, eventually.

It's mostly a matter of which should we get to first in order to unblock tagged components the fastest? The former seems more approachable to me, especially if we restrain ourselves to only whatever subset is needed for tagging and partial updates (if that's at all possible) -- but I can be convinced otherwise.
(It should also be trivial to find many many more upsides and downsides for both of those, I haven't had any time to think it through really.)

[jleibs]

Thanks for the write up! The motivation for .update() makes sense as a mechanism of controlling tag-application.

The concept of "most-specific at timestamp wins" is already how I was imagining things would work, but I agree it's still nice to give users a way to avoid the ambiguity in the first place.

[nikolausWest]

My suggestion is to call it .components() rather than update() to be more explicit about what is happening. Update sounds like it's mutating the archetype