Pipelines and Scopes for better bulk operations

[Ogeon]

Here's something that has been brewing in my mind for a bit. The idea is to improve bulk actions by reusing memory and not repeatedly calculate constants. I'm calling them "pipelines" and "scopes" here, but something else may be more descriptive.

Pipelines

The idea is to use the same technique as iterators to cache values, such as conversion matrices, and chain computations together. This should help the compiler to know what can be reused and what need to be calculated for every value.

Here's an example of how it could work:

// Contains all the matrices and other constants:
let my_pipeline = palette::pipeline::into_color::<Xyz>()
    .darken(0.5)
    .into_color::<Lch>()
    .hue_shift(30.0)
    .into_color::<Srgb>();

for color in &mut colors {
    *color = my_pipeline.apply(color);
}

Note that I haven't yet checked the impact this would have. It may very well be unnecessary if the compiler is able to make the same optimization.

Open questions

• Is it even necessary? Even with "scopes"?
  It's useful as an internal component of "scopes", so doesn't hurt to expose it to users.
• API design?
  • Similar to Iterator?
• Available methods in first version?

Scopes

These are sort of similar to pipelines, but for reusing memory. If there's an image buffer that's stored as [f32], it should be possible to use the same memory space to temporarily convert it to any color space that has the same memory layout. The idea with "scopes" is to make it possible to pass a &mut [T] value to a type that converts into another type U and also knows how to convert U back to T. This wrapper type exposes &mut [U] while in scope, and then converts the values back to T when dropped. All without requiring additional allocations. This should only be allowed if T and U have the same layout, similar to how the Pixel trait works.

Here's the same example as for "pipelines" but with scopes:

{
    let xyz_colors = palette::scope::into_color::<Xyz>(&mut colors);
    
    for color in xyz_colors {
        *color = color.darken(0.5);
    }
    
    let lch_colors = palette::scope::into_color::<Lch>(&mut xyz_colors); // or perhaps xyz_colors.into_color::<Lch>();
    
    for color in lch_colors {
        *color = color.hue_shift(30.0);
    }
} // The scopes are dropped here, and `colors` is available as Srgb again

Another interesting aspect with "scopes" is that they could possibly help the CPU to branch predict and better cache instructions, since it will apply a conversion to the full buffer before moving on to the next operation.

Open questions

• API design?
  • Make use of Drop or scope closures?
• Available methods in first version?

I think these two patterns could complete each other and be used together, depending on the use case, but the "scopes" seem to have the strongest use case, from what I have seen, since it can avoid having to allocate memory just to get a temporary buffer with a different color space while maintaining the type safety.

Is this something anyone has been missing or has a concrete use case for? Any similar or alternative ideas? Opinions? Any input is appreciated.

[Ogeon]

Thinking some more about it, I think a generic "scope" can be implemented using an internal "pipeline" to restore the data. Something like this:

struct Scope<'a, T, R>
where
    T: ArrayRepr<ArrayType = R::Output::ArrayType> + Clone,
    R: Pipeline<T>,
{
    buffer: &'a mut [T],
    restore: R,
}

impl<'a, T, R> Scope<'a, T, R>
where
    T: ArrayRepr<ArrayType = R::Output::ArrayType> + Clone,
    R: Pipeline<T>,
{
    fn new<B, C>(buffer: B, convert: C, restore: R) -> Self
    where
        B: AsMut<[R::Output]>,
        C: Pipeline<R::Output, Output = T>
    {
        let buffer = buffer.as_mut();
        
        for color in buffer {
            let source = color.clone();
            let destination = T::from_raw_mut(color.as_raw_mut::<T::ArrayType>());
            *destination = convert.apply(source);
        }

        Self {buffer, restore}
    }
}

impl<'a, T, R> Drop for Scope<'a, T, R>
where
    T: ArrayRepr<ArrayType = R::Output::ArrayType> + Clone,
    R: Pipeline<T>,
{
    fn drop(&mut self) {
        for color in self.buffer {
            let source = color.clone();
            let destination = R::Output::from_raw_mut(color.as_raw_mut::<T::ArrayType>());
            *destination = self.restore.apply(source);
        }
    }
}

// Assuming these traits:
trait Pipeline<I> {
    type Output;

    fn apply(&self, input: I) -> Output;
}

trait ArrayRepr: Pixel { // Pixel will need to have an associated component type
    type ArrayType;
}

impl<T: Pixel> ArrayRepr for T {
    type ArrayType: [T::Component; T::CHANNELS];
}

This assumes a lot, so I should prototype it, but it would in theory be exactly what I had in mind all along.

[Ogeon]

I gave it a try and could get away with only a few modifications: d8ac7cb. The only real obstacle was that type ArrayType: [T::Component; T::CHANNELS]; isn't possible at the moment, so I extended the Pixel derive macro to implement it. But it works!

There are a couple of ergonomic issues with the prototype API. The into_color function call ended up as palette::scope::into_color::<Xyz, _, _>(&mut colors);. The two additional type parameters are the buffer type and the source color space. The buffer could probably be replaced with &'a mut dyn AsMut<[T]>, letting the compiler inline it all, but I don't see how to get rid of the source type in a good way. It may be ok to have it there, though. Another idea could be to not have free functions and put them on the Scope type. That would probably just end up as Scope::new_into_color::<Xyz>(...).

[Ogeon]

I decided to try a Scope trait and make each scope wrapper a struct. This allows [T] to implement Scope, so it's possible to call the methods on slices and arrays, and also makes it easy to add optimized methods on each scope type. In this example, the conversion scope is replaced with another scope to avoid the extra step when restoring:

let mut colors = vec![Srgb::new(1.0f32, 0.5, 0.0), Srgb::new(0.5, 1.0, 0.5)];

{
    let mut xyz_colors = colors.into_color::<Xyz>();
    
    for color in &mut *xyz_colors {
        *color = color.darken(0.5);
    }
    
    let mut lch_colors = xyz_colors.replace_into_color::<Lch>();
    
    for color in &mut *lch_colors {
        *color = color.shift_hue(30.0);
    }
}

I'm liking how this works, but I'm considering a different naming convention. Something like as_color to make it more obvious that it's a temporary conversion. That would also avoid conflicting with a potential IntoColor<Vec<C>> for Vec<T> (which would be super neat and very possible with the same technique) if Scope should still be implemented for slices.