Uniquely identifying TextureViews - Thoughts on a downstream API design problem

[mitchmindtree]

I have a design problem w/ nannou I'm currently thinking about and would love to get your thoughts on! It's about the way users provide texture views to the immediate draw APIs such as:

draw.texture(tex_view)

draw.mesh().textured_points(tex_view, points)

draw.polygon().textured_points(tex_view, points)

and so on.

Each of these draw API calls creates an entry into an inner list of DrawCommands which describe how to draw the whole scene at a very high-level. When it's time to render the contents of a draw instance to a texture, this list of DrawCommands is consumed and the draw::Renderer converts them into RenderCommands that map directly to commands that may be encoded with a wgpu::CommandEncoder.

As a part of this process, the draw::Renderer needs to be able to uniquely distinguish between each of the texture views provided by the user in order to ensure it has one unique bind group ready for each unique texture. This allows us to draw an arbitrary number of different textures in a single render pass by inserting commands to switch between bind groups between draw commands, but only as often as is necessary. Currently however, the wgpu::TextureView type has no way of being uniquely identified, and I'm trying to think of a nice way of going about this.

The first option that comes to mind is to have the user maintain a map of Ids to TextureView, where Ids are submitted to the draw API and the map of texture views is lent to the rendering process. This is not ideal however, as we mostly shift the burden onto the user and have them run the risk of letting their map leak, etc. We could perhaps maintain a map for the user, however we would need to allow for the user to manually clear this map in some way or run the risk of leaking ourselves.

Another option that comes to mind is creating a TextureView wrapper type that does allow for unique identification, e.g. the ID of the source texture, plus the unique descriptor it was created with. Ensuring that the source texture has an easily accessible unique ID should be doable for nannou as it already maintains its own Texture wrapper type as mentioned in #205. This does mean maintaining a wrapper type however, and all the issues that come along with this, e.g. keeping the API up to date with upstream, ensuring users can always access the inner type if necessary, running the risk of users not being able to use nannou TextureView APIs with a wgpu::TextureView. Maybe these are worth the easier-to-use design though!

I just wanted to share in case you had any thoughts or advice on how you would go about approaching this? Or whether perhaps you see enough value in the use case to consider allowing the TextureView to provide a .unique_id() method or Hash impl - it seems the web target is the blocker here? I think this is one of the last missing pieces to make nannou's draw API really shine - would love for it to be both efficient yet also trivial to use when introducing brand new users to creative coding.

[kvark]

The biggest question for me here is whether Nannou is an opaque abstraction, or a transparent one, when it comes to wgpu. If it's opaque, you can just wrap the TextureView in your own thing that can easily figure out uniqueness (e.g. by incrementing an ID on every creation), and there is no need to keep its API aligned with wgpu.

If it's transparent, then it should expose the programming model that is compatible with wgpu, i.e. expect the users to manage bind groups on their own.

[aloucks]

I ran into a similar issue. I've added PartialEq, Eq, and Hash impls to everything that can allow it here: #216

[mitchmindtree]

Sorry I just realised I never responded here!

@kvark nannou for the most part attempts to be transparent, but provides a suite of helper items to simplify common uses, provide some useful defaults, etc. We try to be as transparent as possible in order to:

• Allow the user complete low-level access if they desire, e.g. creating completely custom pipelines but still taking advantage of nannou's application loop to remove a lot of boilerplate. Our examples/wgpu/ directory demonstrates this.
• Guide nannou's architecture around the way winit and wgpu work rather than vice versa.
• Avoid the need for us to maintain a whole graphics layer on top of wgpu :)

We actually re-export wgpu almost in its entirety via our own wgpu module, but with a suite of added helper types. E.g. there are lots of builder types as they are a common pattern in nannou (but you can always retrieve the inner descriptor if you wish), some types to help re-shaping and capturing textures, etc. There are currently only two cases where we wrap a wgpu type - Texture and TextureView (TextureView not yet published), which we've done for the reasons I mention above and in #205 .

[mitchmindtree]

i.e. expect the users to manage bind groups on their own.

Unfortunately we have a very wide target audience, e.g. we aim to provide both:

• a very high-level API that allow us to introduce artists to getting creative with code. E.g. an alternative to Processing, but where you don't have to stop using it as soon as your artworks become more than sketches or you want to add audio. We can barely expect these users to understand the difference between an image and a texture, let alone bind groups. Here are some examples of this high-level kind of API.
• while also allowing experienced users to bypass the higher-level API if they wish and set up their own pipelines if they wish for more serious multi-room installations for museums, festivals, large exhibitions, etc. E.g. the examples/wgpu/ examples I linked above allowing for custom pipelines, but still with access to helper types.

All that said, this is more of our own problem to solve though! I also think our current solution is actually working out nicely all things considered! I mostly just wanted to mention the experience for feedback and in case any other folk run into similar things and are thinking about solutions :)

[kvark]

@mitchmindtree I see, thanks for the information! So nannou attempts to expose "gradual" complexity, where users can opt into more power and lower level interfaces.
At the same time, you are saying that most objects are wrapped up, so these wrappers can have unique IDs that you produce.
I'm happy to hear that everything is working for you so far. I suppose we can close the issue then. Always open to come back to discussing this!