Bevy Render Rework Round 2: SubGraphs, Views, and Shadows

[cart]

This is the second iteration of my Bevy Renderer Rework experiment. For more context on what motivated this, check out the Round One Prototype Discussion.

My goals for this round were:

1. Support re-running render logic according to runtime-defined inputs, such as multiple cameras/views. This needed to include both Extract / Prepare / Queue systems and Render Graph logic.
2. Render logic should still be modular: users should be able to plug in new render logic without needing to care about the wider call context. Ex: if they defined new per-view logic, they shouldn't need to care how those views are created or where those views are executed. Ideally this logic is reusable across multiple render pipelines.
3. "Dynamic render logic" should be driven by ECS data extracted into the Render World.
4. The new system should be tested by at least one relatively complex render feature involving "repeated render logic" to prove that it works.

The code is available on my pipelined-rendering branch.

The Changes

• RenderGraph: largely rewritten to support "dynamic" execution
  • SubGraphs: graphs can now have named "sub graphs"
  • Graph inputs: graphs can require inputs to run
  • Nodes now have "SlotValue" input and outputs (currently supports Buffer, TextureView, Sampler, and Entity values)
  • Graph is now "read only" during execution. Per-node state must be extracted from ECS or passed through the graph
    • Cleaner: state for a given graph run is not stored in the user-facing RenderGraph type. Graphs are for "configuration" only
    • More flexible: can execute the same graph node more than once (important for implementing SubGraphs)
    • Runtime node input/output state is owned and managed by the graph executor
  • New graph executor with support for running subgraphs. Much simpler implementation. Renderers are now 100% responsible for deciding how to execute the graph.
  • New Node trait
    • RenderGraphContext: ergonomically exposes current inputs, outputs, and read-only graph + node info. Can be used to queue subgraph runs
    • Error handling
  • Graph error handling: graph runs now return errors and propagates node run errors
• Views
  • Views are just entities that are extracted with an ExtractedView component.
  • They define a projection, a transform, a width, and a height
  • Views are often passed in as SubGraph inputs to tell that SubGraph to render from a particular View.
  • Draw functions now take a View entity as input
• New "Core Pipeline"
  • Similar to the old "base" render graph. It provides a minimal, largely empty pipeline that Plugins can add logic to. By making this well defined with statically exported hooks, we can encourage people to write compatible render logic while still exposing full pipeline access.
  • MainPassDriverNode: lives in the "root" graph and selectively executes 2D and 3D subgraphs for each active camera
    • Doesn't run subgraphs if there is no relevant camera.
  • node::MAIN_PASS_DEPENDENCIES
    • an EmptyNode which can be added as a dependent (via a "node edge") to ensure a node runs before the "main pass subgraphs"
• Render Phase Improvements
  • Render phases are now "generic" and use types to determine context. Ex: RenderPhase is now RenderPhase<Transparent3dPhase>
  • Render phases are now components instead of global resources. They should be added to relevant View entities.
• Shadows
  • Traditional "textbook" shadow map implementation (used wgpu example as a reference)
  • Each extracted View with a 3d render phase component selects which lights are relevant to it (currently just whatever the Query returns first). New "light view" entities are generated for each View, which each have their own RenderPhase<ShadowPhase>. Meshes relevant to that light view are then queued up into that phase (currently all meshes are relevant, but ultimately this should use visibility information).
  • The draw_3d_graph sub graph has a ShadowPassNode that queries for "light views" relevant to the current view passed into draw_3d_graph and draws each light view's RenderPhase<ShadowPhase>. After these phases are finished, the MainPass3dNode executes and draws current view's RenderPhase<Transparent3dPhase>.
  • The shadow pass for each "per-view light" could easily have been a separate SubGraph (and we might ultimately want to do this if the render logic for shadows gets more complex / needs to be more modular), but currently it is way simpler to just drive it all from a single node
• Began PBR shader port
  • Ported most pbr logic. PBR material parameters are currently hard-coded as variables in the shader and PBR textures aren't supported yet. But lighting works as expected!
  • Integrated with shadow support
• Bevy ECS now supports re-using Entity ids across Worlds
  • During the Extract step, extracted entities can be spawned using their "app entity id"
  • All "app entity ids" from the current app update are reserved. New entities spawned will be allocated "after" app entity ids by default, but commands.get_or_spawn(entity) can be used to "spawn into" an app world entity id.
  • This removes the need to "map" between "app entities" and "render entities". It allows you to treat the render world as a "separate isolated partition of the main world".
  • It means that multiple extract systems can run in parallel, but still "extract" onto the same entity. Ex: you can have an "extract_view_system" and a "extract_custom_view_data_system" that both extract data for the same "view entity".
  • Will likely come in handy for things like "reusing computed visibility data in a render context".
  • This largely removes the need for the "spawn_and_forget" hack used in the previous renderer prototype
  • Currently this only really works for "pipelined-style" workflows like bevy_render2 (ex: "Run App Logic -> Reset all Render World Entities -> Extract -> ... -> Render")
• Merged in initial Compute Shader PR
  • Nice work @StarArawn!
• New 3d_scene_pipelined Example
  • Illustrates the new renderer, including lights + shadows
• Pluginize bevy_render2
  • TexturePlugin, MeshPlugin, CameraPlugin, ViewPlugin, RenderCommandPlugin.
  • Core RenderPlugin is now the absolute bare minimum (although it currently registers the new plugins for simplicity)
• Safe 100% Read-only "manually driven" query api:
  • get_manual and iter_manual can be run with read-only references to read-only Queries (archetypes must be updated manually at another point in time with a mutable reference)
  • This is useful for RenderGraph Nodes, now that they cannot mutate internal state during a run
• Window Improvements
  • Added RawWindowHandle to Window api, which fully decouples renderers from window backends
  • Moved all window extraction logic to WindowRenderPlugin
• TextureCache
  • holds on to textures across frames, identified by hashes of their descriptor. This enables re-using textures despite entities/components being cleared every frame
• Exposed TextureViews:
  • expose wgpu texture views in RenderResourceContext. There is now a TextureId and TextureViewId
• Crevice Arrays
  • Merged this crevice array pr into local fork and ported glam impls to it: As{Std140, Std430} for arrays of As{Std140, Std430} LPGhatguy/crevice#27
  • (motivated by the need for "light array" uniforms)
• Rebased on main: picked up a few render and ECS changes

Sub Graph Api

All RenderGraphs can now have "Sub Graphs". These are "just" named RenderGraphs owned by the main graph. This creates a nested hierarchy of graphs.

let mut main_render_graph = RenderGraph::default();
main_render_graph.add_sub_graph(draw_3d_graph::NAME, RenderGraph::default());

Graphs can now require inputs, which can be added like this:

let input_node_id = draw_3d_graph.set_input(vec![
    SlotInfo::new(draw_3d_graph::input::VIEW_ENTITY, SlotType::Entity),
    SlotInfo::new(draw_3d_graph::input::RENDER_TARGET, SlotType::TextureView),
    SlotInfo::new(draw_3d_graph::input::DEPTH, SlotType::TextureView),
]);

// The returned input_node_id is just like any other graph node. We can use it to wire up inputs to other graph nodes 
draw_3d_graph
    .add_slot_edge(
        input_node_id,
        draw_3d_graph::input::RENDER_TARGET,
        draw_3d_graph::node::MAIN_PASS,
        MainPass3dNode::IN_COLOR_ATTACHMENT,
    )
    .unwrap();

Sub Graphs are not run by default. They must be run by a graph Node when the graph is executed.

impl Node for MyNode {
    fn run(
        &self, graph: &mut RenderGraphContext, render_context: &mut dyn RenderContext, world: &World,
    ) -> Result<(), NodeRunError> {
        let view_entity = get_view_entity(world);
        let view_texture = get_view_texture(world);
        graph.run_sub_graph(
            "draw_3d_graph",
            vec![
                SlotValue::Entity(view_entity),
                SlotValue::TextureView(view_texture),
            ],
        )?;
    }
}

(note that currently inputs to sub-graph runs must be specified in the order they are defined, but ultimately they will be an unordered map to avoid incorrect-ordering errors)

3D Scene Example

Multiple lights work as expected! Try adding more!

Next Steps

• Collect and respond to feedback from you all!
• Visibility
  • Implement basic frustum culling and ensure each extracted View's phase only renders what is "visible"
• Viewports
  • Add support for user-defined "Viewports", which render a scene from a specific camera to a texture of arbitrary size. This texture can then be used as a material in the main pass. Viewports can be used to implement split screen, world space UIs, mirrors, etc.
• Remove bevy_render's RenderContext, RenderPass, and RenderResourceContext in favor of direct wgpu apis
  • The dynamic apis support arbitrary user-defined backends registered at runtime, but they also introduce complexity and overhead. We (myself and the other bevy devs) have come to the conclusion that this abstraction isn't worth the price we're paying for it.
  • This would mean that we could use wgpu's RAII for gpu resources. No more manual Buffer frees!
  • This means that the current bevy_webgl2 backend will break, but @Frizi is working to make the wgpu WebGL backend usable. Ideally this will happen before we make the switch.
• Experiment with using Naga and WGSL for all of our shader needs
  • This will enable us to remove some of the most troublesome non-rust dependencies in our tree: bevy-glsl-to-spirv, shaderc, and spirv-reflect.
  • This will likely require implementing a preprocessor to support "shader defs" and "imports", but that should be pretty straightforward.
• Experiment with handling Asset->GpuType conversion in the Extract step
  • Currently textures and meshes are handled in app systems, which is a bit dirty
• Report potential bug in As{Std140, Std430} for arrays of As{Std140, Std430} LPGhatguy/crevice#27
  • DynamicUniform containing array of structs containing f32 arrays doesn't work as expected
  • DynamicUniform containing array of structs containing VecX MatX does work as expected
• Consider generalizing TextureCache. Could be used for buffers, bind group descriptors, etc
• Find a way to make DrawFunctions read-only and lock-free during graph execution (likely by extending SystemState)
• Consider adding a RenderPlugin abstraction with slightly more structure?
• Better run_sub_graph() input api
  • it shouldn't require correctly-ordered inputs. accept a "named" map instead
• Cleanup 2d camera. Z Offsets in bundles shouldn't be necessary
• Consider using marker components for camera types (ex: 3dCamera, 2dCamera). Use a single ActiveCamera<3dCamera>(Option<Entity>) instead of centralized hashmap.
• Re-add render graph execution parallelism. This should be even easier than before because all state
• Nodes can currently only queue subgraphs to run immediately after they finish running (and before their dependencies run).
  • Pros: gives the renderer full control over how the queued subgraph is executed
  • Cons: prevents us from adding "graph outputs" and giving the calling Node direct access to them. Nodes cannot internally consume the results of the subgraphs they queue up because they haven't been run yet. Instead "dependent nodes" must operate on these outputs. This is probably fine in practice. Shadow maps, Viewports, and "driver node" patterns should all work well with the current impl. Generally "outputs" relevant to the caller are already passed in as inputs (such as what texture to render to).
  • We could consider an async api that yields control to the graph executor and returns outputs. The main reason not to do this is that async traits aren't fun to work with / aren't natively supported.
• Custom Shaders
  • Consider using preprocessors to import "standard bindings". This could cut down on boilerplate in shaders and allow "standard bindings" to have a large number of imported fields.
  • How to easily share binding logic between shaders?
    • Maybe break up "Draw Functions" into "Bind Functions" and "Draw Functions"?
    • Godot-style "shader templates", where the user opts in to a specific context (ex: a PbrMaterial context) and automatically gets bindings for that.
  • How to ensure users define their bind groups with the right indices / don't step on existing bind groups?
  • How to ergonomically convert and bind ECS data as Gpu data
• Consider an UpdateArchetypes trait:
  • Nodes could require this trait and automatically update archetypes instead of users needing to manually drive them in Node::update?
  • Systems, QueryState, and SystemState could implement this
  • easily derivable
• Consider using scope + direct Queue access to automatically populate UniformVec buffers / copy to actual buffer. Should make these abstractions "just work" with a single call, without needing to remember the "reset, add items, write to staging, write to buffer" calls (in that order)
• Add support for DirectionalLights + shadows (use an orthographic projection)
• Configure PointLight view projection based on light properties (instead of hard coding)

[aevyrie]

Subgraphs

This is incredibly minor, but I have a question about the code in this example:

Graphs can now require inputs, which can be added like this: [...]

A node can have many slots, and nodes can be connected with edges. Why is the function called add_slot_edge() instead of add_node_edge()? My understanding from your description is that graphs can expect multiple input data (slots) which can be bundled together (nodes) and connected to describe the data flow between graph nodes (edges).

Errata?

In the "subgraphs don't run by default" section, is it a typo that only two slot inputs are supplied?

graph.run_sub_graph(
        "draw_3d_graph",
        vec![
            SlotValue::Entity(view_entity),
            SlotValue::TextureView(view_texture),
        ],
    )?;

The earlier example appears to specify three?

Named Inputs

(note that currently inputs to sub-graph runs must be specified in the order they are defined, but ultimately they will be an unordered map to avoid incorrect-ordering errors)

Does this mean we can expect to pair inputs, or slots in general, with a name (K,V)? Drawing from SlotInfo, something like this would be nice for readability:

graph.run_sub_graph(
        "draw_3d_graph",
        [
            (VIEW_ENTITY, SlotValue::Entity(view_entity),),
            (RENDER_TARGET, SlotValue::TextureView(view_texture)), 
        ]
    )?;

Frustum Culling

Visibility
Implement basic frustum culling and ensure each extracted View's phase only renders what is "visible"

Can I help out with this? We would need rudimentary bounding box generation, which is a big can of worms. We can create something that's quick and dirty, at the risk of people actually using it.

I think it's a topic that might need an RFC of its own (following bevyengine/rfcs#12 ?), but I also understand the desire to get some quick render perf wins in place. I wrote a frustum culling plugin which lead to writing this bounding box plugin for the reason mentioned above. Let me know if you'd like my input with this, I learned a lot in the process of writing those prototypes.

Getting data back to the CPU

Outside of compute, do you plan to add the ability to copy a buffer back to the CPU? This would be nice for a color picking shader (🙂), or other cases where it would be helpful to render to bytes we can analyze on the CPU. Maybe we could specify output slots in the render graph that would actually be populated by the GPU?

Stuff I like

Some things that stood out to me in this post that I especially appreciate:

• Remove bevy_render's RenderContext, RenderPass, and RenderResourceContext in favor of direct wgpu apis

  I'm glad that we might move towards exposing the wgpu API directly. Having the wgpu learning materials and examples applicable to Bevy is really nice as a rendering novice. I feel like I can meaningfully work towards being able to contribute.

• Viewports/Views!

• Render graph modularity. I'm hoping these changes will make contributions a bit more accessible, and render features a bit less daunting.

[cart]

node can have many slots, and nodes can be connected with edges. Why is the function called add_slot_edge() instead of add_node_edge()? My understanding from your description is that graphs can expect multiple input data (slots) which can be bundled together (nodes) and connected to describe the data flow between graph nodes (edges).

This isn't a change from the current design. A "slot edge" connects a specific "node slot" output to a specific "node slot" input. Slot edges both pass data between nodes and create an ordering. A "node edge" just creates an ordering between nodes (no slots required).

In the "subgraphs don't run by default" section, is it a typo that only two slot inputs are supplied?

Yup this was a typo. I'll fix that now.

Does this mean we can expect to pair inputs, or slots in general, with a name (K,V)? Drawing from SlotInfo, something like this would be nice for readability:

Hmm I don't have strong opinions on api yet, but it will definitely include key->value (aka name->slotvalue) mappings in some form or another.

Can I help out with this? We would need rudimentary bounding box generation, which is a big can of worms. We can create something that's quick and dirty, at the risk of people actually using it.

Absolutely! 3d frustum culling is definitely a bit more complex and should probably still go through an rfc process. I'll probably start with 2d culling so we can quickly build out visibility apis / not get stuck in the details of culling. Definitely feel free to start experimenting here. I have no plans to dig into 3d visibility in the short term.

do you plan to add the ability to copy a buffer back to the CPU?

Yup this is important to support. An easy solution that requires zero changes: add a "render app system" that run after the graph execution, maps and read relevant buffers, and writes to a crossbeam channel resource. The render world has a Sender<MyType> resource and the app world has a Receiver<MyType> resource. As soon as app logic "needs" MyType, it can just block on the Receiver<MyType> channel until rendering finishes. This way, users can decide when, where, and if they want to block. No need to skip frames (unless that is desired). It just introduces a "sync point" at the point where app logic needs the data. But theres no avoiding a sync point for that case.

[aevyrie]

I'll probably start with 2d culling so we can quickly build out visibility apis / not get stuck in the details of culling.

That makes sense 👍
I can experiment with this in the frustum culling plugin once you have some visibility APIs available to test.

read relevant buffers, and writes to a crossbeam channel resource

Could we instead use the built in bevy event system? That way we can make "reactive" components that listen to events in the same way other external events work (mouse, keyboard, etc), without requiring syncs? Maybe more importantly, is this "copy/map buffers to the CPU" something you plan to include in a future render pipeline rework? This is an area I could contribute to testing the API.

[mtsr]

I don't think a functionality like color picker would have to run within a frame. It would be very involved to do and would at best result in reducing a frame or two of latency of your picker. You would have to render your scene twice, even if just for a single pixel viewport.

I think @aevyrie may not mean a color picker in the sense of selecting the final color of a rendered scene, but a unique-color-per-object based object picker.

[StarArawn]

I was thinking about this today and it would be really helpful if the volumes were built to be easily transferred to the GPU. It could be useful for clustered where we'll need light volumes and camera volumes to calculate the partitioned light list.

[superdump]

And for shadow map projections.

[mtsr]

Similar to the old "base" render graph. It provides a minimal, largely empty pipeline that Plugins can add logic to. By making this well defined with statically exported hooks, we can encourage people to write compatible render logic while still exposing full pipeline access.

I would still propose (as I did before) to make this its own crate. This 'core' graph has to by it's very definition be quite opinionated and seems likely to fragment into at least 2 or 3 different versions depending on system specs and API limitations.

One of the problems I ran into with adding PostProcessing, is that it becomes impossible to do ordering relative to passes added by different plugins, without adding some kind of magic constants. So then either my plugin needs to depend on plugins that may or may not be there (such as UI) or, which I would prefer, the 'core' render graph depending on these plugins instead. And bevy_render itself definitely shouldn't depend on these optional plugins.

[alice-i-cecile]

I'd prefer the labels approach over the stage one if at all possible.

[cart]

I would still propose (as I did before) to make this its own crate. This 'core' graph has to by it's very definition be quite opinionated and seems likely to fragment into at least 2 or 3 different versions depending on system specs and API limitations.

Yup I ultimately plan to break out the core graph into its own crate. I just postponed it to avoid boilerplate / make prototyping easier.

Modular dependencies

Render Graph nodes already behave a lot like many-to-many labels. Add an empty node, add the things you want to wait for as ancestors of that node, and connect the "empty" node to whatever needs to wait for "upstream" work. Imo we have already solved this problem with the MAIN_PASS_DEPENDENCIES node in the core pipeline. Any plugin that needs to add work that is done before the main pass just connects its node(s) to MAIN_PASS_DEPENDENCIES node.

For cases where you might have arbitrary dependency chains that must run one at a time and order might matter (ex: PostProcessA -> PostProcessB -> PostProcessC), this is largely a "conventions" issue. I think a PostProcessorDriver node, which reads a Res<PostProcessSubGraphList> and kicks off subgraphs run in the order defined by the resource might be the way to go. Alternatively, we could define a "PostProcessChainStartNode", which downstream plugin developers could look up, then walk "down render graph node linked-list" and insert themselves where relevant in the list.

[mtsr]

This sounds like a similar problem to system ordering inside the schedule. Could we maybe use similar concept of stages in the render pipeline?

Well, it's not just ordering, really. Sometimes plugins would need to be aware of what textures need to be used, etc, that might change depending on the plugins used.

[cart]

As long as they're all working on things like the "main render target" texture, then they can already just consume those in a standard way. If they require additional, but "standardizable" inputs, we can just standardize them. If they have custom requirements to interop with other effects, then they will need to work out compatibility with the other effects manually. I'm guessing most post processing effects will be able to get away with consuming "custom data managed by the effect plugin" and "standard inputs provided by the core pipeline". More complicated custom interop between two effect plugins would be implemented with careful error handling and order control during graph construction.

[superdump]

I think the starting point should be manual connection of things doing work and that the things doing work expose slots. That way people can connect them up how they want to. When we have that, we can build convenience stuff on top that connects things up in sensible ways for common use cases.

[alice-i-cecile]

Views
Views are just entities that are extracted with an ExtractedView component.
They define a projection, a transform, a width, and a height

I really like this general model. What if we want multiple simultaneous views of the same entity though? Imagine split screen multiplayer. This is the same problem we ran into with frustum culling.

Relations presents an obvious solution: make it a relation tied to a particular camera (perhaps reducing projection data duplication too).

[mtsr]

I think what cart describes here is just that different Views (not just Camera's, but also Lights for shadow map rendering, etc) are described by an Entity in the renderer World. This has no relation to entities being rendered.

For shadow map rendering for point lights, this means 6 different Views might be created, one for each face of the shadow map cube. Now these might relate to the original point light by way of some identifier, but the original entity cannot even be accessed after extraction anyway, so it would be just for grouping them, if that's even needed.

[cart]

The only missing piece is filtering down the entities we queue up to draw for each view (and where relevant, filtering down the entity data we extract). Relations are one way to do this, but we could just as easily use a centralized solution (ex a VisibleEntities list per-view). And porting between the two impls should be pretty trivial. I don't think we have much incentive to block on relations or include them in our initial designs. They might end up being suboptimal for this use case anyway.

[mtsr]

The only missing piece is filtering down the entities we queue up to draw for each view (and where relevant, filtering down the entity data we extract). Relations are one way to do this, but we could just as easily use a centralized solution (ex a VisibleEntities list per-view). And porting between the two impls should be pretty trivial. I don't think we have much incentive to block on relations or include them in our initial designs. They might end up being suboptimal for this use case anyway.

And performance of the datastructure is very important here. So we might make different tradeoffs than a general relations implementation. Not saying we shouldn't use relations, just something to be aware of.

[alice-i-cecile]

Consider using marker components for camera types (ex: 3dCamera, 2dCamera). Use a single ActiveCamera<3dCamera>(Option) instead of centralized hashmap.

Strongly in favor of this change. I would also like a way to distinguish between world-space / screen-space coordinates in a unified way as part of this.

Cleanup 2d camera. Z Offsets in bundles shouldn't be necessary

Would this involve a real layering solution in its place? Z coordinates in 2D are currently essential to control the order in which sprites are drawn.

[cart]

Would this involve a real layering solution in its place? Z coordinates in 2D are currently essential to control the order in which sprites are drawn.

Potentially, but I see that as slightly orthogonal. The general motivation behind it is to protect against users "improperly" positioning the 2d camera / allowing us to use default camera values in bundles. We could do this by changing the camera components to literally prevent this, overwriting z offsets in 2d camera transforms to the "correct" value, or setting it to the "correct" value in the "extracted render data".

[FrankenApps]

New graph executor with support for running subgraphs. Much simpler implementation. Renderers are now 100% responsible for deciding how to execute the graph.

Does this also mean, that it will be possible to trigger renders manually? E.g. I only want to render my static scene, when the camera changes...

[cart]

It does 99% of that work. Render logic is now fully decoupled from app logic. The only missing piece is defining how to "signal" that a render needs to happen and only render on that signal (the current prototype always renders after every app update). Fortunately we need to solve this problem to make Bevy work correctly on mobile devices, because we can't trigger renders when the app is minimized. So we'll need to develop a signal that allows us to say "don't render right now".

[mtsr]

Similarly on Windows when a specific window has been resized so that one of it's dimensions is 0.

[superdump]

After a fix from @cart regarding updating archetypes for function systems coerced to exclusive systems or something like that, and a couple more small fixes, I got my project working on top of the pipelined-rendering branch:

I'm now looking around in the code to understand key things that are missing. As the new StandardMaterial has a color member, I had assumed it was working, but the colour is instead hard-coded in the PBR fragment shader. So, I'm going to first take a pass at trying to get the 'flat' values (i.e. no optional textures, only fixed values per mesh) added to the new StandardMaterial that were present in the old one. As such, this will lead me through working with the new bindings. As I am trying to understand the structure of things and the conventions being adopted, I thought it would be useful to document here to help others get going building things. This is what I think I have gathered so far, @cart please correct/comment as you feel appropriate 😃:

• initialisation
  • it looks like reflection is done as before during initialisation using PipelineLayout::from_shader_layouts() https://github.com/cart/bevy/blob/pipelined-rendering/pipelined/bevy_pbr2/src/render/mod.rs#L41-L42
  • if bindings need to be dynamic, this is set on the pipeline layout like this: pipeline_layout.bind_group_mut(0).bindings[0].set_dynamic(true); https://github.com/cart/bevy/blob/pipelined-rendering/pipelined/bevy_pbr2/src/render/mod.rs#L74
  • after modifying any of the binding configuration in the pipeline layout, pipeline_layout.update_bind_group_ids() must be called as it creates bind group descriptor ids based on hashes of the bind group descriptors
• assets
  • the current pattern looks like there are *_resource_system() implemented that take care of creating mesh vertex/index buffers / textures/samplers, staging and copying the data to the gpu, and storing the various ids in a MeshGpuData/TextureGpuData member on the appropriate assets for later use. there is a comment that these GpuData should not be members of the Mesh/Texture - @cart how do you think this should be long term?
• extraction
  • types are defined containing just the data necessary for rendering. this is likely to be raw data in floats, vectors and matrices that still needs uploading, as well as buffer/sampler/texture view ids from assets
  • copy the data from app world resources / asset gpu data / queries against components in the app world into the aforementioned types and insert as new render world resources / components. the Entity is/can be the same in both worlds
• preparation
  • more types are defined, using the crevice crate for handling padding and such for std140, inside UniformVec / DynamicUniformVec as appropriate for the binding
  • write data from components/resources that still needs to be uploaded into the aforementioned types and push into the *UniformVec. for dynamic uniforms, the offset is returned from the DynamicUniformVec.push() call so it can be stored for later reference
  • call .write_to_staging_buffer(&self, render_resources: &RenderResources) on the *UniformVec to stage the data
• nodes
  • in the Node::run() implementation for the node, the *UniformVec.write_to_uniform_buffer(&self, render_context: &mut dyn RenderContext) is called to upload the data to the GPU uniform buffer from the staging buffer
• draw
  • appropriate resources / components are obtained from the render world using a SystemState which 'holds on to persistent state required to drive SystemParam
  • data from these is used to set the pipeline, bindings, vertex/index buffers, and then draw

The sprite functionality shows how textures can be handled and bound as there are no textures in PBR yet.

The explicitness of all of this is quite refreshing for reading and understanding what is happening and I would expect it will make debugging easier too, perhaps after having added some more error handling/verification. I wanted to disable the shadow functionality when trying to get my project working as it looked like it was possibly adding to my confusion of what was initially wrong. I noted a couple of runtime errors already there that allowed me to quickly find what was wrong and fix it, so I think this is looking very positive. It is more boiler plate at the moment, but this is complicated stuff to get the plumbing right. I think we've probably gone from easy to do something wrong and hard to fix it, to easy to do something wrong and easy to fix it. That's great progress. 😃

[superdump]

Just double-checked and all textures are being bound correctly.

[superdump]

@Frizi noted some things on Discord about wgsl:

• bindings are global
• group (glsl: layout set) and binding numbers can be reused in the sense that you can declare multiple bindings that use the same group and binding numbers as long as only one binding with that group and binding number are used in the code path of each entry point
• multiple entry points per file
• even multiple entry points per stage within the same file
• vertex attributes are arguments to the vertex stage entry point
• Vertex stage outputs can be a self-defined struct type

The last few could be useful for supporting normal maps even without a pre-processor as the code within the file can be shared between two entry points, one for normal maps, one without. That's cool.

[superdump]

Oh yeah, I implemented RenderAsset for StandardMaterial and used the RenderAssetPlugin. It’s a very nice abstraction. I may implement it for Mesh too to further simplify the main rendering queue and draw stuff.

[superdump]

Cart implemented it for Mesh. :) I updated my branches again this morning.

[superdump]

The standard-material-textures branch is now merged. Next up is the directional-light-and-shadow branch: cart#6

[StarArawn]

@cart after implementing my own shader in the new renderer I couldn't be more happy with how well the stages work. I think we can do a better job of sharing bindings across shaders though. For example there isn't a great way for me to reuse the standard pbr material in my own shader. One way is to not use the pbr plugin but if I still want lighting/shadows this is difficult. We could add a ZST to the pbr bundle called: StandardMaterialTag which the pbr extract stage looks for. It might also make sense to have a small set of reusable functions for auto creating the layout/bindings for the standard material?

[cart]

Yup I think solving this problem is critical to make user-defined shaders a good experience. I think I want to tackle this problem by solving both "reusing shader code" and "reusing bindings" holistically. Now is definitely a good time to start experimenting. I'll start digging in to the problem asap.

[fintelia]

I'd imagine one use case would be keeping most of the PBR bundle but have custom code to compute the parameters (albedo, roughness, etc.) for fragments and/or using custom code to compute vertex positions.

[mtsr]

I'm trying to work out if we can have a generic ViewPassNode that gets executed for each View in a RenderPhase, since the different PassNodes are so so similar right now. That way just inserting an extra ExtractedView would be enough to get it rendered. And it would make it easier to reuse for new RenderPhases too.

First bit of work doesn't include a subgraph yet, but should (untested, but compiles) iterate over all the Views in a RenderPhase.

<== click to unfold. Folded because of length.

use std::borrow::Borrow;

use crate::{
    camera::RenderTarget,
    color::Color,
    core_pipeline::ViewDepthTexture,
    render_graph::{Node, NodeRunError, RenderGraphContext},
    render_phase::{DrawFunctions, RenderPhase, TrackedRenderPass},
    renderer::RenderContext,
    view::{ExtractedView, ExtractedWindows},
};
use bevy_ecs::{
    prelude::{Entity, QueryState},
    world::World,
};
use wgpu::{
    LoadOp, Operations, RenderPassColorAttachment, RenderPassDepthStencilAttachment,
    RenderPassDescriptor,
};

pub struct ViewPassNode<T: 'static> {
    query: QueryState<(
        Entity,
        &'static ExtractedView,
        &'static RenderTarget,
        Option<&'static ViewDepthTexture>,
        &'static RenderPhase<T>,
    )>,
}

impl<T> ViewPassNode<T> {
    pub fn new(world: &mut World) -> Self {
        Self {
            query: QueryState::new(world),
        }
    }
}

impl<T> Node for ViewPassNode<T> {
    fn update(&mut self, world: &mut World) {
        self.query.update_archetypes(world);
    }

    fn run(
        &self,
        _graph: &mut RenderGraphContext,
        render_context: &mut RenderContext,
        world: &World,
    ) -> Result<(), NodeRunError> {
        let extracted_windows = world.get_resource::<ExtractedWindows>().unwrap();

        for (view_entity, view, render_target, view_depth_texture, render_phase) in
            self.query.iter_manual(world)
        {
            let color_attachment_texture = match render_target {
                RenderTarget::Window(window_id) => extracted_windows
                    .get(window_id)
                    .unwrap()
                    .swap_chain_frame
                    .as_ref()
                    .unwrap(),
                RenderTarget::Texture(texture_view) => texture_view,
            };

            let pass_descriptor = RenderPassDescriptor {
                label: view.name.as_deref(),
                color_attachments: &[RenderPassColorAttachment {
                    view: color_attachment_texture,
                    resolve_target: None,
                    ops: Operations {
                        load: LoadOp::Clear(Color::rgb(0.4, 0.4, 0.4).into()),
                        store: true,
                    },
                }],
                depth_stencil_attachment: view_depth_texture.map(|view_depth_texture| {
                    RenderPassDepthStencilAttachment {
                        view: &view_depth_texture.view,
                        depth_ops: Some(Operations {
                            load: LoadOp::Clear(1.0),
                            store: true,
                        }),
                        stencil_ops: None,
                    }
                }),
            };

            let draw_functions = world.get_resource::<DrawFunctions>().unwrap();
            let render_pass = render_context
                .command_encoder
                .begin_render_pass(&pass_descriptor);
            let mut draw_functions = draw_functions.write();
            let mut tracked_pass = TrackedRenderPass::new(render_pass);
            for drawable in render_phase.drawn_things.iter() {
                let draw_function = draw_functions.get_mut(drawable.draw_function).unwrap();
                draw_function.draw(
                    world,
                    &mut tracked_pass,
                    view_entity,
                    drawable.draw_key,
                    drawable.sort_key,
                );
            }
        }

        Ok(())
    }
}

[mtsr]

I guess going off parallels with the older PassNode, it makes sense to have the PassDescriptor and the Attachments (or RenderTargets or whatever) as parameters to the PassNode.

A question I'm still thinking about is whether this node goes into a subgraph that can be expanded through plugins, or whether this needs to be split and run a subgraph for each view.

[cart]

Yup I think it is worth building a common abstraction here (and the old interface isn't a bad place to start).

[superdump]

directional-light-and-shadow was merged, now I’m improving the shadow biases and trying to transition to an infinite reverse z perspective projection but with that I have had trouble getting rid of all self-shadowing.

I would like to implement calculation of vertex tangents when importing meshes that don’t have them to allow support for normal maps in one shader. One source suggested calculating them in shaders regardless, but someone else in #rendering disagreed and said that it has downsides and pre-calculating is preferred. I guess I’ll go for pre-calculating.

[zicklag]

If we could get a working rewrite of our shaders in Rust, using Rust-GPU, would that be something we want to use? Or do we want to avoid it for now until it get's more stable/usable, etc. and just use WGSL for now?

[cart]

My current take on "officially adopting rust-gpu in bevy":

[cart]

For now, we're adopting naga/WGSL in the new renderer (which also has risk, but way less than rust-gpu because it is a standard backed by many companies and the stack is way simpler and smaller). This enables us to drop the large and complicated bevy-glsl-to-spirv. Ultimately, once naga's GLSL frontend stabilizes we can decide whether or not to switch back. But i'm pretty sure we'll end up supporting everything naga supports. The only real open question is "what will we ultimately use internally and recommend in tutorials".

[cart]

WGSL is our current bet on the answer to that question, but its definitely still open.

[kirawi]

How useable is this right not for a simple 2D game, or should I wait for it to be further along?

[cart]

It is still missing some key 2D features:

• texture atlases / sprite sheets
• ui
• easy custom shaders (custom shaders are currently possible by writing a fully custom pipeline in low-level rendering code)

If you don't need those, you can start building your game now. And we're about to add back all of those features. We've moved renderer work here if you want to follow along: https://github.com/bevyengine/bevy/tree/pipelined-rendering