-
-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement opt-in sharp screen-space reflections for the deferred renderer. #12959
Conversation
The generated |
2cb94e1
to
f44dfb3
Compare
This should be ready for review now. |
renderer. This commit implements *screen-space reflections* (SSR), which approximate real-time reflections based on raymarching through the depth buffer and copying samples from the final rendered frame. Numerous variations and refinements to screen-space reflections exist in the literature. This patch foregoes all of them in favor of implementing the bare minimum, so as to provide a flexible base on which to customize and build in the future. For a general basic overview of screen-space reflections, see [1]. The raymarching shader uses the basic algorithm of tracing forward in large steps (what I call a *major* trace), and then refining that trace in smaller increments via binary search (what I call a *minor* trace). No filtering, whether temporal or spatial, is performed at all; for this reason, SSR currently only operates on very shiny surfaces. No acceleration via the hierarchical Z-buffer is implemented (though note that bevyengine#12899 will add the infrastructure for this). Reflections are traced at full resolution, which is often considered slow. All of these improvements and more can be follow-ups. SSR is built on top of the deferred renderer and is currently only supported in that mode. Forward screen-space reflections are possible albeit uncommon (though e.g. *Doom Eternal* uses them); however, they require tracing from the previous frame, which would add complexity. This patch leaves the door open to implementing SSR in the forward rendering path but doesn't itself have such an implementation. Screen-space reflections *are* supported in WebGL 2. To add screen-space reflections to a camera, use the `ScreenSpaceReflections` component. `DepthPrepass` and `DeferredPrepass` must also be present for the reflections to show up. The `ScreenSpaceReflections` component contains several settings that artists can tweak, and also comes with sensible defaults. A new example, `ssr`, has been added. It's loosely based on the [three.js ocean sample], but all the assets are original. Note that the three.js demo has no screen-space reflections and instead renders a mirror world. Additionally, this patch fixes a random bug I ran across: that the `"TONEMAP_METHOD_ACES_FITTED"` `#define` is incorrectly supplied to the shader as `"TONEMAP_METHOD_ACES_FITTED "` (with an extra space) in some paths. [1]: https://lettier.github.io/3d-game-shaders-for-beginners/screen-space-reflection.html [three.js ocean sample]: https://threejs.org/examples/webgl_shaders_ocean.html
I renamed |
Don't have time for a review, but saw the request. Just wanted to note that specular occlusion should use SSR if it is available. I believe I cited some papers in the shader code that talk about how to use SSR with specular occlusion. This might be out of scope, but it popped in my head when I saw this. |
@aevyrie Yes, I agree. I think that should be a followup as it'll be a decent amount of code. |
The example crashes if any other prepasses are added to the camera, for example
I believe the |
@chronicl Fixed, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works surprisingly well with TAA, I was expecting the reflections to cause issues.
A bit surprised that the water reflection from the envmap is part of SSR, and not just part of the PBR shader, but maybe there's a good reason for that.
Some weird shader defs / things that stand out to me about how SSR is setup on the CPU. Not sure it's entirely right (although, it does work).
This PR is going to conflict with the clearcoat PR, probably want to merge that one first.
/// [`DeferredPrepass`] to the camera as well. | ||
/// | ||
/// SSR currently performs no roughness filtering for glossy reflections, so | ||
/// only very smooth surfaces will reflect objects in screen space. You can |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please link to the StandardMaterial roughness property here.
|
||
for (view, ssr_settings) in views.iter() { | ||
let uniform_offset = match ssr_settings { | ||
None => 0, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's going on here? Why not use UniformComponentPlugin or similar?
crates/bevy_pbr/src/ssr/mod.rs
Outdated
let mut shader_defs = vec![ | ||
"DEPTH_PREPASS".into(), | ||
"DEFERRED_PREPASS".into(), | ||
"SSR".into(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the SSR shader def here for?
crates/bevy_pbr/src/deferred/mod.rs
Outdated
@@ -301,6 +305,10 @@ impl SpecializedRenderPipeline for DeferredLightingLayout { | |||
shader_defs.push("MOTION_VECTOR_PREPASS".into()); | |||
} | |||
|
|||
if key.contains(MeshPipelineKey::SCREEN_SPACE_REFLECTIONS) { | |||
shader_defs.push("SSR".into()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shader_defs.push("SSR".into()); | |
shader_defs.push("SSR".into()); |
SSR -> SCREEN_SPACE_REFLECTIONS?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, good point, that would be more consistent with SSAO
@@ -301,6 +305,10 @@ impl SpecializedRenderPipeline for DeferredLightingLayout { | |||
shader_defs.push("MOTION_VECTOR_PREPASS".into()); | |||
} | |||
|
|||
if key.contains(MeshPipelineKey::SCREEN_SPACE_REFLECTIONS) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be added to meshlet/material_draw_prepare.rs too, iirc. Please test the meshlet example with SSR.
{ | ||
// SSR is only supported in the deferred pipeline, which has no MSAA | ||
// support. Thus we can assume MSAA is off. | ||
let mut mesh_pipeline_view_key = MeshPipelineViewLayoutKey::from(Msaa::Off) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why use MeshPipelineViewLayoutKey here? Also, why check for prepasses at all? All you need is the deferred prepass?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The depth prepass is also needed, otherwise depth isn't generated by the deferred pass
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Like mentioned in discord, there's currently a bug where the image gets brighter when toggling SSR.
There also seems to be an issue with the transmission example. It's probably just a wrong ifdef.
Other than that, the code is high quality and everything looks good.
}; | ||
|
||
// The speed of camera movement. | ||
const CAMERA_KEYBOARD_ZOOM_SPEED: f32 = 0.1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can do it if you feel like it but it's not required at all to merge. (I already made a note for me to do that in a follow-up PR).
We generally try to put things like camera controllers and debug text in a separate submodule so only the setup part that shows how to enable the feature is at the top.
Unless I'm misunderstanding what you are saying. The envmap reflection is still handled by the PBR shader when SSR is not present and when it's present it's still done when the roughness is under a certain threshold. |
`SCREEN_SPACE_REFLECTIONS`
In the SSR example, disabling SSR turns off both the cube reflection, but also the reflection on the water from the env map, which surprises me. |
The reason why SSR delays evaluation of the environment map until the SSR pass is that the specular part of the environment map is only taken into account if the ray misses (after all, the environment map is "infinitely" far away), and we don't know whether the ray is going to miss until we evaluate SSR. In other words, SSR effectively doubles as raymarched occlusion for the environment map. |
Oh I guess doesn't have to be this PR, but it's probably better to use a compute shader for SSR. |
Yeah, definitely not this PR. I'd much rather have it as-is and introduce a compute path later. We'll also want to reuse the depth pyramid once we have that. |
Will have to redo this later. |
Closing as I am burned out and have no motivation to get this landed anymore. |
…erer, with improved raymarching code. (#13418) This commit, a revamp of #12959, implements screen-space reflections (SSR), which approximate real-time reflections based on raymarching through the depth buffer and copying samples from the final rendered frame. This patch is a relatively minimal implementation of SSR, so as to provide a flexible base on which to customize and build in the future. However, it's based on the production-quality [raymarching code by Tomasz Stachowiak](https://gist.github.com/h3r2tic/9c8356bdaefbe80b1a22ae0aaee192db). For a general basic overview of screen-space reflections, see [1](https://lettier.github.io/3d-game-shaders-for-beginners/screen-space-reflection.html). The raymarching shader uses the basic algorithm of tracing forward in large steps, refining that trace in smaller increments via binary search, and then using the secant method. No temporal filtering or roughness blurring, is performed at all; for this reason, SSR currently only operates on very shiny surfaces. No acceleration via the hierarchical Z-buffer is implemented (though note that #12899 will add the infrastructure for this). Reflections are traced at full resolution, which is often considered slow. All of these improvements and more can be follow-ups. SSR is built on top of the deferred renderer and is currently only supported in that mode. Forward screen-space reflections are possible albeit uncommon (though e.g. *Doom Eternal* uses them); however, they require tracing from the previous frame, which would add complexity. This patch leaves the door open to implementing SSR in the forward rendering path but doesn't itself have such an implementation. Screen-space reflections aren't supported in WebGL 2, because they require sampling from the depth buffer, which Naga can't do because of a bug (`sampler2DShadow` is incorrectly generated instead of `sampler2D`; this is the same reason why depth of field is disabled on that platform). To add screen-space reflections to a camera, use the `ScreenSpaceReflectionsBundle` bundle or the `ScreenSpaceReflectionsSettings` component. In addition to `ScreenSpaceReflectionsSettings`, `DepthPrepass` and `DeferredPrepass` must also be present for the reflections to show up. The `ScreenSpaceReflectionsSettings` component contains several settings that artists can tweak, and also comes with sensible defaults. A new example, `ssr`, has been added. It's loosely based on the [three.js ocean sample](https://threejs.org/examples/webgl_shaders_ocean.html), but all the assets are original. Note that the three.js demo has no screen-space reflections and instead renders a mirror world. In contrast to #12959, this demo tests not only a cube but also a more complex model (the flight helmet). ## Changelog ### Added * Screen-space reflections can be enabled for very smooth surfaces by adding the `ScreenSpaceReflections` component to a camera. Deferred rendering must be enabled for the reflections to appear. ![Screenshot 2024-05-18 143555](https://github.com/bevyengine/bevy/assets/157897/b8675b39-8a89-433e-a34e-1b9ee1233267) ![Screenshot 2024-05-18 143606](https://github.com/bevyengine/bevy/assets/157897/cc9e1cd0-9951-464a-9a08-e589210e5606)
This commit implements screen-space reflections (SSR), which approximate real-time reflections based on raymarching through the depth buffer and copying samples from the final rendered frame. Numerous variations and refinements to screen-space reflections exist in the literature. This patch foregoes all of them in favor of implementing the bare minimum, so as to provide a flexible base on which to customize and build in the future.
For a general basic overview of screen-space reflections, see 1. The raymarching shader uses the basic algorithm of tracing forward in large steps (what I call a major trace), and then refining that trace in smaller increments via binary search (what I call a minor trace). No filtering, whether temporal or spatial, is performed at all; for this reason, SSR currently only operates on very shiny surfaces. No acceleration via the hierarchical Z-buffer is implemented (though note that #12899 will add the infrastructure for this). Reflections are traced at full resolution, which is often considered slow. All of these improvements and more can be follow-ups.
SSR is built on top of the deferred renderer and is currently only supported in that mode. Forward screen-space reflections are possible albeit uncommon (though e.g. Doom Eternal uses them); however, they require tracing from the previous frame, which would add complexity. This patch leaves the door open to implementing SSR in the forward rendering path but doesn't itself have such an implementation. Screen-space reflections are supported in WebGL 2.
To add screen-space reflections to a camera, use the
ScreenSpaceReflectionsBundle
bundle or theScreenSpaceReflectionsSettings
component. In addition toScreenSpaceReflectionsSettings
,DepthPrepass
andDeferredPrepass
must also be present for the reflections to show up. TheScreenSpaceReflectionsSettings
component contains several settings that artists can tweak, and also comes with sensible defaults.A new example,
ssr
, has been added. It's loosely based on the three.js ocean sample, but all the assets are original. Note that the three.js demo has no screen-space reflections and instead renders a mirror world.Additionally, this patch fixes a random bug I ran across: that the
"TONEMAP_METHOD_ACES_FITTED"
#define
is incorrectly supplied to the shader as"TONEMAP_METHOD_ACES_FITTED "
(with an extra space) in some paths.Changelog
Added
ScreenSpaceReflections
component to a camera. Deferred rendering must be enabled for the reflections to appear.