forked from bevyengine/bevy
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
EnvironmentMapLight, BRDF Improvements (bevyengine#7051)
(Before)  (After)   # Objective - Improve lighting; especially reflections. - Closes bevyengine#4581. ## Solution - Implement environment maps, providing better ambient light. - Add microfacet multibounce approximation for specular highlights from Filament. - Occlusion is no longer incorrectly applied to direct lighting. It now only applies to diffuse indirect light. Unsure if it's also supposed to apply to specular indirect light - the glTF specification just says "indirect light". In the case of ambient occlusion, for instance, that's usually only calculated as diffuse though. For now, I'm choosing to apply this just to indirect diffuse light, and not specular. - Modified the PBR example to use an environment map, and have labels. - Added `FallbackImageCubemap`. ## Implementation - IBL technique references can be found in environment_map.wgsl. - It's more accurate to use a LUT for the scale/bias. Filament has a good reference on generating this LUT. For now, I just used an analytic approximation. - For now, environment maps must first be prefiltered outside of bevy using a 3rd party tool. See the `EnvironmentMap` documentation. - Eventually, we should have our own prefiltering code, so that we can have dynamically changing environment maps, as well as let users drop in an HDR image and use asset preprocessing to create the needed textures using only bevy. --- ## Changelog - Added an `EnvironmentMapLight` camera component that adds additional ambient light to a scene. - StandardMaterials will now appear brighter and more saturated at high roughness, due to internal material changes. This is more physically correct. - Fixed StandardMaterial occlusion being incorrectly applied to direct lighting. - Added `FallbackImageCubemap`. Co-authored-by: IceSentry <c.giguere42@gmail.com> Co-authored-by: James Liu <contact@jamessliu.com> Co-authored-by: Rob Parrett <robparrett@gmail.com>
- Loading branch information
1 parent
84def11
commit e9f5929
Showing
20 changed files
with
522 additions
and
80 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| The pisa_*.ktx2 files were generated from https://github.com/KhronosGroup/glTF-Sample-Environments/blob/master/pisa.hdr using the following tools and commands: | ||
| - IBL environment map prefiltering to cubemaps: https://github.com/KhronosGroup/glTF-IBL-Sampler | ||
| - Diffuse: ./cli -inputPath pisa.hdr -outCubeMap pisa_diffuse.ktx2 -distribution Lambertian -cubeMapResolution 32 | ||
| - Specular: ./cli -inputPath pisa.hdr -outCubeMap pisa_specular.ktx2 -distribution GGX -cubeMapResolution 512 | ||
| - Converting to rgb9e5 format with zstd 'supercompression': https://github.com/DGriffin91/bevy_mod_environment_map_tools | ||
| - cargo run --release -- --inputs pisa_diffuse.ktx2,pisa_specular.ktx2 --outputs pisa_diffuse_rgb9e5_zstd.ktx2,pisa_specular_rgb9e5_zstd.ktx2 |
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| #define_import_path bevy_pbr::environment_map | ||
|
|
||
|
|
||
| struct EnvironmentMapLight { | ||
| diffuse: vec3<f32>, | ||
| specular: vec3<f32>, | ||
| }; | ||
|
|
||
| fn environment_map_light( | ||
| perceptual_roughness: f32, | ||
| roughness: f32, | ||
| diffuse_color: vec3<f32>, | ||
| NdotV: f32, | ||
| f_ab: vec2<f32>, | ||
| N: vec3<f32>, | ||
| R: vec3<f32>, | ||
| F0: vec3<f32>, | ||
| ) -> EnvironmentMapLight { | ||
|
|
||
| // Split-sum approximation for image based lighting: https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf | ||
| let smallest_specular_mip_level = textureNumLevels(environment_map_specular) - 1i; | ||
| let radiance_level = perceptual_roughness * f32(smallest_specular_mip_level); | ||
| let irradiance = textureSample(environment_map_diffuse, environment_map_sampler, N).rgb; | ||
| let radiance = textureSampleLevel(environment_map_specular, environment_map_sampler, R, radiance_level).rgb; | ||
|
|
||
| // Multiscattering approximation: https://www.jcgt.org/published/0008/01/03/paper.pdf | ||
| // Useful reference: https://bruop.github.io/ibl | ||
| let Fr = max(vec3(1.0 - roughness), F0) - F0; | ||
| let kS = F0 + Fr * pow(1.0 - NdotV, 5.0); | ||
| let FssEss = kS * f_ab.x + f_ab.y; | ||
| let Ess = f_ab.x + f_ab.y; | ||
| let Ems = 1.0 - Ess; | ||
| let Favg = F0 + (1.0 - F0) / 21.0; | ||
| let Fms = FssEss * Favg / (1.0 - Ems * Favg); | ||
| let FmsEms = Fms * Ems; | ||
| let Edss = 1.0 - (FssEss + FmsEms); | ||
| let kD = diffuse_color * Edss; | ||
|
|
||
| var out: EnvironmentMapLight; | ||
| out.diffuse = (FmsEms + kD) * irradiance; | ||
| out.specular = FssEss * radiance; | ||
| return out; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| use bevy_app::{App, Plugin}; | ||
| use bevy_asset::{load_internal_asset, Handle, HandleUntyped}; | ||
| use bevy_core_pipeline::prelude::Camera3d; | ||
| use bevy_ecs::{prelude::Component, query::With}; | ||
| use bevy_reflect::{Reflect, TypeUuid}; | ||
| use bevy_render::{ | ||
| extract_component::{ExtractComponent, ExtractComponentPlugin}, | ||
| render_asset::RenderAssets, | ||
| render_resource::{ | ||
| BindGroupEntry, BindGroupLayoutEntry, BindingResource, BindingType, SamplerBindingType, | ||
| Shader, ShaderStages, TextureSampleType, TextureViewDimension, | ||
| }, | ||
| texture::{FallbackImageCubemap, Image}, | ||
| }; | ||
|
|
||
| pub const ENVIRONMENT_MAP_SHADER_HANDLE: HandleUntyped = | ||
| HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 154476556247605696); | ||
|
|
||
| pub struct EnvironmentMapPlugin; | ||
|
|
||
| impl Plugin for EnvironmentMapPlugin { | ||
| fn build(&self, app: &mut App) { | ||
| load_internal_asset!( | ||
| app, | ||
| ENVIRONMENT_MAP_SHADER_HANDLE, | ||
| "environment_map.wgsl", | ||
| Shader::from_wgsl | ||
| ); | ||
|
|
||
| app.register_type::<EnvironmentMapLight>() | ||
| .add_plugin(ExtractComponentPlugin::<EnvironmentMapLight>::default()); | ||
| } | ||
| } | ||
|
|
||
| /// Environment map based ambient lighting representing light from distant scenery. | ||
| /// | ||
| /// When added to a 3D camera, this component adds indirect light | ||
| /// to every point of the scene (including inside, enclosed areas) based on | ||
| /// an environment cubemap texture. This is similiar to [`crate::AmbientLight`], but | ||
| /// higher quality, and is intended for outdoor scenes. | ||
| /// | ||
| /// The environment map must be prefiltered into a diffuse and specular cubemap based on the | ||
| /// [split-sum approximation](https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf). | ||
| /// | ||
| /// To prefilter your environment map, you can use `KhronosGroup`'s [glTF-IBL-Sampler](https://github.com/KhronosGroup/glTF-IBL-Sampler). | ||
| /// The diffuse map uses the Lambertian distribution, and the specular map uses the GGX distribution. | ||
| /// | ||
| /// `KhronosGroup` also has several prefiltered environment maps that can be found [here](https://github.com/KhronosGroup/glTF-Sample-Environments). | ||
| #[derive(Component, Reflect, Clone)] | ||
| pub struct EnvironmentMapLight { | ||
| pub diffuse_map: Handle<Image>, | ||
| pub specular_map: Handle<Image>, | ||
| } | ||
|
|
||
| impl EnvironmentMapLight { | ||
| /// Whether or not all textures neccesary to use the environment map | ||
| /// have been loaded by the asset server. | ||
| pub fn is_loaded(&self, images: &RenderAssets<Image>) -> bool { | ||
| images.get(&self.diffuse_map).is_some() && images.get(&self.specular_map).is_some() | ||
| } | ||
| } | ||
|
|
||
| impl ExtractComponent for EnvironmentMapLight { | ||
| type Query = &'static Self; | ||
| type Filter = With<Camera3d>; | ||
| type Out = Self; | ||
|
|
||
| fn extract_component(item: bevy_ecs::query::QueryItem<'_, Self::Query>) -> Option<Self::Out> { | ||
| Some(item.clone()) | ||
| } | ||
| } | ||
|
|
||
| pub fn get_bindings<'a>( | ||
| environment_map_light: Option<&EnvironmentMapLight>, | ||
| images: &'a RenderAssets<Image>, | ||
| fallback_image_cubemap: &'a FallbackImageCubemap, | ||
| bindings: [u32; 3], | ||
| ) -> [BindGroupEntry<'a>; 3] { | ||
| let (diffuse_map, specular_map) = match ( | ||
| environment_map_light.and_then(|env_map| images.get(&env_map.diffuse_map)), | ||
| environment_map_light.and_then(|env_map| images.get(&env_map.specular_map)), | ||
| ) { | ||
| (Some(diffuse_map), Some(specular_map)) => { | ||
| (&diffuse_map.texture_view, &specular_map.texture_view) | ||
| } | ||
| _ => ( | ||
| &fallback_image_cubemap.texture_view, | ||
| &fallback_image_cubemap.texture_view, | ||
| ), | ||
| }; | ||
|
|
||
| [ | ||
| BindGroupEntry { | ||
| binding: bindings[0], | ||
| resource: BindingResource::TextureView(diffuse_map), | ||
| }, | ||
| BindGroupEntry { | ||
| binding: bindings[1], | ||
| resource: BindingResource::TextureView(specular_map), | ||
| }, | ||
| BindGroupEntry { | ||
| binding: bindings[2], | ||
| resource: BindingResource::Sampler(&fallback_image_cubemap.sampler), | ||
| }, | ||
| ] | ||
| } | ||
|
|
||
| pub fn get_bind_group_layout_entries(bindings: [u32; 3]) -> [BindGroupLayoutEntry; 3] { | ||
| [ | ||
| BindGroupLayoutEntry { | ||
| binding: bindings[0], | ||
| visibility: ShaderStages::FRAGMENT, | ||
| ty: BindingType::Texture { | ||
| sample_type: TextureSampleType::Float { filterable: true }, | ||
| view_dimension: TextureViewDimension::Cube, | ||
| multisampled: false, | ||
| }, | ||
| count: None, | ||
| }, | ||
| BindGroupLayoutEntry { | ||
| binding: bindings[1], | ||
| visibility: ShaderStages::FRAGMENT, | ||
| ty: BindingType::Texture { | ||
| sample_type: TextureSampleType::Float { filterable: true }, | ||
| view_dimension: TextureViewDimension::Cube, | ||
| multisampled: false, | ||
| }, | ||
| count: None, | ||
| }, | ||
| BindGroupLayoutEntry { | ||
| binding: bindings[2], | ||
| visibility: ShaderStages::FRAGMENT, | ||
| ty: BindingType::Sampler(SamplerBindingType::Filtering), | ||
| count: None, | ||
| }, | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.