HDR EnvMap example with high quality, flexible PMREM Generator. #8237

bhouston · 2016-02-26T21:17:49Z

I've cleaned up the earlier PR #7902 from @spidersharma03 that addressed issue #7402 in a number of ways:

it now uses the encodings on the textures for both input encodings and output encodings (which has been merged into dev except for the output encoding PR Output encoding #8232.)
it automatically sets appropriate encodings, and filters based on designed texture type, such for Unsigned Byte it uses RGBE, where as for Float and Half it uses LinearRGB.
I have pulled out a new file to make THREE.toHalf() accessible to others.
I have pulled out a THREE.Encoding class to allow for easy CPU-based encoding/decoding, although it only contains @spidersharma03's RGBE encoders for float and half.

To refresh from the other, this is based upon @spidersharma03's real-time filtered cubemap generator, that also creates single texture cube map hierarchy with both roughness levels and mipmaps. It is pretty awesome.

I do believe this is the critical missing piece (e.g. #7402) that completes @WestLangley's Standard material implementation.

The results this PR products look like this and it should look like this on all browsers (if there are not any bugs):

Interactive:

http://ci.threejs.org/api/pullrequests/8237/examples/index.html#webgl_materials_envmaps_hdr

PS. Anyone know how to get shadows working when using the effect compositor? There should be shadows here but I do not see any.

…the THREE namespace, move into examples.

@spidersharma03

cube_uv_reflection map shader. (written by @spidersharma03, modified by @bhouston)

@spidersharma03

…/float fixed. (written by @spidersharma03, modified by @bhouston)

…it into pmrem2

…ort.

…lors are not encoded and we need to get perfect blends.

…ding

# Conflicts: # src/renderers/shaders/ShaderLib.js # src/renderers/webgl/WebGLPrograms.js

…ng wrong when using RGBE encoding.

…ject.

WestLangley · 2016-02-26T21:30:29Z

Many thanks to @bhouston and @spidersharma03 for their work on improving MeshStandardMaterial!

Which PRs do you now want merged and which ones are outdated?

Also, what is still on the near-term to-do list, as far as you are concerned?

bhouston · 2016-02-26T21:34:58Z

@WestLangley, this PR contains the non-merged PR #8232. Outstanding issues is to figure out if this new code, of which there is a fair bit, works on all devices. It is designed to render PBR with HDR IBL maps on all devices that are WebGL 1.0 compliant without any extensions.

I think it needs to auto-fallback to UnsignedByte when Float / Half textures are not available - which it does not do. We also have to test the UnsignedByte RGBE workflow to ensure there are no artifacts.

UPDATE: I was able to modify the PR so it always uses RGBE with UnsignedByte and it always looks good. Thus the main issue in this comment is addressed.

bhouston · 2016-02-26T21:38:56Z

This PR does seem to work on Firefox, Chrome and IE with equivalent results:

It also works well on Chrome on OSX. Now I need to check mobile browsers, I suspect there will be some complications.

…devices.

bhouston · 2016-02-27T02:54:13Z

The latest changes to this PR ensure that it looks good and runs at fast frame rates on Nexus 5 and a Samsung Galaxy TabPRO as well as Ubuntu 15.04. I still need to test this on the iPad and iPhone as well as a few more mobile devices.

bhouston · 2016-02-27T10:29:16Z

BTW @MiiBond this PR may be of interest.

bhouston · 2016-02-27T10:29:56Z

BTW @spite, this PR may be of interest as well.

bhouston · 2016-02-27T10:53:46Z

@WestLangley @mrdoob This PR also re-enables indirect light probe irradiance when you use it with the PMREM generated CubeUV layout because this PR gets rid of the seams on all devices:

https://github.com/mrdoob/three.js/pull/8237/files#diff-29f010d669a36f5263d5351f4702e64fR104

:)

mrdoob · 2016-02-27T12:51:33Z

Super sweet! I can confirm that it works on iOS too! 😁

HDR EnvMap example with high quality, flexible PMREM Generator.

mrdoob · 2016-02-27T12:51:40Z

Many thanks!

mrdoob · 2016-02-27T14:09:18Z

PS. Anyone know how to get shadows working when using the effect compositor? There should be shadows here but I do not see any.

They work if you enabled them 2cb798d 😉

spite · 2016-02-27T18:21:54Z

This looks great! Congratulations! 👍

With HDR looks even better than what i was testing! https://www.clicktorelease.com/tmp/snapshot-fsk2-big.jpg

bhouston · 2016-02-27T19:18:15Z

@spite, nice! Are those beautiful graphics using your code or this new code?

bhouston · 2016-02-27T19:18:48Z

@spite, one need thing about the approach in this PR is that it requires zero extensions, while giving what should be near perfect results.

spite · 2016-02-29T12:22:51Z

@bhouston those are snapshots from my tests, also without extensions. very cool stuff you've got there!

mattdesl · 2016-03-01T15:25:51Z

examples/js/pmrem/PMREMGenerator.js

+    renderTarget.texture.minFilter = this.sourceTexture.minFilter;
+    renderTarget.texture.magFilter = this.sourceTexture.magFilter;
+		this.cubeLods.push( renderTarget );
+		size = Math.max( 16, size / 2 );


@benaadams @spidersharma03 Any particular reason the mipmaps are not any smaller than 16x16? Should this be configurable so that PMREMGenerator can be used as a general purpose cube map filtering tool? Right now it seems tailored toward the UV Packer (which may not be necessary in WebGL2).

phtaylor · 2016-03-08T00:28:00Z

great stuff Ben!

spidersharma03 · 2016-03-08T17:01:31Z

@mattdesl @mrdoob, Restriction to 16x16 was done as there were artifacts with roughness maps lower than 16x16. The line::
this.numLods = Math.log2( cubeTextureLods[ 0 ].width ) - 2;
is used to calculate the number of roughness levels(0-1 range), for a given texture size. Yeah, right now it is tailored for packed CubeUV in a way that the above number of roughness levels are fitted in the cubeuv texture. However I created an example(attached here)where cube textures are directly used instead of cube uv. This is based on an older version, but I guess, should give some hints on how to use the convolved cubemaps directly. This example simply was made to test the convolved cube maps. Though, to be able to use the cube maps directly, all of them should be passed to the shader, and interpolation between two levels should be done inside the shader.
PMREM_Test.txt

Regarding the multiplication by 0.9 and some other stuff in the code::
More thought and work is needed here. Right now it's a kind of a hack to use the previously convolved map to convolve the current one. I tried to use the original map to convolve all the lods, but for many textures(specially the high frequency) even a high number of samples(1024) dosen't lead to satisfactory results.
By using the previous convolved maps, a lower number of samples are generally sufficient, however the math goes wrong as the distribution function tries to sample a larger area than what it should be. So I simply scaled the roughness by 0.9(totally empirical) to try to visually match the original result. The condition "if(i <5)" is also an attemt to make the result match the original result. This method requires the most amount of thinking I guess.
We will put more comments in the code where explanation is needed.

Ben Houston added 29 commits February 12, 2016 13:36

clean up PMREMGEnerator, incorporate global encodings.glsl. Add into …

49f49be

…the THREE namespace, move into examples.

example is starting to work with the new code.

837cd59

mostly working.

6ae79e9

using sRGB encoding when loading PNG images.

96e3f85

add HDR images to examples.

4ad84ca

Added Float, Half float buffer support for HDR. Fixed a bug in

8a630a7

cube_uv_reflection map shader. (written by @spidersharma03, modified by @bhouston)

Bug fix for rgbe/float/half buffers. Inconsistency in rgbe/half float…

818a453

…/float fixed. (written by @spidersharma03, modified by @bhouston)

Merge branch 'texture_encoding' of git@github.com:bhouston/three.js.g…

0540e0e

…it into pmrem2

add support for linearToOutputTexel texture.encoding.

a763534

further adoption of linearToOutputTexel in primary materials.

052254b

remove linear_to_gamma_fragment, replaced by new output encoding supp…

9ae10fc

…ort.

move fog_fragment after encoding, the reason is the background/fog co…

c47accd

…lors are not encoded and we need to get perfect blends.

move getTextureEncodingFromMap to WebGLPrograms enclosure.

10d75a1

have linearToOutputTexel the last encoding function as it is used last.

f40a9f5

update ShaderSkin to use new linearToOutputTexel function.

3c4face

Merge branch 'dev' of git@github.com:mrdoob/three.js into output_enco…

2cb466a

…ding

add comment that HDR-focused encodings do not support transparency

3bd62a1

Merge branch 'pmrem2' into output_encoding

53bf896

# Conflicts: # src/renderers/shaders/ShaderLib.js # src/renderers/webgl/WebGLPrograms.js

more work on PMREM - split out Encodings.js, Half.js

4bf3b58

fix accessing internal WebGLRenderer current render target.

4051cb6

more work on PMREM

f3d4aae

add proper source for toHalf.

54d2c68

float and half HDR CUbe Maps load and are properly generated. Somethi…

4c1eaac

…ng wrong when using RGBE encoding.

clean up envMap hdr example and related files.

363c0b7

avoid interference with backwards compatibility gamma input/output.

f390c49

cleaning up example of envmap_hdr

e154426

more example cleanup.

6c6f566

add a ground plane to make it obvious there is a spot light on the ob…

a21fd99

…ject.

remove unused code.

55a5c38

better resizing of envMap_hdr example.

53dc66c

Ben Houston added 4 commits February 26, 2016 20:34

auto-detect available buffer types.

707d3a4

fix bug in encodings.glsl

964e08b

remove anti-aliasing + alpha from WebGLRenderer in envmap hdr example.

589e84a

use Unsigned for envmap hdr demo, seems to look nice and work on all …

208b281

…devices.

Ben Houston added 2 commits February 27, 2016 05:44

rename PMREM_CubeUVPacker -> PMREMCubeUVPacker

6fb992d

better description of example

2207b21

mrdoob added a commit that referenced this pull request Feb 27, 2016

Merge pull request #8237 from bhouston/pmrem4

c2b8ba5

HDR EnvMap example with high quality, flexible PMREM Generator.

mrdoob merged commit c2b8ba5 into mrdoob:dev Feb 27, 2016

bhouston mentioned this pull request Feb 27, 2016

Adopting a Physically-based Microfacet BRDF model in Three.JS #5847

Closed

mattdesl reviewed Mar 1, 2016
View reviewed changes

bhouston deleted the pmrem4 branch March 14, 2016 13:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDR EnvMap example with high quality, flexible PMREM Generator. #8237

HDR EnvMap example with high quality, flexible PMREM Generator. #8237

bhouston commented Feb 26, 2016

WestLangley commented Feb 26, 2016

bhouston commented Feb 26, 2016

bhouston commented Feb 26, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

mrdoob commented Feb 27, 2016

mrdoob commented Feb 27, 2016

mrdoob commented Feb 27, 2016

spite commented Feb 27, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

spite commented Feb 29, 2016

mattdesl Mar 1, 2016

phtaylor commented Mar 8, 2016

spidersharma03 commented Mar 8, 2016

HDR EnvMap example with high quality, flexible PMREM Generator. #8237

HDR EnvMap example with high quality, flexible PMREM Generator. #8237

Conversation

bhouston commented Feb 26, 2016

WestLangley commented Feb 26, 2016

bhouston commented Feb 26, 2016

bhouston commented Feb 26, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

mrdoob commented Feb 27, 2016

mrdoob commented Feb 27, 2016

mrdoob commented Feb 27, 2016

spite commented Feb 27, 2016

bhouston commented Feb 27, 2016

bhouston commented Feb 27, 2016

spite commented Feb 29, 2016

mattdesl Mar 1, 2016

Choose a reason for hiding this comment

phtaylor commented Mar 8, 2016

spidersharma03 commented Mar 8, 2016