PMREMGenerator basic version #7902
Conversation
|
Niice! Yeah, we do have custom mipmap for textures, but maybe we don't have support for cubetextures yet? |
|
nice to see the PR @spidersharma03! BTW Exocortex / Clara.io has been working with the talented @spidersharma03 recently and is financially sponsoring this PR. :) |
|
@bhouston sweet! 😊 |
We have to, every other major engine supports it. At least the ability to load in manual mipmaps if nothing else. But given that in WebGL 1.0 it isn't possible to explicitly specify the LOD level manually without an extension, it isn't that useful. The most desired solution should be the flat single texture layout for all the reasons mentioned here by @tschw here: Thus I'd suggest that we start with the flat single-texture latong layout in this PR, at least to start with. I think the laylong layout it is a temporary layout because of the inherent distortion in latlong laytouts - we should eventually do the tetrahedral layout of @pyalot or the cubeToUv2 layout of @tschw. But given that we never likely will save these generated layouts (it is not efficient to have to download them, only the source textures should be downloaded for bandwidth efficient sake), we can change that easily going forward without breaking anything - thus we can easily start with this can get this integrated. @spidersharma03's support of allowing LOD in addition to roughness is a great one and falls into line with what @pyalot suggested here: #5847 (comment) |
Thank you, @bhouston ! |
|
|
||
| } | ||
|
|
||
| this.camera = new THREE.OrthographicCamera( - 1, 1, - 1, 1, - 0.01, 1000 ); |
There was a problem hiding this comment.
this.camera = new THREE.OrthographicCamera( - 1, 1, 1, - 1, 0, 1000 );
Is the camera inverted on purpose? Did you compensate somewhere else?
Also, it is not necessary, but it avoids confusion to keep the frustum in front of the camera, so the near plane distance should be >= 0.
|
@spidersharma03 Thank you for this! Would you be willing to share your working testbeds so we can see the code in action? Can you provide links in the code to any papers you referenced? |
|
@WestLangley Thanks for the points!. Yeah, you are right. I was simply doing the inversion in the shader to compensate for this. This is not required if the camera is set up properly like this. Also negative distance is not required. So this works:: I will share all the references, and also I will detail some points where more work is needed, and where the code is experimental. |
|
An Update on the current work. I am able to successfully pack the textures in a cubeUV format(it's a separate class, similar to LatLonPacker), as well as sample them correctly. The class takes array of cubeTextures lod generated by PMREMGenerator, and creates a single packed cubeUV texture on the GPU. Here is a preview of the packing:: But it really can be in any order. The good thing is that we can(and I will :-)) simply also pack the mip maps as well the same way I did for the LatLon maps, and do the trilinear interpolation in order to avoid the artifacts. |
|
I have finished the packing of cubeUV format. I have also added the support of mip maps. Here is the preview:: I have also modified the dynamic cube map demo from threejs examples here:: |
|
Nice. Thanks!
I think what would be beneficial would be debug examples that render the single and packed cubeUV formats to the screen. You apparently already have those testbeds. Typing |
|
@WestLangley The count has increased as right now I am drawing all the quads(for each roughness 6 quads, and for each mip level 6) of cubeUV format as a plane mesh. It's an overkill, but for debugging it was easy for me.This can be changed later by sending a single buffer and sending the necessary info into the shader via vertex attributes. |
Looking good! 😮 |
|
So great there is a first implementation of this much needed component 👍 Looking forward to read deeper into it. |
|
I still need to explain on some parts of the code , and where it is experimental. Will do it soon, and then we can make further improvements/corrections with the help of all experts here:-) |
…acked format * Modified PMREMGenerator * Add PMREM_CubeUVPacker.js * Added Test scenes
|
I have updated the code, and committed the cubeUVPacker. Also have added a few test scenes. |
* unnecessary check removed for mip level * minor refactor
|
Looking good! @WestLangley should I merge this so you can move the code to the right place? |
|
Right now I have written all the shaders in the respective files itself, But they should be moved to the shader library I guess. Not much aware of three.JS shader architecture. |
Since @bhouston is sponsoring this, I will defer to his decision. This appears to be more of a stand-alone demo than something that can be plugged into @bhouston What do you view as the pattern a user will follow to use this feature? |
|
@WestLangley I think that for the most part people will not use this in real-time on a per-frame basis (they could but it is more of an advanced usage scenario), it is more of a data preparation thing that you do while loading a scene. Thus it can start as an example for now, like the NodeMaterials. My optimal scenario for using it as a data preparation system would be to 1, load a single environment image from a server (as cube2UV format or latlong, probably cube2UV is more efficient on a per pixel basis), 2. run it through this tool to prepare the mipmap chain, 3. use it as the envMap for the MeshStandardMaterial with appropriate options. I think that we should likely advocate using this tool for all examples that use MeshStandardMaterial with environment maps, because otherwise the roughness isn't done correctly. But it can stay in examples for now, as it is a data prep tool and fully optional. |
|
@bhouston Thanks. That sound good! |
|
Given that it is already in the /src/extras area, it is already not included in the default build. I think that it is likely easy to merge in with its current state. I am sure myself and @tschw will likely have improvements to bring in but this is already in a really good state. |
|
The shader for sampling the cubeUV format is right now in the html example files, which is not good. Where should that be moved ? |
|
I think the two major improvements should be the following::
|
I think I would put everything in |
I think, thats good @mrdoob. |
|
Thanks a lot @spidersharma03! |
Hi Guys,
I have written a simple class PMREMGenerator for Convolving the Environment maps corresponding to various roughness values. This is just a start and I was doing some experiments to see if we can do it at run time, generating LatLon/cubeUV maps from the convolved cube maps etc. I think it requires some refactoring as well, but I think, that can be done later when this class is completely ready for use.
Some of the code is still experimental, as I was trying to make it run at every frame. I think, that if we go for Physically believable approach, rather than Physically accurate, it's quite possible, as it can be possible to make the sample count as low as 32, in which case I get 60 fps on my Mac Book Pro(Intel HD Graphics 4000 1024 MB). That way, it could be possible to generate dynamic filtered cube maps.
Here is a very basic result, which I created using my custom shader where for each sphere, one of the convolved cube map texture is applied::
Now I need some help from ThreeJS experts regarding how to use the convolved cube maps which this class generates as an array of WebGLRenderTargetCube.
I think the best way would be, to be able to directly write these convolved cube maps into the mip map level of a cube texture. Once that is done, I think the code is already there in ThreeJS which can handle the selection of appropriate lod from a given roughness value.
Other option would be to read back the data from each of the WebGLRenderTargetCube, and then manually upload the mip maps for the cube texture.
I am not sure, of how to do the above two options in ThreeJS currently so, if someone could shed some light on the way to achieve that, it would be great.
Apart from this, I also have written a simple LaLon Map converter(Also runs on the GPU, and not Included here right now), which creates a packed map of LaLon maps of varying roughness, and mip levels, from the array of WebGLRenderTargetCube generated by this class. Here is a preview of that::
Basically for each LatLon map, which corresponds to a given roughness, I also store the mip maps. These mip maps helps a lot in alleviating the moire patterns while minification. I have written a custom shader to sample from this latlon map for various roughness values and also removing the moire patterns by selecting the appropriate mip levels, and then doing a trilinear interpolation to blend the results.
I think that LatLon maps have this advantage over the cube maps, as we can't have the mip map levels for each of the roughness.
Probably a cubeUV format can also solve this problem, and I am experimenting with that currently.
But I think first of all, if we can use these array of WebGLRenderTargetCube textures for lod(roughness) selection, it would be a good first step forward. Thanks for your support.