Add random rotation to cubemap roughness sampling

[JFonS]

Adds a random rotation offset to each texel in order to break the aliasing pattern. It still doesn't get rid of fireflies, but it makes them more uniform and therefore less noticeable.

Comparison at various roughness and metallic levels:
Left is before, right is after.

[Calinou]

With a "worst case" HDRI for radiance map filtering (due to a bright sun), I can confirm this reduces overall fireflies:

1,024 GGX steps (default)

Before	After

4,096 GGX steps

Before	After

The RealTime filter mode (not pictured here) is still more suited in this case, with no visible fireflies whatsoever. Further improvements to the high-quality mode could be made as clayjohn said:

I found that prefiltering the HDRI was really helpful in 3.x. basically you create mipmaps from the HDRI and then use mipmaps when you filter to smooth out the samples.

Clamping the brightness of the reflection (before filtering) is also an option, but it'll have to be a project setting or Environment property to be adjustable by the user. Another alternative is to tonemap to a reduced dynamic range, perform filtering and restore the original dynamic range. See also godotengine/godot-proposals#886.

Testing project: test_radiance_map_high_quality.zip

[clayjohn]

I really like this! I think it should really help once we add in a few other things to fix the firefly artifacts. I hesitate to merge it right away though because it will stop us from being able to precompute the inner loop. As described here we can precompute the inner loop on CPU (or with compute shaders we can do it in shader and store the results in local memory) and massively speed up the process.

I think we should hold off until we have implemented the improvements outlined in #43763 (comment). If we can't get quality to an acceptable level, then we should add this.

[JFonS]

If I understand correctly, pre-computing the sample directions doesn't really clash with adding a random per-vexel rotation. You can just get the sample direction from the pre-computed set and apply the rotation afterwards.

What we really need to see is whether the rotation is actually needed or pre-filtering the samples is already good enough to break the aliasing.

[clayjohn]

If I understand correctly, pre-computing the sample directions doesn't really clash with adding a random per-vexel rotation. You can just get the sample direction from the pre-computed set and apply the rotation afterwards.

We could cache the hammersley sequence, but the blog I linked above goes so far as to pre-calculate ndotl and filter out samples accordingly. At any rate, that isn't a barrier to this, once we see how things look with pre-filtering we may decide that this needed regardless of the potential for optimizations.

What we really need to see is whether the rotation is actually needed or pre-filtering the samples is already good enough to break the aliasing.

Agreed.

[JFonS]

Setting as draft since we decided to wait on the pre-filtering before deciding if we need this or not.

@clayjohn I see what you mean now. I didn't remember I implemented the rotation by offseting the Hammersley sequence result, but all this does is rotate the sample direction around the normal, so it has no effect on ndotl. If we end up adding pre-computation and the filtering is not enough (so many ifs, I know :)) this should be re-implemented by creating a rotation matrix outside the for and uisng it to rotate the pre-comuted sample directions. The only goal is to break the aliasing pattern that appears from re-using the same exact sampling pattern in all texels.

[clayjohn]

I did a quick test using prefiltering and this is the result on the environment map @Calinou uses above (all the images use metallic = 1; roughness = 0.25).

As you see below, prefiltering seems to take care of most artifacts

Using 32 samples results in great quality reflections and it is only a few times more expensive than the realtime option (for reference realtime takes about 1.5 ms on my hardware and high quality now takes ~5ms instead of 60ms).

Using 1024 samples (~65ms):

Using 32 samples (~5ms):

Old version using 32 samples (~4ms):

Old version with 1024 samples (~60ms):

[JFonS]

I tested random rotation on top of #58177, and it still helps break the blocky patterns in some cases, but at the expense of adding more noise.

So we would need to have a final blurring step to reduce the grain. A better random rotation sequence can probably also help.

Closing this PR as I don't think it's worth working on it right now, but random rotation + blur could still make sense as an option for static environment maps.

[clayjohn]

I tested random rotation on top of #58177, and it still helps break the blocky patterns in some cases, but at the expense of adding more noise.

That looks really good! I bet we could also remove the artifacts by taking more samples during filtering.