Add access to the viewport's G-buffers (and depth).

[SIsilicon]

With this addition, the G buffers in viewports, and the depth buffer, are now accessible textures that can be used in shaders.

Using the new buffer_mode property in ViewportTexture, the following buffers are now accessible, including the default Color buffer.

• Depth
• Diffuse + Ambient Factor
• Specular + Metallic
• Normal + Roughness
• Subsurface

It is now possible to do various mid-processing effects, such as this.

There is also an expose_gbuffer variable in Viewport that enables access to the G-buffers, with some exceptions.

Bugsquad edit: This closes godotengine/godot-proposals#798.

[Zireael07]

Regarding CI - it failed on clang format and/or documentation, if I am reading it correctly.

[SIsilicon]

Yeah, the error it's giving is kinda confusing, this is my first time using the Clang. I can't tell what exactly is not right with the formatting in the code parts it's highlighting. :/

[SIsilicon]

Well I'm confused. Can someone explain to me what this is supposed to mean?

I don't understand that error.

[Calinou]

@SIsilicon This is probably due to trailing whitespace. Make sure you don't have trailing whitespace in your local copy.

PS: If you want to stage all changes in a repository but ignore whitespace changes, add the following alias to Git by editing the .gitconfig file:

[alias]
	addnw = !sh -c 'git diff -U0 -w --no-color "$@" | git apply --cached --ignore-whitespace --unidiff-zero -'

Then use git addnw anywhere in a Git repository.

[SIsilicon]

Sorry for the commits. I have no idea how to do the check locally myself.
Anyway, there is one more thing I discovered I needed to do. Apparently, Android and iOS failed to build because of usage of texture swizzling. I'll have to have that pre-processed out. See you tomorrow!

[SIsilicon]

Ok, thanks for pointing all that out @clayjohn . I guess I was a little messy with my code. 😅
As for the wasted memory thing, I guess I should take your suggestions, plus I'll rename force_mrt to expose_gbuffer since that name makes more sense with the proposed changes.

[clayjohn]

Looking good so far!

As you'll see from my comments, I am concerned about the constant memory/performance impact this PR would have. However, I think any performance issues can be addressed with a slight API change.

Right now the extra textures are created regardless of whether the user is using mrts, and then if mrts are used the blits take place automatically, even if the textures are not read. Even if a user does not use mrts, they can force mrts in order to copy the textures. The problem there is that they are then forced to calculate mrts even if there scene doesn't benefit from them.

I suggest instead exposing a boolean expose_textures, the textures would only be exposed when expose_textures is true. Further, you would only expose the textures that would be used (so mrts will only be exposed when mrts are used). Your block in render_scene would become (in psuedo-code):

if (expose_textures) {
    //blit color
    //blit depth
    if (use_mrts) {
        //blit mrts
    }
}

Note: this is just my view of a better API, making the requested changes doesn't guarantee that this will be merged. The final decision is up to reduz and not me.

[SIsilicon]

Technically, color is already exposed, and depth already had its own texture, so they wouldn't need to be explicitly exposed. This would make more sense for the gbuffers however, as as you said before, their textures may not always be accessed. So how about this?

// Scene storage
func set_expose_gbuffer()  {
    Clear render target
    Update render target depending on expose_gbuffer.
    // This includes creating the textures.
}

...

// Scene render
if (expose_gbuffer) {
    //blit gbuffers
}

Again, thanks for the feedback. I'll make the necessary changes tomorrow. 🙂

[clayjohn]

Yea, that sounds good. I think lots of people are familiar with the term "g buffers".

The important thing is making sure that this change is zero cost for people who won't use it.

[nsrosenqvist]

Don't mean to add noise, just wondering if there is anything still blocking this? I need this for a feature and it looks like it just keeps getting pushed back

[Calinou]

Don't mean to add noise, just wondering if there is anything still blocking this? I need this for a feature and it looks like it just keeps getting pushed back

This feature needs to be reimplemented on the master branch, as we don't merge 3.x features that don't have an equivalent in the master branch (unless the feature makes no sense to be supported there). Otherwise, people upgrading from Godot 3.x to Godot 4.0 could end up relying on features that are no longer available in Godot 4.0 because they weren't reimplemented in time.

As said above, SIsilicon does not have access to a GPU that runs Vulkan. This means they can't look into implementing this in master. Given the current GPU shortage situation, this is not something they can resolve soon 🙁

Even if merging this PR for 3.x was a goal, it would need to be rebased – there are a lot of conflicts to resolve. (To clarify, this feature is desired as long as it doesn't incur a performance penalty for those who don't use it.)

[nsrosenqvist]

Don't mean to add noise, just wondering if there is anything still blocking this? I need this for a feature and it looks like it just keeps getting pushed back

This feature needs to be reimplemented on the master branch, as we don't merge 3.x features that don't have an equivalent in the master branch (unless the feature makes no sense to be supported there). Otherwise, people upgrading from Godot 3.x to Godot 4.0 could end up relying on features that are no longer available in Godot 4.0 because they weren't reimplemented in time. As said above, SIsilicon does not have access to a GPU that runs Vulkan. This means they can't look into implementing this in master.

Even if merging this PR for 3.x was a goal, it would need to be rebased – there are a lot of conflicts to resolve.

I understand, thanks for clarifying the situation!

[nsrosenqvist]

Even if merging this PR for 3.x was a goal, it would need to be rebased – there are a lot of conflicts to resolve. (To clarify, this feature is desired as long as it doesn't incur a performance penalty for those who don't use it.)

Actually my GPU doesn't support vulkan either, so it was support in the 3.x series I was hoping for 🙂 but it wouldn't be merged if a viable implementation existed for 4.0 too? Any way one can throw money at this and maybe someone else will take a look at vulkan? Does the Godot project use bounty source or equivalent?

[Calinou]

but it wouldn't be merged if a viable implementation existed for 4.0 too?

This PR would most likely be merged in 3.x if a master version of this PR was merged. I can't make any guarantees about it though.

Anyway one can throw money at this and maybe someone else will take a look at vulkan? Does the Godot project use bounty source or equivalent?

Godot does not use a bounty system, and there are no plans to add one.

[bfloch]

@SIsilicon This is really amazing work.

Small remarks:

• I noticed that render_target_v_flip is not respected.
• I don't seem to get any Normals if a shader has Transparency enabled in a spatial shader. Any idea why?
  EDIT: Just saw Bad normals only when material is set to transparent #23546 which might be related

Also: Would there be a way to access the buffers directly in a screen shader, without having to create ViewportTexture uniforms?

[Zireael07]

Just mentioning that this PR would make effects such as https://godotengine.org/qa/123831/how-to-do-outline-and-overlay-of-3d-meshes possible (or significantly easier and more performant)

[Gloomsfield]

hello!! any updates on this? (for godot 4 specifically)

[Calinou]

hello!! any updates on this? (for godot 4 specifically)

This PR needs a reimplementation from scratch for Godot 4, but SIsilicon no longer seems to be available to do this. Also, a reimplementation would need significant design changes to follow what's being done in #80214.

[Gloomsfield]

This PR needs a reimplementation from scratch for Godot 4, but SIsilicon no longer seems to be available to do this. Also, a reimplementation would need significant design changes to follow what's being done in #80214.

i see, thank you!!