Object3D: Introduce onBeforeShadow().

[Samsy]

Object3D.onBeforeRender not triggered when depthmaterial for shadows are rendered.

renderBufferDirect method does not trigger the onBeforeRender function

Three.js version

• Dev
• r96
• ...

Browser

• All of them
• Chrome
• Firefox
• Internet Explorer

OS

• All of them
• Windows
• macOS
• Linux
• Android
• iOS

Hardware Requirements (graphics card, VR Device, ...)

[Mugen87]

WebGLShadowMap has a function renderObject() which could call onBeforeRender() for the current object. But I'm not sure this should happening since it's an internal rendering for creating shadow maps. What is your use case for this?

[Samsy]

Let say you got 20 meshes sharing the same material but needs differents values of uniforms.

Using onBeforeRender function is useful to refresh thoses uniforms in it, like changing color value, noise or what ever that is associated to each mesh just before they are rendered ( avoiding the creation of 20 materials for 20 meshs, as you just need to refresh the uniforms )

That works, but not when shadow are rendered. If you got some uniforms that influences mesh position for example, using a same customdepthmaterial shared across all thoses meshes, uniforms won't get refresh per-mesh because onBeforeRender is never called when rendering shadow, and that would be useful !!

[Mugen87]

Yeah, that makes sense. Let's see what the others think about this issue.

[WestLangley]

Let say you have 20 meshes sharing the same material but need different values of uniforms.

That is what cloning the material is for. They still share a single shader.

using a same customdepthmaterial shared across all thoses meshes, uniforms won't get refresh per-mesh because onBeforeRender is never called when rendering shadow

Right, but each object can clone the custom depth material, correct?

--

The other solution, presumably, would be to add support for object.onBeforeShadow() and object.onAfterShadow(), but I have not given it enough thought to know if there is a compelling reason to add that feature.

[Samsy]

Main problem with the cloning material thing is, it also clones textures and you end up having as much textures as clone.

Textures in material * number of clones.

For few reasons It would be nice to have more controls over those things. I mean the current system is working, the only thing missing in to get that onBeforeRender function working when shadows are rendered.

[WestLangley]

Perhaps you can provide a simple live example to make it clear what you are doing as a workaround, and what you would prefer to be able to do. Is this about CustomDepthMaterial only?

[Samsy]

I can do an example as soon as I got time for sure.
Yes this is only about when a mesh is rendered with a customDepthMaterial / depthMaterial using internal shadows logic, to be able to get the callback onBeforeRender to act the same and be triggered as if it was a classic mesh

[NataliaDSmirnova]

@Mugen87 ,

may I describe the case where some kind of onBeforeShadow function is strongly required?
I have an object consisted of several simple z-sprites (e.g. spheres). I utilize inversed modelViewMatrix for calculation of proper depth-value in a fragment shader. The inversed modelViewMatrix is set via a uniform and updated in onBeforeRender function.
However, the onBeforeRender function isn't called inside shadowmap construction step. Thus I have got the wrong shadow in shadowmap.
It'll be great to have some onBeforeShadow function or to get an adviсe to overcome the problem.

[WestLangley]

@NataliaDSmirnova

Can you update your uniforms prior to calling renderer.render(), instead of in sprite.onBeforeRender()?

---

@Samsy wrote:

I can do an example as soon as I got time for sure.

It has been awhile... Now would be a good time, if you are still interested.

[NataliaDSmirnova]

@WestLangley
No, uniform update prior to renderer.render() will not help. Shadowmap is built inside renderer.render(). WebGLShadowmap.render constructs new "camera" for every light source, so there are several viewModelMatrix for shadow step and they differ from the one during common render step. My situation in details:

Scene: several objects consisted of sprites. Each object has its own worldMatrix, consequently its own viewModelMatrix. Now I calculate invViewModelMatrix in onBeforeRender for proper calculation of fragment depth.

Algorithm:

renderer.render(scene) {

   shadowMap.render {
      for every light source {
         build viewModelMatrix from light source
         for every Object {                           
                                                  <<<<< require onBeforeShadow to set invViewModelMatrix
            render Object to shadow map    
         }
      }
   }

   for every Object {
      onBeforeRender  //sets invViewModelMatrix from current camera                          
      render Object to frame buffer (using shadowmaps)    
   }
}  

The onBeforeShadow function widens functionality, and let the developer use threejs shadowmap technique for rendering complex objects.
The feature helps me a lot and I'll be happy for it.

[Mugen87]

I'm okay with adding onBeforeShadow() since your use case sounds valid to me. Let's see what @mrdoob thinks about this.

[WestLangley]

@NataliaDSmirnova I have no opinion on onBeforeShadow(), but I do not understand your use case.

Scene: several objects consisted of sprites.
... render Object to shadow map

Do your sprites cast shadows?

@Mugen87 Maybe you can explain since you understand it.

[NataliaDSmirnova]

@WestLangley

Do your sprites cast shadows?

Yes, my sprites cast shadows and receive shadows.

[WestLangley]

@NataliaDSmirnova Sprites do not cast shadows. Have you hacked the library itself? You want us to add this feature to the library to support your hack?

[Mugen87]

Maybe she is using meshes with PlaneBufferGeoemtry or a thin BoxBufferGeometry and orienting the resulting sprites manually.

[NataliaDSmirnova]

@WestLangley , @Mugen87
My object is a set of custom sprites, that visualize small spheres. I create a Mesh from my own RawShaderMaterial with my own fragment and vertex shader for orienting the sprites and InstancedBufferGeometry to draw a lot of sprites in different places. Maybe this is important information.

I am sorry for confusing you. I haven't hacked your library, only used its basiс functionality, to built objects, that I need. I haven't used THREE.Sprite.

[WestLangley]

@NataliaDSmirnova I assume you doing something like this three.js example which demonstrates instancing with shadows, and shows how to set a custom depth material.

Perhaps it would be a good idea if you provide a simple live example so it is clear what you are doing.

Sorry, I cannot approve something I do not understand. It's not for a lack of trying...

[NataliaDSmirnova]

@WestLangley
Thank you for advice. I hope a more simple example shows the problem: https://jsfiddle.net/u5trma69/3/. It's made from webgl_shadowmap_viewer example. All that I've added is cube.onBeforeRender, that makes cube's matrix identity. You can see, that it's rendered at zero, but onBeforeRender isn't triggered on shadow pass. Consequently, the shadow is drawn at a wrong place.

As for my case, I need to turn sprites perpendicular to camera direction to get correct spheres image and turn them perpendicular to the light direction to form correct shadowmap for the spheres. The first task is done using onBeforeRender, so I ask for additional triggering onBeforeRender (or better adding special onBeforeShadow) in shadowpass just before rendering the object into shadowmap.

[WestLangley]

@NataliaDSmirnova wrote:

I create a Mesh from my own RawShaderMaterial with my own fragment and vertex shader for orienting the sprites and InstancedBufferGeometry to draw a lot of sprites in different places. Maybe this is important information.

I wish you would show your example with instancing. :/

I expect you should be doing billboarding in your vertex shader. That way each instance will face the camera or shadow camera automatically. onBeforeRender()is likely unneeded in your case.

[NataliaDSmirnova]

@WestLangley
You are right, we're doing billboarding in the vertex shader but in addition to that, we're performing more calculations in the pixel shader. These calculations also require per-object uniforms, and now onBeforeRender provides the way to do it (instead of the former dynamic uniforms). The uniforms depend on the current camera and the object, and probably could be calculated in the vertex shader, but it is much cheaper to calculate them once for an object than for each vertex of the object.

Such objects cast wrong shadows because the uniforms are not set on per-object/per-camera basis (onBeforeRender is not called during the shadow pass). We see this as an accidental omission in API design and thus are reporting it as a problem.

Of course, I could prepare a full working example for your convenience but it would take much more time (which I probably lack) than showing the design flaw through a simpler example. Even with such an example, you could possibly suggest workarounds and kludges for this specific example. That's not our point. At last, we could implement our own shadowing technique or even a 3D engine ourselves 😄
I'm just trying to help you make three.js more consistent and convenient.

Please let me know if it makes sense.

[WestLangley]

I give up. I did my best...

[NataliaDSmirnova]

Hello! It's been a while since we discussed the question.
I hope we can continue the conversation. We've prepared an example. But first, I want to re-describe the situation.
We use your library in an application, visualizing different molecules.

The geometry may have thousands of cylinders and spheres and we decided to render every one of them with an "analytical sprite" in addition to instancing to speedup. Analytical sprite also may be called a "procedural sprite" and is a little bit more complex than common sprite. It's a quad, perpendicular to camera view but it doesn't use a texture. Quad is used as a placeholder, positioned with instancing, and the cylinder/sphere image is produced on the fly in a pixel shader.
If you ask why we can't use just a simple quad with texture, it is because we need correct intersections.

We create our own Material for procedural sprites and have got a perfect result: image quality + performance. We use onBeforeRender function to update uniforms needed for math inside fragment shader.
And everything is good except for the fact that our sprites can't cast shadows now, because of one small inconsistency in threejs. There is Object.onBeforeRender function, but it isn't called before rendering an object inside a shadowmap step. So we can't render sprites into shadowmap correctly, because some uniforms are not updated.

We suggest to add Object3D.onBeforeShadow and call it inside WebGLShadowMap.renderObject.

In more detail, there is an example https://jsfiddle.net/NataliaDSmirnova/pqeyft5o/ (If you need I'll create a PR with live example file in threejs/examples directory and changes in library). There are three modes, you can change the used one in line 3.
GEO mode just renders cylinders with common cylinder geometry and shader, shadows are ok.
SPRITE mode renders quads with texture, shadows are ok, but the resulting image has problems!
Z_SPRITE mode renders procedural sprites, image is ok, shadows are absent.

What I would ask you is to add Object3D.onBeforeShadow(...){} function and use it inside WebGLShadowMap.renderObject before _renderer.renderBufferDirect call. At the moment, if you add this code, the shadows for Z_SPRITE mode appear.

If you need, I'll create a PR for the feature.

[WestLangley]

If you need, I'll create a PR for the feature.

Go ahead and file a PR so @mrdoob can decide.

I do not think you need to provide an example, but it would be nice if you would show your source code for .onBeforeShadow() so @mrdoob will be able understand your use case.

[WestLangley]

@NataliaDSmirnova Hmm... your technique looks very similar to that of #11349. :-)

[NataliaDSmirnova]

@NataliaDSmirnova Hmm... your technique looks very similar to that of #11349. :-)

Your lines are very cool 👍

Yes, our idea is close, but I think we've realized a closer effect for wide lines. See, how it looks: https://miew.app/?l=pdb:1CRN&p=macro&r=0&m=LN 😄

[WestLangley]

@NataliaDSmirnova Would you be willing to show that example with shadows added?

[NataliaDSmirnova]

@NataliaDSmirnova Would you be willing to show that example with shadows added?

I've added a link to an online example with shadows into PR #17932. It uses modified three.js, I've put temporarily on our site. By default, the mode with procedural sprites is set.
Here it is https://jsfiddle.net/NataliaDSmirnova/e4awmk62/

[gkjohnson]

Unfortunately #25933 from @RenaudRohlinger was closed but this function is something that's needed to support proper frustum culling support with shadows for BatchedMesh. Right now frustum culling will occur every frame based on the render camera on onBeforeRender and then that set of non-culled geometries will be used in the shadow rendering. See https://twitter.com/ycwhk/status/1728213475527360921.

[RenaudRohlinger]

Oops didn't see I made this PR with my dev branch, not sure if I still got it somewhere I will check.