Cycles : VDB volumes refactor

[boberfly]

Generally describe what this PR will do, and why it is needed

• Refactored the VDB loading/rendering code for Cycles
• It now defers voxel grid creation into the scene-lock
• Shader modification now requires an expensive object rebuild, but now it reliably will update in a live render (this has been addressed)
• The unit-test I can't figure out how to load in a vdb, need help on that one

Related issues

• Fixes for cycles vdb support #4815

Dependencies

Not hard-dependencies, but I did apply this based on these two, so I am not sure how it'll go without them.

• GafferCycles : Refactor the problematic reset code #5101
• Cycles : Update to 3.5.0 #5239
• https://projects.blender.org/blender/cycles/pulls/6 https://github.com/boberfly/dependencies/blob/cycles/Cycles/patches/volume_updates.patch

Breaking changes

• GeometryAlgo now doesn't have a scene argument for the convert functions, as these were only for volumes and now scene is only needed in convertVoxelGrids

Checklist

• I have read the contribution guidelines.
• I have updated the documentation, if applicable.
• I have tested my change(s) in the test suite, and added new test cases where necessary.
• My code follows the Gaffer project's prevailing coding style and conventions.

TODO

• I think we can do half-precision on vdb's, I'll add it to this when I figure that out. - Should be done now
• Unit-test needs fixing.

[boberfly]

Just a note for myself - after a few shader tweaks and/or attributes changes, there is a segfault here, or more specifically in Cycles itself:
https://github.com/GafferHQ/gaffer/blob/main/src/GafferCycles/IECoreCyclesPreview/Renderer.cpp#L826

When any old shaders assigned get de-referenced. Something to figure out and I'm not too sure yet why that is...

[boberfly]

Some more investigations:
https://projects.blender.org/blender/cycles/pulls/6

With this MR pull, I can remove including the shader hash + expensive rebuild of a shader per-shader update. My hunch why we run into some of these shader issues vs Blender is Blender tends to keep the same ccl::Shader pointer but rather updates the contents by changing the graph inside.

@johnhaddon I think I saw in GafferArnold that you have a mechanism to update already created shaders, is that something that we could do there also as an alternative approach? I wasn't sure how to determine when a shader is being updated vs created as you'd normally just get a shader and if the hash doesn't match, it requires a new one created.

[johnhaddon]

I think I saw in GafferArnold that you have a mechanism to update already created shaders, is that something that we could do there also as an alternative approach?

Sorry for the ludicrous delay in replying to this. Yes, we do have something like that in IECoreArnold - ShaderNetworkAlgo::update(). We use it to update the network for light shaders, while keeping the terminal shader intact (because that is the light itself, and needs to remain stable for light linking). This is possible for lights because they always have their own unique shader network, rather than an "instanced" one - they don't go through the ShaderCache at all.

[boberfly]

Just adding here that the custom cycles patch I use to update shaders that's not a broken link:
https://projects.blender.org/boberfly/cycles/src/branch/volume_shader

Just to add also, this looks pretty exciting and a much-needed revamp of the volume integrator in cycles:
https://projects.blender.org/blender/blender/pulls/128389