Houdini Solaris - Writing out multiple dependent publishes in one go

[BigRoy]

Issue

Houdini 18 Solaris allows to author USD files in its own procedural LOP (lighting operators) "stage" that results in a .usd file and can write out intermittent USD files to disk too.

The tricky thing here is that the authoring allows to create a procedural graph that can have the intermediate "Layers" it produces written out to disk as .usd and then in the end generate the final .usd file that references them too.

For example, in one go it can produce:

- asset.usd
    - geom.usd
    - look.usd

This is also described here:

In the USD render node, you specify a file path (in the Output file parameter) for the "top-level" USD file, containing data from the root layer.

In addition to this file, the node writes any layers that have their file path metadata set to their own USD files.

When working in the Stage in Houdini it makes a lot of sense, however in Avalon-context this is (as far as I know) new behavior where dependencies are also published along. Because to write a correct USD output file these paths need to be correctly reconfigured to their published counterparts

- /asset//v005/asset.usd
    - /geom/v002/geom.usd
    - /look/v001/look.usd

The odd bit still is that you can't explicitly save only the intermediate layers, as far as I have seen. For example setting the geom.usd output path for the layer with configurelayer it doesn't have its own "save to disk" button. It'll only get written too when a save is done down the line.

Ideas

Custom Output Processor

Likely the solution is in implementing a Custom Output Processor so that any path can be processed into a valid published path. However, that would mean that the path itself would be able to define a publish path, e.g. $HIP/usd/asset.usd and $HIP/usd/geom.usd would somehow need to translate into a valid subset for the current asset.

The trickier thing there is the versioning. When the layer save path metadata is set it's still about to save that particular file, meaning that it would actually need to "publish" it to become a valid published path and at the time of generating the files I'm not sure there's much control on going through a full Publish process.

Embed these dependent layers as a resource for the publish (worst-case?)

Worst-case scenario we'd have to resort to just embedding those USD files, e.g.

- /asset/v001/asset.usd
    - /asset/v001/resources/geom.usd
    - /asset/v002/resources/shade.usd

However, forcing them to be embedded into this particular "usd" and only exposing the asset.usd basically makes it similar to just flattening the full generated content into a single file - which is exactly the opposite behavior of what you'd like with USD. In the end you'll want to have access to the geom.usd and shade.usd explicitly too - at least, that's definitely how I had seen it so far.

---

Any ideas on potential workflows for this? I'd love to start seeing production scenarios built with the USD tools in Solaris to get a clear idea of what's best to implement for Avalon.

[mkolar]

When working in the Stage in Houdini it makes a lot of sense, however in Avalon-context this is (as far as I know) new behavior where dependencies are also published along. Because to write a correct USD output file these paths need to be correctly reconfigured to their published counterparts

Well it's kinda the same problem as choosing whether to publishes textures first, or with the look. The former being cleaner technically, but it's a pain in production. USD by definition holds a lot of information that we normally keep in the database, with this will introduce quite a lot of double information and goes way beyond just dependencies in the .usd files.

Especially these asset.usd files act very much like aggregates of multiple subsets at the end. You can easily have (simplifed)

• asset.usd
  • modelVariants
    • modelDefault.usd
    • modelBroken.usd
  • shadingVariants
    • shadeBlue.usd
    • shaderRed.usd

now each of those usd files corresponds to a subset in avalon let's say, but the asset.usd knows about them without ever looking in the db and should be able to resolve their paths without DB as well. In fact versions could be held as variants in the individual subset usds. However I think that within the bound of avalon and if we want to keep full publishing validation we'll still need to author and publish these from bottom up. Otherwise as you write, we might not have enough control over the validation.

In the houdini example itself. I'd probably stop the publishing if I find that there are sublayers, that haven't gone through publishing themselves. Or if you want to allow that again, publish them as subsets from the bottom up.

But the versioning of USD is a separate beast likely more related to the resolver issue

[BigRoy]

Thanks, that is a good breakdown helpful for explaining what we're facing.

For now I just want to add a link to this USD-Interest topic about "Generating and updating asset.usd / shot.usd (pipeline)" that I recently started and is getting some good answers. It would be nice, if anyone here has some questions regarding what is being discussed there to join the conversation there too.