piggy-back on apptainer SIF image cache

[tomeichlersmith]

This resolves #141 for apptainer but not the other singularity flavors.

apptainer has an in-container environment variable APPTAINER_CONTAINER that holds the full path to the SIF image that was used to launch the container.¹ This environment variable is not documented, but is exactly what we want since it allows us to use apptainer to cache a local copy of the SIF image built from an OCI image and then we just launch it to retrieve the path to this cached image.

The other singularity flavors are supported as before, manually building the SIF image ourselves, meaning different denvs must have their own copy of the image even if the two are identical and owned by the same user.

Leaving this as draft in order to

• see if a similar environment variable can be found for the other singularity flavors
• is there a better way to do this? I think this is the "correct" way to do this as far as I can tell from looking through the documentation and code. apptainer/singularity really don't want folks to rely on the cache especially when running multiple containers from the same image which makes sense and motivates me to update the documentation accordingly.
• add test that makes sure users can invalidate the cache and the image is rebuilt seamlessly
• update documentation: as pointed out in apptainer's cache documentation, if the filesystem the cache is residing in does not support atomic renaming, then running the image from that location may see performance issues (reading between the lines here). I'd like to point this out in the denv docs. denv supports running from a manually built SIF image as well, so if users are preparing a cluster-wide processing campaign, they should manually build and fix a SIF that they can then provide denv.

Footnotes

1. https://github.com/apptainer/apptainer/blob/e6fc6f0d5862a372241a8bec50b980036e37975e/internal/pkg/runtime/launch/launcher_linux.go#L526 ↩

[tomeichlersmith]

It seems like original recipe singularity has a similarly-named environment variable

https://github.com/apptainer/singularity/blob/9dceb4240c12b4cff1da94630d422a3422b39fcf/cmd/internal/cli/actions_linux.go#L263-L269

and SyLabs' extra crispy version has one as well

https://github.com/sylabs/singularity/blob/9646c92a88346a64ed48744f701699f32c574a03/internal/pkg/runtime/launcher/native/launcher_linux.go#L450

I originally limited myself to environment variables that were documented but since APPTAINER_CONTAINER isn't documented, I am now pushing a commit to use SINGULARITY_CONTAINER as well. Let's see if it works.

Edit: seems like it works! Still want to implement some tests for this.