Allow setting .lmod.d location or use XDG convention by default

[klmr]

lmod currently creates a hidden directory (.lmod.d) in the user’s home directory that is used for various purposes, notably for the list of default modules and the spider cache. Am I correct in thinking that there is currently no way for users to tell lmod to create this folder at a different location (after installation, that is)?

Many users prefer for applications not to write into their home directories (see e.g. tools such as xdg-ninja, which help reclaiming the home directory). In fact, the XDG base directory specification formalises a convention for different default locations that applications should use on Linux/Unix systems (macOS and Windows have different conventions). Ideally lmod would follow OS conventions (so, on Linux, for example, use ${XDG_CONFIG_HOME-$HOME/.config}/lmod/default and ${XDG_CACHE_HOME-$HOME/.cache}/lmod for the default modules and the user spider cache, respectively). However, as a fallback the location of the .lmod.d folder should be explicitly configurable if users prefer to keep their home directory clean.

[rtmclay]

It is possible to have a system configuration to move every users to ~/.lmod.d directory to a different location. It is also possible to switch lmod to use ~/.config/lmod/* and ~/.cache/lmod/* directories.

What is not possible to have individual users control where the config and cache files are located. Many sites, including ours, load modules on behalf of users. The current default file: ~/.lmod.d/default, if it exists, will be read by Lmod from an /etc/profile.d/*.sh or similar file BEFORE a users startup file (i.e. ~/.bashrc, ~/.zshrc ) is sourced. So there is no way that a user could set $XDG_CONFIG_HOME before this happens.

[rtmclay]

I'm not against this (or for it either). It is just that I want you understand the issues here.

[klmr]

The current default file: ~/.lmod.d/default, if it exists, will be read by Lmod from an /etc/profile.d/*.sh or similar file BEFORE a users startup file (i.e. ~/.bashrc, ~/.zshrc ) is sourced.

Ah, I wasn't aware of that part. Indeed, that's unfortunate.

I'd still say that lmod could/should use a better default path (i.e. what's used if there's no system configuration saying otherwise): the default installation shouldn't touch the user's home directory to the extent avoidable, and while there are certainly users who use a different path, the vast majority of users will use ~/.config for XDG_CONFIG_HOME, and ~/.cache for XDG_CACHE_HOME (and on macOS the question doesn't pose itself, since the preferred locations are fixed).

[rtmclay]

Here is my thinking on this. This issue will be discussed at the September Lmod Zoom meeting (see https://github.com/TACC/Lmod/wiki for details). So the cache file change will show up soon. But changing the way collections work is a big change. This has to be seamless. Lmod is used widely and sites run versions from Lmod 6* to Lmod 8*. Sometimes sites upgrade and then downgrade Lmod. Sites use shared home file systems with different version of Lmod on each cluster. This has to be supported. The way collections are handle may not happen and if it does it won't happen until September.

There are two issues here: the user cache files and collections

• User Cache files:
  • switch ~/.lmod.d/.cache/* to ~/.cache/lmod* immediately
    cache files are only valid for 24 hours anyway.
• Collections:
  • Write collections to both ~/.lmod.d/ AND ~/.config/lmod for now
  • At a later date (maybe when Lmod 9.0 is released) only write to ~/.config/lmod
  • Chose ~/.config/lmod over ~/.lmod.d/ when reading collection
  • Always read collections from ~/.lmod.d if not available from ~/.config/lmod.

Any comments?

[klmr]

Yes, this looks fairly workable. In particular, the last item should ensure backwards compatibility, and is done successfully by other software that is switching over to using the XDG convention — however, the fact that multiple versions of lmod could be running in parallel on different machines on a cluster makes this more complicated.

[rtmclay]

The testing branch of Lmod is available for testing this feature. Lmod will now write to both ~/.lmod.d and ~/.config/lmod. It will continue to read from both directories. Sites can change this to write to only the ~/.config/lmod directory by configuring "--with-useDotConfigOnly=yes'. This is safe as long as a site doesn't go backwards in Lmod versions or has a shared home files system with multiple version of Lmod.

[klmr]

Awesome, thanks.