Indirect zsh broken from bash starting from 8.7.42

[bartoldeman]

Describe the bug
given a bash login shell I get FPATH=/cvmfs/soft.computecanada.ca/custom/software/lmod/lmod/init/ksh_funcs.
This in turn breaks zsh when started from bash as it can't find functions any more from its default, particularly if .zshrc is empty you get an error straightaway.

It's triggered by this block in bash.in; KSH_VERSION and FPATH are empty when it is encountered and BASH_VERSION is set (5.1.8(1)-release)
https://github.com/TACC/Lmod/blob/main/init/bash.in#L41C1-L46C4

before 8.7.42 this block wasn't there; a different block didn't change FPATH at all, because it wasn't triggered:
https://github.com/TACC/Lmod/blob/8.7.41/init/bash.in#L26C1-L29C24

To Reproduce

in bash:

$ echo $FPATH
/cvmfs/soft.computecanada.ca/custom/software/lmod/lmod/init/ksh_funcs
$ rm ~/.zshrc
$ zsh
/cvmfs/soft.computecanada.ca/gentoo/2023/x86-64-v3/usr/share/zsh/5.9/scripts/newuser:6: zsh-newuser-install: function definition file not found

Expected behavior
zsh should start normally

Desktop (please complete the following information):

Modules based on Lua: Version 8.7.47 2024-07-22 10:04 -04:00
    by Robert McLay mclay@tacc.utexas.edu

Changes from Default Configuration
----------------------------------

Name                           Where Set  Default                                                                           Value
----                           ---------  -------                                                                           -----
LFS_VERSION                    D          1.6.3                                                                             1.8.0
LMOD_ADMIN_FILE                E          /cvmfs/soft.computecanada.ca/custom/software/lmod/etc/admin.list                  /cvmfs/soft.computecanada.ca/config/lmod/admin.list
LMOD_AVAIL_EXTENSIONS          E          yes                                                                               no
LMOD_AVAIL_STYLE               E          <system>                                                                          grouped:system
LMOD_CASE_INDEPENDENT_SORTING  C          no                                                                                yes
LMOD_DUPLICATE_PATHS           C          no                                                                                yes
LMOD_LANG                      E          en                                                                                C.UTF-8
LMOD_LD_LIBRARY_PATH           E          false                                                                             /.singularity.d/libs
LMOD_MODULERC                  E          /cvmfs/soft.computecanada.ca/custom/software/lmod/etc/rc                          :/cvmfs/soft.computecanada.ca/config/lmod//modulerc
LMOD_PACKAGE_PATH              D          nil                                                                               /cvmfs/soft.computecanada.ca/config/lmod/
LMOD_PAGER                     C          less                                                                              /usr/bin/less
LMOD_RC                        E                                                                                            /cvmfs/soft.computecanada.ca/config/lmod/lmodrc.lua:/cvmfs/soft.computecanada.ca/config/lmod/lmodrc/cache_avx2_intel.lua
LMOD_REDIRECT                  C          no                                                                                yes
LMOD_SHORT_TIME                E          2                                                                                 3600
LMOD_SITEPACKAGE_LOCATION      Other      /cvmfs/soft.computecanada.ca/custom/software/lmod/8.7.47/libexec/SitePackage.lua  /cvmfs/soft.computecanada.ca/config/lmod/SitePackage.lua
LMOD_SYSTEM_DEFAULT_MODULES    E          __unknown__                                                                       StdEnv mii
LMOD_TCLSH                     C          tclsh                                                                             /cvmfs/soft.computecanada.ca/custom/software/tcl/bin/tclsh
MODULEPATH_ROOT                E                                                                                            /cvmfs/soft.computecanada.ca/easybuild/modules
PATH_TO_LUA                    C          lua                                                                               /cvmfs/soft.computecanada.ca/custom/software/lua/bin/lua


Where Set -> D: default, E: environment, C: configuration
             lmod_cfg: lmod_config.lua SitePkg: SitePackage StdPkg: StandardPackage
             Other: Set somewhere outside of normal locations

[rtmclay]

I was able to reproduce this issue. I have created a branch called "IS718-FPATH" which has my fix. This fix works for me. Please test this branch out to see if it fixes it for you.

Thanks very much for the bug report!

[bartoldeman]

@rtmclay thanks for the fix! But due to our unusual setup it doesn't work. We can workaround that however.

The problem is that our zsh is provided by a module, and that module isn't loaded yet at shell init time. So zsh can't be found then and FPATH is still set to .../lmod/lmod/init/ksh_funcs only.

We can, of course, change the module that provides zsh to do the FPATH manipulation itself.

[rtmclay]

I don't run your setup so I can't reproduce exactly what you have but:

vato(4766)% bash -l
mclay@vato(164)$ unset __LMOD_SET_ZSH_FPATH 
mclay@vato(165)$ export FPATH=/opt/apps/lmod/lmod/init/ksh_funcs
mclay@vato(166)$ zsh -l
vato(2383)% 

As you can see I run zsh as my shell. Then I start bash. unset the one-time flag and export FPATH to point to the KSH_FUNCS. Then I start zsh and it works. Make sure that you have this in your init/zsh file:

__build_FPATH_for_zsh_ksh()
{
  zsh_fpath=$(unset FPATH; zsh -f -c 'echo $FPATH')
  export FPATH=$(/opt/apps/lmod/lmod/libexec/addto --append FPATH ${zsh_fpath} /opt/apps/lmod/lmod/init/ksh_funcs)
  unset zsh_fpath
}  

The unsetting of FPATH fixes the problem for me.

[bartoldeman]

Our zsh -l configuration is indeed broken at the moment and does not reinitialize Lmod. I'll fix that.

However the user that complained saw the behaviour with plain zsh (without -l), see above. What do you see with plain zsh in the command sequence here:

vato(4766)% bash -l
mclay@vato(164)$ unset __LMOD_SET_ZSH_FPATH 
mclay@vato(165)$ export FPATH=/opt/apps/lmod/lmod/init/ksh_funcs
mclay@vato(166)$ zsh
vato(2383)% echo $FPATH

? (Yes I do realize zsh -l is the proper thing to execute if you want Lmod to function properly, this is mostly out of curiosity)

[rtmclay]

For a different reason. I have remove the guard variable (__LMOD_SET_ZSH_FPATH) as they don't work in another setting. With the newest update to the IS718-FPATH branch I get for the account rtmbash which runs bash as its shell:

rtmbash@vato:~$ export FPATH=/opt/apps/lmod/lmod/init/ksh_funcs
rtmbash@vato:~$ zsh
vato% echo $ZSH_VERSION
5.8.1
vato%

[bartoldeman]

I'm interested in what happens to FPATH, not in ZSH_VERSION :)
Mostly to see if functions such as zsh-newuser-install can be found or not.

I'll test IS718-FPATH in any case to see what happens.

[rtmclay]

I have merge the IS718-FPATH branch on to the main branch. Thanks for submitting this issue!

[bartoldeman]

Thank you for the fixes. My solution (that only applies to our local situation) is to mimic the bash logic in a module file via

local fpath = capture("unset FPATH; " .. pathJoin(root, "usr/bin/zsh -f -c 'echo $FPATH'")):gsub("\n$","")
prepend_path("FPATH", fpath)

(https://github.com/ComputeCanada/software-stack-custom/pull/78/files)

[JCGoran]

The change in question:
https://github.com/TACC/Lmod/blob/main/init/bash.in#L41C1-L46C4
also broke our CI since we run it with -e, in which case type zsh returns a non-zero exit code, suddenly reporting an error without any details whatsoever. What should be used to avoid this is:

if type zsh > /dev/null 2>&1; then

instead of:

type zsh > /dev/null 2>&1
if [ "$?" = 0 ]; then

[rtmclay]

I have updated the IS718-FPATH branch to change from using the exit status to changes similar to the one you have suggested. This was done in two places. Please test the latest update of this branch to see if works for you.

[jose-d]

after patch of /usr/share/lmod/lmod/init/bash file with content of current (main) https://github.com/TACC/Lmod/blob/main/init/bash.in we do not see the silent error of our set -e bash scripts performing module load XXXX anymore, so, looks good.

[rtmclay]

I am closing this issue. If there are any new issues that come up this issue can be reopened.