Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop load storm safe guard for Environment Modules v4.2.4+ #4373

Merged
merged 1 commit into from
Aug 28, 2024

Conversation

xdelaruelle
Copy link
Contributor

Environment Modules v4+ safely handles automatic module load by not reloading already loaded module. Same goes when unloading module: already unloaded dependency will not be evaluated again.

As a result, no safe guard test is required and it should even be avoided to get the module dependency correctly tracked.

If module dependency declaration is skipped, no relation binds the loaded modules. Thus unloading the dependency will not lead to the auto unload of the dependent module. Without "is-loaded" safe guard, dependency is always declared and loaded environment is kept consistent by auto_handling mechanism [1]

A new ModuleTool property named "supports_safe_auto_load" is introduced. When enabled, module load safe guard code is not generated like if depends_on is enabled.

"supports_safe_auto_load" is enabled for EnviromentModules 4.2.4+. Prior v4 versions were not handling safely module load of module parent name if already loaded module were not corresponding to the default version.

[1] https://modules.readthedocs.io/en/latest/module.html#envvar-MODULES_AUTO_HANDLING

@boegel boegel added the change label Nov 8, 2023
@boegel boegel added this to the 4.x milestone Nov 8, 2023
@boegel
Copy link
Member

boegel commented Nov 8, 2023

@xdelaruelle Thanks a lot for the PR!

Correct me if I'm wrong, but this will effectively also change the behavior of module files that were created by EasyBuild with Modules v4.2.4+, since unloading a module file will now also trigger an unload of the modules that got loaded as dependency.

That may surprise people, so I wonder if we should make this opt-in, rather than auto-enabling this based on the version of Modules...

@boegel boegel changed the title No load storm safe guard for Environment Modules drop load storm safe guard for Environment Modules v4.2.4+ Nov 8, 2023
@xdelaruelle
Copy link
Contributor Author

@xdelaruelle Thanks a lot for the PR!

Correct me if I'm wrong, but this will effectively also change the behavior of module files that were created by EasyBuild with Modules v4.2.4+, since unloading a module file will now also trigger an unload of the modules that got loaded as dependency.

That may surprise people, so I wonder if we should make this opt-in, rather than auto-enabling this based on the version of Modules...

@boegel It will auto-unload dependency modules only if they are not needed by other loaded modules and if these dependency modules have been auto-loaded. So there is a change, but it should not disturb users as they have not explicitly asked for the dependency.

I would prefer to enable this by default rather making it an option to enable as with current load storm safe guard code users may end up with inconsistent environment:

With load storm safe guard (you can unload M4 and keep libtool, a dependent module, loaded):

$ ml M4 libtool
$ ml -M4
$ ml
Currently Loaded Modulefiles:
 1) libtool/2.4.7

Without load storm safe guard (dependent module is unloaded):

$ ml M4 libtool
$ ml -M4
Unloading M4/1.4.18
  Unloading dependent: libtool/2.4.7

@boegel boegel added the EasyBuild-5.0 EasyBuild 5.0 label Dec 26, 2023
@boegel boegel modified the milestones: 4.x, 5.0 Dec 26, 2023
@boegel
Copy link
Member

boegel commented Dec 26, 2023

@xdelaruelle There's a window of opportunity for "breaking" changes since we're working towards EasyBuild v5.0, so I propose we apply this change then (either in the 5.0.x branch where we're currently preparing EasyBuild v5.0, or in develop once it's clear that there won't be any further EasyBuild 4.x releases after EasyBuild v4.9.0).

@xdelaruelle xdelaruelle changed the base branch from develop to 5.0.x January 3, 2024 05:20
@xdelaruelle
Copy link
Contributor Author

@boegel I have just changed this pull request's target to 5.0.x and rebased it to this branch.

@xdelaruelle xdelaruelle force-pushed the safe_auto_load branch 3 times, most recently from b3f2498 to aeb9d46 Compare February 23, 2024 17:52
@boegel boegel added the EasyBuild-5.0-blocker Blocker for EasyBuild 5.0 label Jun 5, 2024
@xdelaruelle
Copy link
Contributor Author

@boegel, @branfosj I have rebased this pull request on the current 5.0.x branch. Please let me know if something should be changed for this PR to get merged.

@easybuilders easybuilders deleted a comment from boegelbot Aug 27, 2024
@boegel
Copy link
Member

boegel commented Aug 27, 2024

@xdelaruelle Another one that needs fixing of merge conflict that was introduced recently... (maybe we should sync up via Slack on these related PRs, to get them merged more smoothly...)

Environment Modules v4+ safely handles automatic module load by not
reloading already loaded module. Same goes when unloading module:
already unloaded dependency will not be evaluated again.

As a result, no safe guard test is required and it should even be
avoided to get the module dependency correctly tracked.

If module dependency declaration is skipped, no relation binds the
loaded modules. Thus unloading the dependency will not lead to the auto
unload of the dependent module. Without "is-loaded" safe guard,
dependency is always declared and loaded environment is kept consistent
by auto_handling mechanism [1]

A new ModuleTool property named "supports_safe_auto_load" is introduced.
When enabled, module load safe guard code is not generated like if
depends_on is enabled.

"supports_safe_auto_load" is enabled for EnviromentModules 4.2.4+.
Prior v4 versions were not handling safely module load of module parent
name if already loaded module were not corresponding to the default
version.

[1] https://modules.readthedocs.io/en/latest/module.html#envvar-MODULES_AUTO_HANDLING
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit f8c9913 into easybuilders:5.0.x Aug 28, 2024
35 checks passed
@xdelaruelle xdelaruelle deleted the safe_auto_load branch August 31, 2024 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
change EasyBuild-5.0-blocker Blocker for EasyBuild 5.0 EasyBuild-5.0 EasyBuild 5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants