lift invalidating of module caches into helper method that can be used by easyblocks #2571
Conversation
boegel
changed the title
…idate_module_caches_for lift relevant code block into 'invalidate_module_caches' helper method, which can be used easily by easyblocks that overwrite make_module_step
boegel
changed the title
|
After trying to come up with a test to verify the fix done in 3354ade, I noticed there wasn't actually a problem at all in framework... The actual bug was that the So, I've rolled back the call to I've added the test I was working on as well, since it doesn't hurt; as far as I can tell there was no test yet that accurately verifies that the module caches are reset in between builds... |
boegel
force-pushed
the
reset_module_caches
branch
5 times, most recently
from
71c8505 to
3faea7b
Compare
easybuild/tools/modules.py
Outdated
| @@ -1106,7 +1106,7 @@ class Lmod(ModulesTool): | |||
| """Interface to Lmod.""" | |||
| COMMAND = 'lmod' | |||
| COMMAND_ENVIRONMENT = 'LMOD_CMD' | |||
| REQ_VERSION = '5.8' | |||
| REQ_VERSION = '6.6.3' | |||
There was a problem hiding this comment.
This is a substantial bump in the required Lmod version, but since 6.6.3 was released on Nov 13th 2016, I don't expect this to be a big problem.
A sufficiently recent Lmod version is needed in order for .modulerc to be picked up by Lmod, which is needed for the ModuleAlias easyblock (see easybuilders/easybuild-easyblocks#1503).
In theory, Lmod 6.0 should be sufficient for that, but several (unrelated) tests failed when using older Lmod 6.x versions.
Lmod 6.6.3 is the last 6.x release, so this way EasyBuild is still compatible with both Lmod 6.x & 7.x.
| modulerc_txt = modgen.modulerc(module_version=module_version_spec) | ||
| one_moddir = os.path.join(self.test_installpath, 'modules', 'all', 'one') | ||
| write_file(os.path.join(one_moddir, '.modulerc'), modulerc_txt) | ||
|
|
There was a problem hiding this comment.
What happens if you repeat 1446 at this point (1.0.2 doesn't exist yet but the rc file does)?
There was a problem hiding this comment.
Well, it won't make a difference because of the modules cache (the result for avail one/1.0 & show one/1.0 is cached).
And even if we clear the cache, it will still fail, since the underlying module doesn't exist yet (I'll add that to the test).
While testing the
ModulesAliaseasyblock (cfr. easybuilders/easybuild-easyblocks#1503), I noticed that the caching we do formodule showcommands is a bit too aggressive...Module aliases show up in the output of
module --terse availin such a way that a fallback is needed tomodule showto recognize that a module alias is already in place.The result of a
module showcommand is cached the first time if it is run, but if multiple builds are being performed in the same session, this is a problem since the later build will see the outdated/incorrect result in a module alias was install in an early build.For example, if
BazelhasJava/1.8specified as a dependency to be resolved by a module alias that still needs to be installed, theJava/1.8alias that is in place by the timeBazelis installed will not be observed because the originalmodule show Java/1.8that came back empty is cached.Clearing the
module availandmodule showcaches before every new build is started deals with this issue.