--include-easyblocks[-from-pr] fails for multiple easyblocks related by inheritance

[Flamefire]

I recently updated PythonPackage and TensorFlow easyblocks in one PR and tried to submit test reports but it failed:

File "/home/h3/s3248973/.local/EasyBuildDev/easybuild-framework/easybuild/main.py", line 118, in build_and_install_software
(ec_res['success'], app_log, err) = build_and_install_one(ec, init_env)
File "/home/s3248973/.local/EasyBuildDev/easybuild-framework/easybuild/framework/easyblock.py", line 3697, in build_and_install_one
result = app.run_all_steps(run_test_cases=run_test_cases)
File "/home/s3248973/.local/EasyBuildDev/easybuild-framework/easybuild/framework/easyblock.py", line 3585, in run_all_steps
self.run_step(step_name, step_methods)
File "/home/s3248973/.local/EasyBuildDev/easybuild-framework/easybuild/framework/easyblock.py", line 3438, in run_step
step_method(self)()
File "/home/s3248973/.local/EasyBuildDev/easybuild-easyblocks/easybuild/easyblocks/generic/pythonbundle.py", line 128, in extensions_step
super(PythonBundle, self).extensions_step(*args, **kwargs)
File "/home/s3248973/.local/EasyBuildDev/easybuild-framework/easybuild/framework/easyblock.py", line 2402, in extensions_step
self.init_ext_instances()
File "/home/s3248973/.local/EasyBuildDev/easybuild-framework/easybuild/framework/easyblock.py", line 2332, in init_ext_instances
inst = cls(self, ext)
File "/tmp/easybuild-tmp/included-easyblocks-iBqB1u/easybuild/easyblocks/tensorflow.py", line 239, in init
super(EB_TensorFlow, self).init(*args, **kwargs)
File "/tmp/easybuild-tmp/included-easyblocks-iBqB1u/easybuild/easyblocks/generic/pythonpackage.py", line 278, in init
super(PythonPackage, self).init(*args, **kwargs)
TypeError: must be type, not None

It seems that PythonPackage turns out as None which I can't really explain. Splitting up the PR into 1 easyblock each seemingly works. See easybuilders/easybuild-easyblocks#2516

[migueldiascosta]

@Flamefire I think (or maybe hope) we can narrow this down a bit

• on one hand, the problem doesn't seem to be with any multiple easyblocks. For instance, I don't see a problem including all 3 easyblocks from remove cuda_compute_capabilities from custom easyconfig parameters for Clang, LAMMPS and TensorFlow (now supported as general easyconfig parameter) easybuild-easyblocks#2433 (I know that PR is already merged, but the logs show they are all imported from the PR)

• on the other hand, the problem doesn't seem to be specific to the github integration, I get the same error if I try to --include-easyblocks=*.py with local copies of pythonpackage.py and tensorflow.py (can you check?)

My guess is that this bug is specific to the case where the included easyblocks are related by inheritance and possibly with the fact that we are still (even after #3544) messing with sys.modules at

easybuild-framework/easybuild/tools/include.py

Lines 134 to 141 in f8cc137

	# force re-import if the specified modules was already imported;
	# this is required to ensure that an easyblock that is included via --include-easyblocks-from-pr
	# gets preference over one that is included via --include-easyblocks
	if pymod_spec in sys.modules:
	del sys.modules[pymod_spec]
	try:
	pymod = __import__(pymod_spec, fromlist=[pypkg])

(i.e., we reimport PythonPackage as a new object, but we deleted the PythonPackage object that was the super class of TensorFlow (?) )

[Flamefire]

I get the same error if I try to --include-easyblocks=*.py with local copies of pythonpackage.py and tensorflow.py (can you check?)

Looks like you are absolutely correct. Same here, and fix created changing the mentioned code. Also added a reproducer test. It's a bit involved and only happens in Python 2 as far as I can tell