Feature Request: Recurrsively track the imported source files

[Jarvis73]

I have three source files:

• main.py

from sacred import Experiment
import dataset

ex = Experiment("ExpName")

@ex.automain
def main(_run):
    print(_run.experiment_info['sources'])

• dataset.py

import utils

• utils.py

# Some util functions

When I run the main.py, the output is

[('main.py', '8f7e3a8cdd0d92b7255dae98fe255dc2'), ('dataset.py', 'd8f74eeb22a5f209e4c8b12258282689')]

But the utils.py is not included.

I think recursively tracking the imported source files is a general way to organize my experiments. But now I have to manually add the utils.py by ex.add_source_file("utils.py"), which is not quite convenient.

[thequilo]

I agree this would be a good feature. I'm not that familiar with dependency gathering, but I think that it shouldn't be too complicated.

In general, I would recommend putting all your sources in a git repository. Then, sacred can track the state of that git repository. But even then, it could be that you import another custom package in your code that depends on other things that are currently not tracked.

Let's add this feature as an option for dependency gathering!

[Qwlouse]

Try:

from sacred import SETTINGS

SETTINGS["DISCOVER_DEPENDENCIES"] = "sys"

Sacred supports four dependency gathering strategies:

• "none": do not gather dependencies at all
• "imported": collect all modules that are imported from the main script (default)
• "sys": collect all modules that are cached by sys.modules (<- this should capture recursive imports too)
• "dir": collect all python files in the directory or any subdirectories of the main script

[Jarvis73]

@Qwlouse
Thank you for your reply. I tested the sys mode, and it did solve my question.

@thequilo
By the way, I think maybe the sys mode is more natural to be the default option than imported. Because only if all the "recursively imported source files" are reserved, one can reproduce the experiments.