added basic feature to turn experiment config into shell script #150
Conversation
…emove old (unsupported) module tests
require version 1.6.4 of fair
runFACTS.py
Outdated
| @@ -38,6 +37,11 @@ def run_experiment(exp_dir, debug_mode, resourcedir = None): | |||
| # Exit | |||
| sys.exit(0) | |||
|
|
|||
| # Print out shell script if in shell script mode | |||
| if makeshellscript: | |||
| print_experimentsteps_script(experimentsteps) | |||
There was a problem hiding this comment.
Running the print_experimentsteps_script function at this point will require the onemodule directory to include the location.lst file. Maybe there is a way to run this after line 86-90 or we need to be clear to the user that makeshellscript = True requires location.lst to be present in exp_dir
There was a problem hiding this comment.
Actually, even if I create a location.lst file in experiments/onemodule/ the test run still fails with
FileNotFoundError: [Errno 2] No such file or directory: '/scratch/jt1187/test.1673016583/location.lst'
I think it is not being copied to the working directory.
| print('#!/bin/bash\n') | ||
|
|
||
| print('if [ -z "$WORKDIR" ]; then ') | ||
| print(' WORKDIR=/scratch/`whoami`/test.`date +%s`') |
There was a problem hiding this comment.
It would be useful to include the name of the module. I think that could be done by extracting pipeline_id and add it to the working directory name.
|
Ok, I've addressed the location.lst issue and also set up separate directories for each pipeline. This is guaranteed to break cross-module interactions, since we do not handle in the script the copy_input_data and copy_output_data commands used for passing data between modules. But I am merging the shellscript functionality in as an experimental feature for now. |
…on/runFACTS2shell added basic feature to turn experiment config into shell script
I've retired the old module test scripts, the preferred replacement approach should be:
experiments/onemodule)python runFACTS.py --shellscript experiments/onemodule > test.shsource test.sh