diff --git a/docs/dev/devguide.md b/docs/dev/devguide.md index 73c51bb6..0d1a02a1 100644 --- a/docs/dev/devguide.md +++ b/docs/dev/devguide.md @@ -90,4 +90,54 @@ INFO [job test.cwl] completed success ] ``` -As can be seen from the output json blob the order returned is not the same as input order. \ No newline at end of file +As can be seen from the output json blob the order returned is not the same as input order. + + +## Partial Failures +When the partial failures feature is enabled although the subprocess for the workflow step itself will pass, the post-processing javascript can potentially crash as seen below. The Sophios compiler only semantically understands Sophios/CWL. It is theoretically impossible to correct mistakes in the embedded JS of any arbitrary workflow. The corresponding cwl snippet is also shown. +``` +outputs: + + topology_changed: + type: boolean + outputBinding: + glob: valid.txt + loadContents: true + outputEval: | + ${ + // Read the contents of the file + const lines = self[0].contents.split("\n"); + // Read boolean value from the first line + const valid = lines[0].trim() === "True"; + return valid; + } +``` +``` +stdout was: '' +stderr was: 'evalmachine.:45 + const lines = self[0].contents.split("\n"); + ^ +TypeError: Cannot read properties of undefined (reading 'contents') +``` +To fix this the developer needs to add a javascript snippet to check if the self object being globbed exists, shown below. +``` +outputs: + + topology_changed: + type: boolean + outputBinding: + glob: valid.txt + loadContents: true + outputEval: | + ${ + // check if self[0] exists + if (!self[0]) { + return null; + } + // Read the contents of the file + const lines = self[0].contents.split("\n"); + // Read boolean value from the first line + const valid = lines[0].trim() === "True"; + return valid; + } +``` \ No newline at end of file diff --git a/docs/userguide.md b/docs/userguide.md index 1cf5b702..63ec9855 100644 --- a/docs/userguide.md +++ b/docs/userguide.md @@ -76,4 +76,15 @@ steps: Note that this is one key difference between WIC and CWL. In CWL, all inputs must be given in a separate file. In WIC, inputs can be given inline with !ii and after compilation they will be automatically extracted into the separate file. -(NOTE: raw CWL is still supported with the --allow_raw_cwl flag.) \ No newline at end of file +(NOTE: raw CWL is still supported with the --allow_raw_cwl flag.) + +## Partial Failures + +In running workflows at scale, sometimes it is the case that one of the workflow steps may crash due to a bug causing the entire workflow to crash. In this case can use `--partial_failure_enable` flag. For special cases when the exit status of a workflow step isn't 1, and a different error code is returned (for example 142), then the user can supply the error code to wic as a success code to prevent workflow from crashing with `--partial_failure_success_codes 0 1 142`. By default partial failure flag will consider only 0 and 1 as success codes. An example line snippet of the error code being printed is shown below. +``` +[1;30mWARNING[0m [33m[job compare_extract_protein_pdbbind__step__4__topology_check] exited with status: 139[0m +``` + +## Parallelization + +In order to utilize scattering features in cwl, the user needs to provide the flag `--parallel`. Additionally cwltool has various issues regarding scattering features such as deadlocks and thus it is preferred to use toil in this case `--cwl_runner toil-cwl-runner`. \ No newline at end of file