add user docs

docs/dev/devguide.md

@@ -90,4 +90,54 @@ INFO [job test.cwl] completed success
     ]
 ```
 
-As can be seen from the output json blob the order returned is not the same as input order.
+As can be seen from the output json blob the order returned is not the same as input order.
+
+
+## Partial Failures
+When the partial failures feature is enabled although the subprocess for the workflow step itself will pass, the post-processing javascript can potentially crash as seen below. The Sophios compiler only semantically understands Sophios/CWL. It is theoretically impossible to correct mistakes in the embedded JS of any arbitrary workflow. The corresponding cwl snippet is also shown.
+```
+outputs:
+
+  topology_changed:
+    type: boolean
+    outputBinding:
+      glob: valid.txt
+      loadContents: true
+      outputEval: |
+        ${
+          // Read the contents of the file
+          const lines = self[0].contents.split("\n");
+          // Read boolean value from the first line
+          const valid = lines[0].trim() === "True";
+          return valid;
+        }
+```
+```
+stdout was: ''
+stderr was: 'evalmachine.<anonymous>:45
+  const lines = self[0].contents.split("\n");
+                        ^
+TypeError: Cannot read properties of undefined (reading 'contents')
+```
+To fix this the developer needs to add a javascript snippet to check if the self object being globbed exists, shown below.
+```
+outputs:
+
+  topology_changed:
+    type: boolean
+    outputBinding:
+      glob: valid.txt
+      loadContents: true
+      outputEval: |
+        ${
+          // check if self[0] exists
+          if (!self[0]) {
+            return null;
+          }
+          // Read the contents of the file
+          const lines = self[0].contents.split("\n");
+          // Read boolean value from the first line
+          const valid = lines[0].trim() === "True";
+          return valid;
+        }
+```

docs/userguide.md

@@ -76,4 +76,15 @@ steps:
 
 Note that this is one key difference between WIC and CWL. In CWL, all inputs must be given in a separate file. In WIC, inputs can be given inline with !ii and after compilation they will be automatically extracted into the separate file.
 
-(NOTE: raw CWL is still supported with the --allow_raw_cwl flag.)
+(NOTE: raw CWL is still supported with the --allow_raw_cwl flag.)
+
+## Partial Failures
+
+In running workflows at scale, sometimes it is the case that one of the workflow steps may crash due to a bug causing the entire workflow to crash. In this case can use `--partial_failure_enable` flag. For special cases when the exit status of a workflow step isn't 1, and a different error code is returned (for example 142), then the user can supply the error code to wic as a success code to prevent workflow from crashing with `--partial_failure_success_codes 0 1 142`. By default partial failure flag will consider only 0 and 1 as success codes. An example line snippet of the error code being printed is shown below.
+```
+[1;30mWARNING[0m [33m[job compare_extract_protein_pdbbind__step__4__topology_check] exited with status: 139[0m
+```
+
+## Parallelization
+
+In order to utilize scattering features in cwl, the user needs to provide the flag `--parallel`. Additionally cwltool has various issues regarding scattering features such as deadlocks and thus it is preferred to use toil in this case `--cwl_runner toil-cwl-runner`.