-
Document updates
- Add option to reverse legend for stacked barplots.
- Update heatmap methods to allow for case where heatmap is not generated.
- Add option to write table figure to file.
- Make the project name optional.
- Remove ticks and labels from x-axis where there are too many samples to fit.
- Remove full path from DADA2 version in log (full path not needed and runs out of space).
- Add ellpise to legends on plots when needed.
- Remove xlim and xticks from grouped barplot if too many samples.
- Increase margins/figures/legends to fit longer variable names.
- Increase the size of the margins to MLA format (1 inch all), decrese max legend length, and decrease heatmap sizes.
- Update zscore in heatmap to allow for metadata rows.
-
Grid updates
- Update check for user id to use getpass instead of pwd for python3.
- Make grid scratch optional by default.
- Modify grid scratch option commands to catch error in actions (prevents copy of partial targets from scratch to output folder). This will also resolve slurm grid status stating task completed (based on file copy) when the task failed based on task actions.
- Add default check to determine Cannon vs Fasse grid paritions.
- Add --grid-submit-sleep option plus defaults to resolve fasse autofs mounting issues with too many jobs submitted at once.
- Add another exception to allow for sinfo "Unable to contact slurm controller error".
-
Fix error with reporter message format in python 3.9.
-
Update requirements to allow for newer versions of dependencies.
-
Format error messages so they are easier to read.
-
Document processing
- Change to command line call of pweave for document processing (to remove overlap of scope).
- Add import functionality to document template.
- Update default custom colors for document plots.
- Update plots to increase default sizes, allow for write to file, include latex size in doc write (to increase document size).
- Remove duplicate y-axis ticks for grouped plots.
- Add another pickle method to allow for the pickle of document tasks (to allow document-based workflows to run in parallel).
-
SLURM grid
- Added two new grid options "--grid-max-time" and "--grid-max-mem" to set the max time and memory used for each grid task.
- Update grid template to allow for singularity environment settings.
- Modify topological sort function call to work with latest networkx version.
- Allow for nsf working temp files by allowing for errors in remove of tmp folder and files from zip archive function.
- Update slurm memkill state to match new RC SLURM state and update decodes for python3 to slurm grid submit checks.
- Change hclust2 input file format to work with latest hclust2 version.
- Update hclust2 exe check to be python 2 compliant.
- Suppress stdout in hclust2 exe name check.
- Update hclust2 doc method to allow for pip executable.
- Updates to documentation methods for python 3 compatibility.
- Fix issue running without grid scratch set (addition of / to empty string).
- Subclass pweave processor class to modify function to not remove spaces at the end of strings generated from code chunks. This change allows for endlines after markdown images which is needed for pandoc to generate figure captions.
- Update pandoc option to allow for use with latest version.
- Remove extra decode to resolve new error from prior fix for python 3.
- Decode version for tracked executable to resolve reruns for tasks with executable depends in python 3.
- Changes to grid module for python 3.
- For grid scratch, rerun get task result after grid job to get final output files in target output folder resolving issue with function targets not being recorded in database after run success.
- Add grid scratch option for slurm.
- Add config option.
- Update aws batch task script to return error message to grid (in addition to messages already in log).
- Print workflow description and version to log.
- Add to file_size function the ability to get s3 file sizes.
- Fix small variable typo in grid class introduced with AWS grid changes.
- Small fix to allow for logs generated with python2 to be used in reports generated in python3.
- Updates for python 3 compatibility.
- Add paging for AWS batch queue (for large numbers of input files).
- Fix texts so they comply with AWS batch modifications.
- Fix to prevent error with visualization (part 3 of fixes from AWS batch merge: Allow targets/depends to not have temp_files method)
- Fix to prevent error with custom arguments (part 2: only set tmpdir once and fix function calls)
- Fix to prevent error with custom arguments (part 1: set tmpdir after custom args are added)
- Added grid option for AWS batch with input/output files written to S3
- Changes to colorbar plots to handle NAs
- Add colorbar for continuous data in pcoa plots
- Update max labels on plots as we can fit a few more sample names
- Increase space in heatmap for metadata labels
- Allow for optional data files in the archiving of a document.
- Add distance function keyword to show_hclust2 doc method
- Add method to plot multiple poca colored by abundance
- In document show_pcoa, set default to not apply arcsin-sqrt transform and add caption for this case
- Detect new rc exceeded memory limit reported in error file to slurm class
- Make cli argument keys a list for python3 compatibility
- In document formatter, allow for older versions of pandoc by only using width in figure if newer version of pandoc is installed
- Add read of variables from log to LoggerReporter class method read_log.
- Add metadata keyword to hclust2 method.
- Add new method plot_stacked_barchart_grouped.
- Update pweave section to use custom formatter class (to fix pweave issue of duplicating plots in single code block).
- Add metadata option to pcoa plot.
- Include any document dependencies in the output data folder so they can be bundled with the report archive.
- Update sge default partition to match new broad configuration.
- The API is now hosted at Readthedocs. The html has been removed from the source since it is now automatically generated.
- Bitbucket pipelines have been setup for automatic testing for python2 and python3 installs.
- A new option was added to allow users to provide commands to set up the environment of the grid tasks.
- Jupyter notebook examples have been added.
- For builds based on targets, relative paths are now allowed to targets.
- The dpi and font size for the document heatmaps were modified to improve rendering.
- Unique names are allowed for tasks in groups.
- Instead of raising errors, document heatmaps and grouped barcharts print errors to the document to make debugging easier.
- AnADAMA2 is now pip installable.