From 352c7ee278911d37074719ab906ec4c10eac362e Mon Sep 17 00:00:00 2001 From: Janet Gainer-Dewar Date: Wed, 15 May 2024 11:20:39 -0400 Subject: [PATCH 1/4] More release notes --- CHANGELOG.md | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 50af73e8260..1bdfd3da9aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,13 +9,33 @@ be found [here](https://cromwell.readthedocs.io/en/stable/backends/HPC/#optional ## 87 Release Notes +### GCP Batch + + * Added NVidia driver install ([#7235](https://github.com/broadinstitute/cromwell/pull/7235})) + * Fixed Docker mounting volumes with extra colon ([#7240](https://github.com/broadinstitute/cromwell/pull/7240)) + * Fixed issue with multiple zones defined in config ([#7240](https://github.com/broadinstitute/cromwell/pull/7240)) + * Fixed Batch label regex ([#7355](https://github.com/broadinstitute/cromwell/pull/7355)) + +### Progress toward WDL 1.1 Support + +WDL 1.1 support is in progress. Users that would like to try out the current partial support can do so by using +WDL version `development-1.1`. As of Cromwell 87, `development-1.1` includes: + * Engine functions: + * Added `suffix` ([#7363](https://github.com/broadinstitute/cromwell/pull/7363)) + * Added `unzip` ([#7363](https://github.com/broadinstitute/cromwell/pull/7368)) + * Added `quote` and `squote` ([#7375](https://github.com/broadinstitute/cromwell/pull/7375)) + * Updated `sub` to expect POSIX-flavor regex ([#7374](https://github.com/broadinstitute/cromwell/pull/7374)) + * Struct literals can be included in WDLs ([#7391](https://github.com/broadinstitute/cromwell/pull/7391)) ([#7402](https://github.com/broadinstitute/cromwell/pull/7402)) + * Added `returnCodes` runtime attribute ([#7389](https://github.com/broadinstitute/cromwell/pull/7389)) + ### `upgrade` command removed from Womtool -Womtool previously supported a `womtool upgrade` command for upgrading draft-2 WDLs to 1.0. With WDL 1.1 soon to become the latest supported version, this functionality is retiring. +Womtool previously supported a `womtool upgrade` command for upgrading draft-2 WDLs to 1.0. With WDL 1.1 soon to +become the latest supported version, this functionality is retiring. ([#7382](https://github.com/broadinstitute/cromwell/pull/7382)) ### Replacement of `gsutil` with `gcloud storage` -In this release, all **localization** functionality on the GCP backend migrates to use the more modern and performant `gcloud storage`. With sufficiently powerful worker VMs, Cromwell can now localize at up to 1200 MB/s [0][1][2]. +In this release ([#7359](https://github.com/broadinstitute/cromwell/pull/7359)), all **localization** functionality on the GCP backend migrates to use the more modern and performant `gcloud storage`. With sufficiently powerful worker VMs, Cromwell can now localize at up to 1200 MB/s [0][1][2]. In a future release, **delocalization** will also migrate to `gcloud storage`. As part of that upcoming change, we are considering turning on [parallel composite uploads](https://cromwell.readthedocs.io/en/stable/backends/Google/#parallel-composite-uploads) by default to maximize performance. Delocalized composite objects will no longer have an md5 checksum in their metadata; refer to the matrix below [3]. If you have compatibility concerns for your workflow, please [submit an issue](https://github.com/broadinstitute/cromwell/issues). @@ -32,6 +52,17 @@ In a future release, **delocalization** will also migrate to `gcloud storage`. A [3] Cromwell itself uses crc32c hashes for call caching and is not affected + +### Other Improvements + * In certain cases DRS downloads have been found to hang forever Cromwell will now time these out ([#7416](https://github.com/broadinstitute/cromwell/pull/7416)) + * Increased default Akka `client.parsing.max-response-reason-length` to 1024 ([#7406](https://github.com/broadinstitute/cromwell/pull/7406)) + * Workflow Completion Callback bodies now include fully-qualified output names ([#7234](https://github.com/broadinstitute/cromwell/pull/7234)) + * Improved workflow abort error handling ([#7245](https://github.com/broadinstitute/cromwell/pull/7245)) + * Improved logging for troubleshooting ([#7246](https://github.com/broadinstitute/cromwell/pull/7246)) ([#7253](https://github.com/broadinstitute/cromwell/pull/7253)) ([#7388](https://github.com/broadinstitute/cromwell/pull/7388)) + * Support for Intel Ice Lake chips in PAPI backend ([#7252](https://github.com/broadinstitute/cromwell/pull/7252)) + * Fix workflows getting stuck in Aborting when WDL has a type error ([#7385](https://github.com/broadinstitute/cromwell/pull/7385)) + * Updates to dependencies to fix security vulnerabilities. + ## 86 Release Notes ### GCP Batch From 4c9766e5caabf47a8cab284d9dfe4f19a06f3302 Mon Sep 17 00:00:00 2001 From: Janet Gainer-Dewar Date: Wed, 15 May 2024 11:39:44 -0400 Subject: [PATCH 2/4] Specify Nvidia version Co-authored-by: Adam Nichols --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ffc858f6e2..ced0859cd6c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,7 +15,7 @@ The `genomics` configuration entry was renamed to `batch`, see [ReadTheDocs](htt ### GCP Batch - * Added NVidia driver install ([#7235](https://github.com/broadinstitute/cromwell/pull/7235})) + * Added Nvidia driver install (default 418) ([#7235](https://github.com/broadinstitute/cromwell/pull/7235})) * Fixed Docker mounting volumes with extra colon ([#7240](https://github.com/broadinstitute/cromwell/pull/7240)) * Fixed issue with multiple zones defined in config ([#7240](https://github.com/broadinstitute/cromwell/pull/7240)) * Fixed Batch label regex ([#7355](https://github.com/broadinstitute/cromwell/pull/7355)) From ac6954ffb77f11687746983b02e55902d34b6d4c Mon Sep 17 00:00:00 2001 From: Janet Gainer-Dewar Date: Wed, 15 May 2024 11:40:04 -0400 Subject: [PATCH 3/4] Fix lingo leakage Co-authored-by: Adam Nichols --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ced0859cd6c..78ccc24c75a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,7 +63,7 @@ In a future release, **delocalization** will also migrate to `gcloud storage`. A * Workflow Completion Callback bodies now include fully-qualified output names ([#7234](https://github.com/broadinstitute/cromwell/pull/7234)) * Improved workflow abort error handling ([#7245](https://github.com/broadinstitute/cromwell/pull/7245)) * Improved logging for troubleshooting ([#7246](https://github.com/broadinstitute/cromwell/pull/7246)) ([#7253](https://github.com/broadinstitute/cromwell/pull/7253)) ([#7388](https://github.com/broadinstitute/cromwell/pull/7388)) - * Support for Intel Ice Lake chips in PAPI backend ([#7252](https://github.com/broadinstitute/cromwell/pull/7252)) + * Support for Intel Ice Lake chips in Life Sciences backend ([#7252](https://github.com/broadinstitute/cromwell/pull/7252)) * Fix workflows getting stuck in Aborting when WDL has a type error ([#7385](https://github.com/broadinstitute/cromwell/pull/7385)) * Updates to dependencies to fix security vulnerabilities. From 92309521652805c0074fd5dd5efeda2dc94a3063 Mon Sep 17 00:00:00 2001 From: Janet Gainer-Dewar Date: Wed, 15 May 2024 11:40:25 -0400 Subject: [PATCH 4/4] Punctuation Co-authored-by: Adam Nichols --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 78ccc24c75a..f17c5fd23a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,7 +58,7 @@ In a future release, **delocalization** will also migrate to `gcloud storage`. A ### Other Improvements - * In certain cases DRS downloads have been found to hang forever Cromwell will now time these out ([#7416](https://github.com/broadinstitute/cromwell/pull/7416)) + * In certain cases DRS downloads have been found to hang forever. Cromwell will now time these out. ([#7416](https://github.com/broadinstitute/cromwell/pull/7416)) * Increased default Akka `client.parsing.max-response-reason-length` to 1024 ([#7406](https://github.com/broadinstitute/cromwell/pull/7406)) * Workflow Completion Callback bodies now include fully-qualified output names ([#7234](https://github.com/broadinstitute/cromwell/pull/7234)) * Improved workflow abort error handling ([#7245](https://github.com/broadinstitute/cromwell/pull/7245))