Version 1.1 Spec

[jdidion]

This is a draft of v1.1. I encourage and implore as many people as possible to review this document in as much detail as possible. I have tried to make it polished but I'm sure some things fell through the cracks.

My process for creating this specification was as follows:

1. Start with the current development version of the spec.
2. Merge in all accepted but unmerged PRs.
3. Manually go through all PRs to the development spec from the very beginning to make sure no content was lost due to bad merges.
4. Remove all reference to Directory type and hints section.
5. Reorganize the document into three main sections and two appendecies, and reorganize within each section to put next to each other all the related information and to reduce redundancy.
6. Review all of the text and revise where necessary to ensure consistent and correct use of terms. (Please flag any areas where my view of what is correct is wrong)
7. Perform spell checking, check for broken links, etc.

With input from the group on Slack, I have tried to correctly resolve all the conflicts I found between different parts of the spec, and to add additional clarification where needed. I was also fairly aggressive about removing wrong, redundant, or unnecessarily verbose text. Anywhere I have been too aggressive, please let me know.

I also took the liberty of adding three new pieces of content that have not previously been PR'd:

1. A section on equality and inequality of compound types. It is my understanding that this has previously been implicit, and is supported in Cromwell, so I wanted to make it explicit.
2. Serialization to JSON is undefined for Pairs and for Maps with non-String keys. This make sense, because there is no unambiguous way to represent such types in JSON. Therefore, I added suggested conversions to serializable types (and back to WDL types after deserialization).
3. To convert a Map with a non-String key, the best conversion is to a struct with two array members (keys and values). However, there is no way to write this conversion without an unzip function. Thus, I have added an unzip function to the spec.

struct IntStringMap {
  Array[Int] keys
  Array[String] values
}
Map[Int, String] m = {0: "a", 1: "b"}
Pair[Array[Int], Array[String]] u = unzip(m)
IntStringMap i = IntStringMap {
  keys: u.left,
  values: u.right
}
# after deserialization, we can convert back to Map
Map[Int, String] m2 = as_map(zip(i.keys, i.values))

I hope that items 1 and 2 are uncontroversial. For item 3, I realize we may not want to add another function to the standard library without going through the standard PR process. If that's the case, I'll remove unzip from this draft and I will update the serialization guide to state that serialization to/from JSON for Pairs and for Maps with non-String keys is not possible. I will then add a PR that we can review and decide whether to include in 1.1 or push it to development.

[jdidion]

Voting is open. Obviously I'm +1.

[patmagee]

Huge and enthusiastic +1 for me
👍

[mlin]

Two trivial corrections noted, otherwise 👍 from me thanks!

[geoffjentry]

👍

[rhpvorderman]

👍 From me as well. Thanks for tackling this huge task!

[DavyCats]

👍

[pieterlukasse]

This is in! I'm happy 👍

[aednichols]

👍

[cjllanwarne]

👍 phenomenal work. Thanks so much to @jdidion in particular and to everyone else who has put in so much time and effort contributing and reviewing to get this WDL 1.1 SPEC to this state!

[vdauwera]

👍 Amazing! Huge props to @jdidion and everyone who participated, this is going to be so great for the community. 💯

[vortexing]

👍🏼
👏🏼👏🏼👏🏼👏🏼👏🏼 (just putting my 2 cents in again)