Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Richer workflow inputs. #9073

Closed
wants to merge 2 commits into from

Conversation

jmchilton
Copy link
Member

@jmchilton jmchilton commented Dec 6, 2019

  • Allow parameter inputs to be optional.
  • Allow parameter inputs to have default values (if optional).
  • Allow data and collection parameter to be optional.
  • Unimplemented but new abstractions should allow having users specify extensions manually if they'd like for data and collection parameters.

The previous workflow Module interface would more or less persist Galaxy's "Tool State" directly into the database. The tool state framework persists state in an un-typed fashion (e.g. the optional value for data inputs would be 'True' instead of True). The Format 2 YAML that we would like to publish to Galaxy and export from Galaxy would much more cleanly represent this as actual boolean values.

The problem gets even worse with parameter inputs where we want the default value to be settable only if optional is True. This requires a conditional in the "Tool Form" language - so the persisted value would be nested structures without typing. For instance - instead of {default: 5, parameter_type: "integer", "optional": true} that we might want to store into the database or read from a workflow definition file, the framework would require something like {"parameter_type": "integer", "parameter_definition": {"optional": {"optional": "True", {"specify_default": {"specify_default": "True", "default": "5"}}}}}.

My preference may have been to have plugins defined in the client that just produce the minimal JSON - but I didn't know how to implement that and Sam's preference was that the client use the tool form framework for everything on the side panel anyway though.

As a result, this commit adds abstractions to separate module tool state (for communicating with client and tied to tool form rendering) from abstract export state - loaded from the API when not coming from the tool and stored in database.

To summarize workflow module interface changes...

  • module.get_inputs can build complex tool form parameters (unchanged)
  • module.state is always the tool form state (unchanged)
  • module.get_export_state is added to get the abstract state for export and for reasoning about internally
  • module.recover_state now assumes recovery from abstract step state until from_tool_form=True is supplied
  • API entry points tied to the tool form (build_module) just pass through from_tool_form by default.
  • The client now sends from_tool_form into the API whenever it serializes the workflow state for re-saving

More expansive, cleaned up version of #8807 (and #8232) - work stemming from the CWL branch of Galaxy. Implements #7775, #8218, #8576. xref #1325. Needed for #9030.

- Allow parameter inputs to be optional.
- Allow parameter inputs to have default values (if optional).
- Allow data and collection parameter to be optional.
- Unimplemented but new abstractions should allow having users specify extensions manually if they'd like for data and collection parameters.

The previous workflow Module interface would more or less persist Galaxy's "Tool State" directly into the database. The tool state framework persists state in an un-typed fashion (e.g. the optional value for data inputs would be 'True' instead of True). The Format 2 YAML that we would like to publish to Galaxy and export from Galaxy would much more cleanly represent this as actual boolean values.

The problem gets even worse with parameter inputs where we want the default value to be settable only if optional is True. This requires a conditional in the "Tool Form" language - so the persisted value would be nested structures without typing. For instance - instead of ``{default: 5, parameter_type: "integer", "optional": true}`` that we might want to store into the database or read from a workflow definition file, the framework would require something like ``{"parameter_type": "integer", "parameter_defintion": {"optional": {"optional": "True", {"specify_optional": {"specify_default": "True", "default": "5"}}}}}``.

My preference may have been to have plugins defined in the client that just produce the minimal JSON - but I didn't know how to implement that and Sam's preference was that the client use the tool form framework for everything on the side panel anyway though.

As a result, this commit adds abstractions to separate module tool state (for commiunicating with client and tied to tool form rendering) from abstract export state - loaded from the API when not coming from the tool and stored in database.

To summarize workflow module changes...

- module.get_inputs can build complex tool form parameteras (unchanged)
- module.state is always the tool form state (unchanged)
- module.get_export_state is added to get the abstract state for export and for reasoning about internally
- module.recover_state now assumes recovery from abstract step state until from_tool_form=True is supplied
- API entry points tied to the tool form (build_module) just pass through from_tool_form by default.
- The client now sends from_tool_form into the API whenever it serializes the workflow state for re-saving
@jmchilton jmchilton changed the title [WIP] Richer workflow inputs. Richer workflow inputs. Dec 7, 2019
@galaxybot galaxybot added this to the 20.01 milestone Dec 7, 2019
@jmchilton jmchilton closed this Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants