add OpenAPI schema field in I/O of process descriptions

[fmigneault]

Changes

Increase conformance with https://github.com/opengeospatial/ogcapi-processes by returning OpenAPI schema in process description and parsing them during deployment.

When deploying a process, any complementary details defined in OAS schema are combined with other WPS/CWL definitions to resolve the final I/O definitions in each context.

Implementation Notes

• Using these changes, process descriptions are OGC compliant by offering the schema of each I/O.
• I/O schema are not yet validated. Following PR eventually.
• JSON objects submitted as value literally/embedded in the execute request of some I/O will not yet work to reach all the way down to CWL / Dorker application. Some JSON file generation from this content is needed to mount the content as file-volume for use by the app.
• Raw-data/encoded-string values are not yet converted to File for CWL. Complex I/O must still be given by HTTP/S3 reference for the moment. Again, some following PR at some point to address this.

Experiments in combination with CubeWerx for interop testing of TEPs (quoting/billing/workflow) could help guide which features to prioritize next.

References

fixes #245
fixes https://crim-ca.atlassian.net/browse/CUBW-7

[codecov]

Codecov Report

Merging #429 (56c428a) into master (dea9255) will increase coverage by 0.11%.
The diff coverage is 79.71%.

@@            Coverage Diff             @@
##           master     #429      +/-   ##
==========================================
+ Coverage   80.79%   80.90%   +0.11%     
==========================================
  Files          76       76              
  Lines       13421    14044     +623     
  Branches     2868     3058     +190     
==========================================
+ Hits        10843    11362     +519     
- Misses       1924     1981      +57     
- Partials      654      701      +47     

Impacted Files	Coverage Δ
weaver/typedefs.py	0.00% <ø> (ø)
weaver/wps_restapi/api.py	77.71% <ø> (ø)
weaver/wps_restapi/utils.py	87.69% <ø> (ø)
weaver/utils.py	80.20% <50.00%> (-1.15%)	⬇️
weaver/processes/utils.py	76.29% <52.63%> (-0.31%)	⬇️
weaver/wps_restapi/colander_extras.py	80.09% <68.75%> (-0.53%)	⬇️
weaver/processes/convert.py	79.83% <77.52%> (+1.25%)	⬆️
weaver/datatype.py	75.06% <100.00%> (+0.39%)	⬆️
weaver/formats.py	87.31% <100.00%> (+0.09%)	⬆️
weaver/processes/constants.py	100.00% <100.00%> (ø)
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dea9255...56c428a. Read the comment docs.

[dbyrns]

It looks good, but I wonder if every functions in processes/concert.py are somehow convered by the new tests in test_wps_package.py ? There's a lot of stuff there and it looks like it could be very easy to make a mistake.

[fmigneault]

It looks good, but I wonder if every functions in processes/concert.py are somehow convered by the new tests in test_wps_package.py ? There's a lot of stuff there and it looks like it could be very easy to make a mistake.

Seems mostly covered (indirectly via existing tests), but a few misses here and there.
https://app.codecov.io/gh/crim-ca/weaver/compare/429/changes#D1L1361

[dbyrns]

LGTM