Fix failing shape selection process

[Zeitsperre]

Overview

This PR fixes an issue introduced in 5fce8da

Changes:

• Removed the write+bytes setting from wps_hydrobasins_shape_selection.py that was causing write failure for xml strings.

Related Issue / Discussion

Additional Information

Links to other issues or sources.

[huard]

Isn't there a risk that this will break the notebook this change was for ?

[Zeitsperre]

The "wb" flag is currently breaking the build tests currently. https://travis-ci.org/github/Ouranosinc/raven/jobs/666001344?utm_medium=notification&utm_source=github_status

I'll verify that the notebooks still function. If not, we need to find a solution for both.

[Zeitsperre]

@huard I can't find the notebook that needed the changes implemented previously, which one was it? It also seems as though the notebook build is failing due incompatibilities between salem and pyroj. Would it be worth making a new example given that salem hasn't released a new version in a year?

[huard]

I thought it was the gridded_subset_notebook. But if you can get all the notebooks to pass, I won't complain.
What would you replace salem with ?

[Zeitsperre]

@huard I think the functions that are being used in salem that it's having a hard time with are related mainly to advancements in GeoPandas. I'll see what it is exactly that is being done, but I feel like we could use this as an opportunity to get our feet wet with rioxarray. There has been pretty continuous development since I last mentioned it and since we're only using it in a notebook (ie: not core to any services) I feel that we can take the risk.

In terms of the notebooks, some of them are failing due to calls to localhost that don't seem to resolve when Travis CI runs them (yet pass on my laptop). I can see whether we can replace them with PAVICS prod server calls, if that's an acceptable solution. If I run into problems, I'll speak with @tlvu.

[huard]

@Zeitsperre Sounds good !

[tlvu]

@Zeitsperre Try to merge the sanitize_output.cfg of Raven with the one here https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/master/notebooks/output-sanitize.cfg and use it like so https://github.com/bird-house/finch/blob/4e9d2baaad0166a76c3e49481934c1eb56af007a/Makefile#L108-L112

localhost is handled properly there. Delete the local version of Raven to avoid multiple copy of the same file.

The Finch PR bird-house/finch#112 that added that test-notebooks target FYI.

[Zeitsperre]

Things are coming along here but it looks as though there's an issue really deep dealing with mimetypes:

ValueError                                Traceback (most recent call last)
<ipython-input-8-1084d56c28fe> in <module>
      1 # Use the geoserver to extract the land cover over the appropriate bounding box (automatic)
----> 2 resp = wps.nalcms_zonal_stats(shape=feature_url, select_all_touching=True, band=1, simple_categories=True)

</home/tjs/miniconda3/envs/raven/lib/python3.6/site-packages/birdy/client/base.py-11> in nalcms_zonal_stats(self, shape, raster, simple_categories, band, select_all_touching)

~/miniconda3/envs/raven/lib/python3.6/site-packages/birdy/client/base.py in _execute(self, pid, **kwargs)
    281     def _execute(self, pid, **kwargs):
    282         """Execute the process."""
--> 283         wps_inputs = self._build_inputs(pid, **kwargs)
    284         wps_outputs = [
    285             (o.identifier, "ComplexData" in o.dataType)

~/miniconda3/envs/raven/lib/python3.6/site-packages/birdy/client/base.py in _build_inputs(self, pid, **kwargs)
    251 
    252                     # Guess the mimetype of the input value
--> 253                     mimetype, encoding = guess_type(value, supported_mimetypes)
    254 
    255                     if encoding is None:

~/miniconda3/envs/raven/lib/python3.6/site-packages/birdy/utils.py in guess_type(url, supported)
    167     else:
    168         if mime not in supported:
--> 169             raise ValueError(f"mimetype {mime} not in supported mimetypes {supported}.")
    170 
    171     return mime, enc

ValueError: mimetype application/geo+json not in supported mimetypes ['application/vnd.geo+json', 'application/gml+xml', 'application/json', 'application/x-zipped-shp'].

Something is specifying files as application/geo+json where it should be noted as application/vnd.geo+json and it's causing failures at the Birdy level. Worth opening an issue in Birdy?

[Zeitsperre]

@huard any advice? I think you were the last person to adjust the encoding handling in Birdy (bird-house/birdy@be1a3c0).

[huard]

geopython/pywps#528
geopython/pywps#529

[Zeitsperre]

@huard Putting work for this on hold for now. I'll be watching those PRs in the meantime. Thanks for the help!

[Zeitsperre]

Looking into this again now that the upstream issues have been fixed.

[Zeitsperre]

@huard I think I'm good to merge this now and open an issue about the failing mimetype error. I can't advance on it until a new PyWPS is released and the fix tht was needed for this PR is present here already. Work for you?

[tlvu]

The default Raven URL should have been the one on PAVICS, not the localhost one !