Backport master to 4.4 #574

cehbrecht · 2021-02-08T17:04:37Z

Overview

This branch backports the changes on master to pywps 4.4.

Related Issue / Discussion

#568

Additional Information

List of integrated PRs:

Added validation for GPX files (Added validation for GPX files, fixes issue (#535) #537)
fix geojson mimetype (Fix: changed geojson mimetype #529)
bumpverion (Use bump2version to track version changes #527)
allow custom UOM reference for literal IO (allow custom UOM reference for literal IO #523)
removed unused wpsserver.py (removed unused wpsserver.py #521)
skip py27 (Issue 477 skip py27 #520)
fix tests with language option (fix tests with language option #519)
Add NcML to supported FORMATS (Add NcML to supported FORMATS #515)

Skipped PRs:

jobqueue (initial watchdog #505)
update docs for processes parameter (Issue 516 docs config #518)

Contribution Agreement

(as per https://github.com/geopython/pywps/blob/master/CONTRIBUTING.rst#contributions-and-licensing)

I'd like to contribute [feature X|bugfix Y|docs|something else] to PyWPS. I confirm that my contributions to PyWPS will be compatible with the PyWPS license guidelines at the time of contribution.
[ x] I have already previously agreed to the PyWPS Contributions and Licensing Guidelines

* added ncml to supported formats * added ncml schema

* removed PY2 usage * removed flufl.enum * removed pathlib dependency

* Set up bump2version

coveralls · 2021-02-08T17:38:49Z

Coverage remained the same at 0.0% when pulling 4f15fc6 on cehbrecht:backport-to-4.4 into d593f4a on geopython:pywps-4.4.

cehbrecht · 2021-02-08T18:15:24Z

test with emu ... still seems to work.

cehbrecht · 2021-02-08T18:23:28Z

@tlvu @Zeitsperre Maybe you like to check this PR? We can still improve on pywps-4.4 also after merge ...

Zeitsperre · 2021-02-08T19:40:35Z

@cehbrecht ~~Would you like me to merge #573 into pywps-4.4 before this is merged?~~

I have opened a new PR: #575. Feel free to merge at your leisure!

Zeitsperre

Looks good to me. For the Mmmetype list, I can't help but notice that the GeoTIFF mimetype is a matter of intense debate (opengeospatial/geotiff#34). Does it make sense to include Format('image/tiff', extension='.tiff', encoding='base64') as well? What are your thoughts?

Also, cosmetic opinion, but would you be open to isort-ing the library imports to clean them up a bit?

pywps/inout/formats/__init__.py

pywps/app/Process.py

pywps/exceptions.py

setup.cfg

cehbrecht · 2021-02-09T14:33:09Z

@Zeitsperre @fmigneault Thanks for the review. Please give the pywps-4.4 branch a try ... when there are no complains I can make the 4.4.0 release.

tlvu · 2021-02-10T21:04:10Z

@cehbrecht sorry I took a day off yesterday, was not able to look at this sooner.

So basically if we diff this branch with master, the only differences would be the jobqueue feature? Everything else are backported? We could merge this branch back into master and it should NOT make a difference?

tlvu · 2021-02-10T21:13:38Z

Here is how master would change today if pywps-4.4 is merged into master: 20e1e25...e10c99c

Looks like all the changes from 4.2.4 forward are missing on master. Should be a good thing to have on master.

tlvu · 2021-02-10T21:19:57Z

Here is the reverse diff (if master would merge into pywps-4.4) e10c99c...20e1e25.

Looks like mostly the jobqueue but there are other changes that I am not familiar with.

fmigneault · 2021-02-11T05:28:41Z

I feel it would be a great idea to have first the resolution of python 2/3 with minor patches in 4.4.
Then, another (maybe major?) version for jobqueue and co. features.
This will greatly align with OGC-API / owslib eventual work (opengeospatial/joint-ogc-osgeo-asf-sprint-2021#2) that will require the job items.

I have started pointing at pywps-4.4 with this PR crim-ca/weaver#214
And so far it doesn't look like there are any major incompatibilities.

I fear though that the alembic database migration and all other job queue elements would cause a lot of problems on my side, while being unnecessary in weaver's case as jobs already have their own reporting/handling.

cehbrecht · 2021-02-11T15:04:10Z

I fear though that the alembic database migration and all other job queue elements would cause a lot of problems on my side, while being unnecessary in weaver's case as jobs already have their own reporting/handling.

@fmigneault In the next attempt for the job queue implementation I would prefer to life without alembic database migration. The database is essentially used for the queue and this is just temporary storage.

cehbrecht · 2021-02-11T15:10:22Z

Here is how master would change today if pywps-4.4 is merged into master: 20e1e25...e10c99c

Looks like all the changes from 4.2.4 forward are missing on master. Should be a good thing to have on master.

I could create a dev-branch from the current master and re-integrate the pywps-4.4 branch. The master should then be the same as pywps-4.4.

tlvu · 2021-02-11T15:11:19Z

I feel it would be a great idea to have first the resolution of python 2/3 with minor patches in 4.4.
Then, another (maybe major?) version for jobqueue and co. features.

I am not suggesting merging master to pywps-4.4 so no jobqueue on 4.4. I am only suggesting merge pywps-4.4 to master so master still contain all the fixes (return to the normal workflow before this divergence).

The 2 compare links I generated is simply for us to audit the changes between master and pywps-4.4 branch.

tlvu · 2021-02-11T15:19:31Z

Here is how master would change today if pywps-4.4 is merged into master: 20e1e25...e10c99c
Looks like all the changes from 4.2.4 forward are missing on master. Should be a good thing to have on master.

I could create a dev-branch from the current master and re-integrate the pywps-4.4 branch. The master should then be the same as pywps-4.4.

I think we simply just need to return to the regular workflow. Correct me if I am wrong but we used to merge all branches back to master so master always have all the cumulative changes?

So when we create new branches from master, we ensure the new branch is not missing anything from any previous published releases?

If what I understood is correct, then after the merge of pywps-4.4 to master, we can fix up the jobqueue on master to your liking @cehbrecht then cut a new branch from there for new release. This assume on master the only thing new vs pywps-4.4 is the jobqueue and nothing else (hence the usefulness of the compare links to see the diff between the branches).

cehbrecht · 2021-02-11T16:13:56Z

I feel it would be a great idea to have first the resolution of python 2/3 with minor patches in 4.4.
Then, another (maybe major?) version for jobqueue and co. features.

I am not suggesting merging master to pywps-4.4 so no jobqueue on 4.4. I am only suggesting merge pywps-4.4 to master so master still contain all the fixes (return to the normal workflow before this divergence).

The 2 compare links I generated is simply for us to audit the changes between master and pywps-4.4 branch.

Right. The job-queue is not on pywps-4.4 ... only in the history of master.

cehbrecht · 2021-02-11T16:18:21Z

If what I understood is correct, then after the merge of pywps-4.4 to master, we can fix up the jobqueue on master to your liking @cehbrecht then cut a new branch from there for new release.

When me make a new attempt on the job-queue I would do this on a dev branch. We can merge it to master when we are really sure we want to keep it 😄

The next things that could happen on master (after re-integration of pywps-4.4):

re-formatting using black?
clean up tests
work on rest api

tlvu · 2021-02-11T17:24:57Z

When me make a new attempt on the job-queue I would do this on a dev branch.

Correct. I did not literally meant any commits direct on master. I mean PR to master instead of PR to pywps-4.4.

Sorry if my message was not clear. Nothing should land on master without a PR.

huard and others added 13 commits February 8, 2021 17:31

Add NcML to supported FORMATS (#515)

07f4bb5

* added ncml to supported formats * added ncml schema

fix tests with language option

73bf3a3

Issue 477 skip py27 (#520)

e7adf66

* removed PY2 usage * removed flufl.enum * removed pathlib dependency

removed unused wpsserver.py

89dbb58

allow custom UOM reference for literal IO

2b5662e

Use bump2version to track version changes (#527)

cbd9b08

* Set up bump2version

fix: changed geojson mimetype. previous value was obsolete.

337ae84

adding plural to units

d04c77d

typo

768efd9

another typo

4171583

adding feet

ac32374

removed PY2

41a4997

add metre to UOMs

ff61cc9

cehbrecht marked this pull request as draft February 8, 2021 17:20

fixed bbox test

b5dda54

cehbrecht marked this pull request as ready for review February 8, 2021 18:14

remove py27 in tox.ini

cafb028

cehbrecht requested a review from tlvu February 8, 2021 18:21

Zeitsperre approved these changes Feb 8, 2021

View reviewed changes

pywps/inout/formats/__init__.py Show resolved Hide resolved

pywps/app/Process.py Show resolved Hide resolved

fmigneault reviewed Feb 9, 2021

View reviewed changes

pywps/exceptions.py Show resolved Hide resolved

fmigneault reviewed Feb 9, 2021

View reviewed changes

setup.cfg Outdated Show resolved Hide resolved

use version 4.4.0

4f15fc6

cehbrecht merged commit e10c99c into geopython:pywps-4.4 Feb 9, 2021

cehbrecht deleted the backport-to-4.4 branch February 9, 2021 14:29

cehbrecht mentioned this pull request Feb 10, 2021

Long pending changes of python version compatibilities #568

Closed

cehbrecht mentioned this pull request Feb 11, 2021

release 4.4.0 #577

Merged

1 task

fmigneault mentioned this pull request Feb 12, 2021

Weaver 2 crim-ca/weaver#214

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport master to 4.4 #574

Backport master to 4.4 #574

cehbrecht commented Feb 8, 2021

coveralls commented Feb 8, 2021 •

edited

Loading

cehbrecht commented Feb 8, 2021 •

edited

Loading

cehbrecht commented Feb 8, 2021

Zeitsperre commented Feb 8, 2021 •

edited

Loading

Zeitsperre left a comment

cehbrecht commented Feb 9, 2021

tlvu commented Feb 10, 2021

tlvu commented Feb 10, 2021

tlvu commented Feb 10, 2021

fmigneault commented Feb 11, 2021

cehbrecht commented Feb 11, 2021

cehbrecht commented Feb 11, 2021

tlvu commented Feb 11, 2021

tlvu commented Feb 11, 2021 •

edited

Loading

cehbrecht commented Feb 11, 2021

cehbrecht commented Feb 11, 2021

tlvu commented Feb 11, 2021

Backport master to 4.4 #574

Backport master to 4.4 #574

Conversation

cehbrecht commented Feb 8, 2021

Overview

Related Issue / Discussion

Additional Information

Contribution Agreement

coveralls commented Feb 8, 2021 • edited Loading

cehbrecht commented Feb 8, 2021 • edited Loading

cehbrecht commented Feb 8, 2021

Zeitsperre commented Feb 8, 2021 • edited Loading

Zeitsperre left a comment

Choose a reason for hiding this comment

cehbrecht commented Feb 9, 2021

tlvu commented Feb 10, 2021

tlvu commented Feb 10, 2021

tlvu commented Feb 10, 2021

fmigneault commented Feb 11, 2021

cehbrecht commented Feb 11, 2021

cehbrecht commented Feb 11, 2021

tlvu commented Feb 11, 2021

tlvu commented Feb 11, 2021 • edited Loading

cehbrecht commented Feb 11, 2021

cehbrecht commented Feb 11, 2021

tlvu commented Feb 11, 2021

coveralls commented Feb 8, 2021 •

edited

Loading

cehbrecht commented Feb 8, 2021 •

edited

Loading

Zeitsperre commented Feb 8, 2021 •

edited

Loading

tlvu commented Feb 11, 2021 •

edited

Loading