-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add secure Geoserver access #242
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for submitting a PR.
Make sure you have looked at CONTRIBUTING guidelines.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that it's a non-breaking change and that it's goal is to test the geoserver services protection I don't see any issue in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would like to ask for a favor, could you make this change an optional component (under https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components dir)?
The reason is I already planned to eventually break out every components into separate components to allow for each org to enable only the parts they need so I'd rather have any new changes aligned with the future plan.
You can look into https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components/generic_bird for sample how it's done. It has hooks into the front Nginx proxy and Magpie as well so it's a pretty complete example to based upon to save you time.
Document your new optional component the README.rst at the root of the optional-components/
dir.
Also, do not forget to prepare the https://github.com/bird-house/birdhouse-deploy/blob/master/CHANGES.md file so the PR is complete for eventual release.
The full release procedure is documented here https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse#release-procedure
Sorry for the last minute refactoring request. DavidB has briefly mentioned this upcoming feature during our previous bi-weekly but I forgot to inquire about the implementation whether it will be a "component".
Will be implementing as optional component
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/885/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/886/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/887/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/888/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/889/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh sorry I did not see you had done the optional-component.
Everything looks good, one minor thing missing is updating the doc of the optional-components https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/optional-components/README.rst to list and describe your component.
See many example there for the format.
Should do this after merging with master. I think FrancisM has added a new optional-component lately as well.
@tlvu Just to be sure, since I don't do merges in this repo often, all that's left is to run bumpversion before merging the pull request? |
https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/README.rst#release-procedure (added recently after you opened this PR so you might not be aware). You are basically at this last step "Only when you are ready to merge the PR immediately, ...". Follow from there. If something unclear, let me know. |
@francisPLT |
Yes, I just saw that, working on it |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/996/Result : success BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-216.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/808/NOTEBOOK TEST RESULTS |
@@ -242,7 +242,7 @@ PLATFORMS = { | |||
'info': { | |||
'name': 'PAVICS', | |||
'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.', | |||
'version': '1.21.1', | |||
'version': '1.22.0', | |||
'releaseTime': '2022-08-24T15:43:51Z', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fmigneault release time did not update. Is there a special option we need to run bumpversion
with, for release time to update? If yes, we need to doc this special option. I just checked one of my previous release, the release time did not update for me as well. Or need a special version of bumpversion
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like the portion https://github.com/bird-house/birdhouse-deploy/blob/master/.bumpversion.cfg#L28-L29 cannot update itself because bumpversion rewrites the .bumpversion.cfg
file after applying changes using what it initially read. So the update works the first time when datetime match, but fail on following cases because the new datetime was not applied.
I think we could hack around this using a separate file that contains only the release time. Bumpversion could modify that file without problem. It would need to be mounted in the docker image for the python config to read it.
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/997/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
@francisPLT FYI, the tag should be on the merge commit created by merging this PR, not on the bump version commit. It's okay, just for next time. |
@francisPLT Just to be clear, do not touch the tag at the wrong location anymore (FrancisM said to remove the tag at the wrong location and redo it but I've merged this to our production fork already and sync the tags, so too late to fix the wrong location of the tag). |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1000/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1001/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
@francisPLT Will need to apply this new component to |
## Overview Because `bumpversion` cannot update its own `.bumpversion.cfg` file to apply the new `releaseTime` value (since it rewrites the config with its own config handler once changes were applied), use a workaround to produce the same effect. Use Makefile to parse the current release time, then run `bumpversion` as normal, retrieve the new release time that was applied by `bumpversion`, then replace the old->new release time in the config for future calls. To test, a few bumps were done (note: second tag on the line is the "active version" on that commit) ``` * eb537ef Bump version: 1.22.3 → 1.22.4 * f41fb55 Bump version: 1.22.2 → 1.22.3 * 81470bd Bump version: 1.22.1 → 1.22.2 ``` The diffs are as follows, showing correct behaviour to replace new `releaseTime` everywhere: <details><summary> diff 1.22.2 → 1.22.3 </summary> <p> ```diff diff --git a/.bumpversion.cfg b/.bumpversion.cfg index 36e736a..8a68e80 100644 --- a/.bumpversion.cfg +++ b/.bumpversion.cfg @@ -1,23 +1,23 @@ [bumpversion] -current_version = 1.22.2 +current_version = 1.22.3 commit = True tag = False tag_name = {new_version} parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+) (?P<releaseTime>.*)? -serialize = +serialize = {major}.{minor}.{patch} {utcnow:%Y-%m-%dT%H:%M:%SZ} [bumpversion:file:CHANGES.md] -search = +search = [Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest) ------------------------------------------------------------------------------------------------------------------ -replace = +replace = [Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest) ------------------------------------------------------------------------------------------------------------------ - + [//]: # (list changes here, using '-' for each new entry, remove this when items are added) - + [{new_version}](https://github.com/bird-house/birdhouse-deploy/tree/{new_version}) ({utcnow:%Y-%m-%d}) ------------------------------------------------------------------------------------------------------------------ @@ -30,11 +30,11 @@ search = {current_version} replace = {new_version} [bumpversion:file:RELEASE.txt] -search = {current_version} 2022-08-25T03:24:48Z +search = {current_version} 2022-08-25T03:27:35Z replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ} [bumpversion:part:releaseTime] -values = 2022-08-25T03:24:48Z +values = 2022-08-25T03:27:35Z [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template] search = 'version': '{current_version}' diff --git a/CHANGES.md b/CHANGES.md index 5a0f6f9..357ccfc 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -16,6 +16,11 @@ [//]: # (list changes here, using '-' for each new entry, remove this when items are added) +[1.22.3](https://github.com/bird-house/birdhouse-deploy/tree/1.22.3) (2022-08-25) +------------------------------------------------------------------------------------------------------------------ + +[//]: # (list changes here, using '-' for each new entry, remove this when items are added) + [1.22.2](https://github.com/bird-house/birdhouse-deploy/tree/1.22.2) (2022-08-25) ------------------------------------------------------------------------------------------------------------------ diff --git a/Makefile b/Makefile index dbe3146..35367e9 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # Generic variables override SHELL := bash override APP_NAME := birdhouse-deploy -override APP_VERSION := 1.22.2 +override APP_VERSION := 1.22.3 # utility to remove comments after value of an option variable override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g") diff --git a/README.rst b/README.rst index 8b79614..533d425 100644 --- a/README.rst +++ b/README.rst @@ -14,13 +14,13 @@ for a full-fledged production platform. * - releases - | |latest-version| |commits-since| -.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.2.svg +.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.3.svg :alt: Commits since latest release - :target: 1.22.2...master + :target: 1.22.3...master -.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.2-blue.svg?style=flat +.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.3-blue.svg?style=flat :alt: Latest Tag - :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.2 + :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.3 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest :alt: ReadTheDocs Build Status (latest version) diff --git a/RELEASE.txt b/RELEASE.txt index bc149ee..c7dc42e 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -1 +1 @@ -1.22.2 2022-08-25T03:24:48Z +1.22.3 2022-08-25T03:27:35Z diff --git a/birdhouse/config/canarie-api/docker_configuration.py.template b/birdhouse/config/canarie-api/docker_configuration.py.template index 111582f..7b37a80 100644 --- a/birdhouse/config/canarie-api/docker_configuration.py.template +++ b/birdhouse/config/canarie-api/docker_configuration.py.template @@ -17,8 +17,8 @@ SERVICES = { 'info': { 'name': 'Node', 'synopsis': 'Nodes are data, compute and index endpoints accessed through the PAVICS platform or external clients. The Node service is the backend that allows: data storage, harvesting, indexation and discovery of local and federated data; authentication and authorization; server registration and management. Node service is therefore composed of several other services.', - 'version': '1.22.2', - 'releaseTime': '2022-08-25T03:24:48Z', + 'version': '1.22.3', + 'releaseTime': '2022-08-25T03:27:35Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', @@ -242,8 +242,8 @@ PLATFORMS = { 'info': { 'name': 'PAVICS', 'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.', - 'version': '1.22.2', - 'releaseTime': '2022-08-25T03:24:48Z', + 'version': '1.22.3', + 'releaseTime': '2022-08-25T03:27:35Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', ``` </p> </details> <details><summary> diff 1.22.3 → 1.22.4 </summary> <p> ```diff diff --git a/.bumpversion.cfg b/.bumpversion.cfg index 8a68e80..f0638ae 100644 --- a/.bumpversion.cfg +++ b/.bumpversion.cfg @@ -1,5 +1,5 @@ [bumpversion] -current_version = 1.22.3 +current_version = 1.22.4 commit = True tag = False tag_name = {new_version} @@ -30,11 +30,11 @@ search = {current_version} replace = {new_version} [bumpversion:file:RELEASE.txt] -search = {current_version} 2022-08-25T03:27:35Z +search = {current_version} 2022-08-25T03:27:50Z replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ} [bumpversion:part:releaseTime] -values = 2022-08-25T03:27:35Z +values = 2022-08-25T03:27:50Z [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template] search = 'version': '{current_version}' diff --git a/CHANGES.md b/CHANGES.md index 357ccfc..467ee61 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -16,6 +16,11 @@ [//]: # (list changes here, using '-' for each new entry, remove this when items are added) +[1.22.4](https://github.com/bird-house/birdhouse-deploy/tree/1.22.4) (2022-08-25) +------------------------------------------------------------------------------------------------------------------ + +[//]: # (list changes here, using '-' for each new entry, remove this when items are added) + [1.22.3](https://github.com/bird-house/birdhouse-deploy/tree/1.22.3) (2022-08-25) ------------------------------------------------------------------------------------------------------------------ diff --git a/Makefile b/Makefile index 35367e9..c374b08 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # Generic variables override SHELL := bash override APP_NAME := birdhouse-deploy -override APP_VERSION := 1.22.3 +override APP_VERSION := 1.22.4 # utility to remove comments after value of an option variable override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g") diff --git a/README.rst b/README.rst index 533d425..e74149c 100644 --- a/README.rst +++ b/README.rst @@ -14,13 +14,13 @@ for a full-fledged production platform. * - releases - | |latest-version| |commits-since| -.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.3.svg +.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.4.svg :alt: Commits since latest release - :target: 1.22.3...master + :target: 1.22.4...master -.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.3-blue.svg?style=flat +.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.4-blue.svg?style=flat :alt: Latest Tag - :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.3 + :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.4 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest :alt: ReadTheDocs Build Status (latest version) diff --git a/RELEASE.txt b/RELEASE.txt index c7dc42e..902af90 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -1 +1 @@ -1.22.3 2022-08-25T03:27:35Z +1.22.4 2022-08-25T03:27:50Z diff --git a/birdhouse/config/canarie-api/docker_configuration.py.template b/birdhouse/config/canarie-api/docker_configuration.py.template index 7b37a80..536a706 100644 --- a/birdhouse/config/canarie-api/docker_configuration.py.template +++ b/birdhouse/config/canarie-api/docker_configuration.py.template @@ -17,8 +17,8 @@ SERVICES = { 'info': { 'name': 'Node', 'synopsis': 'Nodes are data, compute and index endpoints accessed through the PAVICS platform or external clients. The Node service is the backend that allows: data storage, harvesting, indexation and discovery of local and federated data; authentication and authorization; server registration and management. Node service is therefore composed of several other services.', - 'version': '1.22.3', - 'releaseTime': '2022-08-25T03:27:35Z', + 'version': '1.22.4', + 'releaseTime': '2022-08-25T03:27:50Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', @@ -242,8 +242,8 @@ PLATFORMS = { 'info': { 'name': 'PAVICS', 'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.', - 'version': '1.22.3', - 'releaseTime': '2022-08-25T03:27:35Z', + 'version': '1.22.4', + 'releaseTime': '2022-08-25T03:27:50Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', ``` </p> </details> ## Changes **Non-breaking changes** - Adds Makefile to run basic DevOps maintenance commands on the repository - Adds RELEASE.txt with the active release tag and datetime - Replace `now:` directives by `utcnow:` to report time properly according to employed ISO format **Breaking changes** - n/a ## Related Issue / Discussion - Resolves problem described here: #242 (comment)
Overview
I added a new provider and location as an optional component,
test-geoserver-secured-access
, in order to test secured access to Geoserver before we completely move Geoserver behind Twitcher (undetermined date).The old
/geoserver
path is still available, so current workflows should not be affected.The new
/geoserver-secured
path is now available for testing once the optional component is activated.To test the
geoserver-secured
service through Magpie, each workspace needs to be added to the new service and then permissions can be set on a per-workspace or even layer basis.Service configuration:
Ex: for the Anonymous Group configuration:
From the above example, a
GetFeature
request for a layer in thepublic
workspace will succeed for any user using any of these two request types:Whereas access to the
private
workspace will require a user or group be given explicit permissions.Mind you, I'm not an expert with Nginx, so I based these changes from the existing configurations.
Changes
Non-breaking changes
geoserver-secured
/geoserver-secured
Breaking changes
Related Issue / Discussion
Future considerations
{BASE_URL}/geoserver/web
). I thought I had found a way a quick way to manage permissions to it, but further testing proved me wrong. All the internal redirections while navigating the web console will probably create issues, and the UI itself did not appreciate the proxy/redirect. I have not looked into this further for the time being.