7000 mpconfig pid doi hdl #8828

poikilotherm · 2022-06-30T21:28:53Z

What this PR does / why we need it:
This is a larger cleanup of many PID provider settings.

It decouples the settings from different provider from each other by adding new names and scopes.
It will reuse existing configuration values to maintain backward compatibility!
The docs have been consolidated and changed as necessary to line out more of the differences and which things are important for what
Sane defaults have been incorporated, pointing to test and non-production resources

Relates to #7000

Which issue(s) this PR closes:
None.

Special notes for your reviewer:
This should be done but might need discussion if we want changes to the installer. (Password aliases etc)

Missing Handle refactoring
Missing a release notes file explaining the changes
Maybe change installer to use new configuration names
HandleNet properties need aliases for old names!
Add tests for aliases to be functional (linking not testable before as no aliases)
Maybe extend like done for Solr (7000 mpconfig solr #8825) to enable protocol, provider, authority choose etc via MPCONFIG in backward compatible manner? (Much easier config for containers and those settings are VERY static - these shall never change for an installation or you'll be in trouble. Database Settings were introduced for dynamically changing settings? 🤔 ) Too large for this PR.

Suggestions on how to test this:
Play around with the settings described in the docs section and watch it work. It's not much of a change despite the renaming.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Not really.

Is there a release notes update needed for this change?:
Not included. Let me know if we should write one now or latter (more mpconfig stuff coming). Technically, as the settings have aliases, nothing should change. There is no breaking change, just deprecated names.

Additional documentation:
Included.

poikilotherm · 2022-08-22T20:52:25Z

Good catch by @landreev - the HandleNet properties are missing their aliases!

…CONFIG IQSS#7000 - Add scopes and settings to JvmSettings for strictness and fast retrieval with MPCONFIG - Rename settings to live under scope "dataverse.pid.ezid|datacite", so they are more aligned to our other settings. This could change again in the future, depending on what other devs feel where they should be located. - Add aliases to make settings backward compatible - in the current state of Dataverse code, only one PID provider can be used at the same time and the double alias is with the deprecated, non-public EZID provider. - Replace SystemConfig.getDataciteRestApiUrlString() with aliases and a default in microprofile-config.properties - Inline with the current setup and docs, make the DataCite MDS API URl default to the Test Fabrica, mds.test.datacite.org

Test execution in PersistentIdentifierServiceBeanTest requires to construct an EZID provider. The providers constructor tries to login with the service, which is not relevant for the test. Providing EZID configuration settings via a src/test/resource/META-INF/microprofile-config.properties file makes the warnings go away about missing config options and also avoids sending requests to the real service for every test execution.

- Refactoring the DOI related JVM settings and cross links - Updated some facts and external links in their descriptions

Add necessary changes to lookup Handle.Net PID provider details via MicroProfile instead of system properties. The options where scoped within JvmSettings and renamed for better readability.

- Documented the renaming and option to retrieve the data via MPCONFIG - Added cross links to other sections - Added references to tech docs of Handle.Net - Add more context information to the three settings

…7000

coveralls · 2022-09-19T15:25:52Z

Coverage: 20.184% (+0.002%) from 20.182% when pulling b7d64df on poikilotherm:7000-mpconfig-pid-doi-hdl into 3a5883c on IQSS:develop.

This adds the missing MPCONFIG option to configure the base URL for the permalink provider, which has been added after the initial work for providers happened.

This relates to IQSS#9486, but with the switch of the properties to come from MPCONFIG, this is much easier to do. We do create the provider object in any case, because otherwise the stateless EJB mechanism will be very unhappy. We do not login if there is no password and username given, so avoid the exceptions. We only log a warning if only one of both is given (which is clearly an error).

poikilotherm · 2023-03-31T16:50:17Z

I'm putting this back on the table. @mreekie automation dropped this into "Ready for review", could you move it to the sprint column? Thx! (I am not allowed to do this for some unclear reason...?)

About sizing: this does not need a resizing, it's still a 10.

The overall important added stuff was that Permalink provider is now configurable via MPCONFIG as well as the others.
And the EZID provider is created in any case, but no login is attempted if no username and password is given. This was not possible before because we reused the same property name for all places. As this is changed now, it can properly detect if to attempt login or not. The rest are some cosmetic changes.

mreekie · 2023-03-31T17:38:36Z

I'm putting this back on the table. @mreekie automation dropped this into "Ready for review", could you move it to the sprint column? Thx! (I am not allowed to do this for some unclear reason...?)

About sizing: this does not need a resizing, it's still a 10.

The overall important added stuff was that Permalink provider is now configurable via MPCONFIG as well as the others. And the EZID provider is created in any case, but no login is attempted if no username and password is given. This was not possible before because we reused the same property name for all places. As this is changed now, it can properly detect if to attempt login or not. The rest are some cosmetic changes.

@poikilotherm This is fine. Let's leave it be.

pdurbin

@poikilotherm great work on refactoring and cleaning up the code. Off to QA!

@kcondon the code looks good but please advised that I only did the lightest of testing, just check the defaults for the FAKE and perma providers. Both worked just fine as shown in the screenshot below. I did not touch DataCite, Handle, or EZID.

pdurbin · 2023-03-31T21:07:46Z

src/main/java/edu/harvard/iq/dataverse/api/Pids.java

+        // FIXME: Even before changing to MPCONFIG retrieval, this was pinned to be DataCite specific!
+        //        Should this be extended to EZID and other PID systems like Handle?
+        String baseUrl = JvmSettings.DATACITE_REST_API_URL.lookup();
+        String username = JvmSettings.DATACITE_USERNAME.lookup();
+        String password = JvmSettings.DATACITE_PASSWORD.lookup();


This is fair. I added the PIDs API but hard coded it to DataCite. I would like to see it evolve to be useful across multiple PID providers.

kcondon · 2023-04-03T19:03:43Z

Issues found:

Existing, known good handle config fails with ui error and stack trace: (fixed)
hdl_create_err.txt
Seems like it really needs new hdl passphrase property. I am not using one for test/qa.

This avoids a "NoSuchElementException" caused by a failing lookup. As we do a check for the necessity of a decryption anyway, this is the logical place to require the presence of the key.

poikilotherm · 2023-04-04T07:19:12Z

@kcondon I fixed this with b7d64df, only looking up the passphrase if it is required. Hope that helps 😄

poikilotherm · 2023-04-05T05:43:35Z

Woo-hoo!!! And another one in! Thanks for testing in depth @kcondon and the positive review @pdurbin ! 🥳

This was referenced Jun 30, 2022

7000 mpconfig for JVM settings #8810

Closed

7000 mpconfig basic infra #8823

Merged

poikilotherm force-pushed the 7000-mpconfig-pid-doi-hdl branch 2 times, most recently from 2a58db7 to 1f04c0e Compare July 4, 2022 13:38

poikilotherm marked this pull request as ready for review July 4, 2022 17:00

This was referenced Aug 5, 2022

Broken link and description of DataCite Proxy no longer used by ADA #8899

Closed

Remove broken link and description of DataCite Proxy no longer used by ADA #8888

Merged

poikilotherm marked this pull request as draft August 22, 2022 20:51

poikilotherm added 7 commits September 19, 2022 17:16

docs(api): fix wrong position of PID API anchor

65d81dd

docs(settings): update DOI provider JVM options IQSS#7000

2969e34

- Refactoring the DOI related JVM settings and cross links - Updated some facts and external links in their descriptions

feat(settings): switch Handle.Net properties to MPCONFIG IQSS#7000

2b6d01d

Add necessary changes to lookup Handle.Net PID provider details via MicroProfile instead of system properties. The options where scoped within JvmSettings and renamed for better readability.

docs(settings): update config guide with new Handle options IQSS#7000

2c9f246

- Documented the renaming and option to retrieve the data via MPCONFIG - Added cross links to other sections - Added references to tech docs of Handle.Net - Add more context information to the three settings

docs(settings): add release note detail about MPCONFIG for PIDs IQSS#…

0db9648

…7000

poikilotherm force-pushed the 7000-mpconfig-pid-doi-hdl branch from 1f04c0e to 0db9648 Compare September 19, 2022 15:23

mreekie added the bk2211 label Nov 1, 2022

mreekie added the bklog: NeedsDiscussion label Mar 29, 2023

Merge branch 'develop' into 7000-mpconfig-pid-doi-hdl

f3969cf

poikilotherm force-pushed the 7000-mpconfig-pid-doi-hdl branch from a14ab28 to f3969cf Compare March 31, 2023 12:17

poikilotherm added 4 commits March 31, 2023 14:27

refactor(pid): make provider info generation use Java 11 List.of()

e2d50b1

feat(pid): enable permalink provider to use MPCONFIG IQSS#7000

2eab5f1

This adds the missing MPCONFIG option to configure the base URL for the permalink provider, which has been added after the initial work for providers happened.

doc(pid): rename permalink base url setting and add note about MPCONFIG

a4524f1

poikilotherm marked this pull request as ready for review March 31, 2023 16:46

doc(pid): add release note about MPCONFIG for PID providers

b96c9b6

pdurbin self-assigned this Mar 31, 2023

pdurbin added 2 commits March 31, 2023 16:55

list new settings in release note IQSS#7000

c92a635

various tweaks to PID provider docs IQSS#7000

b3b0b5d

pdurbin approved these changes Mar 31, 2023

View reviewed changes

pdurbin removed their assignment Mar 31, 2023

kcondon self-assigned this Apr 3, 2023

fix(pid): make Handle provider ask for passphrase only if necessary

b7d64df

This avoids a "NoSuchElementException" caused by a failing lookup. As we do a check for the necessity of a decryption anyway, this is the logical place to require the presence of the key.

kcondon merged commit 82cef32 into IQSS:develop Apr 4, 2023

pdurbin added this to the 5.14 milestone May 10, 2023

poikilotherm deleted the 7000-mpconfig-pid-doi-hdl branch October 2, 2024 06:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

7000 mpconfig pid doi hdl #8828

7000 mpconfig pid doi hdl #8828

poikilotherm commented Jun 30, 2022 •

edited

Loading

poikilotherm commented Aug 22, 2022

coveralls commented Sep 19, 2022 •

edited

Loading

poikilotherm commented Mar 31, 2023 •

edited

Loading

mreekie commented Mar 31, 2023

pdurbin left a comment

pdurbin Mar 31, 2023

kcondon commented Apr 3, 2023 •

edited

Loading

poikilotherm commented Apr 4, 2023

poikilotherm commented Apr 5, 2023 •

edited

Loading

7000 mpconfig pid doi hdl #8828

7000 mpconfig pid doi hdl #8828

Conversation

poikilotherm commented Jun 30, 2022 • edited Loading

poikilotherm commented Aug 22, 2022

coveralls commented Sep 19, 2022 • edited Loading

poikilotherm commented Mar 31, 2023 • edited Loading

mreekie commented Mar 31, 2023

pdurbin left a comment

Choose a reason for hiding this comment

pdurbin Mar 31, 2023

Choose a reason for hiding this comment

kcondon commented Apr 3, 2023 • edited Loading

poikilotherm commented Apr 4, 2023

poikilotherm commented Apr 5, 2023 • edited Loading

poikilotherm commented Jun 30, 2022 •

edited

Loading

coveralls commented Sep 19, 2022 •

edited

Loading

poikilotherm commented Mar 31, 2023 •

edited

Loading

kcondon commented Apr 3, 2023 •

edited

Loading

poikilotherm commented Apr 5, 2023 •

edited

Loading