Update the codemeta metadata block to add some more structure for machine actionability

[doigl]

What this PR does / why we need it:
Actually, the fields MemoryRequirements and ProcessorRequirements and StorageRequirements are just free text fields, what makes it difficult to use them in an automated process to provide the right resources for running a jupyter notebook or a container. Adding subfields to these fields with controlled vocabularies would it make it easier to differentiate between different types and identify the right amount of resources like memory.

In addition to these changes, this pull request also adds new subfields for softwareRequirements and softwareSuggestions to distinguish between InfoUrl (documentation page) and URL (DownloadURL) for dependencies, and adjusts the termURI of contIntegration to contiousIntegration (codemeta v3.0)

Which issue(s) this PR closes:

• Closes Suggestion: Update the CodeMeta Metadata Block to add some more structure for machine actionability #10859

Special notes for your reviewer:

This pull requests only changes codemeta.tsv file. As it introduces new subfields for the metadata fields storageRequirements and memoryRequirements (that were simple fields and no compound fields before), existing metadata in these fields have to be migrated (manually?).

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Metadata form:

Rendered metadata:

Is there a release notes update needed for this change?:
Yes, this should be mentioned in the release notes- if applied - together with a description how to migrate existing metadata in the changed fields.

Additional documentation:

…d storageRequirements, added new subfield for softwareRequirements and softwareSuggestions to distinguish between InfoUrl (documentation page) and URL (DownloadURL) for dependencies, adjusted termURI of contIntegration to codemeta v3.0 in codemeta.tsv

What this PR does / why we need it:

Which issue(s) this PR closes:

• Closes #

Special notes for your reviewer:

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Additional documentation:

[pdurbin]

Overall, this looks like a nice incremental improvement. @doigl I'm leaving some specific feedback below. Thanks!

[pdurbin]

I'm confused. Should this be 3.0 instead of 2.0? Or are we not there yet?

Also, is it important to remove the "v"? In c5a6a8f @poikilotherm added a "v" to src/main/java/propertyFiles/codeMeta20.properties.

[doigl]

Is addressed, thanks for pointing out. The name of the metadatablock is still codeMeta20, but the displayName is now "Software Metadata (CodeMeta v3.0)"

[pdurbin]

Thanks for fixing the display name.

But what about the name? Is the plan to stick with "codeMeta20" forever? Or will we someday switch to "codeMeta30" or "codeMeta40"? Should we consider shortening it to just "codeMeta" (or "codemeta")?

[doigl]

I would be in favour of renaming the block to codemeta, but then, we really need detailed upgrade instructions to avoid a dublication of the metadatablock
@poikilotherm what was the original reason to include the version into the block name?

[pdurbin]

These are just some screenshots as of 28ff8ed for my reference or others interested in the PR. Overall, the fields look good to me.

displayOnCreate fields

all fields

[pdurbin]

I'm just going to add this comment here at the top, but yes, a release note should be added. Please see https://guides.dataverse.org/en/6.5/developers/version-control.html#writing-release-note-snippets

@doigl I noticed you wrote this: "As it introduces new subfields for the metadata fields storageRequirements and memoryRequirements (that were simple fields and no compound fields before), existing metadata in these fields have to be migrated (manually?)."

Do you plan to provide an SQL upgrade script? If not, perhaps tell people they are on their own since the block is experimental? 🤔 🤷

[doigl]

@pdurbin I added a release note, but so far not with an upgrade script. This SQL-statement
select dvo.identifier, dt.name as name, dfv.value as val from datasetfield as df, datasetfieldtype as dt, datasetfieldvalue as dfv, dvobject as dvo, datasetversion as dv where df.id = dfv.datasetfield_id and df.datasetfieldtype_id = dt.id and dvo.id = dv.dataset_id and df.datasetversion_id = dv.id and name IN ('memoryRequirements', 'storageRequirements');
identifies the datasets with values in memoryRequirements and storageRequirements and the following:
select upper(substring(dfv.value from '[kmgtpKMGTP][Bb]')) as unit, substring (dfv.value from '\d{1,4}') as numb_val, upper(substring(dfv.value from 'RAM|Ram|ram|GPU|Gpu|gpu|NPU|Npu|npu')) as ramtype, dvo.identifier, dt.name as name, dfv.value as val from datasetfield as df, datasetfieldtype as dt, datasetfieldvalue as dfv, dvobject as dvo, datasetversion as dv where df.id = dfv.datasetfield_id and df.datasetfieldtype_id = dt.id and dvo.id = dv.dataset_id and df.datasetversion_id = dv.id and name IN ('memoryRequirements', 'storageRequirements'); extracts the information for the subfields.

But I'm a bit hesistant (and perhaps just not experienced enough to really dare), to try to automatically generate the insert statements for the subfields. For our installation I would perhaps rather try to add theses by a script using the API.

What do you mean? Have you had similar changes in metadata blocks before and a good way how to handle this?

[pdurbin]

I can understand your hesitancy. I just made this PR to suggest some changes to the release note:

• improve release note TIK-NFL/dataverse#25

I like the idea of at least showing people which datasets are affected, so I copied that part of the SQL into the note. Actually, I just realized there are two. Maybe you can add that?

[pdurbin]

@doigl I see you incorporated ideas form the PR 25 into 415be06 so I'll resolve this and move the PR into "ready for QA". Thanks!

[pdurbin]

Some more feedback.

[pdurbin]

I can understand your hesitancy. I just made this PR to suggest some changes to the release note:

• improve release note TIK-NFL/dataverse#25

I like the idea of at least showing people which datasets are affected, so I copied that part of the SQL into the note. Actually, I just realized there are two. Maybe you can add that?

[pdurbin]

I haven't done much testing but I think this is ready for QA. Approved.