Upgrade all harvest sources to DCATUS v1.1

[rshewitt]

User Story

In order to ensure harvest sources conform to DCATUS v1.1, datagov wants to upgrade DCATUS v1.0 harvest sources to v1.1.

Acceptance Criteria

• GIVEN the existing 15 packages using DCATUS v1.0 on catalog
  WHEN an upgrade to v1.1 happens
  THEN no more packages use v1.0.

Background

• we aren't sure we want/need to support DCAT1.0 in the new harvester.
• currently, there's 15 harvest sources which use v1.0, all of them were harvested in 2020, and by the same organization "U.S. Department of Health & Human Services"
• there's a ticket to redo and improve what is effectively the import stage of the datajson ckan extension as a new function in the harvester 2.0 pipeline.
  • by upgrading all harvest sources to v1.1 we can omit any functionality in the import stage function which references v1.0.
  • considering the last harvest date for these packages was in 2020 it could suggest the v1.0 functionality isn't being used.

Security Considerations (required)

none

Sketch

• review v1.1 update resource, field mappings, changelog.
• identify fields/data needing to be added/updated.
• update packages to conform to new schema.

[btylerburton]

would this be done by engaging with the agency who's hosting these?

[jbrown-xentity]

Possibly. We could run it through validation and see if it passes, if it does we're probably ok to just change the harvest source type. When it was harvested, that was when we cut over from CKAN2.3 to 2.8, everything was reharvested in Nov2020-Jan2021 I believe. So it's probably that no one has looked at the harvest source in a really long time, so not sure if there's anyone to work with there or not. Also not sure if this was a one-time harvest and the source is gone, or if it really hasn't been touched in 3 years...

[rshewitt]

we intend to use v1.1 moving forward. holding off on upgrading the existing datasets using v1.0.