-
Notifications
You must be signed in to change notification settings - Fork 490
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Harvesting #813
Comments
Assigning to @eaquigley to decide what exactly to do with this ticket. For 4.0 we have decided that we will support the infrastructure for harvested datasets (and have migrated harvested data), but not yet support new harvests. |
For issue discoverability: this issue concerns OAI-PMH, the Open Archives Initiatives Protocol for Metadata Harvesting. |
For consideration: ResourceSync is a set of protocols based on Sitemaps for publishing information about, links to or dumps of changed resources - both data and metadata. As it was developed by the people from the Open Archives Initiative, some consider this the new OAI-PMH. |
And ResourceSync was already mentioned in #900. |
@scolapasta @eaquigley please also keep in mind that we need to add a new Dataverse to our harvest per this request from the UBC Abacus Dataverse team. Should I create a separate ticket for this?
Ticket tracked in RT already: https://help.hmdc.harvard.edu/Ticket/Display.html?id=196381 |
@scolapasta @mcrosas @posixeleni do we have a list of all installations and others that we will be harvesting from? wanted to check due to @posixeleni comment above. also, @scolapasta should i have this ticket still or should it be going to someone else? we've already decided the behavior for displaying harvested materials (is searchable, has a search card with the harvested icon and text, will open in a new tab at the location where it is from) |
I just realised this issue is not about providing an OAI-PMH endpoint for dataset metadata. Or is it? Should another issue be opened for it? |
@bencomp yes, supporting harvesting includes supporting OAI-PMH for dataset metadata. |
One of our partners needs the OAI-PMH endpoint to be available to meet project responsibilities. Is there any new information about the milestone this will be part of? |
The plan for now is to do this for 4.3. We'll be reviewing 4.2 and 4.3 issues in this week and next (in our weekly "Issues review" meetings) and make a decision then. |
@landreev - I met with @eaquigley to review the Usability Testing Feedback and these items should be fixed for this release:
|
There was one serious bug found during the usability session that is NOT on this list: After creating a new, or editing and saving an existing client, clicking on the "edit" button resulted in an empty popup; the page would stay in that state until reloaded. |
@djbrooke |
Any chance we could drop number 4 from the list:
and just go on pretending it has never been brought up in the first place? Unless somebody could give me a single reason anyone would ever assume that this is NOT their local time, but, say, London or Central Australian time? |
Item 5:
it's more complex than just a "number of datasets" - it could be various numbers of datasets related to this set: "N number of datasets found; M already exported; K marked as deleted". But, I agree, "Statistics" is not the best term. I have changed it to just "Datasets" - would that be ok? |
Item number one:
I can do this is in about 5 seconds, most likely... but where is "navigation"? (edit: OK, just spoke to @mheppler about this; "... in the navigation" means make it one of the buttons in the upper right corner, next to "About", "Guides", "Support" and "Account") |
Number 3: This is the way I have resolved this:
However, it will still not be immediately obvious to the admin using the page, that the dataset failed to get harvested for this specific reason - because it's already present in another dataset. To find that, they would need to go and read the harvest log. ... You can consult the detailed harvesting log (found in ...) to learn more about the harvesting failures. For example, please note that an attempt to harvest datasets that are already present in another dataverse, will result in these datasets being skipped and listed as failed. (Deleting harvested datasets from dataverse A will make it possible to harvest them into dataverse B). (Alternatively, a new statistics field could be added - in addition to "harvested", "deleted" and "failed", something like "exist in another dataverse, skipped" - to be shown in the results immediately. If you feel it is necessary for this release, I could put it in place today) |
@mheppler @djbrooke I wasn't talking about not implementing these (with the exception of one, the time zone) If you feel that any of this is a low-hanging fruit that you could easily fix for 4.5, by all means do so. But I am strongly against actually holding the release on account of cosmetic changes, at this point. I've said this about a million times already, but one more time: this functionality is for admins only; these pages will be used by admins only, and very rarely - compared to the workhorse pages used by all the users every day. |
Status of my UI updates from the list above:
|
…ts/Server create/edit popups. [ref #813]
… help/text and labels to the bundle. [ref #813]
OK, looks good. Have completed baseline testing of Harvesting, functionality is delivered and issues found are opened in new individual tickets. Closing. |
In DVN 3.x we had harvesting: https://github.com/IQSS/dvn/blob/develop/doc/sphinx/source/dataverse-user-main.rst#harvesting-section
There should be an option to show federated dataverses in the featured dataverses UI developed in #750.
The text was updated successfully, but these errors were encountered: