Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move harvestingclient_id from the dataset to dvobject and use it directly for files #9686

Open
cmbz opened this issue Jun 30, 2023 · 11 comments · Fixed by #10214
Open

Move harvestingclient_id from the dataset to dvobject and use it directly for files #9686

cmbz opened this issue Jun 30, 2023 · 11 comments · Fixed by #10214
Assignees
@sekmiller
Labels
Feature: Harvesting NIH OTA: 1.4.1 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 prdOwnThis is an item synched from the product ... pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues pm.GREI-d-2.4.1B NIH AIM:4 YR:2 TASK:1B | 2.4.1B | (started yr1) Resolve OAI-PMH harvesting issues Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) Status: Needs Input Applied to issues in need of input from someone currently unavailable Type: Suggestion an idea

Comments

@cmbz
Copy link

cmbz commented Jun 30, 2023

Description

  • Move the column harvestingclient_id from the dataset table in the database to the common dvobject.
  • This decision arrived at during a tech hour where it was decided not to redesign the concept of a harvested file, but instead make it easier to distinguish between harvested vs. 'real' files. See comment:
    Revisit/reimplement the concept of a "Harvested file".  #8629 (comment)

Any open or closed issues related to this feature request?
@cmbz cmbz added pm.GREI-d-2.4.1B NIH AIM:4 YR:2 TASK:1B | 2.4.1B | (started yr1) Resolve OAI-PMH harvesting issues Feature: Harvesting NIH OTA: 1.4.1 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 prdOwnThis is an item synched from the product ... pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues labels Jun 30, 2023
@cmbz cmbz mentioned this issue Jun 30, 2023
Closed
3 tasks
@cmbz
Copy link
Author

cmbz commented Jun 30, 2023

Moved to @siacus backlog column to expedite lingering NIH GREI Year 1 work.

@cmbz cmbz added this to the 6.1 milestone Sep 25, 2023
@cmbz
Copy link
Author

cmbz commented Sep 25, 2023

2023/09/25: Added to 6.1 milestone as per conversation during prioritization meeting.

@pdurbin pdurbin added the Type: Suggestion an idea label Oct 12, 2023
@cmbz
Copy link
Author

cmbz commented Oct 23, 2023

2023/10/23: Sized at 10 during sizing meeting.

@cmbz cmbz added the Size: 10 A percentage of a sprint. 7 hours. label Oct 23, 2023
@sekmiller sekmiller self-assigned this Nov 15, 2023
sekmiller added a commit that referenced this issue Nov 16, 2023
@sekmiller
#9686 preliminary check in
74d36b6
sekmiller added a commit that referenced this issue Nov 16, 2023
@sekmiller
#9686 rename migration script
5c04512
sekmiller added a commit that referenced this issue Nov 20, 2023
@sekmiller
#9686 update metrics queries
5ecfd49
sekmiller added a commit that referenced this issue Nov 20, 2023
@sekmiller
#9686 update metrics IT
f69c229
sekmiller added a commit that referenced this issue Nov 27, 2023
@sekmiller
#9686 fix metrics service bean
0cbdb1c
@cmbz cmbz added Size: 3 A percentage of a sprint. 2.1 hours. and removed Size: 10 A percentage of a sprint. 7 hours. labels Nov 27, 2023
sekmiller added a commit that referenced this issue Nov 27, 2023
@sekmiller
#9686 rename migration script
6ce88a9
@sekmiller sekmiller mentioned this issue Nov 27, 2023
Closed
sekmiller added a commit that referenced this issue Nov 30, 2023
@sekmiller
#9686 changes from CR
a6fd3f5
@scolapasta scolapasta changed the title Move the column harvestingclient_id from the dataset table in the database to the common dvobject Move harvestingclient_id from the dataset to dvobject and use it directly for files Nov 30, 2023
@scolapasta
Copy link
Contributor

The initial PR for this looks good and does what the issue states, that is move the column from one table to another (and modified the code which cares about where this info is for datasets appropriately).

However, while reviewing @landreev, @sekmiller and I agreed it would be better if the scope of this issue also encompassed making the relevant changes to the harvesting code to use the moved column for files, that being the motive for this new column. (note: we debated merging the PR and creating a new issue, but in this case it makes more sense to just keep the issue as is intact).

So I'm closing the PR and moving this issue to 6.2. With the new scope, it will also need to be resized.

@scolapasta scolapasta removed the Size: 3 A percentage of a sprint. 2.1 hours. label Nov 30, 2023
@scolapasta scolapasta removed this from the 6.1 milestone Nov 30, 2023
@scolapasta scolapasta added this to the 6.2 milestone Nov 30, 2023
@scolapasta
Copy link
Contributor

Also, note, this branch:

https://github.com/IQSS/dataverse/tree/9686-move-harvesting-client-id

should be the starting point to continue this work.

@cmbz
Copy link
Author

cmbz commented Dec 18, 2023 •
edited
Loading

2023/12/18

  • Sized at a 33 during the sizing meeting.

@cmbz cmbz added Size: 10 A percentage of a sprint. 7 hours. Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) and removed Size: 10 A percentage of a sprint. 7 hours. labels Dec 18, 2023
sekmiller added a commit that referenced this issue Jan 4, 2024
@sekmiller
#9686 update migration script
2ab5ba9
sekmiller added a commit that referenced this issue Jan 5, 2024
@sekmiller
#9686 assign harvest client id to harvested files
d017bf6
@sekmiller sekmiller mentioned this issue Jan 5, 2024
Merged
sekmiller added a commit that referenced this issue Jan 9, 2024
@sekmiller
#9686 add migration to harvested files
291811e
sekmiller added a commit that referenced this issue Jan 9, 2024
@sekmiller
#9686 add a release note
03f4a06
sekmiller added a commit that referenced this issue Jan 23, 2024
@sekmiller
#9686 display harvesting client info on cards of harvested objects
a28e15a
sekmiller added a commit that referenced this issue Jan 23, 2024
@sekmiller
#9686 fix script names
88bae3b
@sekmiller
Copy link
Contributor

Re-opening while discussions on how to proceed are ongoing

@sekmiller sekmiller reopened this Jan 25, 2024
@pdurbin pdurbin removed this from the 6.2 milestone Jan 27, 2024
@cmbz
Copy link
Author

cmbz commented Jan 31, 2024 •
edited
Loading

2024/01/31

  • Adding waiting tag for now. Awaiting additional team input.

@cmbz cmbz added the Status: Needs Input Applied to issues in need of input from someone currently unavailable label Jan 31, 2024
@cmbz cmbz mentioned this issue Mar 12, 2024
Open
35 tasks
@cmbz
Copy link
Author

cmbz commented Mar 13, 2024

2024/03/13
@landreev will discuss at a future Tech Hours and decide what to do about this specific issue (e.g., close/keep open)

@cmbz
Copy link
Author

cmbz commented Jul 10, 2024

2024/07/10

@pdurbin
Copy link
Member

pdurbin commented Jul 11, 2024

I don't know if it helps or confuses things but @poikilotherm and I were talking about it a bit in Slack yesterday: https://iqss.slack.com/archives/C010LA04BCG/p1720617633778099

@poikilotherm poikilotherm mentioned this issue Jul 14, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sekmiller sekmiller

Labels
Feature: Harvesting NIH OTA: 1.4.1 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 prdOwnThis is an item synched from the product ... pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues pm.GREI-d-2.4.1B NIH AIM:4 YR:2 TASK:1B | 2.4.1B | (started yr1) Resolve OAI-PMH harvesting issues Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) Status: Needs Input Applied to issues in need of input from someone currently unavailable Type: Suggestion an idea
Projects
IQSS Dataverse Project
Status: On Hold ⌛
Recherche Data Gouv
Status: 🔍 Interest
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

9686 move harvesting client IQSS/dataverse
9686 move harvesting client id IQSS/dataverse
4 participants
@pdurbin @sekmiller @scolapasta @cmbz