Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not ingest Jamendo tracks that do not allow downloads #3530

Closed
stacimc opened this issue Dec 15, 2023 · 1 comment · Fixed by #3618
Closed

Do not ingest Jamendo tracks that do not allow downloads #3530

stacimc opened this issue Dec 15, 2023 · 1 comment · Fixed by #3618
Assignees
@stacimc
Labels
💻 aspect: code Concerns the software code in the repository ✨ goal: improvement Improvement to an existing user-facing feature 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@stacimc
Copy link
Collaborator

stacimc commented Dec 15, 2023

Problem

Related to #3499, see that discussion for additional context.

#3524 started tracking which Jamendo audio tracks do not allow downloads by storing the audiodownload_allowed field in the meta_data column. Once this information is populated, we can do the following:

  • Run the delete_records DAG to temporarily move records with audiodownload_allowed set to False to the DeletedAudio table
  • Update the Jamendo DAG to discard records with audiodownload_allowed set to False

This issue is blocked until a Jamendo DagRun has completed, in order to backfill the meta_data column so we can select the relevant tracks.
@stacimc stacimc added 🟨 priority: medium Not blocking but should be addressed soon ✨ goal: improvement Improvement to an existing user-facing feature 💻 aspect: code Concerns the software code in the repository ⛔ status: blocked Blocked & therefore, not ready for work 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Dec 15, 2023
@openverse-bot openverse-bot moved this to ⛔ Blocked in Openverse Backlog Dec 15, 2023
@AetherUnbound AetherUnbound mentioned this issue Dec 21, 2023
Closed
@stacimc stacimc self-assigned this Jan 2, 2024
@stacimc stacimc removed the ⛔ status: blocked Blocked & therefore, not ready for work label Jan 2, 2024
@stacimc
Copy link
Collaborator Author

stacimc commented Jan 2, 2024

The Jamendo DagRun completed. There are 22,122 records with audiodownload_allowed set to False.

@openverse-bot openverse-bot moved this from ⛔ Blocked to 📋 Backlog in Openverse Backlog Jan 2, 2024
@stacimc stacimc mentioned this issue Jan 2, 2024
Merged
8 tasks
@openverse-bot openverse-bot moved this from 📋 Backlog to 🏗 In Progress in Openverse Backlog Jan 2, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to ✅ Done in Openverse Backlog Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stacimc stacimc

Labels
💻 aspect: code Concerns the software code in the repository ✨ goal: improvement Improvement to an existing user-facing feature 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Openverse Backlog
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Do not ingest Jamendo records with downloads disabled WordPress/openverse
1 participant
@stacimc