Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: extract config for finngen ingestion to yaml file #15

Merged
merged 4 commits into from
Sep 9, 2024

Conversation

project-defiant
Copy link
Collaborator

Context

Move all configuration of the finngen_ingestion dag to the separate config.
This dag will has it's own lifecycle depending on the finngen data releases, so it's a good idea to have it run on demand only when requred and cache the produced results:

  • finngen credible sets (ingested by gentropy StudyLocus class)
  • finngen study index (ingested by gentropy StudyIndex class)
    inside the staging gcs path gs://finngen_data/ with appropriate release prefix (r11)

Implementations

  • extracted constants from the finngen_ingestion.py to finngen_ingestion.yaml config file.
  • Explicitly declared parameters that were used as default in gentropy finngen steps.
  • Renamed dag

@project-defiant project-defiant force-pushed the szsz-finngen-extract-configuration branch from c2b7cb1 to 6318cc2 Compare September 9, 2024 10:19
@project-defiant
Copy link
Collaborator Author

@javfg ready for 🥊 2

@javfg
Copy link
Member

javfg commented Sep 9, 2024

All good this time around!

@project-defiant project-defiant merged commit e8ff21b into dev Sep 9, 2024
2 checks passed
@project-defiant project-defiant deleted the szsz-finngen-extract-configuration branch September 9, 2024 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants