Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup scripts/variantstore [VS-414] #7834

Merged
merged 1 commit into from
May 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .dockstore.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,17 +110,13 @@ workflows:
- name: GvsExtractCallset
subclass: WDL
primaryDescriptorPath: /scripts/variantstore/wdl/GvsExtractCallset.wdl
testParameterFiles:
- /scripts/variantstore/wdl/GvsExtractCallset.example.inputs.json
filters:
branches:
- master
- ah_var_store
- name: GvsImportGenomes
subclass: WDL
primaryDescriptorPath: /scripts/variantstore/wdl/GvsImportGenomes.wdl
testParameterFiles:
- /scripts/variantstore/wdl/GvsImportGenomes.example.inputs.json
filters:
branches:
- master
Expand All @@ -137,8 +133,6 @@ workflows:
- name: GvsCreateVAT
subclass: WDL
primaryDescriptorPath: /scripts/variantstore/wdl/GvsCreateVAT.wdl
testParameterFiles:
- /scripts/variantstore/wdl/GvsCreateVAT.example.inputs.json
filters:
branches:
- master
Expand Down
2 changes: 1 addition & 1 deletion scripts/variantstore/AOU_GVS_WORKSPACE.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,4 +153,4 @@ TODO:

## 5. Your VCF is ready!!

The sharded VCF outut files are listed in the `ExtractTask.output_vcf` workflow output, and the associated index files are listed in `ExtractTask.output_vcf_index`
The sharded VCF output files are listed in the `ExtractTask.output_vcf` workflow output, and the associated index files are listed in `ExtractTask.output_vcf_index`
4 changes: 2 additions & 2 deletions scripts/variantstore/variant_annotations_table/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ The pipeline takes in a jointVCF and outputs a table in BigQuery.

### Run GvsCreateVAT:

Most of the inputs are constants — like the reference, or a table schema — and dont require any additional work (and paths can be found in the [example inputs json](/scripts/variantstore/wdl/GvsCreateVAT.example.inputs.json)). However for the specific data being put in the VAT, three inputs need to be created.
Most of the inputs are constants — like the reference, or a table schema — and don't require any additional work, or have defaults set in the WDL). However for the specific data being put in the VAT, three inputs need to be created.

The first two of these inputs are two files — one of the file/vcf/shards you want to use for the VAT, and their corresponding index files. These are labelled as `inputFileofFileNames` and `inputFileofIndexFileNames` and need to be copied into a GCP bucket that this pipeline will have access to (eg. this bucket: `gs://aou-genomics-curation-prod-processing/vat/`) for easy access during the workflow.
The third input is the ancestry file from the ancestry pipeline which will be used to calculate AC, AN and AF for all subpopulations. It needs to be copied into a GCP bucket that this pipeline will have access to. This input has been labelled as the `ancestry_file`.

Most of the other files are specific to where the VAT will live, like the project_id and dataset_name and the table_suffix which will name the VAT itself as vat_`table_suffix` as well as a GCP bucket location, the output_path, for the intermediary files and the VAT export in tsv form.

The provided [example inputs json](/scripts/variantstore/wdl/GvsCreateVAT.example.inputs.json) indicates all of the inputs.
All optional inputs are provided with default values.


### Notes:
Expand Down
9 changes: 0 additions & 9 deletions scripts/variantstore/wdl/GvsAssignIds.example.inputs.json

This file was deleted.

This file was deleted.

This file was deleted.

15 changes: 0 additions & 15 deletions scripts/variantstore/wdl/GvsCreateVAT.example.inputs.json

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

8 changes: 0 additions & 8 deletions scripts/variantstore/wdl/GvsImportGenomes.example.inputs.json

This file was deleted.

2 changes: 1 addition & 1 deletion scripts/variantstore/wdl/WDL_TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Some data (including All of Us data) will be processed using a service account (SA) that has access to the relevant data (GCP storage) and BigQuery (BQ) resources.

Our WLDs are written so that they can be run EITHER (1) as the user (no `service_account_json` file provided as input) OR (2) as the SA (by providing a `service_account_json` input file to the WDL).
Our WDLs are written so that they can be run EITHER (1) as the user (no `service_account_json` file provided as input) OR (2) as the SA (by providing a `service_account_json` input file to the WDL).

## Manual steps to create testing resources
To test running WDLs using a SA, we have created a project, SA, buckets, and BQ datasets that mirror the permissions in the AoU setup. These include:
Expand Down
Loading