Code to extract data from Rave.
Run pip3 install -r requirements.txt
to install dependencies
To run the python scripts for extracting the Gmb data from the Rave database and transforming Gmb data, the user can use the command
python3 gmb_extraction.py config/gmb_config_example.yaml
To run the python script for extracting the Gmb data from the Rave database only, the user can use the command
python3 gmb_extraction.py config/gmb_config.yaml --extract-only
To run the python script for transforming the Gmb data which is downloaded from the s3 bucket, the user can use the command
python3 gmb_transformation.py config/gmb_config_example.yaml s3_bucket_subfolder
To run the python script for transforming the Gmb data which is in the local folder, the user can use the command
python3 gmb_transformation.py config/gmb_config_example.yaml s3_bucket_subfolder --not_download_data
After running the code, the user can check the log for more validation related information, and the user also needs to add 3 more files(CLINICALTRIAL.tsv, SITE.tsv, internal-whitelist.tsv) before uploading data through the data-loader.
-
Notifications
You must be signed in to change notification settings - Fork 1
CBIIT/gmb-etl
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
No description or website provided.
Topics
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published