Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Isaric followup #175

Merged
merged 21 commits into from
Oct 19, 2023
Merged

Isaric followup #175

merged 21 commits into from
Oct 19, 2023

Conversation

pipliggins
Copy link
Collaborator

2 parsers included, as the French followup data follows a very different naming convention to the others.

The visit table should be removed/commented out for the Initial*/Ongoing* data tables using the isaric-tier1 parser, as they contain follow-up data only.

isaric/parsers/isaric-tier1.toml Outdated Show resolved Hide resolved
isaric/parsers/isaric-tier1.toml Outdated Show resolved Hide resolved
# required fields - "visit_id", "subject_id", "country_iso3", "start_date", "outcome", "date_outcome"

[visit]
outcome = "recovered"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sadiekelly how are long COVID patients (if present in these studies) assigned outcome?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abhidg outcome can be discharged for those patients who have a discharge date given in flw_first_disch. Other patients were not admitted. Do we want to consider the survey completion as a 'visit', and find a relevant term that can be omop-ed for these (follow up visit completed - not sure what is available, I can look)? date of visit is then the date of survey completion, but that would likely be all that could be populated

isaric/parsers/isaric-tier1.toml Outdated Show resolved Hide resolved
output/ISARIC FOLLOWUP/adtl-output_tier1_initial.md Outdated Show resolved Hide resolved
@github-actions
Copy link

Summary of the missing optional fields for the isairc-followup-fr parser:

table missing total_fields percentage_coverage
subject 47 53 11.320755%
visit 57 57 0.000000%
observation 38 67 43.283582%
SUBJECT                                         
subject_id                                      False
dataset_id                                      False
enrolment_date                                  False
age                                             False
date_of_birth                                   False
dob_year                                        False
dob_month                                       False
dob_day                                         False
sex_at_birth                                    False
sex                                             False
ethnicity                                       False
works_microbiology_lab                          False
works_healthcare                                False
pregnancy                                       False
pregnancy_date_of_delivery                      False
pregnancy_birth_weight_kg                       False
pregnancy_outcome                               False
pregnancy_gestational_outcome                   False
pregnancy_whether_breastfed                     False
pregnancy_post_partum                           False
pregnancy_gestational_age_weeks                 False
preterm_infant                                  False
has_chronic_hematologic_disease                 False
has_asplenia                                    False
has_tuberculosis                                False
has_dementia                                    False
has_obesity                                     False
has_rheumatologic_disorder                      False
has_hiv                                         False
has_hypertension                                False
has_malignant_neoplasm                          False
has_malnutrition                                False
has_smoking                                     False
has_asthma                                      False
has_chronic_cardiac_disease                     False
has_chronic_respiratory_disease                 False
has_chronic_kidney_disease                      False
has_diabetes                                    False
diabetes_type                                   False
has_liver_disease                               False
has_apnoea                                      False
has_inflammatory_bowel_disease                  False
has_rare_disease_inborn_metabolism_error        False
has_solid_organ_transplant                      False
has_tuberculosis_past                           False
has_hiv_art                                     False
has_immunosuppression                           False
has_comorbidity_other                           False
VISIT                                           
visit_id                                        False
subject_id                                      False
dataset_id                                      False
country_iso3                                    False
start_date                                      False
pathogen_test_date                              False
phase                                           False
icu_admission                                   False
icu_admission_dates                             False
transfer_from_other_facility                    False
treatment_dialysis                              False
treatment_inotropes_vasopressors                False
treatment_ecmo                                  False
treatment_corticosteroid                        False
treatment_corticosteroid_type                   False
treatment_corticosteroid_type_other             False
treatment_oxygen_therapy                        False
treatment_oxygen_mask_prongs                    False
treatment_prone_position                        False
treatment_invasive_ventilation                  False
treatment_antifungal_agent                      False
treatment_antifungal_agent_type                 False
treatment_antivirals                            False
treatment_antiviral_type                        False
treatment_antiviral_type_other                  False
treatment_antibiotics                           False
treatment_antibiotics_type                      False
treatment_anticoagulation                       False
treatment_experimental_agent                    False
treatment_experimental_agent_type               False
treatment_experimental_agent_type_other         False
treatment_inhaled_nitric_oxide                  False
treatment_noninvasive_ventilation               False
treatment_ace_inhibitors                        False
treatment_arb                                   False
treatment_antimalarial                          False
treatment_antimalarial_type                     False
treatment_high_flow_nasal_cannula               False
treatment_steroids                              False
treatment_immunosuppressant                     False
treatment_intravenous_fluids                    False
treatment_nsaid                                 False
treatment_neuromuscular_blocking_agents         False
treatment_cpr                                   False
treatment_offlabel                              False
treatment_respiratory_support                   False
treatment_indication_anticoagulation            False
treatment_cardiovascular_support                False
treatment_colchicine                            False
treatment_immunoglobulins                       False
treatment_delirium                              False
treatment_delirium_type                         False
treatment_monoclonal_antibody                   False
treatment_other                                 False
treatment_pacing                                False
outcome                                         False
date_outcome                                    False
OBSERVATION                                     
avpu                                            False
oxygen_flow_volume_max                          False
oxygen_flow_volume_max                          False
clinical_classification_critical_illness_scale  False
total_fluid_output_ml                           False
base_excess                                     False
clinical_frailty_score                          False
inability_to_walk_scale                         False
blantyre_coma_score                             False
mid_upper_arm_circumference_cm                  False
richmond_agitation-sedation_scale               False
riker_sedation-agitation_scale                  False
other_symptom                                   False
mean_arterial_blood_pressure_mmHg               False
pneumonia                                       False
abdominal_pain                                  False
altered_consciousness_confusion                 False
anorexia                                        False
bleeding                                        False
bleeding_haemorrhage                            False
chest_pain                                      False
confusion                                       False
conjunctivitis                                  False
cough_with_haemoptysis                          False
diarrhoea                                       False
ear_pain                                        False
feeding_intolerance_pediatrics                  False
history_of_fever                                False
inability_to_walk                               False
irritability_pediatrics                         False
lower_chest_wall_indrawing                      False
runny_nose                                      False
seizures                                        False
severe_dehydration                              False
skin_rash                                       False
skin_ulcers                                     False
sternal_capillary_refill_time_greater_2s        False
vomiting_nausea                                 False
heart_sounds                                    False

Copy link
Collaborator

@ekamau ekamau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @pipliggins here's my review of the french follow up parser. My main comments are around integration of this follow up data with the acute data collected in the main CORE database, which can be linked by subjid. So thinking whether some data collected here at follow up can be integrated with and used to upd
French inserm follow up parser review 2023-10-10.xlsx
ate the subject table e.g. where the patient has died. It might be easier to discuss more fully, let me know.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sadiekelly - I've made the edits required. Re linking the data; this is something we'll have to do on the database side, after the data is initially parsed, rather that within the parsers themselves. So I've created a 'subject' and 'visit' table for the follow-up data, but when we come to merge the data based on the subject ID's, we can write a hierarchy rule that allows the followup data to overwrite the acute dataset where necessary. Hope that makes sense!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @pipliggins here's my review of the tier 1 freestanding parser. This and the brazil follow up are almost identical and share very similar points for review/discussion, so I'd suggest resolving them and updating both at the same time.
Tier 1 freestanding parser review 2023-10-10.xlsx

Copy link
Collaborator

@sadiekelly sadiekelly Oct 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pipliggins I've realised the tier1 parser should also fit the initial/ongoing surveys - I'll review this parser against those now, apologies for missing this!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sadiekelly no worries - I've just done the Brazil parser, so if you're happy with those changes I'll apply the same logic here where required

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pipliggins here's a (very short) review when comparing Tier1 parser to Initial follow up. I haven't duplicated anything that is already listed in the Tier 1 parser review, so just a couple of points here
Initial follow up parser review 2023-10-10.xlsx

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pipliggins and here's a few comments regarding the ongoing follow up - thanks!
Ongoing follow up parser review 2023-10-12.xlsx

Contains most of the required changes + notes for possible further edits
Untested
@github-actions
Copy link

Summary of the missing optional fields for the isairc-followup-fr parser:

table missing total_fields percentage_coverage
subject 48 54 11.111111%
visit 52 56 7.142857%
observation 40 69 42.028986%
SUBJECT                                         
dataset_id                                      False
enrolment_date                                  False
age                                             False
date_of_birth                                   False
dob_year                                        False
dob_month                                       False
dob_day                                         False
sex_at_birth                                    False
sex                                             False
ethnicity                                       False
works_lab                                       False
works_microbiology_lab                          False
works_healthcare                                False
pregnancy                                       False
pregnancy_date_of_delivery                      False
pregnancy_birth_weight_kg                       False
pregnancy_outcome                               False
pregnancy_gestational_outcome                   False
pregnancy_whether_breastfed                     False
pregnancy_post_partum                           False
pregnancy_gestational_age_weeks                 False
preterm_infant                                  False
has_chronic_hematologic_disease                 False
has_asplenia                                    False
has_tuberculosis                                False
has_dementia                                    False
has_obesity                                     False
has_rheumatologic_disorder                      False
has_hiv                                         False
has_hypertension                                False
has_malignant_neoplasm                          False
has_malnutrition                                False
has_smoking                                     False
has_asthma                                      False
has_chronic_cardiac_disease                     False
has_chronic_respiratory_disease                 False
has_chronic_kidney_disease                      False
has_diabetes                                    False
diabetes_type                                   False
has_liver_disease                               False
has_apnoea                                      False
has_inflammatory_bowel_disease                  False
has_rare_disease_inborn_metabolism_error        False
has_solid_organ_transplant                      False
has_tuberculosis_past                           False
has_hiv_art                                     False
has_immunosuppression                           False
has_comorbidity_other                           False
VISIT                                           
dataset_id                                      False
country_iso3                                    False
start_date                                      False
pathogen_test_date                              False
phase                                           False
icu_admission                                   False
icu_admission_dates                             False
transfer_from_other_facility                    False
treatment_dialysis                              False
treatment_inotropes_vasopressors                False
treatment_ecmo                                  False
treatment_corticosteroid                        False
treatment_corticosteroid_type                   False
treatment_corticosteroid_type_other             False
treatment_oxygen_therapy                        False
treatment_oxygen_mask_prongs                    False
treatment_prone_position                        False
treatment_invasive_ventilation                  False
treatment_antifungal_agent                      False
treatment_antifungal_agent_type                 False
treatment_antivirals                            False
treatment_antiviral_type                        False
treatment_antiviral_type_other                  False
treatment_antibiotics                           False
treatment_antibiotics_type                      False
treatment_anticoagulation                       False
treatment_experimental_agent                    False
treatment_experimental_agent_type               False
treatment_experimental_agent_type_other         False
treatment_inhaled_nitric_oxide                  False
treatment_noninvasive_ventilation               False
treatment_ace_inhibitors                        False
treatment_arb                                   False
treatment_antimalarial                          False
treatment_antimalarial_type                     False
treatment_high_flow_nasal_cannula               False
treatment_steroids                              False
treatment_immunosuppressant                     False
treatment_intravenous_fluids                    False
treatment_nsaid                                 False
treatment_neuromuscular_blocking_agents         False
treatment_cpr                                   False
treatment_offlabel                              False
treatment_respiratory_support                   False
treatment_cardiovascular_support                False
treatment_colchicine                            False
treatment_immunoglobulins                       False
treatment_delirium                              False
treatment_delirium_type                         False
treatment_monoclonal_antibody                   False
treatment_other                                 False
treatment_pacing                                False
OBSERVATION                                     
avpu                                            False
acvpu                                           False
oxygen_flow_volume_max                          False
oxygen_flow_volume_max                          False
clinical_classification_critical_illness_scale  False
total_fluid_output_ml                           False
base_excess                                     False
clinical_frailty_score                          False
inability_to_walk_scale                         False
blantyre_coma_score                             False
mid_upper_arm_circumference_cm                  False
richmond_agitation-sedation_scale               False
riker_sedation-agitation_scale                  False
other_symptom                                   False
mean_arterial_blood_pressure_mmHg               False
pneumonia                                       False
abdominal_pain                                  False
altered_consciousness_confusion                 False
anorexia                                        False
bleeding                                        False
bleeding_haemorrhage                            False
chest_pain                                      False
confusion                                       False
conjunctivitis                                  False
cough_dry                                       False
cough_with_haemoptysis                          False
diarrhoea                                       False
ear_pain                                        False
feeding_intolerance_pediatrics                  False
history_of_fever                                False
inability_to_walk                               False
irritability_pediatrics                         False
lower_chest_wall_indrawing                      False
musculoskeletal_pain                            False
seizures                                        False
severe_dehydration                              False
skin_rash                                       False
skin_ulcers                                     False
sternal_capillary_refill_time_greater_2s        False
vomiting_nausea                                 False
heart_sounds                                    False

@globaldothealth globaldothealth deleted a comment from github-actions bot Oct 19, 2023
@pipliggins pipliggins merged commit f43439c into main Oct 19, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants