Feature/Contacts and StudentContactAssociations #96
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 tasks
rlittle08
reviewed
Jul 23, 2024
| -- we can't use the gen_skey macro here because we're bringing in the deprecated parents endpoint data, which contains a parentReference that won't work | ||
| iff( | ||
| contact_unique_id is not null, | ||
| md5(cast(coalesce(cast(tenant_code as TEXT), '') || '-' || coalesce(cast(lower(contact_unique_id) as TEXT), '') as TEXT)), |
There was a problem hiding this comment.
could you use dbt_utils.surrogate key? and what's the case where contact_unique_id is null?
There was a problem hiding this comment.
yep, good call! updated to use surrogate_key. and it shouldn't ever be null, that's a mistake on my part.
| {{ gen_skey('k_student_xyear') }}, | ||
| api_year as school_year, | ||
| unioned.* | ||
| {{ extract_extension(model_name=this.name, flatten=True) }} |
There was a problem hiding this comment.
will this lose extensions from student_parent_associations?
There was a problem hiding this comment.
I don't think so because the two v_ext columns have been unioned together
There was a problem hiding this comment.
but I think the macro will only extract the extensions configured under this.name, which is stg_ef3__student_contact_associations
There was a problem hiding this comment.
the macro can take a list of model names, so I added extensions configured under parents: {{ extract_extension([model_name=this.name, 'stg_ef3__parents'], flatten=True) }}
I don't have an implementation with extensions to test this on, but it should work based on the documentation of the macro
|
@rlittle08 I added a variable that has to be set to True in implementations that want to use Contacts (to avoid a big breaking change). Current name is |
|
when we merge this, add to migration notes that implementations should copy in this codegen update: https://github.com/edanalytics/edu_project_template/pull/40/files |
rlittle08
approved these changes
Sep 13, 2024
rlittle08
added a commit
that referenced
this pull request
Sep 19, 2024
* New DS5 optional fields for `Section`, `CourseTranscript`, and `StudentSectionAssociation` (#104) * add section_type * bring into line with correct json names * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * fix typo (#90) * fix typo * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * Feature/Contacts and StudentContactAssociations (#96) * redraft * add student contact associations * switch from parent to contact * add comments about name change * can't use gsn_skey * add note about new fields * update changelog * use dbt utils surrogate key * use parents if contacts are not enabled * add docs * remove configurability * extract extenstions configured with parent naming too * fix bad syntax --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * Bugfix/Updated Surrogate Keys to include lower() in student_unique_id inputs (#100) * added lower to student_unique_id in surrogate keys * Update stg_ef3__student_objective_assessments.sql * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * Feature/new DS5 optional fields on staff (#103) * add new optional fields * fix typo * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * flatten student cohort year (#92) * flatten student cohort year * grab school year and term * rename fields * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * explicitly call extract macro from this package (#98) * explicitly call extract macro from this package * Update CHANGELOG.md --------- Co-authored-by: John C. Merfeld <John.Merfeld@gmail.com> Co-authored-by: ejoranlienea <ejoranlien@edanalytics.org> Co-authored-by: Samantha LeBlanc <56237580+sleblanc23@users.noreply.github.com> Co-authored-by: Angelica Lastra <107369214+AngelicaLastra@users.noreply.github.com> Co-authored-by: keen <51969358+keenzarate213@users.noreply.github.com>
rlittle08
added a commit
that referenced
this pull request
Sep 20, 2024
… for grading periods v5.0) (#99) * draft of grading_period key change for v5.0 * update from main (#106) * New DS5 optional fields for `Section`, `CourseTranscript`, and `StudentSectionAssociation` (#104) * add section_type * bring into line with correct json names * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * fix typo (#90) * fix typo * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * Feature/Contacts and StudentContactAssociations (#96) * redraft * add student contact associations * switch from parent to contact * add comments about name change * can't use gsn_skey * add note about new fields * update changelog * use dbt utils surrogate key * use parents if contacts are not enabled * add docs * remove configurability * extract extenstions configured with parent naming too * fix bad syntax --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * Bugfix/Updated Surrogate Keys to include lower() in student_unique_id inputs (#100) * added lower to student_unique_id in surrogate keys * Update stg_ef3__student_objective_assessments.sql * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * Feature/new DS5 optional fields on staff (#103) * add new optional fields * fix typo * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * flatten student cohort year (#92) * flatten student cohort year * grab school year and term * rename fields * Update CHANGELOG.md --------- Co-authored-by: rlittle08 <rlittle@edanalytics.org> * explicitly call extract macro from this package (#98) * explicitly call extract macro from this package * Update CHANGELOG.md --------- Co-authored-by: John C. Merfeld <John.Merfeld@gmail.com> Co-authored-by: ejoranlienea <ejoranlien@edanalytics.org> Co-authored-by: Samantha LeBlanc <56237580+sleblanc23@users.noreply.github.com> Co-authored-by: Angelica Lastra <107369214+AngelicaLastra@users.noreply.github.com> Co-authored-by: keen <51969358+keenzarate213@users.noreply.github.com> * add ods & dm version to every base table --------- Co-authored-by: John C. Merfeld <John.Merfeld@gmail.com> Co-authored-by: ejoranlienea <ejoranlien@edanalytics.org> Co-authored-by: Samantha LeBlanc <56237580+sleblanc23@users.noreply.github.com> Co-authored-by: Angelica Lastra <107369214+AngelicaLastra@users.noreply.github.com> Co-authored-by: keen <51969358+keenzarate213@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description & motivation
In Data Standard v5.0, Parents were renamed to Contacts. The new Contact entity is identical to the previous Parent entity with the addition of an optional GenderIdentity, PreferredFirstName, and PreferredLastName fields. StudentParentAssociations was also renamed to StudentContactAssociations and the parentReference in SurveyResponses was renamed to contactReference.
This PR adds base and stage models for the new resources, which union in the data from the deprecated parent-named resources.
Breaking changes introduced by this PR:
stg_ef3__survey_responsesnow includesk_contactinstead ofk_parent.PR Merge Priority:
Changes to existing files:
gen_skey: addk_contactto key definitionsbase_ef3__parents: add three new fields which were introduced along with the Contacts rename in order to allow the tables to be unioned in stagebase_ef3__survey_responses: coalesce parentReference and contactReference into a singlecontact_referencestg_ef3__survey_responses: rename k_parent to k_contactNew files created:
Base and stage models for
contacts,student_contact_associations, and unpackedcontactslists.Tests and QC done:
Ran in Stadium SCDE; confirmed that columns were populating correctly (where data is available) and staging tables included both Parent and Contact data.
Future ToDos & Questions:
k_contactinstg_ef3__survey_responsesis small, possibly none - we don't have any warehouse models for surveys yet and I didn't see the field referenced in any implementation repos.