Emit JSON schema and BigQuery schema #6
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When setting up a new project using
dfe-analytics
, it's necessary to create a schema in BigQuery that's compatible with the schema in this project.BigQuery uses its own schema format, so we need to map the JSON schema to that format. The BigQuery schema contains less information than the JSON schema (e.g. it lacks enums,
additionalProperties
etc). We keep the JSON schema as the source of truth for testing, and generate a BigQuery schema from it.This PR adds two new rake tasks:
dfe:analytics:schema
anddfe:analytics:big_query_schema
. The BigQuery schema is suitable for copy-pasting into the BigQuery web interface. It could also be used to generate a table via thebq
CLI tool.Changes to the schema!
During this process I noticed that the event-schema.json was not the same as the BigQuery schema (for Apply, at least) and I amended it (relaxing requirements, so not breaking changes)