This module allows you to create opinionated Google Cloud Platform BigQuery datasets and tables. This will allow the user to programmatically create an empty table schema inside of a dataset, ready for loading. Additional user accounts and permissions are necessary to begin querying the newly created table(s).
The current version is 1.X. The following guide is available to assist with upgrades:
Examples of how to use this module are located in the examples directory
This module provisions a dataset and a table with an associated JSON schema.
| Name | Description | Type | Required | Default |
|---|---|---|---|---|
| dataset_id | Unique id for the dataset being provisioned | string | yes | |
| dataset_name | Friendly name for the dataset being provisioned | string | yes | |
| description | Dataset description | string | yes | |
| location | The regional location for the dataset, all Big Query dataset locations are allows | string | yes | US |
| expiration | TTL of tables using the dataset in MS | integer | yes | |
| project_id | Project where the dataset and table are created | string | yes | |
| tables | A list of maps containing table_id and schema files | array | yes | |
| table_id | A attribute in the map of the tables list, that contains a unique id for the table being provisioned | string | yes | |
| time_partitioning | Unique id for the table being provisioned | string | yes | |
| schema | A attribute in the map of the tables list, that contains JSON schema files for the table | string | yes |
| Name | Description |
|---|---|
| dataset_id | Unique id for the dataset being provisioned |
| dataset_name | Friendly name for the dataset being provisioned |
| dataset_project | Project wheree the dataset and table are created |
| table_id | List of unique id for the table(s) being provisioned |
| table_name | List friendly name(s) for the dataset being provisioned |
| dataset_labels | Key value pairs in a map for dataset labels |
| table_labels | Key value pairs in a map for table labels |
- Terraform 0.11.x
- terraform-provider-google plugin v2.5.0
In order to execute this module you must have a Service Account with the following roles:
- roles/bigquery.dataOwner
A helper script for configuring a Service Account is located at (./helpers/setup-sa.sh).
Be sure you have the current Terraform version (0.11.x), you can choose the binary from Terraform releases.
To set this up on your machine, follow the official Kitchen installation instructions.
- Kitchen tests are located: test/integration/full.
- Terraform fixtures are located: test/fixtures/full.
cd /path/to/terraform-google-bigquery
The following command will run all tests for the module:
make
To run kitchen tests on macOS > 10.14.4 xcode will need to be reset
xcode-select --install