Skip to content

Commit

Permalink
Remove support for dict type, add helpful exception message
Browse files Browse the repository at this point in the history
dict type columns cannot be directly loaded to BigQuery, this change makes that more explicit.

---------

Co-authored-by: Austin Weisgrau <62900254+austinweisgrau@users.noreply.github.com>
  • Loading branch information
KasiaHinkson and austinweisgrau authored Jul 11, 2024
1 parent 24728eb commit 118d744
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions parsons/google/google_bigquery.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
"datetime": "DATETIME",
"date": "DATE",
"time": "TIME",
"dict": "RECORD",
"NoneType": "STRING",
"UUID": "STRING",
"timestamp": "TIMESTAMP",
Expand Down Expand Up @@ -1200,7 +1199,14 @@ def _generate_schema_from_parsons_table(self, tbl):
if isinstance(value, datetime.datetime) and value.tzinfo:
best_type = "timestamp"

field_type = self._bigquery_type(best_type)
try:
field_type = self._bigquery_type(best_type)
except KeyError as e:
raise KeyError(
"Column type not supported for load to BigQuery. "
"Consider converting to another type. "
f"[type={best_type}]"
) from e
field = bigquery.schema.SchemaField(stat["name"], field_type)
fields.append(field)
return fields
Expand Down

0 comments on commit 118d744

Please sign in to comment.