Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update destination-s3 to handle the new data types protocol #17645

Closed
Tracked by #17637
edgao opened this issue Oct 5, 2022 · 2 comments · Fixed by airbytehq/json-avro-converter#17 or #20088
Closed
Tracked by #17637

Update destination-s3 to handle the new data types protocol #17645

edgao opened this issue Oct 5, 2022 · 2 comments · Fixed by airbytehq/json-avro-converter#17 or #20088
Assignees
@suhomud
Labels
team/destinations Destinations team's backlog

Comments

@edgao
Copy link
Contributor

edgao commented Oct 5, 2022 •
edited by grishick
Loading

See #17637.

Definition of done:

  • destination-s3 accepts the new protocol messages
  • destination data does not change when protocol message version changes (should write a test for this)
@edgao edgao mentioned this issue Oct 5, 2022
Closed
8 tasks
@edgao edgao changed the title destination-s3 Update destination-s3 to handle the new data types protocol Oct 5, 2022
@edgao edgao mentioned this issue Oct 5, 2022
Closed
@edgao edgao added team/destinations Destinations team's backlog and removed team/triage labels Oct 5, 2022
@edgao edgao mentioned this issue Oct 5, 2022
Closed
@edgao edgao mentioned this issue Oct 27, 2022
Closed
@grishick
Copy link
Contributor

grishick commented Nov 10, 2022 •
edited
Loading

Notes from grooming:

  • unknown how this will affect BigQuery and Snowflake destinations that depend on base-s3 code
  • the balk part of the work may be in JsonToAvroSchemaConverter, which is used in destination-gcs, destination-bigquery-denormalized, and destination-s3
  • may have to publish: destination-s3, destination-gcs, destination-snowflake, destination-bigquery, destination-bigquery-denormalized

@suhomud suhomud self-assigned this Nov 11, 2022
@suhomud suhomud closed this as completed Nov 21, 2022
@suhomud suhomud reopened this Nov 21, 2022
@suhomud suhomud linked a pull request Dec 5, 2022 that will close this issue
Support era and string numbers airbytehq/json-avro-converter#17
Merged
@grishick grishick reopened this Dec 13, 2022
@grishick
Copy link
Contributor

Remaining work in #20088

@grishick grishick linked a pull request Dec 13, 2022 that will close this issue
🎉 Update destination-s3 to handle the new data types protocol #20088
Merged
37 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@suhomud suhomud

Labels
team/destinations Destinations team's backlog
Projects
None yet
Milestone
No milestone
4 participants
@grishick @suhomud @edgao @octavia-squidington-iv