Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When start_date set in meltano.yml and a non datetime Replication Key is set start_date is used in where clause #47

Open
BuzzCutNorman opened this issue May 5, 2023 · 1 comment

Comments

@BuzzCutNorman
Copy link
Owner

If you setup Incremental replication on a stream select a non datetime column as a Replcation Key if you have a start_date set in the meltano.yml then during the initial incremental run it will fail with a pyodbc conversion error.

      start_date: '2022-01-03'
      raw-tags:
        replication-method: INCREMENTAL
        replication-key: Id
sqlalchemy.exc.DataError: (pyodbc.DataError) ('22018', "[22018] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Conversion failed when converting the nvarchar value '2022-01-03' to data type int. (245) (SQLExecDirectW)")
[SQL: SELECT raw.tags.[Id], raw.tags.[TagName], raw.tags.[Count], raw.tags.[ExcerptPostId], raw.tags.[WikiPostId]
FROM raw.tags
WHERE raw.tags.[Id] >= ? ORDER BY raw.tags.[Id]]
[parameters: ('2022-01-03',)]
@BuzzCutNorman
Copy link
Owner Author

I tracked the issue down to _write_starting_replication_value. It doesn't check if the replication key col is of type timestamp before assigning the start_date. I guess it could also check that the replication key is not of type numeric or int.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant