Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apply: error deleting rows due to date/time mismatch #143

Closed
wonder-sk opened this issue Aug 2, 2021 · 2 comments · Fixed by #151
Closed

apply: error deleting rows due to date/time mismatch #143

wonder-sk opened this issue Aug 2, 2021 · 2 comments · Fixed by #151
Labels
bug Something isn't working

Comments

@wonder-sk
Copy link
Contributor

When using db-sync, it was seen (MerginMaps/db-sync#50) that if a table is synced from geopackage to postgresql, some rows are removed and then the diff is applied back on the geopackage, it may happen that applying the diff fails due to subtle difference in how date/time is stored.

The original data may contain date/time values like this: 2021-03-16T00:00:00
Diff returning from postgresql was referring to values including Zulu timezone: 2021-03-16T00:00:00Z

When deleting rows, this mismatch of what is stored in SQLite and what is stored in diff causes the error.

A temporary fix for this issue is to add the "Z" suffix to the source data, in sqlite the batch command may look like this:

UPDATE fruit_monitor SET date = date || 'Z';
@wonder-sk wonder-sk added the bug Something isn't working label Aug 2, 2021
@wonder-sk
Copy link
Contributor Author

We already have special handling of date/time values when it comes to creation of UPDATE statements (https://github.com/lutraconsulting/geodiff/blob/master/geodiff/src/drivers/sqlitedriver.cpp#L497), we should apply that also when creating/applying DELETE statements.

sqlite> select '2021-03-16T00:00:00' = '2021-03-16T00:00:00Z';
0
sqlite> select datetime('2021-03-16T00:00:00') = datetime('2021-03-16T00:00:00Z');
1

@HugoP27
Copy link

HugoP27 commented Sep 17, 2021

Hi @wonder-sk is this scheduled in your development timeline or could we speed it up somehow, maybe fund a part of the development? please let me know. cheers

wonder-sk added a commit that referenced this issue Oct 8, 2021
This fixes the issue when applying a diff with date/time values,
but the representation of date/time values as strings is slightly
different even though they refer to the same date/time.

Fixes both the case with update and the case with delete.
Insert does not need fixing as there is no comparison with previous values.
tomasMizera pushed a commit that referenced this issue Oct 13, 2021
This fixes the issue when applying a diff with date/time values,
but the representation of date/time values as strings is slightly
different even though they refer to the same date/time.

Fixes both the case with update and the case with delete.
Insert does not need fixing as there is no comparison with previous values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants