TDL-9728: Stream ads_insights_age_gender has unexpected datatype for replication key field date_start
#172
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dbshah1212
approved these changes
Nov 12, 2021
karanpanchal-crest
suggested changes
Nov 17, 2021
| @@ -196,11 +196,6 @@ def bookmarks_test(self, expected_streams): | |||
|
|
|||
There was a problem hiding this comment.
@harshpatel4crest Can you add a check in the integration test case to check the format of the date_start and date_stop. You can verify the format for only one record.. That would be enough.
There was a problem hiding this comment.
Added the check for the date.
kpanchal-crest
approved these changes
Nov 17, 2021
karanpanchal-crest
approved these changes
Nov 17, 2021
KrisPersonal
approved these changes
Nov 17, 2021
kspeer825
approved these changes
Nov 18, 2021
Merged
KrisPersonal
pushed a commit
that referenced
this pull request
Nov 19, 2021
* TDL-9728: Stream `ads_insights_age_gender` has unexpected datatype for replication key field `date_start` (#172) * added format as date-time in schema file * added code coverage * added check for date format in the bookmark test * added the check for first sync messages Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-9809: `forced-replication-method` missing from metadata for some streams and TDL-9872: replication keys are not specified as expected in discoverable metadata (#167) * added valid replication keys in catalog * modified the code * TDL-9809: Added replication keys in metadata * adde code coverage * Resolved review comments Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7455: Add tap-tester test to verify replication of deleted records (#168) * TDL-7455: Added archived data integration test * TDL-7455: Updated integration test * added code coverage * Resolved review comment Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7596: Add tap-tester test for attribution window (#169) * added tap tester test for attribution window * updated the code * added code coverage * updated the code according to the comments * updated code to raise error when attribution window is not 1, 7, 28 * test: run invalid attribution window intergation test * updated test case * test: updated test case code * test: test invalid attribution window * test: test invalid attribution window * test: test invalid attribution window * test: test invalid attribution window * test: run invalid attribution window test case * added intergation test for invalid sttribution window Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: savan-chovatiya <80703490+savan-chovatiya@users.noreply.github.com>
KrisPersonal
added a commit
that referenced
this pull request
Nov 23, 2021
…ve requests that are not retrying (#171) * TDL-6148: Added retry for Attribute error of sync batches * TDL-6148: Removed unused imports from unit tests * TDL-13267: Added retry for 500 error of AdCreatives * TDL-6148: Add AttributeError backoff for all sync functions * added code coverage * Resolved review comment * Resolved review comments * Added code comments * Resolved review comment * TDL-9728: Stream `ads_insights_age_gender` has unexpected datatype for replication key field `date_start` (#172) * added format as date-time in schema file * added code coverage * added check for date format in the bookmark test * added the check for first sync messages Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-9809: `forced-replication-method` missing from metadata for some streams and TDL-9872: replication keys are not specified as expected in discoverable metadata (#167) * added valid replication keys in catalog * modified the code * TDL-9809: Added replication keys in metadata * adde code coverage * Resolved review comments Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7455: Add tap-tester test to verify replication of deleted records (#168) * TDL-7455: Added archived data integration test * TDL-7455: Updated integration test * added code coverage * Resolved review comment Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: Harsh <80324346+harshpatel4crest@users.noreply.github.com> Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
KrisPersonal
added a commit
that referenced
this pull request
Dec 14, 2021
* TDL-15863: Added request timeout and unit tests * TDL-15863: Added unit tests * TDL-15863: Removed commented code * TDL-15863: Added request_timeout lookup from config also * added code coverage * TDL-15863: Added float type cast for timeout * Updated comment * added code change for empty string timeout value from config * Added empty string handling in request_timeout param * Resolved review comment * Added if else for request timeout * Added backoff for ConnectionError * TDL-9728: Stream `ads_insights_age_gender` has unexpected datatype for replication key field `date_start` (#172) * added format as date-time in schema file * added code coverage * added check for date format in the bookmark test * added the check for first sync messages Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-9809: `forced-replication-method` missing from metadata for some streams and TDL-9872: replication keys are not specified as expected in discoverable metadata (#167) * added valid replication keys in catalog * modified the code * TDL-9809: Added replication keys in metadata * adde code coverage * Resolved review comments Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7455: Add tap-tester test to verify replication of deleted records (#168) * TDL-7455: Added archived data integration test * TDL-7455: Updated integration test * added code coverage * Resolved review comment Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: Harsh <80324346+harshpatel4crest@users.noreply.github.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
jesuejunior
pushed a commit
to sixcodes/tap-facebook
that referenced
this pull request
Mar 17, 2023
* TDL-9728: Stream `ads_insights_age_gender` has unexpected datatype for replication key field `date_start` (singer-io#172) * added format as date-time in schema file * added code coverage * added check for date format in the bookmark test * added the check for first sync messages Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-9809: `forced-replication-method` missing from metadata for some streams and TDL-9872: replication keys are not specified as expected in discoverable metadata (singer-io#167) * added valid replication keys in catalog * modified the code * TDL-9809: Added replication keys in metadata * adde code coverage * Resolved review comments Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7455: Add tap-tester test to verify replication of deleted records (singer-io#168) * TDL-7455: Added archived data integration test * TDL-7455: Updated integration test * added code coverage * Resolved review comment Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7596: Add tap-tester test for attribution window (singer-io#169) * added tap tester test for attribution window * updated the code * added code coverage * updated the code according to the comments * updated code to raise error when attribution window is not 1, 7, 28 * test: run invalid attribution window intergation test * updated test case * test: updated test case code * test: test invalid attribution window * test: test invalid attribution window * test: test invalid attribution window * test: test invalid attribution window * test: run invalid attribution window test case * added intergation test for invalid sttribution window Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: savan-chovatiya <80703490+savan-chovatiya@users.noreply.github.com>
jesuejunior
pushed a commit
to sixcodes/tap-facebook
that referenced
this pull request
Mar 17, 2023
…ve requests that are not retrying (singer-io#171) * TDL-6148: Added retry for Attribute error of sync batches * TDL-6148: Removed unused imports from unit tests * TDL-13267: Added retry for 500 error of AdCreatives * TDL-6148: Add AttributeError backoff for all sync functions * added code coverage * Resolved review comment * Resolved review comments * Added code comments * Resolved review comment * TDL-9728: Stream `ads_insights_age_gender` has unexpected datatype for replication key field `date_start` (singer-io#172) * added format as date-time in schema file * added code coverage * added check for date format in the bookmark test * added the check for first sync messages Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-9809: `forced-replication-method` missing from metadata for some streams and TDL-9872: replication keys are not specified as expected in discoverable metadata (singer-io#167) * added valid replication keys in catalog * modified the code * TDL-9809: Added replication keys in metadata * adde code coverage * Resolved review comments Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7455: Add tap-tester test to verify replication of deleted records (singer-io#168) * TDL-7455: Added archived data integration test * TDL-7455: Updated integration test * added code coverage * Resolved review comment Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: Harsh <80324346+harshpatel4crest@users.noreply.github.com> Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
jesuejunior
pushed a commit
to sixcodes/tap-facebook
that referenced
this pull request
Mar 17, 2023
* TDL-15863: Added request timeout and unit tests * TDL-15863: Added unit tests * TDL-15863: Removed commented code * TDL-15863: Added request_timeout lookup from config also * added code coverage * TDL-15863: Added float type cast for timeout * Updated comment * added code change for empty string timeout value from config * Added empty string handling in request_timeout param * Resolved review comment * Added if else for request timeout * Added backoff for ConnectionError * TDL-9728: Stream `ads_insights_age_gender` has unexpected datatype for replication key field `date_start` (singer-io#172) * added format as date-time in schema file * added code coverage * added check for date format in the bookmark test * added the check for first sync messages Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-9809: `forced-replication-method` missing from metadata for some streams and TDL-9872: replication keys are not specified as expected in discoverable metadata (singer-io#167) * added valid replication keys in catalog * modified the code * TDL-9809: Added replication keys in metadata * adde code coverage * Resolved review comments Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> * TDL-7455: Add tap-tester test to verify replication of deleted records (singer-io#168) * TDL-7455: Added archived data integration test * TDL-7455: Updated integration test * added code coverage * Resolved review comment Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: namrata270998 <namrata.brahmbhatt@crestdatasys.com> Co-authored-by: harshpatel4_crest <harsh.patel4@crestdatasys.com> Co-authored-by: Harsh <80324346+harshpatel4crest@users.noreply.github.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
TDL-9728: Stream
ads_insights_age_genderhas unexpected datatype for replication key fielddate_startads_insights_age_and_genderandads_insights_hourly_advertiserand added"format": "date-time"to format as "2019-07-22T00:00:00.000000Z"Manual QA steps
Risks
Rollback steps