Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Made datetime attribute insert and match time-zone invariant #301

Merged
merged 19 commits into from
Jun 30, 2023

Conversation

shiladitya-mukherjee
Copy link
Contributor

@shiladitya-mukherjee shiladitya-mukherjee commented Jun 21, 2023

What is the goal of this PR?

Enables the BDD tests to check timezone-invariance of inserting and reading datetime attributes.

What are the changes implemented in this PR?

  • During deserialization of datetime, we incorrectly used fromtimestamp function. This function is environment timezone-sensitive. The correct function to use is utcfromtimestamp which converts the milliseconds since epoch to UTC datetime object. This conforms to the serialization process: we convert datetime object to milliseconds since epoch assuming the datetime is in UTC.
  • Added step which sets environment timezone to time_zone_label
  • Fixed parsing of class AttributeMatcher and ValueMatcher.

Fixes #298.

…alue parsing for datetime fields in AttributeMatcher and ValueMatcher
…zone step for scenarios in BDD, fixed AttributeMatcher and ValueMatcher parsing
@typedb-bot
Copy link
Member

typedb-bot commented Jun 21, 2023

PR Review Checklist

Do not edit the content of this comment. The PR reviewer should simply update this comment by ticking each review item below, as they get completed.


Trivial Change

  • This change is trivial and does not require a code or architecture review.

Code

  • Packages, classes, and methods have a single domain of responsibility.
  • Packages, classes, and methods are grouped into cohesive and consistent domain model.
  • The code is canonical and the minimum required to achieve the goal.
  • Modules, libraries, and APIs are easy to use, robust (foolproof and not errorprone), and tested.
  • Logic and naming has clear narrative that communicates the accurate intent and responsibility of each module (e.g. method, class, etc.).
  • The code is algorithmically efficient and scalable for the whole application.

Architecture

  • Any required refactoring is completed, and the architecture does not introduce technical debt incidentally.
  • Any required build and release automations are updated and/or implemented.
  • Any new components follows a consistent style with respect to the pre-existing codebase.
  • The architecture intuitively reflects the application domain, and is easy to understand.
  • The architecture has a well-defined hierarchy of encapsulated components.
  • The architecture is extensible and scalable.

@shiladitya-mukherjee shiladitya-mukherjee changed the title Fixed datetime insert and match time-zone bug, Implementated set time-zone step for scenarios in BDD, fixed AttributeMatcher and ValueMatcher parsing Fixed datetime insert and match time-zone bug, implementated set time-zone step for scenarios in BDD, fixed AttributeMatcher and ValueMatcher parsing Jun 23, 2023
@shiladitya-mukherjee shiladitya-mukherjee changed the title Fixed datetime insert and match time-zone bug, implementated set time-zone step for scenarios in BDD, fixed AttributeMatcher and ValueMatcher parsing Made datetime attribute insert and match time-zone invariant Jun 23, 2023
Copy link
Member

@krishnangovindraj krishnangovindraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small changes + question about resetting the timezone after the scenario

typedb/concept/value/value.py Show resolved Hide resolved
def step_impl(context: Context, time_zone_label: str):
os.environ["TZ"] = time_zone_label
time.tzset()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we reset it after our test is done, so we don't affect other tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to do it in typedb-behaviour. Can we assume that the server time is always UTC?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we can't assume that.
We could store the time zone and restore it using @Before/@after methods
https://cucumber.io/docs/cucumber/api/?lang=java

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating my comment with what Joshua said:
We should try to put it as an explicit step in the background.

typedb/concept/thing/attribute.py Outdated Show resolved Hide resolved
typedb/common/exception.py Outdated Show resolved Hide resolved
Copy link
Member

@krishnangovindraj krishnangovindraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just a little clean up left.

tests/behaviour/concept/serialization/json/BUILD Outdated Show resolved Hide resolved
tests/behaviour/concept/thing/attribute/attribute_steps.py Outdated Show resolved Hide resolved
tests/behaviour/config/parameters.py Show resolved Hide resolved
tests/behaviour/typeql/typeql_steps.py Outdated Show resolved Hide resolved
@@ -60,9 +60,13 @@ def parseWords(text):
@parse.with_pattern(r"\d\d\d\d-\d\d-\d\d(?: \d\d:\d\d:\d\d)?")
def parse_datetime(text: str) -> datetime:
try:
return datetime.strptime(text, "%Y-%m-%d %H:%M:%S")
return datetime.strptime(text, "%Y-%m-%dT%H:%M:%S")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this won't ever run right? Because the regex above doesn't accept a T ever. Let's discuss why you made this change tomorrow

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function was called also from inside the Python client, on top of it being called by Cucumber.

@typedb typedb deleted a comment from flyingsilverfin Jun 30, 2023
Copy link
Member

@krishnangovindraj krishnangovindraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

remote = "https://github.com/vaticle/typedb-behaviour",
commit = "767bf98fef7383addf42a1ae6e97a44874bb4f0b" # sync-marker: do not remove this comment, this is used for sync-dependencies by @vaticle_typedb_behaviour
remote = "https://github.com/shiladitya-mukherjee/typedb-behaviour",
commit = "ac3006755c13e40bb1404f82ab9e65d4255e3cce" # sync-marker: do not remove this comment, this is used for sync-dependencies by @vaticle_typedb_behaviour
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since typedb-behaviour is now merged, You can now bump this to the new vaticle commit.

@flyingsilverfin flyingsilverfin merged commit e669cf5 into typedb:master Jun 30, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python client datetimes are influenced by time zone
5 participants