-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NEAT-808]🐫 Migration tooling to utils #1009
base: main
Are you sure you want to change the base?
Conversation
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified FilesNo covered modified files...
|
"Class Description": "HIERARCHY", | ||
"Classification": "HIER", | ||
"GL Account": "287839.0000.109.??????.???", | ||
"P&ID": "KL47DSF3001", | ||
"Primary Craft": "ROTATING", | ||
"Primary OTSU": "A1-20.01", | ||
"Priority": "99", | ||
"PublishEventDate": "2024-12-03T21:13:31.186Z", | ||
"Site": "KL47", | ||
"Unit": "DS", | ||
"lastUpdatedTime": "2024-12-03 21:18:45.869", | ||
"locationLocationsID": "105876", | ||
"locationParentID": "12966", | ||
"location_Location": "DS/NH3/DINGS/628-3-I/728-2-FLOT", | ||
"site_location": "KL47-DS/NH3/DINGS/628-3-I/728-2-FLOT", | ||
"site_unit_tag": "KL47-DS-728-2-FLOT", | ||
"unit_tag": "DS-728-2-FLOT", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Inspired by the original data, but changes all letters and numbers.
cognite/neat/_utils/migration.py
Outdated
target: CogniteClient | None = None, | ||
) -> None: | ||
"""Displays the differences between the source and target objects.""" | ||
local_import("deepdiff", "compare") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its a poor pattern to call into the import machinery deep within functions in Python - just because we can do it, doesn't mean we should.
There is also a developer affordance impact of having custom import layers - IDEs and other code level tools don't typically handle things that do imports but aren't import statements well.
I do see that you need this in some places, to switch to fallback codepaths, but that isn't the case here for anything in this module. -> If this function called from the contexts where you don't have deepdiff
available, it will fail at runtime.
So, please either:
- document the failure modes in the docstring, including what it will raise
- or lift the import out of the function so that code which has imported the function knows it is usable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an internal function and we do not want to add an extra dependency on DeepDiff to everyone that uses neat, thus we use a local import and mark this as an optional dependency. The local_import function raises an appropriate error if the dependency is missing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like it will be quite nice. However it is effectively not tested today, and if we're depending on it to tell us about the accuracy of the system, it must be.
I've left some thoughts on the structure and patterns as well.
Currently, this is used internally and as you are saying it is not well tested. The thinking is to extend this one as we continue to use it on our pilot customers. |
# Conflicts: # poetry.lock
Used in the migration work to do quick comparisons between node and source TimeSeries/Asset/Event/FileMetadata/Sequences.
This is not exposed to the end user and will likely not be yet, but we we need it as we do large scale tests across multiple customers.
Bump