Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NEAT-808]🐫 Migration tooling to utils #1009

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Conversation

doctrino
Copy link
Collaborator

@doctrino doctrino commented Feb 25, 2025

Used in the migration work to do quick comparisons between node and source TimeSeries/Asset/Event/FileMetadata/Sequences.

This is not exposed to the end user and will likely not be yet, but we we need it as we do large scale tests across multiple customers.

image

Bump

  • Patch
  • Minor
  • Skip

Copy link

github-actions bot commented Feb 25, 2025

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
14986 11998 80% 60% 🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: d27c49b by action🐍

Comment on lines 20 to 36
"Class Description": "HIERARCHY",
"Classification": "HIER",
"GL Account": "287839.0000.109.??????.???",
"P&ID": "KL47DSF3001",
"Primary Craft": "ROTATING",
"Primary OTSU": "A1-20.01",
"Priority": "99",
"PublishEventDate": "2024-12-03T21:13:31.186Z",
"Site": "KL47",
"Unit": "DS",
"lastUpdatedTime": "2024-12-03 21:18:45.869",
"locationLocationsID": "105876",
"locationParentID": "12966",
"location_Location": "DS/NH3/DINGS/628-3-I/728-2-FLOT",
"site_location": "KL47-DS/NH3/DINGS/628-3-I/728-2-FLOT",
"site_unit_tag": "KL47-DS-728-2-FLOT",
"unit_tag": "DS-728-2-FLOT",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inspired by the original data, but changes all letters and numbers.

nikokaoja
nikokaoja previously approved these changes Feb 25, 2025
nikokaoja
nikokaoja previously approved these changes Feb 25, 2025
@doctrino doctrino requested review from a team and rbtcollins and removed request for a team February 25, 2025 10:42
@doctrino doctrino added auto-bake Autogenerate manifests with Baker if there is a diff auto-merge Bulldozer auto-merge and removed auto-bake Autogenerate manifests with Baker if there is a diff labels Feb 25, 2025
target: CogniteClient | None = None,
) -> None:
"""Displays the differences between the source and target objects."""
local_import("deepdiff", "compare")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a poor pattern to call into the import machinery deep within functions in Python - just because we can do it, doesn't mean we should.

There is also a developer affordance impact of having custom import layers - IDEs and other code level tools don't typically handle things that do imports but aren't import statements well.

I do see that you need this in some places, to switch to fallback codepaths, but that isn't the case here for anything in this module. -> If this function called from the contexts where you don't have deepdiff available, it will fail at runtime.

So, please either:

  • document the failure modes in the docstring, including what it will raise
  • or lift the import out of the function so that code which has imported the function knows it is usable.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an internal function and we do not want to add an extra dependency on DeepDiff to everyone that uses neat, thus we use a local import and mark this as an optional dependency. The local_import function raises an appropriate error if the dependency is missing.

Copy link

@rbtcollins rbtcollins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it will be quite nice. However it is effectively not tested today, and if we're depending on it to tell us about the accuracy of the system, it must be.

I've left some thoughts on the structure and patterns as well.

@doctrino
Copy link
Collaborator Author

This looks like it will be quite nice. However it is effectively not tested today, and if we're depending on it to tell us about the accuracy of the system, it must be.

I've left some thoughts on the structure and patterns as well.

Currently, this is used internally and as you are saying it is not well tested. The thinking is to extend this one as we continue to use it on our pilot customers.

nikokaoja
nikokaoja previously approved these changes Feb 26, 2025
@doctrino doctrino enabled auto-merge (squash) February 28, 2025 07:06
@doctrino doctrino changed the title 🐫 Migration tooling to utils [NEAT-808]🐫 Migration tooling to utils Mar 1, 2025
@doctrino doctrino requested a review from nikokaoja March 1, 2025 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants