Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add maxLength validation keywords matching nvarchar values in archive_catalog #448

Merged
merged 3 commits into from
Sep 30, 2024

Conversation

braingram
Copy link
Collaborator

@braingram braingram commented Sep 20, 2024

Closes spacetelescope/roman_datamodels#342

Add maxLength validation keywords matching nvarchar values in archive_catalog portions of the schemas. A few cases were not handled:

  • nvarchar(max) as max is undetermined
  • nvarchar datatype for non-string values. I don't see any way to enforce these as the thing being validated is not a string. Examples are:

The TVAC and FPS schemas are not updated by this PR.

A test was added to check that these match which walks the schemas looking for archive_catalog and if it contains a nvarchar (that isn't max) and if the subschema has type: string the test checks the subschema has a maxLength keyword and that it's value matches the one in nvarchar. As this PR fixes the schemas no errors are seen in the CI but some examples of errors are:

  • For a missing maxLength: AssertionError: archive_catalog has nvarchar, schema asdf://stsci.edu/datamodels/roman/schemas/aperture-1.0.0 is missing maxLength
  • For a mismatch: AssertionError: archive_catalog nvarchar does not match maxLength in schema asdf://stsci.edu/datamodels/roman/schemas/aperture-1.0.0

Regression tests running with this PR and roman_datamodels dev (to pick up changes in spacetelescope/roman_datamodels#388):
https://github.com/spacetelescope/RegressionTests/actions/runs/11060303121

Prior to merging this PR I think it makes sense to set the romancal required version for rad and roman_datamodels to dev (so that subsequent work on rad and roman_datamodels can be tested).

Tasks

  • Update or add relevant rad tests.
  • Update relevant docstrings and / or docs/ page.
  • Does this PR change any schema files?
    • Schema changes were discussed at RAD Review Board meeting.
  • Does this PR change any API used downstream? (If not, label with no-changelog-entry-needed.)
News fragment change types:
  • changes/<PR#>.feature.rst: new feature
  • changes/<PR#>.bugfix.rst: fixes an issue
  • changes/<PR#>.doc.rst: documentation change
  • changes/<PR#>.removal.rst: deprecation or removal of public API
  • changes/<PR#>.misc.rst: infrastructure or miscellaneous change

@braingram braingram changed the title Varchar go vroom Add maxLength validation keywords matching nvarchar values in archive_catalog Sep 20, 2024
Copy link

codecov bot commented Sep 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.81%. Comparing base (f6c4abc) to head (057bf92).
Report is 166 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #448      +/-   ##
==========================================
+ Coverage   95.45%   95.81%   +0.35%     
==========================================
  Files           4        4              
  Lines         198      215      +17     
==========================================
+ Hits          189      206      +17     
  Misses          9        9              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@schlafly
Copy link
Collaborator

Thanks Brett. That's great. I had forgotten the read_pattern and rotation_matrix cases. I think you're right to ignore these as those lengths were picked to be very large and it's hard to see how they will cause problems. It's also good not to touch TVAC & FPS.

@braingram

This comment has been minimized.

@schlafly

This comment has been minimized.

@braingram

This comment has been minimized.

@braingram braingram marked this pull request as ready for review September 26, 2024 22:06
@braingram braingram requested a review from PaulHuwe September 26, 2024 22:06
Copy link
Collaborator

@PaulHuwe PaulHuwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on pointing to dev for tickets (other PRs will need it as well).

@braingram braingram merged commit 8a4452a into spacetelescope:main Sep 30, 2024
13 checks passed
@braingram braingram deleted the varchar_go_vroom branch September 30, 2024 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add String Length Enforcement to Validate
5 participants