-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(oscal): link remapper method component defn #879
base: main
Are you sure you want to change the base?
Conversation
…component-definitions
…results' into 810-partial-component-compositions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel a bit uncomfortable with this implementation, but not so much that I'd block it - it's partly based on not entirely understanding what's going on in here and partly based on my entirely fuzzy mental image of a future refactor, which isn't exactly a valid consideration 😂
Yeah I feel this too 😆 - I really wasn't sure how to do this optimally/reusably... So I think the bigger picture idea is that you can have an OSCAL model (this is just for component definition, but really any OSCAL Model) that can have links to other resources (remote or local). If we want to sort of package up all this data we may need to re-write the paths. Additionally, in the case of a component definition, it can import other component definitions (which contain these links) and those internal references will need to be re-written when the data gets imported to the importer model (ala, what we do with Compose right now, although those don't resolve paths, that function just puts all validations in the backmatter, which is not great for some of the use cases we've discussed with the SSP, etc). The initial thought on this was to brute force look through all the spots where you'd possibly need to change a reference, and just do it. Although you can see from the list of items in |
My discomfort is because this feels like an implementation for iterating over raw json, but we have concrete types, so I'd expect something that iterated over each type. Here's some probably-awful pseudo code: func (c ComponentDefinition) RewritePaths (paths, anythingelse) error { Though for the actual implementation (and this is where my handwavey future ideas come into play, so again THIS MIGHT NOT MAKE SENSE for right now/ever) I'd probably have each type with links (every field inside the ComponentDefinition) fulfill an interface that includes a RewritePaths method, so we could handle each separately. |
Ok fair enough, yeah agreed maybe it would be better to just write a super long function and have all the types checked by type and not with reflect. This does get a little tedious as well because of all the pointers... To be honest, this is probably something I could coerce chat-gipity to give me so I didn't have to write it manually 🤣 For the future idea, I truly don't think I know enough about go-oscal to know if that's do-able. But maybe worth investigating more... I suppose for the near term we're probably trading the two options: using the reflect implementation or write a big function. (selfishly/lazily I do think this implementation is easier to test/understand) |
@brandtkeller if you have any weigh-ins on this TIA! |
Reading back through the issue and then this implementation - I am weighing the "what problem does this solve now" with "are the surrounding assumptions correct". Yes the solutions complexity obfuscates some of the operations - I had to spend some time with debugging to get a clear-ish picture. But we have an original problem statement - composing component definitions that import other component definitions and remapping paths. Relevant problem for us - as each imported component-definition has potential links to Lula Validations or potentially other artifacts (which is why I appreciated the breadth of support in this PR). Apologies in advance. I believe we see the intent of a Do we see ourselves having any mechanism of getting away from a |
@brandtkeller @mildwonkey - Updated this PR to remove the reflect method (as that sounded broadly unpopular 😅 ) in favor of the verbose model walk/rewrite in place |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the re-write - debugging traceability really improved my perspective.
I'm a bit of a mess - this is all notional work not tied into execution currently - do we have issue(s) that add this to current execution?
#896 may be the execution implementation? |
That's right - trying to decide if that PR should also include instrumenting the method in the generate SSP function or if that should be separate. But that's really the endgame here: enable the SSP having the right links to pass through to the Assessment Plan (so then we can do a thing with that). |
…hub.com/defenseunicorns/lula into 878-link-remapper-method-component-defn
6d4a4ad
Alright, I know you all already approved this but I think @mildwonkey was right about refactoring that huge function (ultimately I do see reuse across models, so the goal was to make that more reusable/clear/testable) - with that, I decided to not be lazy and made the following updates:
|
@@ -549,3 +656,11 @@ func FuzzReadValidationsFromYaml(f *testing.F) { | |||
common.ReadValidationsFromYaml(a) | |||
}) | |||
} | |||
|
|||
func FuzzRemapPath(f *testing.F) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉 nice!!!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 you may be able to round-trip this: (If this is valid) you could re-map back to the original path and add a check that last returned path is equal to the original test input.
Something like this?
gotPath := remap path (a, b, c)
reversePath := (gotPath, c, b)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I love the changes! I left one incredibly minor comment on handlePath
but I think this is looking great.
return handlePath(filepath.Join(url.Host, url.RequestURI()), baseDir) | ||
} | ||
|
||
func handlePath(path, baseDir string) string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method name isn'g telling me anything, and it's not obvious to me why it returns an empty string if given an abs path but a full path to a directory if given a relative path. Can you add some comments that explains why this is the behavior? And maybe rename it - returnBaseDirIfNotAbsPath
seems like a more accurate description of what's happening (though I still wonder why)
@@ -102,6 +102,74 @@ func (c *ComponentDefinition) HandleExisting(path string) error { | |||
return nil | |||
} | |||
|
|||
// RewritePaths finds all the paths in the component definition relative to the baseDir and updates them to be relative to the newDir | |||
// baseDir and newDir must be absolute paths | |||
func (c *ComponentDefinition) RewritePaths(baseDir string, newDir string) (err error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I love the changes here ❤️
Description
Added a new method to component definition -
RewritePaths
Intent of this method is to facilitate aggregation of data in component definitions (e.g., via Compose functionality) where the references contained therein may be relative to said component definition. This method should also have extensibility to be used for a future
packaging
type workflow where we may wish to pull disaggregated OSCAL models into a centralized, opinionated structure. Hence, this method may make sense to add to the interface when it's been implemented across models.Notes to reviewers
GetLocalFileDir
-> before it was doing a file existence check which I think is probably not needed in this function itself. Removal allowed for easier testing. Also needed to update this function to check for the opaque structure (before was just checking host) - see net/url for referenceRelated Issue
Fixes #878
Type of change
Checklist before merging