Allow scanning unpacked container filesystems #1485
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jedevc it looks like you might just need to update a test (the ID is calculated based on the fields in the struct and one was added): https://github.com/anchore/syft/actions/runs/3958281654/jobs/6779661340#step:7:165 |
jedevc
commented
Jan 19, 2023
We can use the already existing file tree to peform symlink resolution for FilesByPath, instead of traversing the symlinks again. This moves all of the symlink logic into the indexing code, and then we can rely on syft's resolution algorithm over the index in this part of the codebase. Signed-off-by: Justin Chadwell <me@jedevc.com>
The new base parameter is an optional parameter for the directory resolver that resolves all symlinks relative to this root. There are two intended use cases: - base = "/". The previous behavior, symlinks are resolved relative to the root filesystem. - base = path. Symlinks are resolved relative to the target filesystem, allowing correct behavior when scanning unpacked container filesystems on disk. Signed-off-by: Justin Chadwell <me@jedevc.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
jedevc
force-pushed
the
symlink-base-parameter
branch
from
d176a76 to
d1d5fa1
Compare
wagoodman
approved these changes
Jan 30, 2023
This was referenced Jan 30, 2023
This was referenced Feb 6, 2023
|
Would love to the see this feature make it to the CLI. We have a use case for wanting to run syft on a root filesystem and have it work just like syft does against an existing docker image today. |
|
I don't currently have the bandwidth to go and add the feature, but it should be a fairly simple plumbing exercise now that the underlying code is there, so happy for someone else to pick that up 🎉 |
GijsCalis
pushed a commit
to GijsCalis/syft
that referenced
this pull request
Feb 19, 2024
* source: avoid second-step of symlink resolution in directory resolver We can use the already existing file tree to peform symlink resolution for FilesByPath, instead of traversing the symlinks again. This moves all of the symlink logic into the indexing code, and then we can rely on syft's resolution algorithm over the index in this part of the codebase. Signed-off-by: Justin Chadwell <me@jedevc.com> * source: add base parameter to directory resolver The new base parameter is an optional parameter for the directory resolver that resolves all symlinks relative to this root. There are two intended use cases: - base = "/". The previous behavior, symlinks are resolved relative to the root filesystem. - base = path. Symlinks are resolved relative to the target filesystem, allowing correct behavior when scanning unpacked container filesystems on disk. Signed-off-by: Justin Chadwell <me@jedevc.com> * source: add tests for new base parameter Signed-off-by: Justin Chadwell <me@jedevc.com> --------- Signed-off-by: Justin Chadwell <me@jedevc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🐛 Fixes #1359 🎉
This PR is an attempt to explore how to modify the existing directory resolver to allow for resolving symlinks in an unpacked container filesystem. To do this, we add a new optional
baseparameter for the directory resolver that resolves all symlinks relative to this base. There are two intended use cases:base = "/". The previous behavior, symlinks are resolved relative to the root filesystem.base = path. Symlinks are resolved relative to the target filesystem, allowing correct behavior when scanning unpacked container filesystems on disk.Currently, this only modifies the API surface for the source, which would probably be sufficient for my use case in buildkit-syft-scanner.
I've marked this as in-draft since I've only done some preliminary testing, but the following things probably need working out before this could be ready:
root-dirinput type, or similar? Not quite sure on naming, I don't have any particular preference here.filepath.EvalSymlinkswithcfs.RootPathfrom containerd/continuity. The logic of these functions is fairly similar, however, the containerd equivalent allows bounding the result to a specified directory.This logic is only needed for 1 of the symlink resolution components - the other is much simpler, since syft does the heavy lifting. I'm not sure how the maintainers feel about adding this as a new dependency? Maybe there's a simpler solution I'm not seeing 👀
Update: I can remove this dependency, since we can avoid this second instance of doing symlink resolution - we can just use the file tree that we've already indexed over, which should be more efficient anyways. I think that this is a valid use? It seems to work in the cases I tried it on.
cc @wagoodman @kzantow