feat(security-scanning): image for scanning

[Jose-Matsuda]

Temporary measure to bypass credential setting on new repo.
Does not do any updates, will just print data to console.

Dependent on StatCan/aaw#960 as step b will not actually pull the image in it's entirety, it will just be a .marker file, and will thus not be scanned by XRAY.

[vexingly]

It's a bit hard to follow all of the files (the letters helped), but they will make the future steps easier and provide verbose details of each step!

Seems like you have covered every detail I could think of, looks good!

I'd be interested to see some output of the log when it matches a critical CVE at this stage if you have something.

[Jose-Matsuda]

Yeah I agree it's a bit hard to follow, its a lot easier when you look at the outputs of each step. There is some example output here https://github.com/Jose-Matsuda/artifactory-cleanup/tree/main/2022/z-Example-outputs (though it might not match completely anymore, this was the general idea).

At this point the the outputs would just be the affected images, or images not found.

[vexingly]

Will this watch build up over time with a large number of items in the list or is it flushed out periodically? Alternatively is there a time component to the filter that could be used if necessary to limit to recent scans?

[Jose-Matsuda]

fair play, yeah we could use a created_from field to ensure that we only grab 'new' violations that are generated with a specific pull, that could be included in a later update to the Dockerfile as this isn't fully complete of course, just the initial implementation is here

[vexingly]

Yeah I agree it's a bit hard to follow, its a lot easier when you look at the outputs of each step. There is some example output here https://github.com/Jose-Matsuda/artifactory-cleanup/tree/main/2022/z-Example-outputs (though it might not match completely anymore, this was the general idea).

At this point the the outputs would just be the affected images, or images not found.

Those are really helpful and interesting, thanks! One thing from the example that I am wondering about, any concerns when it picks up an artifact using the 'latest' tag? Could we get into a scenario where the vulnerability report 'latest' is not the same as the one in a notebook? Not sure if its a valid concern or not possible in an actual notebook.

[Jose-Matsuda]

Yeah I agree it's a bit hard to follow, its a lot easier when you look at the outputs of each step. There is some example output here https://github.com/Jose-Matsuda/artifactory-cleanup/tree/main/2022/z-Example-outputs (though it might not match completely anymore, this was the general idea).
At this point the the outputs would just be the affected images, or images not found.

Those are really helpful and interesting, thanks! One thing from the example that I am wondering about, any concerns when it picks up an artifact using the 'latest' tag? Could we get into a scenario where the vulnerability report 'latest' is not the same as the one in a notebook? Not sure if its a valid concern or not possible in an actual notebook.

Very Valid concern. So after the discussion we had with StatCan/aaw#961, we did decide to eventually move to a pattern that will actually have a long lived tag (similar to latest). And using that tag, we will also have say weekly updates to the everyone's notebook images to get on the latest version of what we have deployed in aaw-kubeflow-containers. Admittedly, before this and the research I was not considering using a long lasting tag.

With the long lived tag (say a v1) when we push say jupyterlab-cpu:v1 to the ACR, it will clobber and annihilate the previous jupyterlab-cpu:v1 tag which makes me now see where you are coming from.
There can be a case where images on a notebook's pods can be different from what we pull from the ACR. For example, on a Monday we fix a critical CVE on kubeflow-containers and push to main and that will update the v1 tag. Then this cronjob runs nightly and then bam it pulls the jupyterlab-cpu:v1 and while scanning that for CVE's.

For our current implementation it is not a problem, but when we do move towards that we will need to consider pulling by the digest and not the tag dockerimage@sha256: