Add Container Security Scanning to CI/CD Process

[lewisdaly]

Goal:

As an OSS Maintainer
I want to enable container security scans in the CI/CD Process
so that containers built and released by Mojaloop releases for services are secure and any security issues can be identified

Tasks:

• Locally compare sysdig and anchore-cli tools

It looks like sysdig and anchore-cli do quite different things (and can actually play quite nicely together). Sysdig is more for runtime container security management, while anchore-cli is for build time security scanning

• Pending the above decision, integrate the chosen tool into the CI/CD process of the account-lookup-service
• Evaluate the above, and if we are happy, go ahead and integrate into the CI/CD process of the following:
  • bulk-api-adapter [PR]
  • central-event-processor [PR]
  • central-ledger [PR]
  • central-settlement [PR]
  • email-notifier [PR]
  • ml-api-adapter [PR]
  • quoting-service [PR]
  • mojaloop-simulator [PR]
    - [ ] central-directory? Out of scope
    - [ ] simulator? Out of scope

Acceptance Criteria:

• Container scans are added across all codebases which produce Docker images
• Appropriate reports are accessible in CircleCI artifacts

Pull Requests:

• Feature/1003 add container scans account-lookup-service#103
• feature/1003 tidying circleci config and removing unused branches account-lookup-service#165
• feature/1003 Add Anchore image scanning, upgrade to CircleCI 2.1 central-event-processor#89
• feature/1003 Add Anchore image scanning, upgrade to CircleCI 2.1 bulk-api-adapter#16
• Feature/1003 container scan policy central-settlement#195
• Feature/1003 add container scans central-ledger#498
• feature/1003 add container scans email-notifier#85
• Feature/1003 add container scans 3 ml-api-adapter#291
• Feature/1003 add container scans quoting-service#103
• Add anchore image scanning, update circleci config mojaloop-simulator#22

Follow-up:

• Expand this scanning capability to all currently deployed versions of the given mojaloop containers. This exists beyond the scope of an individual release.
• Add container scanning to the Helm releases?
• Create separate stories for issues identified during initial scans

Dependencies:

• N/A

Accountability:

• Owner: @lewisdaly
• QA/Review: TBC

[lewisdaly]

I've enabled the scans for a branch of the account-lookup-service:

https://circleci.com/gh/mojaloop/account-lookup-service/1596#artifacts/containers/0

There's a few issues we need to figure out, such as:

• if policy_failure in the image-scan step is set to true, the build artifacts aren't generated

[lewisdaly]

Here's an example of the updated CI workflow:

I've separated out the build and publish steps, so that we can insert the docker image license-scan and image-scan in parallel here.

[elnyry-sam-k]

@lewisdaly thanks for the updates..

Looking at the snapshot, is the container security scanning enabled only for releases/snapshots (according to the snapshot) or even PRs?

[lewisdaly]

I have it set up only for releases/snapshots, but that is because historically, we only build a docker image for the release/snapshot tags.

Is that fine? Or we could change it to build a docker image on every branch/tag/PR, and only publish the docker images if we are building for a release/snapshot.

[lewisdaly]

I should note that the image scan is quite a long process (5 mins +), so putting it on every PR might get annoying.

[elnyry-sam-k]

In fact that was what I was leaning towards as well - to have it only for snapshots / releases and not for PRs, since the build happens only for those anyway..

[rasputtintin]

This is great work. Can we look at:
1 - Policy Settings so we can configure our baselines
2 - Reporting / Alerts to allow us do scheduled scans and have reports emailed to relevant teams to re-mediate
3 - Explore ways of tracking issues over time so we can report on progress / trends.

[lewisdaly]

@rasputtintin here is a link to the default policy that is being applied:

https://github.com/anchore/hub/blob/master/sources/bundles/anchore_default_bundle.json

I'm not exactly sure how it works, but perhaps we can continue the integration on the other repos with just the default policy for now.

[lewisdaly]

@elnyry Marked central-directory and simulator as out of scope since I want to focus our attention on services that not be deprecated in the near future.