Parser for AWS Inspector2 findings

[siniysv]

A parser for AWS Inspector2 ("new" Inspector) findings exported with aws inspector2 list-findings in JSON format.

Checklist

This checklist is for your information.

• Make sure to rebase your PR against the very latest dev.
• Features/Changes should be submitted against the dev.
• Bugfixes should be submitted against the bugfix branch.
• Give a meaningful name to your PR, as it may end up being used in the release notes.
• Your code is flake8 compliant.
• Your code is python 3.11 compliant.
• If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
• Model changes must include the necessary migrations in the dojo/db_migrations folder.
• Add applicable tests to the unit tests.
• Add the proper label to categorize your PR.

[sc-8014]

[dryrunsecurity]

DryRun Security Summary

The pull request primarily focuses on enhancing the integration between the DefectDojo application security platform and the AWS Inspector2 security scanning service, including documentation updates, configuration changes, a new AWS Inspector2 parser implementation, and expanded test suite.

Expand for full summary

Summary:

The changes in this pull request primarily focus on enhancing the integration between the DefectDojo application security platform and the AWS Inspector2 security scanning service. The key changes include:

1. Documentation Update: The addition of documentation for the AWS Inspector2 Scanner integration, providing details on the file format, example commands, and how to use the parser to import findings into DefectDojo.

2. Configuration Updates: Changes to the application's default configuration file (settings.dist.py) to add support for the AWS Inspector2 Scanner, including mapping attributes, deduplication algorithms, and hash code fields.

3. AWS Inspector2 Parser Implementation: The introduction of a new parser (dojo/tools/aws_inspector2/parser.py) that can process the JSON-formatted AWS Inspector2 scan reports and create corresponding Findings and Endpoints in DefectDojo.

4. Test Suite Expansion: The addition of several new unit tests (unittests/scans/aws_inspector2/*.json and unittests/tools/test_aws_inspector2_parser.py) to ensure the reliability and correctness of the AWS Inspector2 Scanner integration.

From an application security perspective, these changes are positive as they enhance the security capabilities of the DefectDojo platform by allowing it to ingest and process findings from the AWS Inspector2 security scanning service. This provides security teams with a more comprehensive view of their attack surface and potential vulnerabilities.

The changes to the configuration and the implementation of the AWS Inspector2 parser demonstrate a thoughtful approach to integrating the new scanner, including considerations for deduplication, severity mapping, and the extraction of detailed vulnerability information. The expanded test suite also helps ensure the reliability and robustness of the integration, which is crucial for maintaining the overall security of the application.

Files Changed:

1. docs/content/en/integrations/parsers/file/aws_inspector2.md: Added documentation for the AWS Inspector2 Scanner integration.
2. dojo/settings/.settings.dist.py.sha256sum: Updated the SHA256 hash value for the .settings.dist.py file.
3. dojo/settings/settings.dist.py: Added support for the AWS Inspector2 Scanner integration.
4. dojo/tools/aws_inspector2/parser.py: Implemented the parser for AWS Inspector2 scan reports.
5. unittests/scans/aws_inspector2/empty_with_error.json: Created an empty JSON file with an error.
6. unittests/scans/aws_inspector2/aws_inspector2_zero_vul.json: Created a JSON file with no vulnerabilities.
7. unittests/scans/aws_inspector2/aws_inspector2_one_vul.json: Created a JSON file with a single vulnerability.
8. unittests/scans/aws_inspector2/aws_inspector2_many_vul.json: Created a JSON file with multiple vulnerabilities.
9. unittests/tools/test_aws_inspector2_parser.py: Added new unit tests for the AWS Inspector2 parser.

Code Analysis

We ran 9 analyzers against 10 files and 0 analyzers had findings. 9 analyzers had no findings.

Riskiness

🟢 Risk threshold not exceeded.

View PR in the DryRun Dashboard.

[manuel-sommer]

A lot of AWS users use AWS Security Hub to retrieve all findings. Could you also advance this parser: https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/tools/awssecurityhub/inspector.py ?

Are the returned AWS Security Hub findings of Inspector2 equal to the returned findings of Inspector2 or are they differently formatted?

[mtesauro]

It's definitely preferred to have a single parser that understands multiple formats vs multiple parsers - if nothing else, it's easier for people to use who might not know the difference between Inspector JSON versions.

[siniysv]

thank you for your attention, I'll try to answer below.
Security Hub collects findings from inspector and a few other AWS services, then it converts the findings into one common format (ASFF - AWS Security Finding Format https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-findings-format.html) and provides API to retrieve the findings in that format. AWS Inspector provides own API with own finding format. It is not necessary to use Security Hub in order to use Inspector since those are 2 different AWS services. Answering to your @manuel-sommer question - yes, they are differently formatted.

I would extend the existing parser If I had a way to easily detect what finding format is used in an uploaded json, although the name of the parser would be misleading since AWS Security Hub and AWS Inspector are two different services.

If you believe DefectDojo needs only Security Hub parser because it is indeed an AWS service that can collect findings from other services (full list here https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-internal-providers.html), and there is no need to support standalone AWS service report import, I'm totally okay with this approach. Thank you.

[mtesauro]

Understood about AWS Security Hub vs AWS Inspector.

I would make sense in this case to have a parser just for AWS Inspector not that you've provided more details - TBH, I can't keep up with all the AWS whatever services.

You will need get the ruff linter happy (as well as the other tests) before the PR will be reviewed and merged (assuming we get 4 approvals).

I'm going to re-kick the tests now - if you want to do that yourself, you can close & open the PR to get them to restart assuming this isn't your first PR against DefectDojo.

[manuel-sommer]

Either add more fields as the title is probably not unique enough, or only use uniqueidfromtool. Furthermore, you forgot to edit the settings.dist.py hash sum https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/settings/.settings.dist.py.sha256sum

[Maffooch]

Suggested change

	},
	{
	"fields": {
	"name": "AWS Inspector2"
	},
	"model": "dojo.test_type",
	"pk": 8

This file is no longer modified when adding new parsers

[Maffooch]

It would make the code much easier to read by converting individual if cases into a function to process each resource type. This is

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[mtesauro]

Approved