Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new Mend Platform API 3.0 file types to existing Mend parser #11259

Merged
merged 64 commits into from
Nov 22, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
716f94a
Add new Mend Platform API 3.0 parser
testaccount90009 Nov 14, 2024
b2fe0b8
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
7e8a2fd
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
82e46ee
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
4f45fa6
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
2eec598
Update parser.py
testaccount90009 Nov 14, 2024
4cd5bb3
Update parser.py
testaccount90009 Nov 14, 2024
6faf27f
add component path as file_path
testaccount90009 Nov 14, 2024
5d6de56
updated parser
testaccount90009 Nov 14, 2024
0cbce00
Update parser.py
testaccount90009 Nov 14, 2024
2127d81
Update parser.py
testaccount90009 Nov 14, 2024
f1aa0ec
refactor
testaccount90009 Nov 14, 2024
bdc721a
Update parser.py
testaccount90009 Nov 14, 2024
fd76af6
Update parser.py
testaccount90009 Nov 14, 2024
9f3d41e
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
726f8c3
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
3977cd9
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
66d00c5
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
9605700
Update test_mend_platform_api3_parser.py
testaccount90009 Nov 14, 2024
8003d1f
Update parser.py
testaccount90009 Nov 14, 2024
e2911e2
change single quotes to double quotes - reformat broken json
testaccount90009 Nov 14, 2024
1e2cc67
Try to refactor to harden Mend parser instead of creating additional …
testaccount90009 Nov 14, 2024
bf8f9ce
Update parser.py
testaccount90009 Nov 14, 2024
f119ef7
Merge branch 'DefectDojo:dev' into dev
testaccount90009 Nov 14, 2024
64eaa50
adding author update
testaccount90009 Nov 14, 2024
5a9f279
Update parser.py
testaccount90009 Nov 14, 2024
52776b1
Add elif for "component" in content for list of Findings
testaccount90009 Nov 14, 2024
b97c5da
Update parser.py
testaccount90009 Nov 14, 2024
0acd9ee
Update parser.py
testaccount90009 Nov 14, 2024
6451542
Update parser.py
testaccount90009 Nov 14, 2024
132ec4d
Update parser.py
testaccount90009 Nov 14, 2024
7b33df5
Update parser.py
testaccount90009 Nov 14, 2024
e52d08c
Update parser.py
testaccount90009 Nov 14, 2024
2272d46
preserve the original else statement for grabbing severity
testaccount90009 Nov 14, 2024
45e723a
update parser to capture component_node array
testaccount90009 Nov 14, 2024
79d56b8
Update parser.py
testaccount90009 Nov 14, 2024
ee24c36
change how vulnerability description is retrieved
testaccount90009 Nov 14, 2024
1b1b242
Update parser.py
testaccount90009 Nov 14, 2024
b661790
Update parser.py
testaccount90009 Nov 14, 2024
2744a5f
updating vuln count
testaccount90009 Nov 14, 2024
b48a356
Update parser.py
testaccount90009 Nov 14, 2024
9313c2f
Update parser.py
testaccount90009 Nov 14, 2024
9313a86
Update parser.py
testaccount90009 Nov 14, 2024
f137019
Update parser.py
testaccount90009 Nov 14, 2024
c9477f9
Update parser.py
testaccount90009 Nov 14, 2024
503b7a1
Update parser.py
testaccount90009 Nov 14, 2024
f489b0e
Update parser.py
testaccount90009 Nov 14, 2024
6538d8a
Update parser.py
testaccount90009 Nov 14, 2024
d4f162f
Update parser.py
testaccount90009 Nov 14, 2024
29e8dd7
Update parser.py
testaccount90009 Nov 15, 2024
ba9dc02
Merge branch 'DefectDojo:dev' into dev
testaccount90009 Nov 15, 2024
b7c9d3d
Update parser.py
testaccount90009 Nov 18, 2024
75d9e7d
Merge branch 'DefectDojo:dev' into dev
testaccount90009 Nov 18, 2024
7404f44
Merge branch 'DefectDojo:dev' into dev
testaccount90009 Nov 18, 2024
9bd5f03
Merge branch 'DefectDojo:dev' into dev
testaccount90009 Nov 18, 2024
a705a5b
Fix spacing and formatting for easier readability
testaccount90009 Nov 18, 2024
8a396a3
Merge branch 'dev' of https://github.com/testaccount90009/django-Defe…
testaccount90009 Nov 18, 2024
a6c2db3
Rework Mitigation field - slight adjustment in formatting and edited …
testaccount90009 Nov 18, 2024
9e8f540
Fix comma and indents
testaccount90009 Nov 18, 2024
aeabdd4
Fix typo
testaccount90009 Nov 18, 2024
6202162
fix commas
testaccount90009 Nov 18, 2024
d221e3c
fix spacing
testaccount90009 Nov 18, 2024
a4b14f3
fix spacing for readability
testaccount90009 Nov 18, 2024
d2a612d
Merge branch 'DefectDojo:dev' into dev
testaccount90009 Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file.
169 changes: 169 additions & 0 deletions dojo/tools/mend_sca_platform_api3/parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
import hashlib
import json
import logging

from dojo.models import Finding

__author__ = "testaccount90009 aka SH"

logger = logging.getLogger(__name__)


class MendPlatformApi3Parser:
testaccount90009 marked this conversation as resolved.
Show resolved Hide resolved
def get_scan_types(self):
return ["Mend Platform APIv3 Scan"]

def get_label_for_scan_types(self, scan_type):
return "Mend Platform APIv3 Scan"

def get_description_for_scan_types(self, scan_type):
return "Import JSON report"

def get_findings(self, file, test):
if file is None:
return []

data = file.read()
try:
content = json.loads(str(data, "utf-8"))
except Exception:
content = json.loads(data)

def _build_common_output(node, lib_name=None):
# project only available in manual export
# name --> CVE in manual, library name in pipeline
cve = None
component_name = None
component_version = None
impact = None

if "component" in node:
description = (
"**Vulnerability Description** : "
+ node["vulnerability"].get("description", "")
+ "\n\n"
+ "**Component Name** : "
+ node["component"].get("name", "")
+ "\n\n"
+ "**Component Type** : "
+ node["component"].get("componentType", "")
+ "\n\n"
+ "**Root Library** : "
+ str(node["component"].get("rootLibrary", ""))
+ "\n\n"
+ "**Library Type** : "
+ node["component"].get("libraryType", "")
+ "\n\n"
+ "**Location Found** : "
+ node["component"].get("path", "")
+ "\n\n"
+ "**Direct or Transitive Dependency** : "
+ node["component"].get("dependencyType", "")
+ "\n"
)
lib_name = node["component"].get("name")
component_name = node["component"].get("artifactId")
component_version = node["component"].get("version")
impact = node["component"].get("dependencyType")
else:
description = node["vulnerability"].get("description", "")

cve = node.get("name")
if cve is None:
title = "CVE-None | " + lib_name
else:
title = cve + " | " + lib_name

cvss_sev = node.get("vulnerability", {}).get("severity", "UNKNOWN").lower().capitalize()

cvss3_score = node.get("vulnerability", {}).get("score", None)
cvss3_vector = node.get("scoreMetadataVector", None)
severity_justification = "CVSS v3 score: {} ({})".format(
cvss3_score if cvss3_score is not None else "N/A", cvss3_vector if cvss3_vector is not None else "N/A",
)

cwe = 1035 # default OWASP a9 until the report actually has them

# Handling Mitigation (topFix) safely
mitigation = "N/A"
if "topFix" in node:
try:
topfix_node = node.get("topFix", {})
mitigation = "**Resolution** ({}): {}\n".format(
topfix_node.get("date", "N/A"),
topfix_node.get("fixResolution", "N/A"),
)
except Exception as ex:
logger.exception("Error handling topFix node: %s", ex)

filepaths = []
if "sourceFiles" in node:
try:
sourceFiles_node = node.get("sourceFiles", [])
for sfile in sourceFiles_node:
filepaths.append(sfile.get("localPath", ""))
except Exception as ex:
logger.exception("Error handling sourceFiles for vulnerability: %s", ex)

locations = []
if "locations" in node:
try:
locations_node = node.get("locations", [])
for location in locations_node:
path = location.get("path", "")
if path:
locations.append(path)
except Exception as ex:
logger.exception("Error handling locations for vulnerability: %s", ex)

if locations:
filepaths = locations
else:
filepaths = filepaths

new_finding = Finding(
title=title,
test=test,
description=description,
severity=cvss_sev,
cwe=cwe,
mitigation=mitigation,
file_path=", ".join(filepaths),
component_name=component_name,
component_version=component_version,
severity_justification=severity_justification,
dynamic_finding=True,
cvssv3=cvss3_vector,
cvssv3_score=float(cvss3_score) if cvss3_score is not None else None,
impact=impact,
)
if cve:
new_finding.unsaved_vulnerability_ids = [cve]

return new_finding

findings = []
if "libraries" in content:
tree_libs = content.get("libraries", [])
for lib_node in tree_libs:
if "response" in lib_node and len(lib_node.get("response", [])) > 0:
for vuln in lib_node.get("response", []):
findings.append(_build_common_output(vuln, lib_node.get("name")))
elif "response" in content:
tree_node = content.get("response", [])
for node in tree_node:
findings.append(_build_common_output(node))

def create_finding_key(f: Finding) -> str:
"""Hashes the finding's description and title to retrieve a key for deduplication."""
return hashlib.md5(
f.description.encode("utf-8") + f.title.encode("utf-8"),
).hexdigest()

dupes = {}
for finding in findings:
dupe_key = create_finding_key(finding)
if dupe_key not in dupes:
dupes[dupe_key] = finding

return list(dupes.values())
Loading
Loading