Adding the new NIST CVE Tracking feature for 3.x release. #365

bminnix · 2024-08-08T17:49:36Z

Closes: #80

This issue was marked as closed, but then briefly discussed in #81 that is open regarding last_modified_date (or similar). See summary below.

What's Changed

Summary:

Creates field for last_modified_date (Issue Feature Request: Last Updated Date field (for the vendor's update timestamps) #80) that is used as a method for NIST to know when a record needs updating.
Adds job that goes through all SoftwareVersion objects and queries NIST for associated CVE records.
- Creates/Updates the CVE record and associates to the SoftwareVersion being queried

New "NIST - Software CVE Search" Job

Job Results

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

progala · 2024-08-12T17:38:09Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+        """Initializing job with extra options."""
+        super().__init__()
+        self.nist_api_key = getenv("NIST_API_KEY")
+        self.sleep_timer = 0.75


Should this be configurable? Feels a bit arbitrary.

progala · 2024-08-12T17:42:47Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+
+        # Set session attributes for retries
+        self.session = Session()
+        retries = Retry(


Should this be configurable?

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

progala · 2024-08-12T18:03:54Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+        """Converts CVE info into DLC Model compatibility."""
+        cve = cve_json
+
+        # cve_base = cve["vulnerabilities"][0]['cve']


Is this still needed?

I believe the last commit resolved this.

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

progala · 2024-08-12T18:05:24Z

nautobot_device_lifecycle_mgmt/migrations/0031_cvelcm_last_modified_date.py

+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("nautobot_device_lifecycle_mgmt", "0030_delete_models_migrated_to_core"),


We'll need to rename/regenerate this. I had to pull in migration for Nautobot 2.3 compatibility into next3.0

This has been regenerated.

Creating a batched commit with several of the syntax update recommendations. Co-authored-by: Przemek Rogala <progala@progala.net>

…ommendations

…mproper upgrade

progala · 2024-08-13T14:55:10Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+    def __init__(self):
+        """Initializing job with extra options."""
+        super().__init__()
+        self.nist_api_key = getenv("NAUTOBOT_DLM_NIST_API_KEY")


Thinking about this again. We should move this to the plugin config and set the default in app init. Then we can do:

self.nist_api_key = PLUGIN_CFG["NAUTOBOT_DLM_NIST_API_KEY"]

Also, we should guard for this being empty and return early with error.

progala · 2024-08-13T14:58:50Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+        self.headers = {"ContentType": "application/json", "apiKey": self.nist_api_key}
+
+        # Set session attributes for retries
+        self.session = Session()


We always send the same headers so let's set them when we init our Session object.

Suggested change

self.headers = {"ContentType": "application/json", "apiKey": self.nist_api_key}

# Set session attributes for retries

self.session = Session()

headers = {"ContentType": "application/json", "apiKey": self.nist_api_key}

# Set session attributes for retries

self.session = Session()

self.session.headers.update(headers)

progala · 2024-08-13T15:30:05Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+    @staticmethod
+    def create_cpe_software_search_urls(vendor: str, platform: str, version: str) -> list:


I don't think this method is needed. We are making call to the external code, we could move it to the caller method.

progala · 2024-08-13T15:51:12Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+                self.associate_software_to_cve(software.id, matching_dlc_cve.id)
+                if str(cve_info["modified_date"][0:10]) != str(matching_dlc_cve.last_modified_date):
+                    self.update_cve(matching_dlc_cve, cve_info)
+                    continue


This is not needed.

Suggested change

continue

progala · 2024-08-13T15:56:32Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+    def associate_software_to_cve(self, software_id, cve_id):
+        """A function to associate software to a CVE."""
+        cve = CVELCM.objects.get(id=cve_id)
+        software = SoftwareVersion.objects.get(id=software_id)


We already have those objects in the method that's calling this one. Perhaps we could get rid of this method and do the association directly in the calling method.

progala · 2024-08-13T16:07:12Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+
+        self.logger.info("Created New CVEs.", extra={"grouping": "CVE Creation"})
+
+    def get_cve_info(self, cpe_software_search_urls: list, software_id=None) -> dict:


We already have SoftwareVersion object in the calling method so it's probably better to use that instead of passing software_id around.

progala · 2024-08-13T16:10:18Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+                    extra={"object": SoftwareVersion.objects.get(id=software_id), "grouping": "CVE Creation"},
+                )
+                cve_list = [cve["cve"] for cve in result["vulnerabilities"]]
+                dlc_cves = [cve.name for cve in CVELCM.objects.all()]


Suggested change

dlc_cves = [cve.name for cve in CVELCM.objects.all()]

dlc_cves = CVELCM.objects.values_list("name", flat=True)

progala · 2024-08-13T16:13:51Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+        if cve_list:
+            for cve in cve_list:
+                cve_name = cve["id"]
+                if cve_name.startswith("CVE"):
+                    if cve_name not in dlc_cves:
+                        processed_cve_info["new"].update({cve_name: self.prep_cve_for_dlc(cve)})
+                    else:
+                        processed_cve_info["existing"].update({cve_name: self.prep_cve_for_dlc(cve)})
+            self.logger.info(
+                "Prepared %s CVE for creation." % len(processed_cve_info["new"]),
+                extra={"object": SoftwareVersion.objects.get(id=software_id), "grouping": "CVE Creation"},
+            )


Suggested change

if cve_list:

for cve in cve_list:

cve_name = cve["id"]

if cve_name.startswith("CVE"):

if cve_name not in dlc_cves:

processed_cve_info["new"].update({cve_name: self.prep_cve_for_dlc(cve)})

else:

processed_cve_info["existing"].update({cve_name: self.prep_cve_for_dlc(cve)})

self.logger.info(

"Prepared %s CVE for creation." % len(processed_cve_info["new"]),

extra={"object": SoftwareVersion.objects.get(id=software_id), "grouping": "CVE Creation"},

)

if not cve_list:

return processed_cve_info

for cve in cve_list:

cve_name = cve["id"]

if not cve_name.startswith("CVE"):

continue

if cve_name not in dlc_cves:

processed_cve_info["new"].update({cve_name: self.prep_cve_for_dlc(cve)})

else:

processed_cve_info["existing"].update({cve_name: self.prep_cve_for_dlc(cve)})

self.logger.info(

"Prepared %s CVE for creation." % len(processed_cve_info["new"]),

extra={"object": SoftwareVersion.objects.get(id=software_id), "grouping": "CVE Creation"},

)

progala · 2024-08-13T16:14:23Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+            dict: Dictionary of returned results if successful.
+        """
+        try:
+            result = self.session.get(url, headers=self.headers)


We'll set headers when we init session.

Suggested change

result = self.session.get(url, headers=self.headers)

result = self.session.get(url)

progala · 2024-08-13T16:15:16Z

nautobot_device_lifecycle_mgmt/jobs/cve_tracking.py

+                "The NIST Service is currently unavailable. Status Code: %s. Try running the job again later.", code
+            )
+
+        return result.json()


Do we need try..except in case JSON decoding fails?

bminnix requested review from jvanderaa and progala as code owners August 8, 2024 17:49

bminnix force-pushed the app-nist-cve-sync3.0 branch 3 times, most recently from 9e9de39 to 7cf3d04 Compare August 10, 2024 06:31

bminnix changed the title ~~Adding the new NIST CVE Tracking feature to the next3.0 base.~~ Adding the new NIST CVE Tracking feature for 3.x release. Aug 10, 2024

bminnix force-pushed the app-nist-cve-sync3.0 branch from 7cf3d04 to d543611 Compare August 12, 2024 16:48

Adding the new NIST CVE Tracking feature for 3.x release.

1d7f8bd

bminnix force-pushed the app-nist-cve-sync3.0 branch from 3dfb613 to 1d7f8bd Compare August 12, 2024 17:40

progala requested changes Aug 12, 2024

View reviewed changes

bminnix and others added 3 commits August 12, 2024 15:15

Apply suggestions from code review

f127877

Creating a batched commit with several of the syntax update recommendations. Co-authored-by: Przemek Rogala <progala@progala.net>

Updating a couple of things for linting before moving on to other rec…

4c45575

…ommendations

Pinning django-debug-toolbar to <4.4, seems to be what's causing an i…

8153f18

…mproper upgrade

progala requested changes Aug 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding the new NIST CVE Tracking feature for 3.x release. #365

Adding the new NIST CVE Tracking feature for 3.x release. #365

bminnix commented Aug 8, 2024

progala Aug 12, 2024

progala Aug 12, 2024 •

edited

Loading

progala Aug 12, 2024

bminnix Aug 12, 2024

progala Aug 12, 2024

bminnix Aug 12, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

progala Aug 13, 2024

		@staticmethod
		def create_cpe_software_search_urls(vendor: str, platform: str, version: str) -> list:


		self.logger.info("Created New CVEs.", extra={"grouping": "CVE Creation"})

		def get_cve_info(self, cpe_software_search_urls: list, software_id=None) -> dict:

	dlc_cves = [cve.name for cve in CVELCM.objects.all()]
	dlc_cves = CVELCM.objects.values_list("name", flat=True)

	result = self.session.get(url, headers=self.headers)
	result = self.session.get(url)

Adding the new NIST CVE Tracking feature for 3.x release. #365

Are you sure you want to change the base?

Adding the new NIST CVE Tracking feature for 3.x release. #365

Conversation

bminnix commented Aug 8, 2024

Closes: #80

What's Changed

Summary:

New "NIST - Software CVE Search" Job

Job Results

Choose a reason for hiding this comment

progala Aug 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

progala Aug 12, 2024 •

edited

Loading