[Bug] SSL verification failing for majority of analyzers.

[redwolfey]

For the majority of the analyzer's I'm getting SSL verification errors from Python. See below for logs.
I think I know what's causing it, they are sitting behind our corp firewall and the traffic is getting routed through ZScaler, which is essentially MiTM-ing it. I have added the ZScaler root cert to the OS (CentOS 7), and everything appears to be working other than these analyzers.

To my knowledge there is know way to add this as a traditional proxy, as it is getting forwarded to ZScaler from our firewall.

To Reproduce
Steps to reproduce the behavior:

1. Be behind ZScaler ZIA
2. Run almost any analyzer.
3. SSL Cert verification errors

Complementary information

Traceback (most recent call last):  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen  \
chunked=chunked,  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, \ in _make_request    self._validate_conn(conn)  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn    conn.connect()  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 360, in connect    \ ssl_context=context,  File "/usr/local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 370, \
 in ssl_wrap_socket    return context.wrap_socket(sock, server_hostname=server_hostname) \
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket    _context=self, _session=session) \
 File "/usr/lib64/python3.6/ssl.py", line 773, in __init__    self.do_handshake()  File \ "/usr/lib64/python3.6/ssl.py", line 1033, in do_handshake    self._sslobj.do_handshake()  File \ "/usr/lib64/python3.6/ssl.py", line 645, in do_handshake    self._sslobj.do_handshake()ssl.SSLError: \[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)During handling of the above \ exception, another exception occurred:Traceback (most recent call last):  File \ "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send    timeout=timeout  \ File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen    \ method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment  \ 
raise MaxRetryError(_pool, url, error or ResponseError(cause))urllib3.exceptions.MaxRetryError: \ HTTPSConnectionPool(host='urlscan.io', port=443): Max retries exceeded with url: /api/v1/search/?q=8.8.8.8 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)'),)) \ 
During handling of the above exception, another exception occurred:Traceback (most recent call last):  File "/etc/cortex/Cortex-Analyzers/analyzers/Urlscan.io/urlscan_analyzer.py", line 59, in <module>    UrlscanAnalyzer().run()  File "/etc/cortex/Cortex-Analyzers/analyzers/Urlscan.io/urlscan_analyzer.py", line 32, in run  \
'indicator': self.search(query)  File "/etc/cortex/Cortex-Analyzers/analyzers/Urlscan.io/urlscan_analyzer.py", line 17, in search    res = Urlscan(indicator).search()  File "/etc/cortex/Cortex-Analyzers/analyzers/Urlscan.io/urlscan.py", line 16,  \ in search    r = requests.get("https://urlscan.io/api/v1/search/", params=payload)  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get    return request('get', url, params=params, **kwargs)  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request    return session.request(method=method, url=url, **kwargs)  \ 
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request  \
resp = self.send(prep, **send_kwargs)  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send   \
r = adapter.send(request, **kwargs)  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 514, in send   \
raise SSLError(e, request=request)requests.exceptions.SSLError: HTTPSConnectionPool(host='urlscan.io', port=443): Max retries exceeded with url: /api/v1/search/?q=8.8.8.8 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)'),))

Cortex Version info

Screenshot of some of the errors. I've tried almost all types of IOCs (IPs, URLs, etc)

Work environment

• Server OS: CentOS 7
• Browse type and version: Chrome 79.0.3945.130
• Cortex version: 3.0.1-1
• Cortex Analyzer/Responder name: Many, see below.
• Cortex Analyzer/Responder version: Many.

Possible solutions
Is there an ability to globally ignore certificate verification? I reviewed #228 and #210, #228 appears to to have solved the issue only for the Cuckoo analyzer, #210 appears to have only solved for MISP analyzer.

Thanks, appreciate any help!

[redwolfey]

Solved by replacing the "cacert.pem" files in the python2.7 and python3.6 certifi folders. These were replaced with a copy of the cert file "ca-bundle.crt" from /etc/ssl/certs/ that was first converted to a .pem.

As a caution: this change will be overwritten whenever the certifi package is updated.