Analyzers using online query fails to use system proxy settings

[cemasirt]

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	CentOS
OS version (client)	7
Cortex Analyzer Name	PassiveTotal, OTXQuery, phishtank etc.
Cortex Analyzer Version	1.7.0
Cortex Version	1.1.4
Browser type & version	N/A

Description

I'm trying to run queries behind a proxy server and even though I can use the proxy with other applications (git, wget etc.) Cortex analyzers fails to do so. I'm triggering the queries from hive.

Steps to Reproduce

1. Define http_proxy and https_proxy env. variables. (also put them in /etv/environment)
2. Run a cortex analyzer which use online query.

Some errors I'm getting on Hive reports:

{
"errorMessage": "Unexpected Error: HTTPSConnectionPool(host='api.passivetotal.org', port=443): Max retries exceeded with url: /v2/enrichment/malware?query=103.31.186.29 (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0xf587d0>, 'Connection to api.passivetotal.org timed out. (connect timeout=30)'))",
"input": {
"dataType": "ip",
"config": {
"username": "xxxxx",
"max_tlp": 1,
"check_tlp": true,
"service": "malware",
"key": "REMOVED"
},
"tlp": 0,
"data": "103.31.186.29"
},
"success": false
}

OTXQuery
{
"errorMessage": "API Error! Please verify data type is correct.",
"input": {
"dataType": "hash",
"config": {
"max_tlp": 3,
"check_tlp": true,
"service": "query",
"key": "REMOVED"
},
"tlp": 0,
"data": "8290bbdd3f18ce676f0ef6b652fa38ae"
},
"success": false
}

{
"errorMessage": "Error: Invalid output\nTraceback (most recent call last):\n File "./phishtank_checkurl.py", line 80, in \n phishtankAnalyzer().run()\n File "./phishtank_checkurl.py", line 51, in run\n r = self.phishtank_checkurl(data)\n File "./phishtank_checkurl.py", line 24, in phishtank_checkurl\n r = requests.post(url, data=postdata)\n File "/usr/lib/python2.7/site-packages/requests/api.py", line 112, in post\n return request('post', url, data=data, json=json, **kwargs)\n File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request\n return session.request(method=method, url=url, **kwargs)\n File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request\n resp = self.send(prep, **send_kwargs)\n File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send\n r = adapter.send(request, **kwargs)\n File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 508, in send\n raise ConnectionError(e, request=request)\nrequests.exceptions.ConnectionError: HTTPSConnectionPool(host='checkurl.phishtank.com', port=443): Max retries exceeded with url: /checkurl/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x1d46f10>: Failed to establish a new connection: [Errno 110] Connection timed out',))\n",
"input": null,
"success": false
}

[cemasirt]

Needed to add the proxy config in /etc/cortex/application.conf.

config { global { proxy { http="http://PROXYIP:PORT", https="http://PROXYIP:PORT" } }