Error trying an example python3 last.py -l 24h

[cakegit007]

Hello again,

as I have now PyMISP up and running I tried an example

python3 last.py -l 24h

But I still have problems fetching these Information. The cert Warning ist okay. But I dont understand the response.
The servers have the same timestamp, i rebootet them, i had an event within the last 24 hours. And the Information from /var/www/MISP/app/tmp/log/error.log

last.py:4: DeprecationWarning:
As our baseline system is the latest Ubuntu LTS, and Ubuntu LTS 22.04 has Python 3.10 available,
we will officially deprecate python versions below 3.10 on January 1st 2024.
Please update your codebase.
from pymisp import ExpandedPyMISP
/home/cake/.local/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.0.0.10'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
/home/cake/.local/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.0.0.10'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
/home/cake/.local/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.0.0.10'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
/home/cake/.local/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '10.0.0.10'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
Unknown error: the response is not in JSON.
Something is broken server-side, please send us everything that follows (careful with the auth key):
Request headers:
{'User-Agent': 'PyMISP 2.4.168.1 - Python 3.8', 'Accept-Encoding': 'br, gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Cookie': 'CAKEPHP=f2umgolpsk1pomriceq09ji4iptc28ha'}
Request body:
None
Response (if any):
{
"name": "Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.",
"message": "Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.",
"url": "/events/restSearch"
}
Traceback (most recent call last):
File "last.py", line 36, in
result = misp.search(publish_timestamp=args.last, limit=args.limit, page=args.page, pythonify=True)
File "/home/cake/.local/lib/python3.8/site-packages/pymisp/api.py", line 2677, in search
normalized_response = self._check_json_response(response)
File "/home/cake/.local/lib/python3.8/site-packages/pymisp/api.py", line 3613, in _check_json_response
r = self._check_response(response, expect_json=True)
File "/home/cake/.local/lib/python3.8/site-packages/pymisp/api.py", line 3631, in _check_response
raise MISPServerError(f'Error code 500:\n{response.text}')
pymisp.exceptions.MISPServerError: Error code 500:
{
"name": "Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.",
"message": "Restsearch queries using GET and no parameters are not allowed. If you have passed parameters via a JSON body, make sure you use POST requests.",
"url": "/events/restSearch"
}

Thanks in advance for any help!

[Rafiot]

I just ran the same request, and I'm not getting that 500 error. Which version of MISP is running on the server?

[cakegit007]

• MISP 2.4.168 from the VM and then Updated (I also tried a fresh install from sources with the same result)

I now switched for testing purpose to Ubuntu 22.04.02

MISP Server:

• Ubuntu 22.04.02 Server
• MISP 2.4.168

PyMISP Server

• Ubuntu 22.04.02 Server

Running the test poetry run pytest --cov=pymisp tests/test_*.py now looks good
Result: 67 passed, 2 skipped, 1 warning in 51.38s

Running the test against my MISP-Server misp2@misp2:~/PyMISP$ poetry run pytest --cov=pymisp tests/testlive_comprehensive.py is not successfully
Result:

platform linux -- Python 3.10.6, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/misp2/PyMISP
plugins: cov-4.0.0
collected 80 items
tests/testlive_comprehensive.py EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE [100%]
ERROR at setup of TestComprehensive.test_add_event_with_attachment
cls = <class 'tests.testlive_comprehensive.TestComprehensive'>
@classmethod
def setUpClass(cls):
cls.maxDiff = None
# Connect as admin
cls.admin_misp_connector = PyMISP(url, key, verifycert, debug=False)
cls.admin_misp_connector.set_server_setting('Security.allow_self_registration', True, force=True)
cls.admin_misp_connector.set_server_setting('debug', 1, force=True)
if not fast_mode:
r = cls.admin_misp_connector.update_misp()
tests/testlive_comprehensive.py:69:
pymisp/api.py:292: in update_misp
return self._check_json_response(response)
pymisp/api.py:3613: in _check_json_response
r = self._check_response(response, expect_json=True)
self = <PyMISP(url=http://10.0.0.37), response = <Response [500]>, lenient_response_type = False, expect_json = True
def _check_response(self, response: requests.Response, lenient_response_type: bool = False, expect_json: bool = False) -> Union[Dict, str]:
"""Check if the response from the server is not an unexpected error"""
if response.status_code >= 500:
headers_without_auth = {i: response.request.headers[i] for i in response.request.headers if i != 'Authorization'}
logger.critical(everything_broken.format(headers_without_auth, response.request.body, response.text))
raise MISPServerError(f'Error code 500:\n{response.text}')
E pymisp.exceptions.MISPServerError: Error code 500:
E {
E "code": 500,
E "name": "View file "Servers/json/ajax/update.ctp" is missing.",
E "message": "View file "Servers/json/ajax/update.ctp" is missing.",
E "url": "/servers/update"
E }

The file /var/www/MISP/app/View/Servers/ajax/update.ctp does not exist in a fresh installation out of the sources:

misp@misp:/var/www/MISP/app/View/Servers$ ls -lah
total 136K
drwxr-x--- 3 www-data www-data 4.0K Mar 7 00:27 .
drwxr-x--- 69 www-data www-data 4.0K Mar 6 22:52 ..
drwxr-x--- 2 www-data www-data 4.0K Mar 6 22:52 ajax
-rwxr-x--- 1 www-data www-data 4.2K Mar 6 22:52 compare_servers.ctp
-rwxr-x--- 1 www-data www-data 370 Mar 6 22:52 create_sync.ctp
-rwxr-x--- 1 www-data www-data 21K Mar 6 22:52 edit.ctp
-rwxr-x--- 1 www-data www-data 1.3K Mar 6 22:52 event_block_rule.ctp
-rwxr-x--- 1 www-data www-data 1.9K Mar 6 22:52 filter_event_index.ctp
-rwxr-x--- 1 www-data www-data 3.4K Mar 6 22:52 id_translator.ctp
-rwxr-x--- 1 www-data www-data 877 Mar 6 22:52 import.ctp
-rwxr-x--- 1 www-data www-data 16K Mar 6 22:52 index.ctp
-rwxr-x--- 1 www-data www-data 3.4K Mar 6 22:52 ondemand_action.ctp
-rwxr-x--- 1 www-data www-data 5.8K Mar 6 22:52 preview_event.ctp
-rwxr-x--- 1 www-data www-data 11K Mar 6 22:52 preview_index.ctp
-rwxr-x--- 1 www-data www-data 1.4K Mar 6 22:52 pull.ctp
-rwxr-x--- 1 www-data www-data 724 Mar 6 22:52 push.ctp
-rwxr-x--- 1 www-data www-data 1.6K Mar 6 22:52 server_settings.ctp
-rwxr-x--- 1 www-data www-data 13K Mar 6 22:52 update_progress.ctp
-rwxr-x--- 1 www-data www-data 1.3K Mar 6 22:52 view_export_format_usage.ctp

Maybe I am doing something completely wrong during installation...?

[Rafiot]

Something went wrong during the installation, the file should be there: https://github.com/MISP/MISP/blob/2.4/app/View/Servers/ajax/update.ctp

[cakegit007]

Thanks for you help... now I have a new error. I installed the misp server twice new on Ubuntu Server 22.04.02.

E               pymisp.exceptions.PyMISPUnexpectedResponse: Unexpected response (size: 1080) from server: <div class="confirmation">
E                   <legend>Update MISP</legend>
E                   <div class="inner">
E                       <form action="https://10.0.0.38/servers/update" style="margin:0px;" id="PromptForm" method="post" accept-charset="utf-8"><div style="display:none;"><input type="hidden" name="_method" value="POST"/></div>        <p>Do you want to pull the latest commit from the 2.4 branch? If you have made local changes to MISP the merge will fail.</p>
E                       <table>
E                           <tr>
E                               <td style="vertical-align:top">
E                                   <span role="button" tabindex="0" aria-label="Publish" title="Publish" id="PromptYesButton" class="btn btn-primary" onClick="submitMISPUpdate();">Yes</span>
E                               </td>
E                               <td style="width:540px;">
E                               </td>
E                               <td style="vertical-align:top;">
E                                   <span role="button" tabindex="0" aria-label="Cancel" title="Cancel" class="btn btn-inverse" id="PromptNoButton" onClick="cancelPrompt();">No</span>
E                               </td>
E                           </tr>
E                       </table>
E                   </form>        </div>
E               </div>

../pymisp/api.py:3656: PyMISPUnexpectedResponse

[cakegit007]

Finally it worked. In fact only changed she Ubuntu Version and installed again both systems.

Tanks for your help!