Improper validation of filter leads to internal server error

[matrixbot]

This issue has been migrated from #14223.

---

Description

The /_matrix/client/v3/publicRooms endpoint supports filtering. Some values trigger a 500.

Steps to reproduce

Either send a null byte in generic_search_term or an invalid value in since. See the HTTP requests below:

POST /_matrix/client/v3/publicRooms HTTP/2
Host: matrix-federation.matrix.org
Authorization: Bearer [redacted]
Accept: application/json
Content-Type: application/json
Content-Length: 50

{ "filter" : { "generic_search_term" : "\u0000" }}

POST /_matrix/client/v3/publicRooms HTTP/2
Host: matrix-federation.matrix.org
Authorization: Bearer [redacted]
Accept: application/json
Content-Type: application/json
Content-Length: 66

{ "filter" : { "generic_search_term" : "a" },
"since": "invalid"}

Homeserver

matrix.org and my homeserver

Synapse Version

v1.69.0 on my homeserver

Installation Method

Docker (matrixdotorg/synapse)

Platform

Debian 11, amd64, on my homeserver.

Relevant log output

When searching for a null byte:

Oct 18 14:23:23 matrix-host matrix-synapse[3424675]: 2022-10-18 12:23:23,865 - synapse.http.server - 123 - ERROR - POST-107945 - Failed handle request via 'PublicRoomListRestServlet': <XForwardedForRequest at [redacted] method='POST' uri='/_matrix/client/v3/publicRooms' clientpr
oto='HTTP/1.0' site='8008'>
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]: Traceback (most recent call last):
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 306, in _async_render_wrapper
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     callback_return = await self._async_render(request)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 512, in _async_render
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     callback_return = await raw_callback_return
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/rest/client/room.py", line 544, in on_POST
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     data = await handler.get_local_public_room_list(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/room_list.py", line 103, in get_local_public_room_list
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return await self._get_public_room_list(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/room_list.py", line 164, in _get_public_room_list
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     results = await self.store.get_largest_public_rooms(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/room.py", line 463, in get_largest_public_rooms
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     ret_val = await self.db_pool.runInteraction(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 881, in runInteraction
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return await delay_cancellation(_runInteraction())
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     result = current_context.run(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return g.throw(self.type, self.value, self.tb)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 848, in _runInteraction
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     result = await self.runWithConnection(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 976, in runWithConnection
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return await make_deferred_yieldable(
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/python/threadpool.py", line 244, in inContext
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     result = inContext.theWork()  # type: ignore[attr-defined]
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/python/context.py", line 117, in callWithContext
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return self.currentContext().callWithContext(ctx, func, *args, **kw)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/python/context.py", line 82, in callWithContext
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return func(*args, **kw)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     result = func(conn, *args, **kw)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 969, in inner_func
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return func(db_conn, *args, **kwargs)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 710, in new_transaction
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     r = func(cursor, *args, **kwargs)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/room.py", line 454, in _get_largest_public_rooms_txn
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     txn.execute(sql, query_args)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 388, in execute
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     self._do_execute(self.txn.execute, sql, *args)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 436, in _do_execute
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]:     return func(sql, *args, **kwargs)
Oct 18 14:23:23 matrix-host matrix-synapse[3424675]: ValueError: A string literal cannot contain NUL (0x00) characters.

When sending an invalid since:

Oct 18 14:25:08 matrix-host matrix-synapse[3424675]: 2022-10-18 12:25:08,411 - synapse.http.server - 123 - ERROR - POST-108361 - Failed handle request via 'PublicRoomListRestServlet': <XForwardedForRequest at [redacted] method='POST' uri='/_matrix/client/v3/publicRooms' clientpr
oto='HTTP/1.0' site='8008'>
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]: Traceback (most recent call last):
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 306, in _async_render_wrapper
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:     callback_return = await self._async_render(request)
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/http/server.py", line 512, in _async_render
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:     callback_return = await raw_callback_return
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/rest/client/room.py", line 544, in on_POST
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:     data = await handler.get_local_public_room_list(
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/room_list.py", line 103, in get_local_public_room_list
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:     return await self._get_public_room_list(
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/room_list.py", line 147, in _get_public_room_list
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:     batch_token = RoomListNextBatch.from_token(since_token)
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "/usr/local/lib/python3.9/site-packages/synapse/handlers/room_list.py", line 498, in from_token
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:     decoded = msgpack.loads(decode_base64(token), raw=False)
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]:   File "msgpack/_unpacker.pyx", line 194, in msgpack._cmsgpack.unpackb
Oct 18 14:25:08 matrix-host matrix-synapse[3424675]: ValueError: 10 exceeds max_map_len(2)

Anything else that would be useful to know?

See this matrix-public-archive issue.

[dklimpel]

The same issue for value of limit (e.g. LIMIT=-1)

[dklimpel]

Related to: #13147