Upgraded to v1.62.0, "Mentions Only" rooms suddenly start having notification counts

[turt2live]

Description

I have most/all of my rooms manually set to Mentions Only, which should mean they don't get notification counts unless I've been mentioned. Somewhat randomly, I've started getting counts on rooms periodically in Element Web, which is powered purely by what is given in /sync from the server. The rooms are not consistent, and the problem only persists for a few messages before going back to behaving properly.

I've also noticed that sometimes the badge gets "stuck": even though I've sent and received new messages, it's almost as if my (implied) read receipt isn't enough to clear the unreadness of the room. There's a 40% chance this particular point is a client issue, but worth mentioning in case that 60% chance sparks ideas server-side.

Steps to reproduce

1. Set a room to Mentions Only
2. Engage in conversation in that room
3. Eventually get a grey badge (indicating unread, unnotified, message count)
4. Continue engaging
5. Badge goes away

Homeserver

t2l.io

Synapse Version

v1.62.0

Installation Method

pip (from PyPI)

Platform

Ubuntu server, hosted in the ✨cloud✨

Relevant log output

Unclear what would be relevant.

Anything else that would be useful to know?

This problem only started happening ~24 hours after I upgraded to v1.62.0 (from v1.61.1). It's happened four times since then though. I updated the same day as the v1.62.0 release.

I don't use the global push rule to set rooms to Mentions Only: when I join a new room, I click the bell on that room and change it there. My default push rules are similar to those of a brand new user: unread messages cause a badge.

The rooms this has been happening in have been in my room list for a long while (years): they are not recent joins. Not sure if relevant, but might be helpful?

[DMRobertson]

The stuckness sounds similar to #13196. I'm not sure if the rooms being notification-only makes a difference @erikjohnston, or if this really is a straight-up duplicate?

[turt2live]

Private rooms by default have notifications enabled, so it sounds like #13196 has turned them off at the global level while I've turned them off at the room level.

[erikjohnston]

@turt2live for a given room that has that problem can you select all rows for that room and your user from event_push_actions and event_push_summary please?

[turt2live]

worth noting that as of writing I don't think I'm seeing the issue in the room, but here's the results of the queries (should I also re-run when in the middle of it happening?)

t2lsynapse_c=# select * from event_push_actions where user_id = '@travis:t2l.io' and room_id = '!bEWtlqtDwCLFIAKAcv:matrix.org';
            room_id             |                   event_id                   |    user_id     | profile_tag | actions | topological_ordering | stream_ordering | notif | highlight | unread
--------------------------------+----------------------------------------------+----------------+-------------+---------+----------------------+-----------------+-------+-----------+--------
 !bEWtlqtDwCLFIAKAcv:matrix.org | $Ff7PGs22mZlJn7FCzOgoC4sVJZY5vB9Fux9AC61U6VM | @travis:t2l.io |             |         |                27621 |        40247976 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $LyMNjGfHWddrgZG8WP-nkUPH1j8dlfcFg4SeX29sG8U | @travis:t2l.io |             |         |                27644 |        40257279 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $-c_GBk0HUNpMsXzGqIkpTTAehwv9dQCEheHRI7JqGRI | @travis:t2l.io |             |         |                27645 |        40257281 |     1 |         1 |      0
 !bEWtlqtDwCLFIAKAcv:matrix.org | $VUDzo2MxSzFfqeRiVYUSR1TMF1A7CNS3qqlq0xtwzE0 | @travis:t2l.io |             |         |                27709 |        40288928 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $M6dTadfpOsb5SZytp8ekVN-9r60SRQ8afDUPn-HqzJ8 | @travis:t2l.io |             |         |                27742 |        40290321 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $kjcH5JMugcuMFtBqys_NBePV7Hn72RLpOnLpVq-CWOA | @travis:t2l.io |             |         |                27756 |        40290488 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $gTG4pPI-Hu64aurywZD0Eist_Z-4_K5cJbeT_EQgqa8 | @travis:t2l.io |             |         |                27757 |        40290508 |     1 |         1 |      0
 !bEWtlqtDwCLFIAKAcv:matrix.org | $AfK1Zh-ugsOl74GEZ_V7-QsuaTDEguy8W2dpMCkiV34 | @travis:t2l.io |             |         |                27782 |        40340455 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $ruGiGUKr0yw6rCun6_fxe8fcFRiof59EqBp2tuTmcWo | @travis:t2l.io |             |         |                27783 |        40340543 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $oqAXc7zSa4877FqYvsynxkoECtH4npsSe8HRoQxx0Ck | @travis:t2l.io |             |         |                27851 |        40367312 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $Jx-DAAv3owk-0uWm-gUp4TElJUqU_tJm3dfGf4ktEEw | @travis:t2l.io |             |         |                27856 |        40368161 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $j85kJULHEZZQ3xm3nKJ-qSDddRHahrdkmqP4PeFwz0k | @travis:t2l.io |             |         |                27871 |        40375740 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $CbHZxd_dCFzSAUCic6bE_Wxch1EdA6BfGxyfn1luQxU | @travis:t2l.io |             |         |                27889 |        40379248 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $JDX1HyzErIINJG5qCOXkdlnup5VNUnq5MiZIhgFsPJU | @travis:t2l.io |             |         |                27896 |        40380412 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $SyldcVcH4eKN4ARjr7gWAqeqGXXra3v0FjlOADDaRWc | @travis:t2l.io |             |         |                27941 |        40383802 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $hhRI9Ci1vHJY7_sXQS1UeESkHs4egca6Kd-gYqKpS5Y | @travis:t2l.io |             |         |                27946 |        40383884 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $4ffLLc2R9QQXXGYs1EhSY-SW65paBPdoa5IvsjmncDU | @travis:t2l.io |             |         |                27955 |        40384088 |     1 |         1 |      1
 !bEWtlqtDwCLFIAKAcv:matrix.org | $Ptk0wn8e3pLd5QVu5V-pLEILhEkAyc4bKIBRdjhVPYc | @travis:t2l.io |             |         |                27960 |        40384113 |     1 |         1 |      1
(18 rows)

t2lsynapse_c=# select * from event_push_summary where user_id = '@travis:t2l.io' and room_id = '!bEWtlqtDwCLFIAKAcv:matrix.org';
    user_id     |            room_id             | notif_count | stream_ordering | unread_count | last_receipt_stream_ordering
----------------+--------------------------------+-------------+-----------------+--------------+------------------------------
 @travis:t2l.io | !bEWtlqtDwCLFIAKAcv:matrix.org |           0 |        40396201 |            0 |                     40395152
(1 row)

[erikjohnston]

should I also re-run when in the middle of it happening?

Yes please! That data is interesting, but will be more interesting to compare with when its broken

[turt2live]

I think I've been seeing #13196 - closing in favour of that.