Revert "Set thread_id column to non-null for event_push_{actions,actions_staging,summary}"

[erikjohnston]

Reverts #15437

Trying to apply this causes a deadlock (?!)

Also, we probably want to move the drop index to the background update.

[erikjohnston]

2023-05-12 10:34:27.145 UTC [matrix event_persister5] DETAIL:  Process 962504 waits for AccessShareLock on relation 10501784 of database 16403; blocked by process 2583817.
        Process 2583817 waits for AccessExclusiveLock on relation 876931 of database 16403; blocked by process 962504.
        Process 962504: 
                    INSERT INTO event_push_actions (
                        room_id, event_id, user_id, actions, stream_ordering,
                        topological_ordering, notif, highlight, unread, thread_id
                    )
                    SELECT '....', event_id, user_id, actions, 3968315820, 53108, notif, highlight, unread, thread_id
                    FROM event_push_actions_staging
                    WHERE event_id = '....'
                
        Process 2583817: COMMIT; BEGIN TRANSACTION; /* Copyright 2022 The Matrix.org Foundation C.I.C
         *
         * Licensed under the Apache License, Version 2.0 (the "License");
         * you may not use this file except in compliance with the License.
         * You may obtain a copy of the License at
         *
         *    http://www.apache.org/licenses/LICENSE-2.0
         *
         * Unless required by applicable law or agreed to in writing, software
         * distributed under the License is distributed on an "AS IS" BASIS,
         * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
         * See the License for the specific language governing permissions and
         * limitations under the License.
         */
        
        LOCK TABLE event_push_actions_staging, event_push_actions, event_push_summary;
        
        -- The thread_id columns can now be made non-nullable, this is done by using a
        -- constraint (and not altering the column) to avoid taking out a full table lock.
        --
        -- We initially add an invalid constraint which guards against new data (this
        -- doesn't lock the table).
        ALTER TABLE event_push_actions_st
2023-05-12 10:34:27.145 UTC [matrix event_persister5] HINT:  See server log for query details.
2023-05-12 10:34:27.145 UTC [matrix event_persister5] STATEMENT:  
                    INSERT INTO event_push_actions (
                        room_id, event_id, user_id, actions, stream_ordering,
                        topological_ordering, notif, highlight, unread, thread_id
                    )
                    SELECT '....', event_id, user_id, actions, 3968315820, 53108, notif, highlight, unread, thread_id
                    FROM event_push_actions_staging
                    WHERE event_id = '...'
                

Note I tried adding an explicit LOCK TABLE in the delta file to try and get around this, but it didn't work

[erikjohnston]

cc @clokep

[erikjohnston]

This also happened with some synchrotron process:

2023-05-12 10:36:07.055 UTC [matrix synchrotron5] ERROR:  deadlock detected at character 46
2023-05-12 10:36:07.055 UTC [matrix synchrotron5] DETAIL:  Process 924941 waits for AccessShareLock on relation 876931 of database 16403; blocked by process 2589469.
        Process 2589469 waits for AccessExclusiveLock on relation 2770566 of database 16403; blocked by process 924941.
        Process 924941: 
                    SELECT COUNT(*), thread_id FROM event_push_actions
                    LEFT JOIN (
                        SELECT thread_id, MAX(stream_ordering) AS threaded_receipt_stream_ordering
                        FROM receipts_linearized
                        LEFT JOIN events USING (room_id, event_id)
                        WHERE
                            user_id = '....'
                            AND room_id = '...'
                            AND stream_ordering > 3508003783
                            AND receipt_type = ANY(ARRAY['m.read','m.read.private'])
                        GROUP BY thread_id
                    ) AS receipts USING (thread_id)
                    WHERE user_id = '...'
                        AND room_id = '...'
                        AND stream_ordering > COALESCE(threaded_receipt_stream_ordering, 3508003783)
                        AND highlight = 1
                    GROUP BY thread_id
                
        Process 2589469: COMMIT; BEGIN TRANSACTION; /* Copyright 2022 The Matrix.org Foundation C.I.C
         *
         * Licensed under the Apache License, Version 2.0 (the "License");
         * you may not use this file except in compliance with the License.
         * You may obtain a copy of the License at
         *
         *    http://www.apache.org/licenses/LICENSE-2.0
         *
         * Unless required by applicable law or agreed to in writing, software
         * distributed under the License is distributed on an "AS IS" BASIS,
         * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
         * See the License for the specific language governing permissions and
         * limitations under the License.
         */
        
        LOCK TABLE event_push_actions_staging, event_push_actions, event_push_summary;
        
        -- The thread_id columns can now be made non-nullable, this is done by using a
        -- constraint (and not altering the column) to avoid taking out a full table lock.
        --
        -- We initially add an invalid constraint which guards against new data (this
        -- doesn't lock the table).
        ALTER TABLE event_push_actions_st
2023-05-12 10:36:07.055 UTC [matrix synchrotron5] HINT:  See server log for query details.