Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

synapse_port_db errors porting sqlite3 database (Pending background updates exist in the SQLite3 database.) #15219

Closed
jrick opened this issue Mar 7, 2023 · 9 comments
Closed

synapse_port_db errors porting sqlite3 database (Pending background updates exist in the SQLite3 database.) #15219

jrick opened this issue Mar 7, 2023 · 9 comments
Labels
A-Database DB stuff like queries, migrations, new/remove columns, indexes, unexpected entries in the db A-Portdb Script for exporting sqlite data into postgres O-Occasional Affects or can be seen by some users regularly or most users rarely S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Release-Blocker Must be resolved before making a release

Comments

@jrick
Copy link

jrick commented Mar 7, 2023

Description

The synapse_port_db script is unable to port my (old, and large) sqlite3 database to postgres. It errors due to finding background updates still pending, however I have confirmed through both the synapse admin API (while synapse is running) and by checking for any rows in the background_updates table (with synapse stopped) that there are no background updates left to run. So I'm not sure why this is erroring here:

synapse/synapse/_scripts/synapse_port_db.py

Lines 689 to 699 in adac949

# Check if all background updates are done, abort if not.
updates_complete = (
await self.sqlite_store.db_pool.updates.has_completed_background_updates()
)
if not updates_complete:
end_error = (
"Pending background updates exist in the SQLite3 database."
" Please start Synapse again and wait until every update has finished"
" before running this script.\n"
)
return

In any case, I modified that script by commenting out that entire check. This allows the script to complete (surprisingly quickly, for a 6.3G database...). Yet, on the postgres side, only the schema and tables have been created, and none of my data has been carried over.

synapse$ sqlite3 homeserver.db 'select count(*) from users;'
1

synapse_db$ psql -U postgres -c 'select count(*) from users;' synapse   # after migrating
Password for user postgres: 
 count 
-------
     0
(1 row)

It seems that there are two issues here, perhaps related. One being it finding background updates when there were none, and the other being that either the rest of my db's data is never discovered or the script is not properly copying it over to the postgres db. Only the migration script appears to have these problems; synapse itself works well with the sqlite3 db.

Steps to reproduce

  • I ran the following:
synctl stop
synapse_port_db --sqlite-database homeserver.db --postgres-config homeserver_psql.yaml

Homeserver

zettaport.com

Synapse Version

1.78.0

Installation Method

pip (from PyPI)

Database

SQLite

Workers

Single process

Platform

OpenBSD/amd64 7.2

Configuration

No response

Relevant log output

First log of an unmodified synapse_port_db:

2023-03-07 11:48:15,120 - synapse.config.key - 136 - WARNING - Synapse requires that a list of trusted key servers are specified in order to
provide signing keys for other servers in the federation.

This homeserver does not have a trusted key server configured in
homeserver.yaml and will fall back to the default of 'matrix.org'.

Trusted key servers should be long-lived and stable which makes matrix.org a
good choice for many admins, but some admins may wish to choose another. To
suppress this warning, the admin should set 'trusted_key_servers' in
homeserver.yaml to their desired key server and 'suppress_key_server_warning'
to 'true'.

In a future release the software-defined default will be removed entirely and
the trusted key server will be defined exclusively by the value of
'trusted_key_servers'.
--------------------------------------------------------------------------------
Preparing sqlite3...
2023-03-07 11:48:15,239 - synapse.storage.prepare_database - 120 - INFO - ('main', 'state'): Checking existing schema version
2023-03-07 11:48:15,243 - synapse.storage.prepare_database - 150 - INFO - ('main', 'state'): Initialising new database
2023-03-07 11:48:15,357 - synapse.storage.prepare_database - 416 - INFO - Applying schema deltas for v73
2023-03-07 11:48:15,360 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/01event_failed_pull_attempts.sql
2023-03-07 11:48:15,363 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/02add_pusher_enabled.sql
2023-03-07 11:48:15,370 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/02room_id_indexes_for_purging.sql
2023-03-07 11:48:15,374 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/03pusher_device_id.sql
2023-03-07 11:48:15,381 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/03users_approved_column.sql
2023-03-07 11:48:15,388 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/04partial_join_details.sql
2023-03-07 11:48:15,401 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/04pending_device_list_updates.sql
2023-03-07 11:48:15,403 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/05old_push_actions.sql.sqlite
2023-03-07 11:48:15,411 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/06thread_notifications_thread_id_idx.sql
2023-03-07 11:48:15,413 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/08thread_receipts_non_null.sql.sqlite
2023-03-07 11:48:15,473 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/09partial_joined_via_destination.sql
2023-03-07 11:48:15,481 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/09threads_table.sql
2023-03-07 11:48:15,484 - synapse.storage.prepare_database - 507 - INFO - Running 73/10_update_sqlite_fts4_tokenizer.py:run_create
2023-03-07 11:48:15,493 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/10login_tokens.sql
2023-03-07 11:48:15,497 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/12refactor_device_list_outbound_pokes.sql
2023-03-07 11:48:15,500 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/13add_device_lists_index.sql
2023-03-07 11:48:15,502 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/20_un_partial_stated_room_stream.sql
2023-03-07 11:48:15,505 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/22_rebuild_user_dir_stats.sql
2023-03-07 11:48:15,507 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/22_un_partial_stated_event_stream.sql
2023-03-07 11:48:15,510 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/23_fix_thread_index.sql
2023-03-07 11:48:15,512 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/24_events_jump_to_date_index.sql
2023-03-07 11:48:15,514 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/25drop_presence.sql
2023-03-07 11:48:15,516 - synapse.storage.prepare_database - 416 - INFO - Applying schema deltas for v74
2023-03-07 11:48:15,517 - synapse.storage.prepare_database - 551 - INFO - Schema now up to date
2023-03-07 11:48:15,521 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for receipts_linearized(stream_id): 1
2023-03-07 11:48:15,524 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for events(stream_ordering): 1
2023-03-07 11:48:15,524 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for events(stream_ordering): -1
2023-03-07 11:48:15,525 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for ex_outlier_stream(event_stream_ordering): -1
2023-03-07 11:48:15,528 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for un_partial_stated_event_stream(stream_id): 1
2023-03-07 11:48:15,530 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for pushers(id): 1
2023-03-07 11:48:15,530 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for deleted_pushers(stream_id): 1
2023-03-07 11:48:15,531 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules_stream(stream_id): 1
2023-03-07 11:48:15,532 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules(id): 1
2023-03-07 11:48:15,533 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules_enable(id): 1
2023-03-07 11:48:15,534 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for room_account_data(stream_id): 1
2023-03-07 11:48:15,534 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for room_tags_revisions(stream_id): 1
2023-03-07 11:48:15,536 - synapse.storage.databases.main.event_push_actions - 1298 - INFO - Searching for stream ordering 1 month ago
2023-03-07 11:48:15,537 - synapse.storage.databases.main.event_push_actions - 1302 - INFO - Found stream ordering 1 month ago: it's 0
2023-03-07 11:48:15,537 - synapse.storage.databases.main.event_push_actions - 1305 - INFO - Searching for stream ordering 1 day ago
2023-03-07 11:48:15,538 - synapse.storage.databases.main.event_push_actions - 1309 - INFO - Found stream ordering 1 day ago: it's 0
Pending background updates exist in the SQLite3 database. Please start Synapse again and wait until every update has finished before running this script.



Second log after commenting out the background_updates check:

2023-03-07 11:50:25,337 - synapse.config.key - 136 - WARNING - Synapse requires that a list of trusted key servers are specified in order to
provide signing keys for other servers in the federation.

This homeserver does not have a trusted key server configured in
homeserver.yaml and will fall back to the default of 'matrix.org'.

Trusted key servers should be long-lived and stable which makes matrix.org a
good choice for many admins, but some admins may wish to choose another. To
suppress this warning, the admin should set 'trusted_key_servers' in
homeserver.yaml to their desired key server and 'suppress_key_server_warning'
to 'true'.

In a future release the software-defined default will be removed entirely and
the trusted key server will be defined exclusively by the value of
'trusted_key_servers'.
--------------------------------------------------------------------------------
Preparing sqlite3...
2023-03-07 11:50:25,455 - synapse.storage.prepare_database - 120 - INFO - ('main', 'state'): Checking existing schema version
2023-03-07 11:50:25,459 - synapse.storage.prepare_database - 150 - INFO - ('main', 'state'): Initialising new database
2023-03-07 11:50:25,563 - synapse.storage.prepare_database - 416 - INFO - Applying schema deltas for v73
2023-03-07 11:50:25,566 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/01event_failed_pull_attempts.sql
2023-03-07 11:50:25,569 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/02add_pusher_enabled.sql
2023-03-07 11:50:25,576 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/02room_id_indexes_for_purging.sql
2023-03-07 11:50:25,579 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/03pusher_device_id.sql
2023-03-07 11:50:25,586 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/03users_approved_column.sql
2023-03-07 11:50:25,593 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/04partial_join_details.sql
2023-03-07 11:50:25,605 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/04pending_device_list_updates.sql
2023-03-07 11:50:25,608 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/05old_push_actions.sql.sqlite
2023-03-07 11:50:25,615 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/06thread_notifications_thread_id_idx.sql
2023-03-07 11:50:25,617 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/08thread_receipts_non_null.sql.sqlite
2023-03-07 11:50:25,677 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/09partial_joined_via_destination.sql
2023-03-07 11:50:25,685 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/09threads_table.sql
2023-03-07 11:50:25,688 - synapse.storage.prepare_database - 507 - INFO - Running 73/10_update_sqlite_fts4_tokenizer.py:run_create
2023-03-07 11:50:25,695 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/10login_tokens.sql
2023-03-07 11:50:25,699 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/12refactor_device_list_outbound_pokes.sql
2023-03-07 11:50:25,702 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/13add_device_lists_index.sql
2023-03-07 11:50:25,703 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/20_un_partial_stated_room_stream.sql
2023-03-07 11:50:25,706 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/22_rebuild_user_dir_stats.sql
2023-03-07 11:50:25,708 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/22_un_partial_stated_event_stream.sql
2023-03-07 11:50:25,711 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/23_fix_thread_index.sql
2023-03-07 11:50:25,713 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/24_events_jump_to_date_index.sql
2023-03-07 11:50:25,715 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/25drop_presence.sql
2023-03-07 11:50:25,717 - synapse.storage.prepare_database - 416 - INFO - Applying schema deltas for v74
2023-03-07 11:50:25,718 - synapse.storage.prepare_database - 551 - INFO - Schema now up to date
2023-03-07 11:50:25,722 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for receipts_linearized(stream_id): 1
2023-03-07 11:50:25,724 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for events(stream_ordering): 1
2023-03-07 11:50:25,725 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for events(stream_ordering): -1
2023-03-07 11:50:25,726 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for ex_outlier_stream(event_stream_ordering): -1
2023-03-07 11:50:25,729 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for un_partial_stated_event_stream(stream_id): 1
2023-03-07 11:50:25,730 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for pushers(id): 1
2023-03-07 11:50:25,731 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for deleted_pushers(stream_id): 1
2023-03-07 11:50:25,732 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules_stream(stream_id): 1
2023-03-07 11:50:25,733 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules(id): 1
2023-03-07 11:50:25,733 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules_enable(id): 1
2023-03-07 11:50:25,734 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for room_account_data(stream_id): 1
2023-03-07 11:50:25,735 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for room_tags_revisions(stream_id): 1
2023-03-07 11:50:25,736 - synapse.storage.databases.main.event_push_actions - 1298 - INFO - Searching for stream ordering 1 month ago
2023-03-07 11:50:25,737 - synapse.storage.databases.main.event_push_actions - 1302 - INFO - Found stream ordering 1 month ago: it's 0
2023-03-07 11:50:25,737 - synapse.storage.databases.main.event_push_actions - 1305 - INFO - Searching for stream ordering 1 day ago
2023-03-07 11:50:25,738 - synapse.storage.databases.main.event_push_actions - 1309 - INFO - Found stream ordering 1 day ago: it's 0
Preparing psycopg2...
2023-03-07 11:50:25,845 - synapse.storage.prepare_database - 120 - INFO - ('main', 'state'): Checking existing schema version
2023-03-07 11:50:25,865 - synapse.storage.prepare_database - 150 - INFO - ('main', 'state'): Initialising new database
2023-03-07 11:50:26,590 - synapse.storage.prepare_database - 416 - INFO - Applying schema deltas for v73
2023-03-07 11:50:26,594 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/01event_failed_pull_attempts.sql
2023-03-07 11:50:26,602 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/02add_pusher_enabled.sql
2023-03-07 11:50:26,605 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/02room_id_indexes_for_purging.sql
2023-03-07 11:50:26,611 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/03pusher_device_id.sql
2023-03-07 11:50:26,614 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/03users_approved_column.sql
2023-03-07 11:50:26,617 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/04partial_join_details.sql
2023-03-07 11:50:26,621 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/04pending_device_list_updates.sql
2023-03-07 11:50:26,629 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/05old_push_actions.sql.postgres
2023-03-07 11:50:26,633 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/06thread_notifications_thread_id_idx.sql
2023-03-07 11:50:26,636 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/08thread_receipts_non_null.sql.postgres
2023-03-07 11:50:26,641 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/09partial_joined_via_destination.sql
2023-03-07 11:50:26,644 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/09threads_table.sql
2023-03-07 11:50:26,653 - synapse.storage.prepare_database - 507 - INFO - Running 73/10_update_sqlite_fts4_tokenizer.py:run_create
2023-03-07 11:50:26,654 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/10login_tokens.sql
2023-03-07 11:50:26,663 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/11event_search_room_id_n_distinct.sql.postgres
2023-03-07 11:50:26,666 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/12refactor_device_list_outbound_pokes.sql
2023-03-07 11:50:26,676 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/13add_device_lists_index.sql
2023-03-07 11:50:26,678 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/20_un_partial_stated_room_stream.sql
2023-03-07 11:50:26,686 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/21_un_partial_stated_room_stream_seq.sql.postgres
2023-03-07 11:50:26,691 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/22_rebuild_user_dir_stats.sql
2023-03-07 11:50:26,695 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/22_un_partial_stated_event_stream.sql
2023-03-07 11:50:26,703 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/23_fix_thread_index.sql
2023-03-07 11:50:26,706 - synapse.storage.prepare_database - 532 - INFO - Applying engine-specific schema 73/23_un_partial_stated_room_stream_seq.sql.postgres
2023-03-07 11:50:26,711 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/24_events_jump_to_date_index.sql
2023-03-07 11:50:26,713 - synapse.storage.prepare_database - 524 - INFO - Applying schema 73/25drop_presence.sql
2023-03-07 11:50:26,717 - synapse.storage.prepare_database - 416 - INFO - Applying schema deltas for v74
2023-03-07 11:50:26,719 - synapse.storage.prepare_database - 551 - INFO - Schema now up to date
2023-03-07 11:50:26,764 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for pushers(id): 1
2023-03-07 11:50:26,765 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for deleted_pushers(stream_id): 1
2023-03-07 11:50:26,767 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules_stream(stream_id): 1
2023-03-07 11:50:26,770 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules(id): 1
2023-03-07 11:50:26,772 - synapse.storage.util.id_generators - 92 - INFO - Initialising stream generator for push_rules_enable(id): 1
2023-03-07 11:50:26,786 - synapse.storage.databases.main.event_push_actions - 1298 - INFO - Searching for stream ordering 1 month ago
2023-03-07 11:50:26,788 - synapse.storage.databases.main.event_push_actions - 1302 - INFO - Found stream ordering 1 month ago: it's 0
2023-03-07 11:50:26,788 - synapse.storage.databases.main.event_push_actions - 1305 - INFO - Searching for stream ordering 1 day ago
2023-03-07 11:50:26,790 - synapse.storage.databases.main.event_push_actions - 1309 - INFO - Found stream ordering 1 day ago: it's 0
Running background updates on PostgreSQL...
2023-03-07 11:50:28,897 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'event_push_actions_thread_id_null'
2023-03-07 11:50:28,948 - synapse.storage.background_updates - 666 - INFO - Adding index event_push_actions_thread_id_null to event_push_actions
2023-03-07 11:50:28,979 - synapse.storage.background_updates - 474 - INFO - Running background update 'event_push_actions_thread_id_null'. Processed 1 items in 30ms. (total_rate=0.03333333333333333/ms, current_rate=0.03333333333333333/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:30,977 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'event_push_summary_thread_id_null'
2023-03-07 11:50:30,981 - synapse.storage.background_updates - 666 - INFO - Adding index event_push_summary_thread_id_null to event_push_summary
2023-03-07 11:50:30,995 - synapse.storage.background_updates - 474 - INFO - Running background update 'event_push_summary_thread_id_null'. Processed 1 items in 13ms. (total_rate=0.07692307692307693/ms, current_rate=0.07692307692307693/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:32,997 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'threads_backfill'
2023-03-07 11:50:33,011 - synapse.storage.background_updates - 474 - INFO - Running background update 'threads_backfill'. Processed 0 items in 10ms. (total_rate=None/ms, current_rate=None/ms, total_updated=0, batch_size=100)
2023-03-07 11:50:35,007 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'device_lists_changes_in_room_by_room_index'
2023-03-07 11:50:35,011 - synapse.storage.background_updates - 666 - INFO - Adding index device_lists_changes_in_room_by_room_idx to device_lists_changes_in_room
2023-03-07 11:50:35,029 - synapse.storage.background_updates - 474 - INFO - Running background update 'device_lists_changes_in_room_by_room_index'. Processed 1 items in 18ms. (total_rate=0.05555555555555555/ms, current_rate=0.05555555555555556/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:37,037 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'populate_stats_process_rooms'
2023-03-07 11:50:37,048 - synapse.storage.background_updates - 474 - INFO - Running background update 'populate_stats_process_rooms'. Processed 1 items in 7ms. (total_rate=0.14285714285714285/ms, current_rate=0.14285714285714285/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:39,047 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'populate_stats_process_users'
2023-03-07 11:50:39,059 - synapse.storage.background_updates - 474 - INFO - Running background update 'populate_stats_process_users'. Processed 1 items in 8ms. (total_rate=0.125/ms, current_rate=0.125/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:39,065 - synapse.storage.background_updates - 399 - INFO - Not starting on bg update populate_user_directory_cleanup until populate_user_directory_process_users is done
2023-03-07 11:50:41,027 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'populate_user_directory_createtables'
2023-03-07 11:50:41,067 - synapse.storage.background_updates - 474 - INFO - Running background update 'populate_user_directory_createtables'. Processed 1 items in 36ms. (total_rate=0.027777777777777776/ms, current_rate=0.02777777777777778/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:41,073 - synapse.storage.background_updates - 399 - INFO - Not starting on bg update populate_user_directory_cleanup until populate_user_directory_process_users is done
2023-03-07 11:50:42,538 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'populate_user_directory_process_rooms'
2023-03-07 11:50:42,559 - synapse.storage.background_updates - 474 - INFO - Running background update 'populate_user_directory_process_rooms'. Processed 1 items in 15ms. (total_rate=0.06666666666666667/ms, current_rate=0.06666666666666667/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:42,565 - synapse.storage.background_updates - 399 - INFO - Not starting on bg update populate_user_directory_cleanup until populate_user_directory_process_users is done
2023-03-07 11:50:44,538 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'populate_user_directory_process_users'
2023-03-07 11:50:44,547 - synapse.storage.background_updates - 474 - INFO - Running background update 'populate_user_directory_process_users'. Processed 1 items in 6ms. (total_rate=0.16666666666666666/ms, current_rate=0.16666666666666666/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:46,537 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'populate_user_directory_cleanup'
2023-03-07 11:50:46,560 - synapse.storage.background_updates - 474 - INFO - Running background update 'populate_user_directory_cleanup'. Processed 1 items in 20ms. (total_rate=0.05/ms, current_rate=0.05/ms, total_updated=1, batch_size=100)
2023-03-07 11:50:48,537 - synapse.storage.background_updates - 431 - INFO - Starting update batch on background update 'events_jump_to_date_index'
2023-03-07 11:50:48,540 - synapse.storage.background_updates - 666 - INFO - Adding index events_jump_to_date_idx to events
2023-03-07 11:50:48,552 - synapse.storage.background_updates - 474 - INFO - Running background update 'events_jump_to_date_index'. Processed 1 items in 12ms. (total_rate=0.08333333333333333/ms, current_rate=0.08333333333333333/ms, total_updated=1, batch_size=100)
Creating port tables...
Setting up sequence generators...
Fetching tables...
2023-03-07 11:50:48,644 - synapse_port_db - 778 - INFO - Found 146 tables
Checking on port progress...
Copying to postgres...
2023-03-07 11:50:50,554 - synapse_port_db - 400 - INFO - Table event_expiry: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,554 - synapse_port_db - 400 - INFO - Table user_directory_stream_pos: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,555 - synapse_port_db - 400 - INFO - Table room_tags: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,556 - synapse_port_db - 400 - INFO - Table erased_users: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,557 - synapse_port_db - 400 - INFO - Table user_stats_current: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,557 - synapse_port_db - 400 - INFO - Table user_external_ids: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,558 - synapse_port_db - 400 - INFO - Table monthly_active_users: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,558 - synapse_port_db - 400 - INFO - Table event_auth_chain_to_calculate: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,558 - synapse_port_db - 400 - INFO - Table threepid_validation_token: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,558 - synapse_port_db - 400 - INFO - Table sessions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,558 - synapse_port_db - 400 - INFO - Table stats_incremental_position: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,559 - synapse_port_db - 400 - INFO - Table refresh_tokens: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,560 - synapse_port_db - 400 - INFO - Table event_json: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,561 - synapse_port_db - 400 - INFO - Table user_daily_visits: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,561 - synapse_port_db - 400 - INFO - Table event_auth_chains: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,563 - synapse_port_db - 400 - INFO - Table local_media_repository_thumbnails: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,563 - synapse_port_db - 400 - INFO - Table state_groups: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,563 - synapse_port_db - 400 - INFO - Table user_directory_search: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,564 - synapse_port_db - 400 - INFO - Table rejections: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,564 - synapse_port_db - 400 - INFO - Table push_rules: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,564 - synapse_port_db - 400 - INFO - Table current_state_events: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,564 - synapse_port_db - 400 - INFO - Table remote_media_cache_thumbnails: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,565 - synapse_port_db - 400 - INFO - Table device_lists_remote_pending: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,565 - synapse_port_db - 400 - INFO - Table e2e_fallback_keys_json: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,565 - synapse_port_db - 400 - INFO - Table devices: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,565 - synapse_port_db - 400 - INFO - Table server_signature_keys: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,565 - synapse_port_db - 400 - INFO - Table state_events: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,566 - synapse_port_db - 400 - INFO - Table device_lists_remote_cache: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,566 - synapse_port_db - 400 - INFO - Table event_auth_chain_links: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,566 - synapse_port_db - 400 - INFO - Table room_stats_earliest_token: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,566 - synapse_port_db - 400 - INFO - Table event_search: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,567 - synapse_port_db - 400 - INFO - Table application_services_state: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,567 - synapse_port_db - 400 - INFO - Table receipts_graph: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,567 - synapse_port_db - 400 - INFO - Table room_retention: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,567 - synapse_port_db - 400 - INFO - Table federation_inbound_events_staging: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,567 - synapse_port_db - 400 - INFO - Table user_filters: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,568 - synapse_port_db - 400 - INFO - Table server_keys_json: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,568 - synapse_port_db - 400 - INFO - Table room_depth: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,568 - synapse_port_db - 400 - INFO - Table insertion_event_edges: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,568 - synapse_port_db - 400 - INFO - Table room_alias_servers: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,569 - synapse_port_db - 400 - INFO - Table room_tags_revisions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,569 - synapse_port_db - 400 - INFO - Table local_media_repository_url_cache: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,570 - synapse_port_db - 400 - INFO - Table users_pending_deactivation: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,570 - synapse_port_db - 400 - INFO - Table device_lists_changes_in_room: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,570 - synapse_port_db - 400 - INFO - Table user_threepids: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,570 - synapse_port_db - 400 - INFO - Table ex_outlier_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,570 - synapse_port_db - 400 - INFO - Table insertion_event_extremities: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,571 - synapse_port_db - 400 - INFO - Table event_forward_extremities: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,571 - synapse_port_db - 400 - INFO - Table users_who_share_private_rooms: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,571 - synapse_port_db - 400 - INFO - Table user_directory: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,571 - synapse_port_db - 400 - INFO - Table threepid_guest_access_tokens: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,571 - synapse_port_db - 400 - INFO - Table worker_locks: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,572 - synapse_port_db - 400 - INFO - Table receipts_linearized: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,572 - synapse_port_db - 400 - INFO - Table destinations: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,572 - synapse_port_db - 400 - INFO - Table federation_stream_position: 0/2 (rows 1-0) already ported
2023-03-07 11:50:50,573 - synapse_port_db - 400 - INFO - Table device_inbox: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,573 - synapse_port_db - 400 - INFO - Table threepid_validation_session: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,573 - synapse_port_db - 400 - INFO - Table local_media_repository: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,573 - synapse_port_db - 400 - INFO - Table users: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,573 - synapse_port_db - 400 - INFO - Table ui_auth_sessions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,574 - synapse_port_db - 400 - INFO - Table device_lists_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,574 - synapse_port_db - 400 - INFO - Table e2e_room_keys: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,574 - synapse_port_db - 400 - INFO - Table stream_positions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,574 - synapse_port_db - 400 - INFO - Table device_federation_outbox: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,575 - synapse_port_db - 400 - INFO - Table user_threepid_id_server: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,575 - synapse_port_db - 400 - INFO - Table rooms: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,575 - synapse_port_db - 400 - INFO - Table push_rules_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,575 - synapse_port_db - 400 - INFO - Table event_push_summary_last_receipt_stream_id: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,576 - synapse_port_db - 400 - INFO - Table applied_module_schemas: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,576 - synapse_port_db - 400 - INFO - Table users_in_public_rooms: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,576 - synapse_port_db - 400 - INFO - Table device_auth_providers: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,576 - synapse_port_db - 400 - INFO - Table push_rules_enable: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,576 - synapse_port_db - 400 - INFO - Table event_push_summary_stream_ordering: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,577 - synapse_port_db - 400 - INFO - Table background_updates: 0/13 (rows 1-0) already ported
2023-03-07 11:50:50,578 - synapse_port_db - 400 - INFO - Table state_group_edges: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,578 - synapse_port_db - 400 - INFO - Table open_id_tokens: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,578 - synapse_port_db - 400 - INFO - Table room_stats_state: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,578 - synapse_port_db - 400 - INFO - Table device_lists_remote_resync: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,578 - synapse_port_db - 400 - INFO - Table presence_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,579 - synapse_port_db - 400 - INFO - Table event_push_actions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,579 - synapse_port_db - 400 - INFO - Table local_current_membership: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,579 - synapse_port_db - 400 - INFO - Table appservice_stream_position: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,579 - synapse_port_db - 400 - INFO - Table event_reports: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,580 - synapse_port_db - 400 - INFO - Table appservice_room_list: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,580 - synapse_port_db - 400 - INFO - Table user_signature_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,580 - synapse_port_db - 400 - INFO - Table account_data: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,580 - synapse_port_db - 400 - INFO - Table redactions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,581 - synapse_port_db - 400 - INFO - Table batch_events: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,581 - synapse_port_db - 400 - INFO - Table event_push_summary: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,581 - synapse_port_db - 400 - INFO - Table event_labels: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,581 - synapse_port_db - 400 - INFO - Table device_lists_outbound_last_success: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,581 - synapse_port_db - 400 - INFO - Table account_validity: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,582 - synapse_port_db - 400 - INFO - Table login_tokens: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,582 - synapse_port_db - 400 - INFO - Table device_lists_changes_converted_stream_position: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,582 - synapse_port_db - 400 - INFO - Table blocked_rooms: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,582 - synapse_port_db - 400 - INFO - Table remote_media_cache: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,583 - synapse_port_db - 400 - INFO - Table event_backward_extremities: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,583 - synapse_port_db - 400 - INFO - Table state_groups_state: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,584 - synapse_port_db - 400 - INFO - Table e2e_one_time_keys_json: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,584 - synapse_port_db - 400 - INFO - Table e2e_cross_signing_signatures: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,584 - synapse_port_db - 400 - INFO - Table stream_ordering_to_exterm: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,585 - synapse_port_db - 400 - INFO - Table event_auth: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,585 - synapse_port_db - 400 - INFO - Table insertion_events: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,585 - synapse_port_db - 400 - INFO - Table e2e_cross_signing_keys: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,585 - synapse_port_db - 400 - INFO - Table device_lists_outbound_pokes: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,586 - synapse_port_db - 400 - INFO - Table room_memberships: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,586 - synapse_port_db - 400 - INFO - Table schema_compat_version: 0/1 (rows 1-0) already ported
2023-03-07 11:50:50,586 - synapse_port_db - 400 - INFO - Table e2e_device_keys_json: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,587 - synapse_port_db - 400 - INFO - Table pushers: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,587 - synapse_port_db - 400 - INFO - Table current_state_delta_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,587 - synapse_port_db - 400 - INFO - Table application_services_txns: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,588 - synapse_port_db - 400 - INFO - Table deleted_pushers: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,588 - synapse_port_db - 400 - INFO - Table device_lists_remote_extremeties: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,588 - synapse_port_db - 400 - INFO - Table room_aliases: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,588 - synapse_port_db - 400 - INFO - Table room_stats_current: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,588 - synapse_port_db - 400 - INFO - Table dehydrated_devices: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,589 - synapse_port_db - 400 - INFO - Table event_to_state_groups: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,589 - synapse_port_db - 400 - INFO - Table event_push_actions_staging: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,589 - synapse_port_db - 400 - INFO - Table threads: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,589 - synapse_port_db - 400 - INFO - Table event_relations: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,590 - synapse_port_db - 400 - INFO - Table e2e_room_keys_versions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,590 - synapse_port_db - 400 - INFO - Table user_ips: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,590 - synapse_port_db - 400 - INFO - Table room_account_data: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,590 - synapse_port_db - 400 - INFO - Table ratelimit_override: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,590 - synapse_port_db - 400 - INFO - Table registration_tokens: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,591 - synapse_port_db - 400 - INFO - Table profiles: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,591 - synapse_port_db - 400 - INFO - Table pusher_throttle: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,591 - synapse_port_db - 400 - INFO - Table received_transactions: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,591 - synapse_port_db - 400 - INFO - Table events: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,591 - synapse_port_db - 400 - INFO - Table ignored_users: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,592 - synapse_port_db - 400 - INFO - Table device_federation_inbox: 0/0 (rows 1-0) already ported
user_directory_stream_pos: 100% (1/1)
stats_incremental_position: 100% (1/1)
event_push_summary_last_receipt_stream_id: 100% (1/1)
federation_stream_position: 100% (2/2)
event_push_summary_stream_ordering: 100% (1/1)
appservice_stream_position: 100% (1/1)
schema_compat_version: 100% (1/1)
device_lists_changes_converted_stream_position: 100% (1/1)
background_updates: 100% (13/13)
2023-03-07 11:50:50,694 - synapse_port_db - 400 - INFO - Table un_partial_stated_room_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,694 - synapse_port_db - 400 - INFO - Table users_to_send_full_presence_to: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,695 - synapse_port_db - 400 - INFO - Table destination_rooms: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,695 - synapse_port_db - 400 - INFO - Table un_partial_stated_event_stream: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,696 - synapse_port_db - 400 - INFO - Table event_failed_pull_attempts: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,696 - synapse_port_db - 400 - INFO - Table event_edges: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,697 - synapse_port_db - 400 - INFO - Table access_tokens: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,697 - synapse_port_db - 400 - INFO - Table ui_auth_sessions_credentials: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,698 - synapse_port_db - 400 - INFO - Table ui_auth_sessions_ips: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,698 - synapse_port_db - 400 - INFO - Table partial_state_rooms: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,699 - synapse_port_db - 400 - INFO - Table event_txn_id: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,699 - synapse_port_db - 400 - INFO - Table partial_state_rooms_servers: 0/0 (rows 1-0) already ported
2023-03-07 11:50:50,700 - synapse_port_db - 400 - INFO - Table partial_state_events: 0/0 (rows 1-0) already ported

Anything else that would be useful to know?

No response
@jrick
Copy link
Author

jrick commented Mar 7, 2023

Curiously, the script also seems to create a sqlite3 db file weirdly named file:homeserver.db?mode=rw. It has some of my homeserver.db data (like access_tokens, and users), but definitely not all of it (the file is only 1.7M large).

@clokep
Copy link
Member

clokep commented Mar 7, 2023

Maybe a regression from #14692?

@clokep
Copy link
Member

clokep commented Mar 7, 2023

Can you run:

python --version

and

python -c "import sqlite3; print(sqlite3.version)"

From wherever you have Synapse installed.

@jrick
Copy link
Author

jrick commented Mar 7, 2023 

synapse$ python3 --version
Python 3.9.16
synapse$ python3 -c "import sqlite3; print(sqlite3.version)"
2.6.0

@jrick
Copy link
Author

jrick commented Mar 7, 2023

Reverted #14692 and the script appears to be running properly now.

@jrick
Copy link
Author

jrick commented Mar 7, 2023

My actual sqlite3 version (not the python package) is 3.39.3, so it should support that URI syntax referenced by that issue. Hmm...

@clokep
Copy link
Member

clokep commented Mar 7, 2023

My actual sqlite3 version (not the python package) is 3.39.3, so it should support that URI syntax referenced by that issue. Hmm...

Oh wrong version. 🤦

python3 -c "import sqlite3; print(sqlite3.sqlite_version)"

should show that, just to be clear.

@DMRobertson
Copy link
Contributor

Reverted #14692 and the script appears to be running properly now.

From https://docs.python.org/3/library/sqlite3.html?highlight=sqlite#sqlite3.connect it looks like we might need to explicitly pass uri=True to sqlite3.connect.

@reivilibre reivilibre added the X-Release-Blocker Must be resolved before making a release label Mar 9, 2023
@DMRobertson DMRobertson added S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. A-Database DB stuff like queries, migrations, new/remove columns, indexes, unexpected entries in the db A-Portdb Script for exporting sqlite data into postgres O-Occasional Affects or can be seen by some users regularly or most users rarely labels Mar 13, 2023
@MatMaul MatMaul removed the X-Release-Blocker Must be resolved before making a release label Mar 14, 2023
@clokep clokep added the X-Release-Blocker Must be resolved before making a release label Mar 20, 2023
@clokep clokep mentioned this issue Mar 20, 2023
Closed
This was referenced Mar 21, 2023
Merged
Merged
@squahtx
Copy link
Contributor

squahtx commented Mar 28, 2023

Fixed by #15301, which reverts the change that introduced the bug.
#15306 re-adds the reverted check with a different implementation.

The revert is in Synapse 1.80rc2, while the new check will be in Synapse 1.81.

@squahtx squahtx closed this as completed Mar 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
A-Database DB stuff like queries, migrations, new/remove columns, indexes, unexpected entries in the db A-Portdb Script for exporting sqlite data into postgres O-Occasional Affects or can be seen by some users regularly or most users rarely S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Release-Blocker Must be resolved before making a release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@clokep @jrick @MatMaul @squahtx @DMRobertson @reivilibre