Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Port script cannot be run twice on the same DB #8619

Closed
erikjohnston opened this issue Oct 21, 2020 · 2 comments · Fixed by #8755
Closed

Port script cannot be run twice on the same DB #8619

erikjohnston opened this issue Oct 21, 2020 · 2 comments · Fixed by #8755
Labels
S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Regression Something broke which worked on a previous release z-p2 (Deprecated Label)

Comments

@erikjohnston
Copy link
Member

The port script should be able to run multiple times on the same DB, to allow incremental porting.

This fails with:

(env) ~/temp/synapse >>> synapse_port_db --sqlite-database matrix.db.snapshot --postgres-config homeserver-postgres.yaml                                                                  [130]
Preparing sqlite3...
2020-10-21 18:04:54,927 - synapse.storage.prepare_database - 102 - INFO - ['main', 'state']: Checking existing schema version
2020-10-21 18:04:54,928 - synapse.storage.prepare_database - 107 - INFO - ['main', 'state']: Existing schema is 58 (+20 deltas)
2020-10-21 18:04:54,929 - synapse.storage.databases.main - 601 - INFO - Checking database for consistency with configuration...
2020-10-21 18:04:54,929 - synapse.storage.prepare_database - 379 - INFO - Applying schema deltas for v58
2020-10-21 18:04:54,929 - synapse.storage.prepare_database - 503 - INFO - Schema now up to date
2020-10-21 18:04:54,931 - synapse.storage.util.id_generators - 57 - INFO - initialising stream generator for events(stream_ordering)
2020-10-21 18:04:54,931 - synapse.storage.util.id_generators - 57 - INFO - initialising stream generator for events(stream_ordering)
2020-10-21 18:04:54,931 - synapse.storage.util.id_generators - 57 - INFO - initialising stream generator for ex_outlier_stream(event_stream_ordering)
Preparing psycopg2...
2020-10-21 18:04:54,946 - synapse.storage.prepare_database - 102 - INFO - ['main', 'state']: Checking existing schema version
2020-10-21 18:04:54,948 - synapse.storage.prepare_database - 107 - INFO - ['main', 'state']: Existing schema is 58 (+22 deltas)
2020-10-21 18:04:54,948 - synapse.storage.databases.main - 601 - INFO - Checking database for consistency with configuration...
2020-10-21 18:04:54,949 - synapse.storage.prepare_database - 379 - INFO - Applying schema deltas for v58
2020-10-21 18:04:54,949 - synapse.storage.prepare_database - 503 - INFO - Schema now up to date
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1081, in emit
    msg = self.format(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 925, in format
    return fmt.format(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 664, in format
    record.message = record.getMessage()
  File "/usr/lib/python3.8/logging/__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not enough arguments for format string
Call stack:
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 1097, in <module>
    start()
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 1092, in start
    reactor.run()
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/base.py", line 1283, in run
    self.mainLoop()
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/base.py", line 1292, in mainLoop
    self.runUntilCurrent()
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/metrics/__init__.py", line 538, in f
    ret = func(*args, **kwargs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/base.py", line 886, in runUntilCurrent
    f(*a, **kw)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/defer.py", line 460, in callback
    self._startRunCallbacks(result)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 547, in run
    self.postgres_store = self.build_db_store(
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 497, in build_db_store
    store = Store(DatabasePool(hs, db_config, engine), db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/client_ips.py", line 34, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/deviceinbox.py", line 286, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/devices.py", line 704, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/events_bg_updates.py", line 32, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/media_repository.py", line 27, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/registration.py", line 784, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/registration.py", line 39, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/room.py", line 866, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/roommember.py", line 826, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/search.py", line 92, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/state/bg_updates.py", line 184, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/state.py", line 323, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/roommember.py", line 57, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/events_worker.py", line 83, in __init__
    self._stream_id_gen = MultiWriterIdGenerator(
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/util/id_generators.py", line 263, in __init__
    self._sequence_gen.check_consistency(
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/util/sequence.py", line 118, in check_consistency
    logger.warning(
Message: 'Postgres sequence %s is behind table %s: %d < %d'
Arguments: (1, 41)
2020-10-21 18:04:54,960 - synapse_port_db - 639 - ERROR - 
Traceback (most recent call last):
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 547, in run
    self.postgres_store = self.build_db_store(
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 497, in build_db_store
    store = Store(DatabasePool(hs, db_config, engine), db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/client_ips.py", line 34, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/deviceinbox.py", line 286, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/devices.py", line 704, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/events_bg_updates.py", line 32, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/media_repository.py", line 27, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/registration.py", line 784, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/registration.py", line 39, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/room.py", line 866, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/roommember.py", line 826, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/search.py", line 92, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/state/bg_updates.py", line 184, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/state.py", line 323, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/roommember.py", line 57, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/events_worker.py", line 83, in __init__
    self._stream_id_gen = MultiWriterIdGenerator(
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/util/id_generators.py", line 263, in __init__
    self._sequence_gen.check_consistency(
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/util/sequence.py", line 123, in check_consistency
    raise IncorrectDatabaseSetup(
synapse.storage.engines._base.IncorrectDatabaseSetup: 
Postgres sequence 'events_stream_seq' is inconsistent with associated
table 'events'. This can happen if Synapse has been downgraded and
then upgraded again, or due to a bad migration.

To fix this error, shut down Synapse (including any and all workers)
and run the following SQL:

    SELECT setval('events_stream_seq', (
        SELECT GREATEST(MAX(stream_ordering), 0) FROM events
    ));

See docs/postgres.md for more information.

Traceback (most recent call last):
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 547, in run
    self.postgres_store = self.build_db_store(
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 497, in build_db_store
    store = Store(DatabasePool(hs, db_config, engine), db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/client_ips.py", line 34, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/deviceinbox.py", line 286, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/devices.py", line 704, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/events_bg_updates.py", line 32, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/media_repository.py", line 27, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/registration.py", line 784, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/registration.py", line 39, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/room.py", line 866, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/roommember.py", line 826, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/search.py", line 92, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/state/bg_updates.py", line 184, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/state.py", line 323, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/roommember.py", line 57, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/databases/main/events_worker.py", line 83, in __init__
    self._stream_id_gen = MultiWriterIdGenerator(
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/util/id_generators.py", line 263, in __init__
    self._sequence_gen.check_consistency(
  File "/home/twilight/temp/synapse/env/lib/python3.8/site-packages/synapse/storage/util/sequence.py", line 123, in check_consistency
    raise IncorrectDatabaseSetup(
synapse.storage.engines._base.IncorrectDatabaseSetup: 
Postgres sequence 'events_stream_seq' is inconsistent with associated
table 'events'. This can happen if Synapse has been downgraded and
then upgraded again, or due to a bad migration.

To fix this error, shut down Synapse (including any and all workers)
and run the following SQL:

    SELECT setval('events_stream_seq', (
        SELECT GREATEST(MAX(stream_ordering), 0) FROM events
    ));

See docs/postgres.md for more information.

Traceback (most recent call last):
  File "/home/twilight/temp/synapse/env/bin/synapse_port_db", line 1104, in <module>
    sys.stderr.write(end_error)
TypeError: write() argument must be str, not IncorrectDatabaseSetup
@clokep clokep added z-bug (Deprecated Label) p1 labels Oct 21, 2020
@clokep clokep added the z-regression (Deprecated Label) label Oct 29, 2020
@richvdh richvdh mentioned this issue Nov 12, 2020
Closed
erikjohnston added a commit that referenced this issue Nov 13, 2020
@erikjohnston
Fix port script so that it can be run again after failure.
8d53639 
If the script fails (or is CTRL-C'ed) between porting some of the events
table and copying of the sequences then the port script will immediately
die if run again due to the postgres DB having inconsistencies between
sequences and tables.

Fixes #8619
@erikjohnston erikjohnston mentioned this issue Nov 13, 2020
Merged
erikjohnston added a commit that referenced this issue Nov 13, 2020
@erikjohnston
Fix port script so that it can be run again after failure. (#8755)
1b15a3d 
If the script fails (or is CTRL-C'ed) between porting some of the events table and copying of the sequences then the port script will immediately die if run again due to the postgres DB having inconsistencies between sequences and tables.

The fix is to move the porting of sequences to before porting the tables, so that there is never a period where the Postgres DB is inconsistent. To do that we need to change how we port the sequences so that it calculates the values from the SQLite DB rather than the Postgres DB.

Fixes #8619
@richvdh richvdh reopened this May 5, 2021
@richvdh
Copy link
Member

richvdh commented May 5, 2021

this seems to have regressed:

Preparing sqlite3...
2021-05-05 16:27:47,280 - synapse.storage.prepare_database - 98 - INFO - ('main', 'state'): Checking existing schema version
2021-05-05 16:27:47,282 - synapse.storage.prepare_database - 103 - INFO - ('main', 'state'): Existing schema is 59 (+15 deltas)
2021-05-05 16:27:47,283 - synapse.storage.databases.main - 351 - INFO - Checking database for consistency with configuration...
2021-05-05 16:27:47,283 - synapse.storage.prepare_database - 374 - INFO - Applying schema deltas for v59
2021-05-05 16:27:47,284 - synapse.storage.prepare_database - 504 - INFO - Schema now up to date
2021-05-05 16:27:47,307 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for pushers(id)
2021-05-05 16:27:47,307 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for deleted_pushers(stream_id)
2021-05-05 16:27:47,308 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for events(stream_ordering)
2021-05-05 16:27:47,309 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for events(stream_ordering)
2021-05-05 16:27:47,309 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for ex_outlier_stream(event_stream_ordering)
Preparing psycopg2...
2021-05-05 16:27:47,356 - synapse.storage.prepare_database - 98 - INFO - ('main', 'state'): Checking existing schema version
2021-05-05 16:27:47,360 - synapse.storage.prepare_database - 103 - INFO - ('main', 'state'): Existing schema is 59 (+19 deltas)
2021-05-05 16:27:47,361 - synapse.storage.databases.main - 351 - INFO - Checking database for consistency with configuration...
2021-05-05 16:27:47,362 - synapse.storage.prepare_database - 374 - INFO - Applying schema deltas for v59
2021-05-05 16:27:47,363 - synapse.storage.prepare_database - 504 - INFO - Schema now up to date
2021-05-05 16:27:47,369 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for pushers(id)
2021-05-05 16:27:47,370 - synapse.storage.util.id_generators - 55 - INFO - initialising stream generator for deleted_pushers(stream_id)
2021-05-05 16:27:47,379 - synapse.storage.util.sequence - 173 - WARNING - Postgres sequence event_auth_chain_id is behind table event_auth_chains: 224 < 225
2021-05-05 16:27:47,379 - synapse_port_db - 715 - ERROR - 
Traceback (most recent call last):
  File "./scripts/synapse_port_db", line 580, in run
    self.postgres_store = self.build_db_store(
  File "./scripts/synapse_port_db", line 530, in build_db_store
    store = Store(DatabasePool(hs, db_config, engine), db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/client_ips.py", line 34, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/deviceinbox.py", line 541, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/devices.py", line 919, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/events_bg_updates.py", line 57, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/media_repository.py", line 47, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/registration.py", line 1082, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/registration.py", line 78, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/cache.py", line 42, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/room.py", line 1016, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/roommember.py", line 959, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/search.py", line 91, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/state/bg_updates.py", line 186, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/state.py", line 322, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/roommember.py", line 67, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/events_worker.py", line 174, in __init__
    self.event_chain_id_gen = build_sequence_generator(
  File "/home/rav/work/synapse/synapse/storage/util/sequence.py", line 283, in build_sequence_generator
    seq.check_consistency(
  File "/home/rav/work/synapse/synapse/storage/util/sequence.py", line 180, in check_consistency
    raise IncorrectDatabaseSetup(
synapse.storage.engines._base.IncorrectDatabaseSetup: 
Postgres sequence 'event_auth_chain_id' is inconsistent with associated
table 'event_auth_chains'. This can happen if Synapse has been downgraded and
then upgraded again, or due to a bad migration.

To fix this error, shut down Synapse (including any and all workers)
and run the following SQL:

    SELECT setval('event_auth_chain_id', (
        SELECT GREATEST(MAX(chain_id), 0) FROM event_auth_chains
    ));

See docs/postgres.md for more information.

Traceback (most recent call last):
  File "./scripts/synapse_port_db", line 580, in run
    self.postgres_store = self.build_db_store(
  File "./scripts/synapse_port_db", line 530, in build_db_store
    store = Store(DatabasePool(hs, db_config, engine), db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/client_ips.py", line 34, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/deviceinbox.py", line 541, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/devices.py", line 919, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/events_bg_updates.py", line 57, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/media_repository.py", line 47, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/registration.py", line 1082, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/registration.py", line 78, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/cache.py", line 42, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/room.py", line 1016, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/roommember.py", line 959, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/search.py", line 91, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/state/bg_updates.py", line 186, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/state.py", line 322, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/roommember.py", line 67, in __init__
    super().__init__(database, db_conn, hs)
  File "/home/rav/work/synapse/synapse/storage/databases/main/events_worker.py", line 174, in __init__
    self.event_chain_id_gen = build_sequence_generator(
  File "/home/rav/work/synapse/synapse/storage/util/sequence.py", line 283, in build_sequence_generator
    seq.check_consistency(
  File "/home/rav/work/synapse/synapse/storage/util/sequence.py", line 180, in check_consistency
    raise IncorrectDatabaseSetup(
synapse.storage.engines._base.IncorrectDatabaseSetup: 
Postgres sequence 'event_auth_chain_id' is inconsistent with associated
table 'event_auth_chains'. This can happen if Synapse has been downgraded and
then upgraded again, or due to a bad migration.

To fix this error, shut down Synapse (including any and all workers)
and run the following SQL:

    SELECT setval('event_auth_chain_id', (
        SELECT GREATEST(MAX(chain_id), 0) FROM event_auth_chains
    ));

See docs/postgres.md for more information.


Postgres sequence 'event_auth_chain_id' is inconsistent with associated
table 'event_auth_chains'. This can happen if Synapse has been downgraded and
then upgraded again, or due to a bad migration.

To fix this error, shut down Synapse (including any and all workers)
and run the following SQL:

    SELECT setval('event_auth_chain_id', (
        SELECT GREATEST(MAX(chain_id), 0) FROM event_auth_chains
    ));

See docs/postgres.md for more information.

@richvdh richvdh added S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Regression Something broke which worked on a previous release and removed z-bug (Deprecated Label) z-p1 z-regression (Deprecated Label) labels May 5, 2021
@callahad callahad added the P2 label May 6, 2021
@richvdh
Copy link
Member

richvdh commented May 13, 2021

It's actually worse than that - Synapse won't start when pointed to the ported database. This isn't really a regression of the original bug, so I'm going to close this in favour of #9979.

@richvdh richvdh closed this as completed May 13, 2021
@kittykat kittykat added the z-p2 (Deprecated Label) label Sep 6, 2022
@axelsimon axelsimon mentioned this issue Jul 31, 2023
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
S-Minor Blocks non-critical functionality, workarounds exist. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Regression Something broke which worked on a previous release z-p2 (Deprecated Label)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix port script so that it can be run again after failure. matrix-org/synapse
6 participants
@callahad @kittykat @neilisfragile @clokep @richvdh @erikjohnston