Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Events with lots of prev_events make backfill very slow #13677

Open
matrixbot opened this issue Dec 20, 2023 · 0 comments
Open

Events with lots of prev_events make backfill very slow #13677

matrixbot opened this issue Dec 20, 2023 · 0 comments

Comments

@matrixbot
Copy link
Collaborator

matrixbot commented Dec 20, 2023

This issue has been migrated from #13677.


I tried to do a backfill in Matrix HQ, and as part as the received events, I received $k59mE1_NBJs-JP4QTza2N7VMahvLp_GlvoYLcJ821s8. This event has 20 prev_events, almost none of which I have in my db.

For each of those 20 events, we then request the full state of the room (taking 24 seconds), and then have to request a lot more individual events for the state or auth_chain. Many of those events get requested each time, because their signing keys have expired, or have been invalidated, so it is impossible to find a good copy of the event (see #13676). This whole process takes at least a minute for each of the 19 missing prev_events.

The whole /messages request took 1787 seconds (nearly 30 minutes).

@matrixbot matrixbot changed the title Dummy issue Events with lots of prev_events make backfill very slow Dec 21, 2023
@matrixbot matrixbot reopened this Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant