Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure implicit SQL transaction is intact during parallel insertions #1830

Closed
zone117x opened this issue Jan 16, 2024 · 1 comment
Closed
Assignees

Comments

@zone117x
Copy link
Member

zone117x commented Jan 16, 2024

PR #1818 introduced parallel insertions during block ingestion.

For reference here is the relevant code:
https://github.com/hirosystems/stacks-blockchain-api/pull/1818/files#diff-6015cbd3e33ffb7064a4076b06232863ccf21ecc15a17f51dc4c4d223e31cd2bR264-R278

The write queries leverage implicit sql transaction handling. This is done via some magic involving AsyncLocalStorage: https://github.com/hirosystems/api-toolkit/blob/83acd010be5f1703e258d567c69c838fb6b05936/src/postgres/base-pg-store.ts#L51-L64

In the optimization PR, queries which previously used async/await in series are now added to a queue with callbacks, where think the async call stack is dropped, and the implicit sql transaction handling may stop working as expected. E.g. each of the callbacks invoked by the queue might be its own async context and therefore its own sql transaction.

This would be a critical bug because any issue with block ingestion would no longer rollback all changes made and corrupt the database.

@github-project-automation github-project-automation bot moved this to 🆕 New in API Board Jan 16, 2024
@smcclellan smcclellan moved this from 🆕 New to 📋 Backlog in API Board Jan 16, 2024
@zone117x
Copy link
Member Author

This is working as expected

@github-project-automation github-project-automation bot moved this from 📋 Backlog to ✅ Done in API Board Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

2 participants