feat: ensure events are inserted into the raw event request table #1925
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Vercel deployment URL: https://stacks-blockchain-i7qr60ei6-blockstack.vercel.app 🚀 |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1925 +/- ##
==========================================
- Coverage 72.07% 72.02% -0.06%
==========================================
Files 93 93
Lines 12026 12027 +1
Branches 2654 2654
==========================================
- Hits 8668 8662 -6
- Misses 3202 3209 +7
Partials 156 156 ☔ View full report in Codecov by Sentry. |
zone117x
commented
Apr 5, 2024
rafaelcr
reviewed
Apr 5, 2024
zone117x
commented
Apr 5, 2024
zone117x
commented
Apr 5, 2024
rafaelcr
approved these changes
Apr 5, 2024
csgui
approved these changes
Apr 5, 2024
Release v7.10.0 stable
## [7.10.0](v7.9.1...v7.10.0) (2024-04-15) ### Features * add nakamoto block time to v2 endpoints ([#1921](#1921)) ([ae6bbe8](ae6bbe8)) * add signer-keys from pox4 events ([#1857](#1857)) ([c17ad23](c17ad23)) * ingest signer_bitvec ([#1900](#1900)) ([aa1750f](aa1750f)) * nakamoto block timestamps ([#1886](#1886)) ([f547832](f547832)) * pox 4 revoke events and signer-key support ([#1829](#1829)) ([5e5650a](5e5650a)), closes [#1849](#1849) * pox stacker & signer cycle details ([#1873](#1873)) ([d2c2805](d2c2805)) * rosetta pox4 stacking support ([#1928](#1928)) ([2ba36f9](2ba36f9)), closes [#1929](#1929) ### Bug Fixes * add nakamoto testnet to openapi docs ([#1910](#1910)) ([01fb971](01fb971)) * batch drop mempool transactions ([#1920](#1920)) ([a7ee96d](a7ee96d)) * cycle signer filter ([#1916](#1916)) ([dc7d600](dc7d600)) * cycles response for empty cycle info ([#1914](#1914)) ([a7a4558](a7a4558)) * delegate-stx burn-op parsing and test fix ([#1939](#1939)) ([73ec0db](73ec0db)) * event-replay readiness for nakamoto & fix for [#1879](#1879) ([#1903](#1903)) ([1572e73](1572e73)) * log message when sql migration is performed ([#1942](#1942)) ([49a4d25](49a4d25)) * other empty result responses ([#1915](#1915)) ([3cd2c64](3cd2c64)) * pox4 stack-stx burn-op handling ([#1936](#1936)) ([9e9a464](9e9a464)) * remove signer columns from tenure-change transactions ([#1845](#1845)) ([8ec726b](8ec726b)) * sql transactional consistency bug with fetching chaintip in various areas ([#1853](#1853)) ([ada8536](ada8536))
…erwise explicit error
csgui
force-pushed
the
fix/ensure-raw-event-request-insert
branch
from
8a81360 to
cb4016d
Compare
|
@zone117x Just validated this fix against an archive synced from genesis. Please check the conversation on this devops issue for more context. Blocks containing a large number of transactions are now included in the archive file. For example, block 137843, which was the root cause of this issue, is now present in the archive file after the fix. Therefore, for a block to be present in the archive file, it must have also been present in the Fix LGTM. |
|
Awesome, thanks for the thorough testing @csgui! |
|
🎉 This PR is included in version 7.11.0-beta.8 🎉 The release is available on: Your semantic-release bot 📦🚀 |
blockstack-devops
pushed a commit
that referenced
this pull request
Jun 7, 2024
## [7.11.0](v7.10.0...v7.11.0) (2024-06-07) ### Features * add average stacks block time to burn block endpoints ([#1963](#1963)) ([31c2eed](31c2eed)) * add burn_block_height to Rosetta endpoints ([#1974](#1974)) ([9648ac8](9648ac8)) * add burn_block_height to transactions ([#1969](#1969)) ([3e2d524](3e2d524)) * add signer_address to pox signer endpoints ([#1975](#1975)) ([7d3444b](7d3444b)) * add total tx count to burn block endpoints ([#1965](#1965)) ([d38b78a](d38b78a)) * average block times endpoint ([#1962](#1962)) ([cd151aa](cd151aa)) * docker build for arm ([#1947](#1947)) ([2c526fc](2c526fc)) * include solo and pooled stackers in signer stacker endpoints ([#1987](#1987)) ([302a5d8](302a5d8)) * support multiple STX faucet source accounts ([#1946](#1946)) ([be5db0c](be5db0c)) * support multiple STX faucet source accounts ([#1946](#1946)) ([5d69c7c](5d69c7c)) ### Bug Fixes * ensure events are inserted into the raw event request table ([#1925](#1925)) ([34a8454](34a8454)) * inconsistent block transaction results in Rosetta response ([#1958](#1958)) ([a5bec61](a5bec61)) * issue with block_time receipt not being written to db ([#1961](#1961)) ([74c06c6](74c06c6)) * pox4 properties missing in various endpoints ([#1977](#1977)) ([521d771](521d771)) * rosetta account endpoint should assume chain tip if block not specified ([#1956](#1956)) ([4bba526](4bba526)) * signer stacker query using string instead of int for cycle_id ([#1991](#1991)) ([5ce9b44](5ce9b44)) * socket-io client must only use websocket transport ([#1976](#1976)) ([85ea5af](85ea5af))
|
🎉 This PR is included in version 7.11.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue found by @csgui where the
event_observer_requeststable can be missing events even if the rest of the event ingestion was successful.This appears to be because the function for inserting into
event_observer_requestswas ran after the event server http request has been completed. So when a large block was received, it may have been ingested into the regular tables, but could run into an OOM error on theevent_observer_requestsinsert.This PR fixes the issue by ensuring the http request is not completed until insert into
event_observer_requestsis completed. Also the amount of JSON parsing/stringifying has been reduced in an attempt to reduce likelihood of an OOM when receiving a large/new_blockevent payload.