Index pending transactions #255
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KronicDeth
added
bug 🐛
…ddressExtraction Move AddressExtraction out of BlockFetcher because it is used in the InternalTransactionFetcher and will be used in the PendingTransactionFetcher.
Pending transactions have nil for "blockNumber" and "blockHash", which broke `quantity_to_integer`, so for those keys, check if `nil` and pass through. For others, check that the value is not `nil` as it gives a better error message with both the key and value instead of just the nil value passed to quantity_to_integer as happened before.
Rename Indexer.BlockFetcher.monitor_task to Indexer.start_monitor, so that all fetchers can use it.
igorffs
reviewed
Jun 6, 2018
| from( | ||
| t in Transaction, | ||
| # exclude pending transactions | ||
| where: not is_nil(t.block_hash) and is_nil(t.internal_transactions_indexed_at), |
There was a problem hiding this comment.
This seems important, wdyt of including a test for it?
igorffs
approved these changes
Jun 6, 2018
Parity doesn't support asking for the internal transactions of pending transactions.
Not including timestamps is a developer error, so it should be caught as soon as possible to allow the code to be fixed in development.
Because the repository transaction for a pending `Explorer.Chain.Transaction`s could `COMMIT` after the repository transaction for that same transaction being collated into a block, writers, it is recommended to use `:nothing` for pending transactions and `:replace_all` for collated transactions, so that collated transactions win.
Use `parity_pendingTransactions` JSONRPC to get the pending transactions and import them. If they conflict with pre-existing transactions, then the previous transactions win, under the assumption that sometimes pending transactions repository transaction will COMMIT after the the realtime index has COMMITed that same transaction being validated as the timeout for transactions is 60 seconds while the block rate is faster than that.
ghost
removed
the
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #250
Changelog
Enhancements
AddressExtraction.extract_addresses.EthereumJSONRPC.request/1inEthereumJSONRPC.Parity.Indexer.BlockFetcher.monitor_tasktoIndexer.start_monitorand make it public, so that all*Fetchers can use it.timestampsoption as soon as possible as not including timestamps is a developer error, so it should be caught as soon as possible to allow the code to be fixed in development.parity_pendingTransactionsJSONRPC to get the pending transactions and import them. If they conflict with pre-existing transactions, then the previous transactions win, under the assumption that sometimes pending transactions repository transaction willCOMMITafter the therealtime index has
COMMITed that same transaction being validated as the timeout for transactions is 60 seconds while the block rate is faster than that.Bug Fixes
AddressExtractionout ofBlockFetcherbecause it is used in theInternalTransactionFetcherandPendingTransactionFetcher.nilfor"blockNumber"and"transactionIndex", which brokequantity_to_integer, so for those keys, check ifniland pass through. For others, check that the value is notnilas it gives a better error message with both the key and value instead of just thenilvalue passed to quantity_to_integer as happened before.:transactions:on_conflictoption that is required when:transactions:paramsas passed because the repository transaction for a pendingExplorer.Chain.Transactions couldCOMMITafter the repository transaction for that same transaction being collated into a block, writers, it is recommended to use:nothingfor pending transactions and:replace_allfor collated transactions, so that collated transactions win.params_optionvalue should be[map], notmapsince it is a list of params for the given schema.adddresses->addressesinaddresses_option.