recover senders manually if not found in db

[alessandromazza98]

Closes #3938

This is an initial draft for closing that issue.

I actually don't know if there is some other work to do in order to properly addressing that issue.

I just created a match statement in order to be able to recover senders also if they are not in the db anymore because SenderRecovery pruned mode is enabled.

[codecov]

Codecov Report

Merging #4091 (02b10ab) into main (4de1ff0) will decrease coverage by 0.30%.
Report is 79 commits behind head on main.
The diff coverage is 76.92%.

❗ Current head 02b10ab differs from pull request most recent head 96cb76d. Consider uploading reports for the commit 96cb76d to get more accurate results

Files Changed	Coverage Δ
...torage/provider/src/providers/database/provider.rs	79.54% <76.92%> (-0.09%)	⬇️

... and 45 files with indirect coverage changes

Flag	Coverage Δ
integration-tests	16.51% <0.00%> (-0.36%)	⬇️
unit-tests	63.92% <76.92%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
reth binary	25.36% <ø> (-0.58%)	⬇️
blockchain tree	83.04% <ø> (+0.21%)	⬆️
pipeline	90.07% <ø> (ø)
storage (db)	74.59% <76.92%> (-0.17%)	⬇️
trie	94.67% <ø> (-0.05%)	⬇️
txpool	48.16% <ø> (+0.49%)	⬆️
networking	77.46% <ø> (-0.27%)	⬇️
rpc	57.48% <ø> (-1.24%)	⬇️
consensus	63.76% <ø> (ø)
revm	32.26% <ø> (ø)
payload builder	6.57% <ø> (ø)
primitives	87.76% <ø> (-0.15%)	⬇️

[joshieDo]

depending on the pruning configuration, there might be senders but not for all transactions. should account for that

[joshieDo]

Because it's also important that the ordering remains the same and the tx IDs of senders and transactions coincide one-by-one.

both TxSenders and Transactions use TxNumber as key to their tables. When they get queried, they'll definitely come sorted, and without any holes.

And in this particular case, they get queried for the same range. (first_transaction..=last_transaction)?
Given that we always receive every Transaction from the query, it's just a question of seeing how many TxSenders are missing and get an iterator range from the end range of the Transaction list.

example:
Transactions:
[tx0, tx1, tx2, tx3, tx4]

TxSenders:
[s_tx0, s_tx1]

You only need to do .range(...) of the last 3 transactions: tx2, tx3, tx4.

And once this PR#4098 gets merged, you can even pass the range iterator to TransactionSigned::recover_signers

[alessandromazza98]

Because it's also important that the ordering remains the same and the tx IDs of senders and transactions coincide one-by-one.

both TxSenders and Transactions use TxNumber as key to their tables. When they get queried, they'll definitely come sorted, and without any holes.

And in this particular case, they get queried for the same range. (first_transaction..=last_transaction)? Given that we always receive every Transaction from the query, it's just a question of seeing how many TxSenders are missing and get an iterator range from the end range of the Transaction list.

example: Transactions: [tx0, tx1, tx2, tx3, tx4]

TxSenders: [s_tx0, s_tx1]

You only need to do .range(...) of the last 3 transactions: tx2, tx3, tx4.

And once this PR#4098 gets merged, you can even pass the range iterator to TransactionSigned::recover_signers

Ok I previously understood it in a wrong way because I thought that txs were ordered in the other sense: the old ones first. So in your example tx0 would be older then tx4.
And of course in this way senders would contain s_tx3, s_tx4 as they would be the latest ones.

If that's not the case as I understand from your comment, I can do something like this to update senders:

// Recover senders manually if not found in db
let start_index = senders.len();
let end_index = transactions.len();
let missing_senders = end_index - start_index;
senders.extend((start_index..end_index).zip((0..missing_senders).map(|_| {
    TransactionSigned::recover_signers(
        transactions.iter().skip(start_index),
        missing_senders,
    )
})));

[alessandromazza98]

Oh I just saw that the PR you mentioned got reverted: #4115

Should I continue without using TransactionSigned::recover_signers? Is it ok ?

[alessandromazza98]

I worked without TransactionSigned::recover_signers since it was reverted.

I just use .skip(senders.len()) because as @joshieDo pointed out to me, transactions and senders are already sorted so I just need to iterate from it.

[joshieDo]

Sorry, recover_signers has been merged again (with its fix), can you change accordingly? #4120

[alessandromazza98]

Sorry, closing this PR and re-opening a new one for this issue because I had some problems in my Vs code.

Here is the new PR: #4280