Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[transaction-controller] Boundless growth of transaction history #4549

Closed
Gudahtt opened this issue Jul 23, 2024 · 3 comments · Fixed by #4555
Closed

[transaction-controller] Boundless growth of transaction history #4549

Gudahtt opened this issue Jul 23, 2024 · 3 comments · Fixed by #4555
Assignees
Labels
bug Something isn't working team-confirmations

Comments

@Gudahtt
Copy link
Member

Gudahtt commented Jul 23, 2024

Each transaction has a history property, which tracks changes to the transaction state. This history doesn't currently have a maximum limit and can grow indefinitely, effectively becoming a memory leak. Unfortunately it can grow rather rapidly as well, because it tracks changes to the transaction warning.error state which can update multiple times in a single pending transaction status check.

We should prevent this history from growing unbounded, and maybe replace or severely limit its size in general.

@Gudahtt Gudahtt added bug Something isn't working team-confirmations labels Jul 23, 2024
@Gudahtt Gudahtt changed the title [transaction-controller] Prevent boundless growth of transaction history [transaction-controller] Boundless growth of transaction history Jul 23, 2024
@Gudahtt
Copy link
Member Author

Gudahtt commented Jul 23, 2024

In one report of this happening, there was a loop of these two updates:

[
  {
    value: '[ethjs-rpc] rpc error with payload {"id":[number],"jsonrpc":"2.0","params":["0x"],"method":"eth_getTransactionReceipt"} { "code": -32603, "message": "Internal JSON-RPC error.", "data": { "code": -32602, "message": "invalid argument 0: hex string has length 0, want 64 for common.Hash", "cause": null }, "stack": ...'
    path: '/warning/error',
    op: 'replace'
  },
  {
    note: 'transactions/pending-tx-tracker#event: tx:warning',
    value: 'There was a problem loading this transaction.',
    path: '/warning/message',
    op: 'replace'
  },
]

In this case it was encountered with a Ledger account, though I'm not sure whether that's relevant.

@Gudahtt
Copy link
Member Author

Gudahtt commented Jul 24, 2024

Trimming this down is tricky because we use the transaction history in the UI.

It was originally added as diagnostic data, but we began using it for the "Transaction activity" view, for example here: https://github.com/MetaMask/metamask-extension/blob/be1c107fd323112a6677e05e2714b6602c4367ab/ui/components/app/transaction-activity-log/transaction-activity-log.util.js#L76 (this is the one example I could find quickly, not sure if there are others).

@Gudahtt
Copy link
Member Author

Gudahtt commented Jul 24, 2024

Here is the related extension issue: MetaMask/metamask-extension#9372

@Gudahtt Gudahtt self-assigned this Jul 24, 2024
Gudahtt added a commit that referenced this issue Jul 24, 2024
The TransactionController has been updated to compress transaction history if
it exceeds the max transaction history size, which for now has been set to 100.
Each time a new entry is added to a transaction history already at max size, we
merge two entries to make room for the new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that we want
to display. If there are no non-displayed entries to compress, history is
allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the transaction
history. While technically it is still unbounded at this level because we don't
strictly limit displayed history entries, we don't know of any cases where
displayed entries can be repeated a significant number of times, so this will
solve that problem in practice.

Fixes #4549
Gudahtt added a commit that referenced this issue Jul 24, 2024
The TransactionController has been updated to compress transaction history if
it exceeds the max transaction history size, which for now has been set to 100.
Each time a new entry is added to a transaction history already at max size, we
merge two entries to make room for the new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that we want
to display. If there are no non-displayed entries to compress, history is
allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the transaction
history. While technically it is still unbounded at this level because we don't
strictly limit displayed history entries, we don't know of any cases where
displayed entries can be repeated a significant number of times, so this will
solve that problem in practice.

Fixes #4549
Gudahtt added a commit that referenced this issue Jul 24, 2024
The TransactionController has been updated to compress transaction history if
it exceeds the max transaction history size, which for now has been set to 100.
Each time a new entry is added to a transaction history already at max size, we
merge two entries to make room for the new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that we want
to display. If there are no non-displayed entries to compress, history is
allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the transaction
history. While technically it is still unbounded at this level because we don't
strictly limit displayed history entries, we don't know of any cases where
displayed entries can be repeated a significant number of times, so this will
solve that problem in practice.

Fixes #4549
Gudahtt added a commit that referenced this issue Jul 24, 2024
The TransactionController has been updated to compress transaction history if
it exceeds the max transaction history size, which for now has been set to 100.
Each time a new entry is added to a transaction history already at max size, we
merge two entries to make room for the new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that we want
to display. If there are no non-displayed entries to compress, history is
allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the transaction
history. While technically it is still unbounded at this level because we don't
strictly limit displayed history entries, we don't know of any cases where
displayed entries can be repeated a significant number of times, so this will
solve that problem in practice.

Fixes #4549
Gudahtt added a commit that referenced this issue Jul 25, 2024
The TransactionController has been updated to compress transaction history if
it exceeds the max transaction history size, which for now has been set to 100.
Each time a new entry is added to a transaction history already at max size, we
merge two entries to make room for the new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that we want
to display. If there are no non-displayed entries to compress, history is
allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the transaction
history. While technically it is still unbounded at this level because we don't
strictly limit displayed history entries, we don't know of any cases where
displayed entries can be repeated a significant number of times, so this will
solve that problem in practice.

Fixes #4549
Gudahtt added a commit that referenced this issue Jul 26, 2024
The TransactionController has been updated to compress transaction history if
it exceeds the max transaction history size, which for now has been set to 100.
Each time a new entry is added to a transaction history already at max size, we
merge two entries to make room for the new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that we want
to display. If there are no non-displayed entries to compress, history is
allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the transaction
history. While technically it is still unbounded at this level because we don't
strictly limit displayed history entries, we don't know of any cases where
displayed entries can be repeated a significant number of times, so this will
solve that problem in practice.

Fixes #4549
AugmentedMode pushed a commit that referenced this issue Jul 30, 2024
## Explanation

The TransactionController has been updated to compress transaction
history if it exceeds the max transaction history size, which for now
has been set to 100. Each time a new entry is added to a transaction
history already at max size, we merge two entries to make room for the
new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that
we want to display. If there are no non-displayed entries to compress,
history is allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the
transaction history. While technically it is still unbounded at this
level because we don't strictly limit displayed history entries, we
don't know of any cases where displayed entries can be repeated a
significant number of times, so this will solve that problem in
practice.

## References

Fixes #4549

## Changelog

For the `@metamask/transaction-controller` package:
```markdown
### Added

- Add `DISPLAYED_TRANSACTION_HISTORY_PATHS` constant, representing the transaction history paths that may be used for display ([#4555](#4555))
  - This was exported so that it might be used to ensure display logic and internal history logic remains in-sync.
  - Any paths listed here will have their timestamps preserved. Unlisted paths may be compressed by the controller to minimize history size, losing the timestamp.
- Add `MAX_TRANSACTION_HISTORY_LENGTH` constant, representing the expected maximum size of the `history` property for a given transaction ([#4555](#4555))
  - Note that this is not strictly enforced, the length may exceed this number of all entries are "displayed" entries, but we expect this to be extremely improbable in practice.

### Fixed

- Prevent transaction history from growing endlessly in size ([#4555](#4555))
```
## Checklist

- [x] I've updated the test suite for new or updated code as appropriate
- [x] I've updated documentation (JSDoc, Markdown, etc.) for new or
updated code as appropriate
- [x] I've highlighted breaking changes using the "BREAKING" category
above as appropriate

---------

Co-authored-by: Matthew Walsh <matthew.walsh@consensys.net>
Gudahtt added a commit that referenced this issue Jul 30, 2024
The TransactionController has been updated to compress transaction
history if it exceeds the max transaction history size, which for now
has been set to 100. Each time a new entry is added to a transaction
history already at max size, we merge two entries to make room for the
new one.

Note that we never compress entries used for display in the transaction
activity log, because compressing those entries might hide events that
we want to display. If there are no non-displayed entries to compress,
history is allowed to exceed the max size.

This is a temporary solution to prevent unbounded growth of the
transaction history. While technically it is still unbounded at this
level because we don't strictly limit displayed history entries, we
don't know of any cases where displayed entries can be repeated a
significant number of times, so this will solve that problem in
practice.

Fixes #4549

For the `@metamask/transaction-controller` package:
```markdown

- Add `DISPLAYED_TRANSACTION_HISTORY_PATHS` constant, representing the transaction history paths that may be used for display ([#4555](#4555))
  - This was exported so that it might be used to ensure display logic and internal history logic remains in-sync.
  - Any paths listed here will have their timestamps preserved. Unlisted paths may be compressed by the controller to minimize history size, losing the timestamp.
- Add `MAX_TRANSACTION_HISTORY_LENGTH` constant, representing the expected maximum size of the `history` property for a given transaction ([#4555](#4555))
  - Note that this is not strictly enforced, the length may exceed this number of all entries are "displayed" entries, but we expect this to be extremely improbable in practice.

- Prevent transaction history from growing endlessly in size ([#4555](#4555))
```

- [x] I've updated the test suite for new or updated code as appropriate
- [x] I've updated documentation (JSDoc, Markdown, etc.) for new or
updated code as appropriate
- [x] I've highlighted breaking changes using the "BREAKING" category
above as appropriate

---------

Co-authored-by: Matthew Walsh <matthew.walsh@consensys.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working team-confirmations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant