Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OBSOLETE] Introduce a chain agnostic generic HEAD type in the core Transaction Manager #8597

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
53a01fd
Initial changes
prashantkumar1982 Feb 27, 2023
4422700
Resolving merge conflicts
prashantkumar1982 Feb 27, 2023
8cb32ca
midpoint
prashantkumar1982 Mar 1, 2023
248a535
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 1, 2023
6b4bcc5
Initial changes with generic HeadView interface
prashantkumar1982 Mar 1, 2023
f4a1437
test fixes
prashantkumar1982 Mar 1, 2023
97aadd4
Merge issues
prashantkumar1982 Mar 2, 2023
08715d7
test fixes
prashantkumar1982 Mar 2, 2023
7c95323
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 2, 2023
ba27396
docker changes testing
prashantkumar1982 Mar 2, 2023
89c5571
mocks
prashantkumar1982 Mar 2, 2023
cf8bbfe
mocks fix
prashantkumar1982 Mar 2, 2023
8166f75
test fixes
prashantkumar1982 Mar 2, 2023
d03175f
Merge
prashantkumar1982 Mar 8, 2023
bdc3f1a
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 8, 2023
1e3573c
merging
prashantkumar1982 Mar 8, 2023
23d0be7
Merge conflicts and Generic types
prashantkumar1982 Mar 8, 2023
13928c6
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 8, 2023
58796ad
Small fixes
prashantkumar1982 Mar 8, 2023
eae70df
Small fixes
prashantkumar1982 Mar 8, 2023
bea18e1
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 8, 2023
8471df7
Build fixes
prashantkumar1982 Mar 9, 2023
15df148
build fixes
prashantkumar1982 Mar 9, 2023
510ceef
mocks
prashantkumar1982 Mar 9, 2023
b10e528
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 9, 2023
a6d24dc
Import fixes
prashantkumar1982 Mar 9, 2023
d15caa9
Fixes
prashantkumar1982 Mar 9, 2023
7db5b1d
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 9, 2023
6249aae
Test fixes
prashantkumar1982 Mar 9, 2023
cd72f52
test fixes
prashantkumar1982 Mar 9, 2023
d9ad2d1
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 9, 2023
a949f0a
refactoring
prashantkumar1982 Mar 9, 2023
ea12d46
Merge branch 'develop' of https://github.com/smartcontractkit/chainli…
prashantkumar1982 Mar 9, 2023
1d43f66
test fixes
prashantkumar1982 Mar 9, 2023
0db994d
Lint fixes
prashantkumar1982 Mar 9, 2023
b49420e
cleanups
prashantkumar1982 Mar 9, 2023
1390b5e
Response to comments
prashantkumar1982 Mar 10, 2023
3f3e4dd
nits
prashantkumar1982 Mar 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion common/txmgr/types/fee_estimator.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ type PriorAttempt[FEE any, HASH any] interface {
//
//go:generate mockery --quiet --name FeeEstimator --output ./mocks/ --case=underscore
type FeeEstimator[HEAD any, FEE any, MAXPRICE any, HASH any] interface {
OnNewLongestChain(context.Context, HEAD)
HeadTrackable[HEAD]
Start(context.Context) error
Close() error

Expand Down
12 changes: 12 additions & 0 deletions common/txmgr/types/head_trackable.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package types

import "context"

// HeadTrackable represents a generic type, implemented by the core txm,
// to be able to receive head events from any chain.
// Chain implementations should notify head events to the core txm via this interface.
//
//go:generate mockery --quiet --name HeadTrackable --output ../mocks/ --case=underscore
type HeadTrackable[HEAD any] interface {
OnNewLongestChain(ctx context.Context, head HeadView[HEAD])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDTY about simplifying this irregular name while we're changing things here?

Suggested change
OnNewLongestChain(ctx context.Context, head HeadView[HEAD])
NewHead(ctx context.Context, head HeadView[HEAD])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is indicating that we found a head which creates a new longest chain, not just any head.
For example, if we had 2 forks, A and B, like this:
A: 4,5,6,7
B:4,5

Now if we see a new head #6 on B fork, it won't trigger this callback. Only trigger on A8, or if we reach B8 before A8.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this for more context:

if prevHead == nil || head.Number > prevHead.Number {
promCurrentHead.WithLabelValues(ht.chainID.String()).Set(float64(head.Number))
headWithChain := ht.headSaver.Chain(head.Hash)
if headWithChain == nil {
return errors.Errorf("HeadTracker#handleNewHighestHead headWithChain was unexpectedly nil")
}
ht.backfillMB.Deliver(headWithChain)
ht.broadcastMB.Deliver(headWithChain)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about NewHighestHead like in that error message?

Suggested change
OnNewLongestChain(ctx context.Context, head HeadView[HEAD])
NewHighestHead(ctx context.Context, head HeadView[HEAD])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I would prefer that name. However note that OnLongestChain name is used in all places that subscribe to HeadBroadcaster, which is way more places than just TxMgr.
If we want to do that, I would prefer to do a repo wide rename in a dedicated PR.

}
32 changes: 32 additions & 0 deletions common/txmgr/types/head_view.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package types

import "github.com/ethereum/go-ethereum/common"

// HeadView provides minimal access to a chain's head, as needed by the TxManager.
// This is a generic interface which ALL chains will implement.
//
//go:generate mockery --quiet --name HeadView --output ./mocks/ --case=underscore
type HeadView[HEAD any] interface {
prashantkumar1982 marked this conversation as resolved.
Show resolved Hide resolved
// BlockNumber is the head's block number
BlockNumber() int64

// ChainLength returns the length of the chain followed by recursively looking up parents
ChainLength() uint32

// EarliestInChain traverses through parents until it finds the earliest one
EarliestInChain() HeadView[HEAD]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this return HEAD type as well? Same is for Parent() down below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No these functions are called from within the EthConfirmer, which doesn't understand the native Head type. It only can understand HeadView.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But HeadView does not hide the head type, it just parameterizes it so it can be swapped out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does hide the native type for EthConfirmer right?
The EthConfirmer needs to be generic, without importing any native types. But it also needs to call EarliestInChain().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generic types are not hidden in any way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interfaces and generics serve two separate purposes, and when they are mixed together like this they undermine one another.


// Hash is the head's block hash
Hash() common.Hash

// Parent is the head's parent block
Parent() HeadView[HEAD]

// HashAtHeight returns the hash of the block at the given height, if it is in the chain.
// If not in chain, returns the zero hash
HashAtHeight(blockNum int64) common.Hash

// GetNativeHead returns the head in the chain's native type
// Chain specific code can retrieve the native type via this function.
GetNativeHead() HEAD
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
GetNativeHead() HEAD
Native() HEAD

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't fully grok how this is meant to work. If a caller knows the type is HEAD, then why aren't they just using that directly in the first place?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also not sure about this call. The chain-specific txm implements the interface to comply with the minimum required functionality based on the HEAD type which is known to it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HeadView was a generic interface with a few functions that all chains implement. But this Native() will return the complete Head object in chain's native type. This is being passed to GasEstimators, which need the whole chain specific head object.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HeadView was a generic interface with a few functions that all chains implement. But this Native() will return the complete Head object in chain's native type. This is being passed to GasEstimators, which need the whole chain specific head object.

What problem does this abstract wrapper solve? All the callers still have to import and reference the native type anyways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Components inside the core Txm don't need to know or import native types.
For example, look at the eth_confirmer.go file.
It uses the methods in the Head interface, without knowing the native type.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Components inside the core Txm don't need to know or import native types.

Then they should use a less complex abstraction that does not impose knowing the native type - i.e. only an interface.

It uses the methods in the Head interface, without knowing the native type.

It does know the type. It is leaking everywhere. If the generic type is not being used, then it should be eliminated from that code scope with a stronger abstraction that assumes less.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there may be another way, which you might prefer.

I create a generic Head interface, similar to the one I currently have here.
Then, the native evmtypes.Head should implement it, and extend it with its own native head fields. Similarly, all chains will implement it, and extend with their own fields.

The txmgr will only use the generic Head interface, and pass it around. Chain-specific code can keep using native Head type.

I didn't do it this way earlier, as I didn't want to change the existing evmtypes.Head, as its being used in a lot of places.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me if it can be done.

}
6 changes: 3 additions & 3 deletions common/txmgr/types/mocks/fee_estimator.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 35 additions & 0 deletions common/txmgr/types/mocks/head_trackable.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

136 changes: 136 additions & 0 deletions common/txmgr/types/mocks/head_view.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 8 additions & 15 deletions core/chains/evm/chain.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"github.com/smartcontractkit/chainlink/core/chains/evm/logpoller"
"github.com/smartcontractkit/chainlink/core/chains/evm/monitor"
"github.com/smartcontractkit/chainlink/core/chains/evm/txmgr"
"github.com/smartcontractkit/chainlink/core/chains/evm/types"
"github.com/smartcontractkit/chainlink/core/logger"
"github.com/smartcontractkit/chainlink/core/services"
"github.com/smartcontractkit/chainlink/core/services/keystore"
Expand All @@ -32,7 +33,7 @@ type Chain interface {
Config() evmconfig.ChainScopedConfig
LogBroadcaster() log.Broadcaster
HeadBroadcaster() httypes.HeadBroadcaster
TxManager() txmgr.TxManager
TxManager() txmgr.TxManager[*types.Head]
HeadTracker() httypes.HeadTracker
Logger() logger.Logger
BalanceMonitor() monitor.BalanceMonitor
Expand All @@ -46,7 +47,7 @@ type chain struct {
id *big.Int
cfg evmconfig.ChainScopedConfig
client evmclient.Client
txm txmgr.TxManager
txm evmTxm
logger logger.Logger
headBroadcaster httypes.HeadBroadcaster
headTracker httypes.HeadTracker
Expand Down Expand Up @@ -114,17 +115,9 @@ func newChain(ctx context.Context, cfg evmconfig.ChainScopedConfig, nodes []*v2.
}
}

var txm txmgr.TxManager
if !cfg.EVMRPCEnabled() {
txm = &txmgr.NullTxManager{ErrMsg: fmt.Sprintf("Ethereum is disabled for chain %d", chainID)}
} else if opts.GenTxManager == nil {
checker := &txmgr.CheckerFactory{Client: client}
txm = txmgr.NewTxm(db, client, cfg, opts.KeyStore, opts.EventBroadcaster, l, checker, logPoller)
} else {
txm = opts.GenTxManager(chainID)
}
var txm = newEvmTxm(db, cfg, client, l, logPoller, opts)
prashantkumar1982 marked this conversation as resolved.
Show resolved Hide resolved

headBroadcaster.Subscribe(txm)
headBroadcaster.Subscribe(&txm)

// Highest seen head height is used as part of the start of LogBroadcaster backfill range
highestSeenHead, err := headSaver.LatestHeadFromDB(ctx)
Expand Down Expand Up @@ -185,7 +178,7 @@ func (c *chain) Start(ctx context.Context) error {
// We do not start the log poller here, it gets
// started after the jobs so they have a chance to apply their filters.
var ms services.MultiStart
if err := ms.Start(ctx, c.txm, c.headBroadcaster, c.headTracker, c.logBroadcaster); err != nil {
if err := ms.Start(ctx, &c.txm, c.headBroadcaster, c.headTracker, c.logBroadcaster); err != nil {
return err
}
if c.balanceMonitor != nil {
Expand All @@ -212,7 +205,7 @@ func (c *chain) Close() error {
merr = multierr.Combine(merr, c.headTracker.Close())
c.logger.Debug("Chain: stopping headBroadcaster")
merr = multierr.Combine(merr, c.headBroadcaster.Close())
c.logger.Debug("Chain: stopping txm")
c.logger.Debug("Chain: stopping evmTxm")
merr = multierr.Combine(merr, c.txm.Close())
c.logger.Debug("Chain: stopping client")
c.client.Close()
Expand Down Expand Up @@ -263,7 +256,7 @@ func (c *chain) Config() evmconfig.ChainScopedConfig { return c.cfg }
func (c *chain) LogBroadcaster() log.Broadcaster { return c.logBroadcaster }
func (c *chain) LogPoller() logpoller.LogPoller { return c.logPoller }
func (c *chain) HeadBroadcaster() httypes.HeadBroadcaster { return c.headBroadcaster }
func (c *chain) TxManager() txmgr.TxManager { return c.txm }
func (c *chain) TxManager() txmgr.TxManager[*types.Head] { return c.txm.TxManagerEvmType }
func (c *chain) HeadTracker() httypes.HeadTracker { return c.headTracker }
func (c *chain) Logger() logger.Logger { return c.logger }
func (c *chain) BalanceMonitor() monitor.BalanceMonitor { return c.balanceMonitor }
Expand Down
2 changes: 1 addition & 1 deletion core/chains/evm/chain_set.go
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ type ChainSetOpts struct {
GenLogBroadcaster func(*big.Int) log.Broadcaster
GenLogPoller func(*big.Int) logpoller.LogPoller
GenHeadTracker func(*big.Int, httypes.HeadBroadcaster) httypes.HeadTracker
GenTxManager func(*big.Int) txmgr.TxManager
GenTxManager func(*big.Int) txmgr.TxManager[*types.Head]
}

// NewTOMLChainSet returns a new ChainSet from TOML configuration.
Expand Down
Loading