Staking pools fetching

[saneery]

Set blockscout to work with staking app in POSDAO network

This PR for fetching staking pools from POSDAO network.

I added:

• Fetcher.StakingPools - fetches current list of pools from the network on per block.
• Import.Runner.StakingPools - inserts new pools list to the database.

Required variables:

• POS_VALIDATORS_CONTRACT
• POS_STAKING_CONTRACT

To run fetcher you need to enable it in the indexer/config/config.exs:

config :indexer, Indexer.Fetcher.StakingPools.Supervisor, disabled?: false

Also you need to disable BlockReward supervisor

config :indexer, Indexer.Fetcher.BlockReward.Supervisor, disabled?: true

[coveralls]

Pull Request Test Coverage Report for Build 3ae0afae-1dc3-41e8-8514-f3e981150d28

• 53 of 64 (82.81%) changed or added relevant lines in 6 files are covered.
• 4 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.2%) to 82.274%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
apps/explorer/lib/explorer/chain/import/runner/staking_pools.ex	6	9	66.67%
apps/explorer/lib/explorer/staking/pools_reader.ex	26	29	89.66%
apps/indexer/lib/indexer/fetcher/staking_pools.ex	18	23	78.26%

Files with Coverage Reduction	New Missed Lines	%
apps/indexer/lib/indexer/block/catchup/fetcher.ex	4	69.23%

Totals
Change from base Build 7cefe622-f206-463b-ab38-d5da6c0ac6a5:	0.2%
Covered Lines:	4530
Relevant Lines:	5506

---

💛 - Coveralls

[zachdaniel]

Might be good to put these in module attributes since they are constants.

[saneery]

@zachdaniel Are you about the contract address?

[zachdaniel]

No, the abi file names.

[zachdaniel]

I approved just based on the code itself, but I'm not really familiar with DApps themselves, so others will definitely want to weigh in on that aspect.

[goodsoft]

From what I see, you make 2 + 9 * N (where N is total count of pools) of eth_call requests once on the launch, and then after every new block import. Method calls can and should be batched together to reduce request overhead.

Also, I'm not sure of how POSDAO works, but is it really necessary to query each and every existing pool after each and every incoming block? Is it maybe possible to gather a list of updated pools from
block content?

[goodsoft]

This will replace any existing name with anonymous, according to import runner, won't it?

[saneery]

In staking app pools are all anonymous

[goodsoft]

Okay, I assumed there is a way to manually specify a name for certain well-known addresses.

[saneery]

@goodsoft

From what I see, you make 2 + 9 * N (where N is total count of pools) of eth_call requests once on the launch, and then after every new block import. Method calls can and should be batched together to reduce request overhead.

Are there examples of how to batch?

Also, I'm not sure of how POSDAO works, but is it really necessary to query each and every existing pool after each and every incoming block? Is it maybe possible to gather a list of updated pools from
block content?

No, from block content only able to get few events but they aren't enough. But as I know in the future will be one contract method for getting full data about a pool by one call.

[goodsoft]

Are there examples of how to batch?

Currently only token balance requests get batched, see apps/explorer/lib/explorer/token/balance_reader.ex. Ideally, all other smart contract queries should be done in the same way, if they happen often enough.

The basic idea is to use Explorer.SmartContract.Reader.query_contracts instead of query_contract.

No, from block content only able to get few events but they aren't enough

Sad. Even more reason to batch requests as much as possible.

[saneery]

@goodsoft I've added batching requests. Could you look at this?

[goodsoft]

It would be ideal to batch queries even more, i.e. include multiple pool queries in one requests.
But it's already 3-4 times less requests than initially.

@vbaranov we'll need to compare app server and node performance before and after deploying this PR. If the amount of requests will cause too much of a burden, we'll need to improve batching.

[vbaranov]

@vbaranov we'll need to compare app server and node performance before and after deploying this PR. If the amount of requests will cause too much of a burden, we'll need to improve batching.

@goodsoft I suppose it shouldn't affect any instance (except POSDAO) as pools fetcher will be switched off by default. Meanwhile, sure, I will check performance on POSDAO instance when we will be ready to deploy it.