Sector index lookup metrics - Testing & Callouts #10188
Replies: 14 comments 4 replies
-
|
If this gets into a proper DB its going to make a massive difference.... I've already built my own external DB around it to work around this issue :) I'm probably towards the 15 minutes for running 1 command. Will post once its done here.
|
Beta Was this translation helpful? Give feedback.
-
|
proper here means: not sqlite ;) |
Beta Was this translation helpful? Give feedback.
-
|
Date: 06.02.2023 Sectors output from lotus-miner info: Sectors:
Command: Time to display output (completely finished): |
Beta Was this translation helpful? Give feedback.
-
|
Date: 06.02.2023 Sectors output from lotus-miner info:
|
Beta Was this translation helpful? Give feedback.
-
|
Sector#Count: 1480 Sector#Count: 233499 |
Beta Was this translation helpful? Give feedback.
-
|
Date: 06.02.2023 Sectors output from lotus-miner info:
I personally like my 3 sectors with the "Faulty" status... Very unique 🥇 |
Beta Was this translation helpful? Give feedback.
-
|
On Calib-miner:
|
Beta Was this translation helpful? Give feedback.
-
|
I'm running a small amount of sectors and only CC for now. Using storage-workers, so lotus-miner cannot access the storage directly. Maybe this is why the results are not showing improvements. |
Beta Was this translation helpful? Give feedback.
-
|
Date: 06.02.2023 Sectors output from
I'm not sure why there's this Command: Time to display output (completely finished):
|
Beta Was this translation helpful? Give feedback.
-
|
I recommend mongodb , easy to embed |
Beta Was this translation helpful? Give feedback.
-
|
Make the user unconscious and does not need to maintain. |
Beta Was this translation helpful? Give feedback.
-
|
Date: 07.02.2023 Sectors output from lotus-miner info: Command: Time to display output (completely finished): |
Beta Was this translation helpful? Give feedback.
-
|
Date: 07.02.2023 (DSS 7) Sectors output from lotus-miner info:
|
Beta Was this translation helpful? Give feedback.
-
|
Sectors:
|
Beta Was this translation helpful? Give feedback.
-
|
Sectors: Command= Avg time to display output: |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
The coming v1.21.0 release includes parallization to the sector list which should speed up the Some initial ideas we have: Fetching just the last xx-amount of sectors/get the sector list for given range/last XX sectors list state changes. (feedback welcome, post in this 🧵) Now for the results, with the v1.21.0-rc1 release I´m currently seeing:
|
Beta Was this translation helpful? Give feedback.
-
Date: 24.03.2023
|
Beta Was this translation helpful? Give feedback.
-
Hey everyone!
As part of the lotus-miner-v2 roadmap 🗺 we are planning to move the sector index to a robust database compared to todays non-persistent in-memory maps. With the move you will be able to shut down the lotus-miner process for upgrades/maintenance while keeping the ability to prove your sectors (windowPoSt) and win block rewards (winningPoSt) as long as you have setup windowPoSt- and winningPoSt-workers.
Moving the sector index to another DB should also speed up sector related commands. ⚡️ To measure this effort we are looking to capture the current timings of these commands. ⏱ We encourage storage providers of all sizes to add their current timings, so we get a clearer picture of how long these commands currently take:
Example reply card:
Date: 06.02.2023
Sectors output from lotus-miner info:
Lotus-miner sectors listLotus-miner sectors list —fastLotus-miner storage list sectorsAnd in about a weeks time or so - do the same exercise to see the time difference with the given sector growth/fall you had in that timeframe:
Date: xx.02.2023
Sectors output from lotus-miner info:
Lotus-miner sectors listLotus-miner sectors list —fastLotus-miner storage list sectorsBeta Was this translation helpful? Give feedback.
All reactions