update to markets v1.2.x

[dirkmc]

Fixes #5133
Fixes #5443

Note: This version of markets still contains several known issues with retrieval, that will be fixed in a subsequent release (For example if the connection goes down during a retrieval, the deal will not recover).

Major improvements:

• Detect when client fails to get an Accept or Complete message from miner during storage deal
• Better error reporting when there's an issue with unsealing / reading data from the blockstore
• Don't double count the amount of transferred data when there's a restart during the transfer

For a detailed list of changes, see the changelog in go-fil-markets (from v1.1.9 - v1.2.3):
https://github.com/filecoin-project/go-fil-markets/blob/master/CHANGELOG.md

Tasks:

• Run testground test that makes a deal between lotus client / provider of master / this branch
• Run manual storage and retrieval deal
  • this branch client -> v1.5.3 miner
    • Storage deal
    • Retrieval deal
  • this branch client -> this branch miner
    • Storage deal
    • Storage deal with miner restart
    • Retrieval deal
• Merge better logging when unsealing fails #5851 (nice to have)
• Merge add lotus client cancel-retrieval cmd to lotus CLI #5871 (nice to have)

[dirkmc]

Testground tests

Client (master) <-> Provider (feat/markets-v1.2.0)

https://ci.testground.ipfs.team/logs?task_id=c1ca6od5p7aahco59v2g

$ LOTUS_VERSION_CLIENT=master LOTUS_VERSION_MINER=d48dc4f testground run composition -f ./_compositions/baseline-k8s-1-1-versions.toml --metadata-commit=d48dc4f --metadata-repo=filecoin-project/lotus --metadata-branch=feat/markets-v1.2.0

$ kubectl exec -it tg-lotus-soup-c1ca6od5p7aahco59v2g-clients-0 sh
# cat testground_dep_list  | grep lotus
github.com/filecoin-project/lotus/testplans/lotus-soup
github.com/filecoin-project/lotus v1.5.2 => github.com/filecoin-project/lotus v1.5.3-rc1.0.20210319182246-43d9cc36a4ea

$ kubectl exec -it tg-lotus-soup-c1ca6od5p7aahco59v2g-miners-0 sh
# cat testground_dep_list  | grep lotus
github.com/filecoin-project/lotus/testplans/lotus-soup
github.com/filecoin-project/lotus v1.5.2 => github.com/filecoin-project/lotus v1.5.3-rc1.0.20210322110805-d48dc4fc916e

Client (feat/markets-v1.2.0) <-> Provider (master)

https://ci.testground.ipfs.team/logs?task_id=c1cabnt5p7aahco59v30

$ LOTUS_VERSION_CLIENT=d48dc4f LOTUS_VERSION_MINER=master testground run composition -f ./_compositions/baseline-k8s-1-1-versions.toml --metadata-commit=d48dc4f --metadata-repo=filecoin-project/lotus --metadata-branch=feat/markets-v1.2.0


$ kubectl exec -it tg-lotus-soup-c1cabnt5p7aahco59v30-clients-0  sh
# cat testground_dep_list  | grep lotus
github.com/filecoin-project/lotus/testplans/lotus-soup
github.com/filecoin-project/lotus v1.5.2 => github.com/filecoin-project/lotus v1.5.3-rc1.0.20210322110805-d48dc4fc916e

$ kubectl exec -it tg-lotus-soup-c1cabnt5p7aahco59v30-miners-0  sh
# cat testground_dep_list  | grep lotus
github.com/filecoin-project/lotus/testplans/lotus-soup
github.com/filecoin-project/lotus v1.5.2 => github.com/filecoin-project/lotus v1.5.3-rc1.0.20210319182246-43d9cc36a4ea

[dirkmc]

Manual testing has revealed a couple of issues:

• push channel monitor doesn't stop transfer completes go-data-transfer#168
  Update: Fixed and tested
• Add mechanism to start push / pull channel data rate monitoring  go-data-transfer#172
  A bigger fix is needed. Suggest we disable data-rate monitoring for retrievals until the next release, as there are already known issues with retrieval accounting that need to be addressed, so restarts while retrieving data will not work yet anyway: Voucher processing go-data-transfer#151
  Update: disabled data-rate monitoring for retrieval for the moment: add config options to enable / disable push or pull monitoring individually go-data-transfer#174

[dirkmc]

It seems that when the client needs to create a payment channel for retrieval, as soon as the payment channel lane is allocated the deal gets stuck in the Ongoing state. Our theory is that the connection to the provider goes down while the provider is waiting for the client to create the payment channel.
We don't believe this is a regression but we would like to do some more testing to be sure.

[dirkmc]

Turns out it is a regression: filecoin-project/go-fil-markets#517

[dirkmc]

After updating to markets v1.2.3 the regression is fixed:

lotus client retrieve --miner f01234 bafykbzacedbytx65vf2n2daoallyvtrc52pguqvpmofgehncs6p5tk2qbg7pa myfile                                                                           
> Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew)                                                                                                                                                 
> Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptance)                                                                                                                           
> Recv: 0 B, Paid 0 FIL, ClientEventDealAccepted (DealStatusAccepted)                                                                                                                                    
> Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelAddingFunds (DealStatusPaymentChannelAddingInitialFunds)                                                                                               
> Recv: 29.72 KiB, Paid 0 FIL, ClientEventBlocksReceived (DealStatusPaymentChannelAddingInitialFunds)                                                                                                    
> Recv: 29.72 KiB, Paid 0 FIL, ClientEventPaymentRequested (DealStatusPaymentChannelAddingInitialFunds)                                                                                                  
> Recv: 1.029 MiB, Paid 0 FIL, ClientEventBlocksReceived (DealStatusPaymentChannelAddingInitialFunds)                                                                                                    
> Recv: 1.029 MiB, Paid 0 FIL, ClientEventPaymentChannelReady (DealStatusPaymentChannelAllocatingLane)                                                                                                   
> Recv: 1.029 MiB, Paid 0 FIL, ClientEventLaneAllocated (DealStatusOngoing)                                                                                                                              
> Recv: 1.029 MiB, Paid 0 FIL, ClientEventPaymentRequested (DealStatusFundsNeeded)                                                                                                                       
> Recv: 1.029 MiB, Paid 0 FIL, ClientEventSendFunds (DealStatusSendFunds)                                                                                                                                
> Recv: 1.029 MiB, Paid 0.000000000002158014 FIL, ClientEventPaymentSent (DealStatusOngoing)                                                                                                             
> Recv: 2.029 MiB, Paid 0.000000000002158014 FIL, ClientEventBlocksReceived (DealStatusOngoing)                                                                                                          
> Recv: 2.029 MiB, Paid 0.000000000002158014 FIL, ClientEventPaymentRequested (DealStatusFundsNeeded)                                                                                                    
> Recv: 3.029 MiB, Paid 0.000000000002158014 FIL, ClientEventBlocksReceived (DealStatusFundsNeeded)                                                                                                      
> Recv: 3.029 MiB, Paid 0.000000000002158014 FIL, ClientEventSendFunds (DealStatusSendFunds)                                                                                                             
> Recv: 3.029 MiB, Paid 0.000000000006352318 FIL, ClientEventPaymentSent (DealStatusOngoing)                                                                                                             
> Recv: 4.029 MiB, Paid 0.000000000006352318 FIL, ClientEventBlocksReceived (DealStatusOngoing)                                                                                                          
...
> Recv: 583 MiB, Paid 0.000000001176563134 FIL, ClientEventBlocksReceived (DealStatusOngoing)
> Recv: 584 MiB, Paid 0.000000001176563134 FIL, ClientEventBlocksReceived (DealStatusOngoing)
> Recv: 584 MiB, Paid 0.000000001176563134 FIL, ClientEventLastPaymentRequested (DealStatusFundsNeededLastPayment)
> Recv: 584.6 MiB, Paid 0.000000001176563134 FIL, ClientEventBlocksReceived (DealStatusFundsNeededLastPayment)
> Recv: 584.6 MiB, Paid 0.000000001176563134 FIL, ClientEventAllBlocksReceived (DealStatusSendFundsLastPayment)
> Recv: 584.6 MiB, Paid 0.000000001226060862 FIL, ClientEventPaymentSent (DealStatusFinalizing)
> Recv: 584.6 MiB, Paid 0.000000001226060862 FIL, ClientEventComplete (DealStatusCompleted)
Success

[dirkmc]

I re-ran the testground tests against the latest commit to verify compatibliity between this branch and lotus master:

Client (master) <-> Provider (feat/markets-v1.2.0)

https://ci.testground.ipfs.team/logs?task_id=c1ea36t5p7aahco5a0b0

$ LOTUS_VERSION_CLIENT=master LOTUS_VERSION_MINER=0134099 testground run composition -f ./_compositions/baseline-k8s-1-1-versions.toml --metadata-commit=0134099 --metadata-repo=filecoin-project/lotus --metadata-branch=feat/markets-v1.2.0

Client (feat/markets-v1.2.0) <-> Provider (master)

https://ci.testground.ipfs.team/logs?task_id=c1ea3255p7aahco5a0ag

LOTUS_VERSION_CLIENT=0134099 LOTUS_VERSION_MINER=master testground run composition -f ./_compositions/baseline-k8s-1-1-versions.toml --metadata-commit=0134099 --metadata-repo=filecoin-project/lotus --metadata-branch=feat/markets-v1.2.0

[raulk]

This has been squashed into https://github.com/filecoin-project/lotus/tree/minerx/staging for minerX early testing.