'withdraw all' fails when 'utxos' specified; leaves UTxOs unavailable

[whitslack]

Issue and Steps to Reproduce

$ newaddr=$(lightning-cli newaddr | jq -r .bech32)

$ lightning-cli withdraw "${newaddr}" all 3333perkb 0 '["(my txid):(vout)"]'
{
   "code": 301,
   "message": "Cannot afford transaction with 138sat sats of fees"
}

Attempting the same withdrawal a second time yields:

{
   "code": -32602,
   "message": "No matching utxo was found from the wallet. You can get a list of the wallet utxos with the `listfunds` RPC call."
}

Indeed, the specified <txid>:<vout> is no longer returned in lightning-cli listfunds. The UTxO does appear again after a dev-rescan-outputs, but obviously this is risky and shouldn't be needed.

So there are really two problems here:

1. C-Lightning should not be marking UTxOs as spent if it didn't actually spend them.
2. C-Lightning apparently doesn't understand how to spend all of a specified set of UTxOs while deducting enough from their total value to satisfy the specified fee rate.

getinfo output

$ lightning-cli getinfo | jq '{binding, version, blockheight, network, "lightning-dir"}'
{
  "binding": [
    {
      "type": "ipv6",
      "address": "::",
      "port": 9735
    },
    {
      "type": "ipv4",
      "address": "0.0.0.0",
      "port": 9735
    }
  ],
  "version": "0.8.1",
  "blockheight": 621608,
  "network": "bitcoin",
  "lightning-dir": "/var/lib/lightning/bitcoin"
}

[whitslack]

Problem 1 also occurs when attempting to fundchannel with a specified UTxO but the channel funding fails because the auto-connection to the target node failed. Specifically, C-Lightning keeps the UTxO reserved, even though no attempt was actually made to spend it.

[darosior]

I experienced it too with fundchannel. And even funnier things happen when you try to fundchannel_cancel the peer.. Giving a look to withdraw now.

You should not specify all, but we shouldn't crash.

[whitslack]

You should not specify all, but we shouldn't crash.

@darosior: Okay, then what is the recommended way to withdraw (or fund a channel with) exactly a set of UTxOs without producing a change output? The all keyword seems like the most logical choice to me. I expect it to take all coins available in the specified set of UTxOs and send them, minus a specified mining fee.

[darosior]

Hmm all specifies to spend all the available funds from the wallet.
But you are right that it would be nice to be able to condition it to the specified utxos..

EDIT: to answer the question clearly, you should set the amount explicitly if you provide a list of utxos.

[whitslack]

to answer the question clearly, you should set the amount explicitly if you provide a list of utxos.

But I don't know what the fee will be. If I specify the whole sum of the UTxOs, then there won't be anything left for the fee, and if I specify too little a sum, then I'll inadvertently generate a change output, which is what I'm trying to avoid.

[darosior]

Yes, that's the well-known problem of fees computation, that's why it's better automated :-)
Though note that we won't create a dust output, which leaves some room for approximation.

[darosior]

@whitslack ok I tried to reproduce and I think I understood the root cause : were you using unconfirmed utxos ?

EDIT: btw I was badly wrong, all is handled along with utxos

[whitslack]

were you using unconfirmed utxos ?

@darosior: No. I was specifying exactly one confirmed UTxO, and I was specifying an explicit feerate using the perkb suffix. I have some tiny, confirmed UTxOs that I am willing to try withdrawing from my C-Lightning wallet if you'd like me to do any tests. Everytime I've tried before, using the all keyword, I get the Cannot afford transaction with ___sat sats of fees error. (Side note: That error message says "sat" twice.)

[darosior]

Weird.. Then I cannot reproduce on current master. Cf #3593 for the unconfirmed case..

[whitslack]

Maybe it's already been fixed since the 0.8.1 release. I'll retest after the next release and update here with my findings.

[niftynei]

@whitslack has this been fixed?

[whitslack]

@niftynei: I don't know. There hasn't been a new release since 0.8.1. I'll update this issue again after there's been a new release and I've retested.

[niftynei]

Oh I see, apologies I didn't quite grok that you were waiting til the next release to test :)

[ZmnSCPxj]

We already handle this correctly, as these tests show, and these tests are in released 0.8.2.1:

lightning/tests/test_wallet.py

Line 172 in 7a7b154

l1.rpc.withdraw(waddr, "all", minconf=0, utxos=uutxos)

lightning/tests/test_connection.py

Line 872 in 7a7b154

l1.rpc.fundchannel(l2.info["id"], "all", utxos=utxos[0:2])

@whitslack please confirm if ok to close.

[whitslack]

I just tested this again on 0.8.2.1 and got a different failure:

{
   "code": -1,
   "message": "Error broadcasting transaction: error code: -26\\nerror message:\\nmin relay fee not met, 109 < 110 (code 66). Unsent tx discarded"
}

I was withdrawing with feerate=1000perkb, which should be the minimum-allowed fee rate. Indeed, bitcoin-cli getmempoolinfo reports:

{
  "loaded": true,
  "size": 15097,
  "bytes": 9207924,
  "usage": 32831760,
  "maxmempool": 64000000,
  "mempoolminfee": 0.00001000,
  "minrelaytxfee": 0.00001000
}

So it looks like there's somewhere that C-Lightning is rounding down where it ought to be rounding up.

On the plus side, the UTxO that I tried to spend is still listed in listfunds after the failure.

[darosior]

FWIW, you can use 253perkw for the moment. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ Le mardi, juin 9, 2020 7:36 PM, Matt Whitlock <notifications@github.com> a écrit :

…

I just tested this again on 0.8.2.1 and got a different failure: { "code": -1, "message": "Error broadcasting transaction: error code: -26\\nerror message:\\nmin relay fee not met, 109 < 110 (code 66). Unsent tx discarded" } I was withdrawing with feerate=1000perkb, which should be the minimum-allowed fee rate. Indeed, bitcoin-cli getmempoolinfo reports: { "loaded": true, "size": 15097, "bytes": 9207924, "usage": 32831760, "maxmempool": 64000000, "mempoolminfee": 0.00001000, "minrelaytxfee": 0.00001000 } So it looks like there's somewhere that C-Lightning is rounding down where it ought to be rounding up. On the plus side, the UTxO that I tried to spend is still listed in listfunds after the failure. — You are receiving this because you were mentioned. Reply to this email directly, [view it on GitHub](#3591 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AFLK3F7OBGXNX4CRGPQHI5DRVZXIBANCNFSM4LI5KLKA).

[whitslack]

253perkw

That worked. And my withdraw spent only the UTxO that I asked it to, and it spent the whole thing without creating a change output. 👍

[darosior]

I'll give the rounding error a look (greping logs on my nodes currently to check it doesn't happen on other code path.. Which might be worrying).

[ZmnSCPxj]

1000perkb == 250perkw. It may have to do with us rounding down consistently as in C, and Bitcoin rounding up consistently because they have magic C++ code to do that or something (or maybe they just consistently round up from sheer code review). In practice we should probably use 253perkw as a minimum and also check that perkb units are greater than 253perkw when converted.

[whitslack]

In practice we should probably use 253perkw as a minimum

Wouldn't that just kick the can? Then when getmempoolinfo says "mempoolminfee": 0.00001500, an attempt to send with feerate=1500perkb may fail.

[ZmnSCPxj]

Hmm, looks like an issue then. Suggest closing this one and opening another?

[ZmnSCPxj]

Filed #3771, closing this.