option_simple_close (features 60/61)

[rustyrussell]

This is a "can't fail!" close protocol, as discussed at the NY Summit, and on @Roasbeef's wishlist. It's about as simple as I could make it: the only complexity comes from allowing each side to indicate whether they want to omit their own output.

It's "taproot ready"(TM) in the sense that shutdown is always sent to trigger it, so that can contain the nonces without any persistence requirement.

I split it into three commits for cleanliness:

1. Introduce the new protocol
2. Remove the requirement that shutdown not be sent multiple times (which was already nonsensical)
3. Remove the older protocols

I recommend reviewing it as separate commits, it'll make more sense!

[ProofOfKeags]

Perhaps this is a misunderstanding on my part for how the document is organized, but wouldn't you still have to support the original closing_signed message in the main BOLT docs here? I understand that maybe you can enforce the new simple close protocol gets used if the channel was opened or upgraded to option_simple_close but the way I'm reading this is that it is deleting the old close method from the protocol completely and I'm pretty sure we can't do that unless this is an extension BOLT right?

Otherwise I tend to like this. The game that has been identified where the closer of the channel pays the fee has the side effect of keeping channels open longer which I believe to be a good thing.

ACK up to aa6deb8

[rustyrussell]

There was an expressed desire in NY to more agressively clean up deprecated parts of the spec, even if they're still in use. It keeps the document cleaner, and avoid having to revisit later (when?) to remove the older parts. However, it's still in the git history.

[ProofOfKeags]

Yeah I understand the desire to remove deprecated stuff. I just wonder what a "valid" implementation looks like here. If we implement this change, and essentially drop support for the old closing scheme from the spec, doesn't that mean that option_simple_close isn't really an option for a new impl? If it is an option but we remove the old closing scheme, doesn't that mean that there is no way to do a coop close anymore?

Maybe this is more of a meta-question of who the main BOLTs are for. My naive (I'm new here!) perspective is that the BOLTs should be the document I should consult if I wanted to bootstrap a new LN impl and that as long as I implement everything in the main BOLTs without any of the optional stuff or extensions then I should be broadly compatible with the network. Through this lense, it would seem that this change would result in the removal of cooperative closes from the main specification. Is my understanding of the purpose of the spec wrong? If not, are we OK with removing coop closes from the table-stakes features of the lightning network?

[TheBlueMatt]

I'm not really sure who would be served by keeping the old stuff around - if someone is using the BOLTs to implement a fresh Lightning client, by the time they're done doing so (which probably takes a year!) hopefully everyone supports the new features and there's no need for them to even be aware the old crap was there. If the BOLTs exist purely for the engineers working on the existing implementations, we already have all the old code, making us read the old stuff while implementing the new stuff is a lot more confusing than just writing out the new stuff IME. I'm not sure who else the BOLTs "should be for".

[t-bast]

Huge concept ACK, I'll really enjoy removing the old-style mutual close from eclair 😁

[vincenzopalazzo]

LGTM.

My two cents on what we should keep inside the BOLTs.

I'd like to suggest cleaning up the feature that is somewhat outdated because having an outdated ways to do the same operation in the BOLTs can make it more challenging to read and understand what is the recent version.

If someone is writing a lightning node from scratch (and I promise that I will do it someday), they should start reading from the current state of the BOLT.

Of course, there is the concern of what happens if the BOLTs change while I am implementing the node? I think we all assume that an implementor will stay updated on the development of the spec (at least we hope).

[whitslack]

This discussion of the game theory hints at an issue I've always had with the "opener pays" rule. I do understand the pathological scenario that the rule is meant to avoid (where a malicious node can drain the funds of a victim node by repeatedly opening and abandoning channels to it), but this new proposal creates a perverse incentive.

Olav opens a channel to Kristján. Some time later, Kristján wants the channel closed. Under this new proposal, Kristján's fee-optimal channel-closing strategy would be to transmit a bogus error message to Olav so that Olav will be forced (by protocol convention) to unilaterally close the channel. Thusly, Kristján would obtain an immediately spendable output and would avoid paying any fees.

[rustyrussell]

That's always true.

But K can also mutual close at <1 sat per byte.

[whitslack]

How can any mutual closing transaction use <1 sat/vB fee rate? The transaction would not propagate.

And you're right that it's always true. It has always irked me. I swear I have seen some cases where nodes to which I have opened channels have sent me error messages for no reason other than to get me to unilaterally close. I think it was a design mistake to have commitment transactions reward the non-closing party with an immediately spendable output. I think commitment transactions should time-lock both outputs to discourage baiting a peer into unilaterally closing.

[t-bast]

That will disappear once we have package relay, because at that point commitment transactions will pay no fees. The fees will have to be paid using CPFP on an ephemeral anchor. Also, if you send bogus errors, nodes will start blacklisting you at some point and won't accept new channels from you: this isn't the case today, but will definitely be in the future.

[RocketNodeLN]

The opener should at least be on the hook for whatever the commitment fee is. Otherwise, as the non-opener, it's cheaper to FC and wait out the time lock. Also, if there's nothing on the non-opener side, is FC the only options since there's nothing on that side to pay the fee.

[ProofOfKeags]

I want to challenge this idea of "it's cheaper to FC and wait out the time lock". Having money locked up carries opportunity cost of the entire output. Yes you may save in fees, but your outputs are tied up in chain resolution and that is a cost in its own right.

Also, if there's nothing on the non-opener side, is FC the only options since there's nothing on that side to pay the fee.

Why would a node be concerned about aggressively closing a channel that has no capital tied up in it?

[RocketNodeLN]

Current coop close is ~168 vB (2 x P2WPKH outputs) to ~192 vB (2 x P2TR outputs). Sweeping the timelock is 121-123 vB.. so ~35-50% smaller tx. I agree with the time cost, but if you are closing due to months of inactivity, what's another up to two weeks. But the opener would be doubly screwed since they would pay the FC (+ anchors) and then have to pay to sweep. I supposed the opener can "deter" this by keeping max commit fees low.

Closing channels should be discouraged, but having dead/inactive channel is worse.

In my mind, game theory would have me coop close at a very low fee if the balance is mostly on the other side or FC if it's mostly on my side and I'm not the opener and commit fee is high enough.

Why would a node be concerned about aggressively closing a channel that has no capital tied up in it?

More of a hypothetical.. how does a closer pay if there's little to nothing on their side.

I agree a change to channel closing is needed, but 100% opener to 100% closer is a pretty drastic change. I would like to see the opener using (some of) the commit fee already reserved and/or splitting the fee based on balance.

[morehouse]

Concept ACK

[morehouse]

This doesn't seem like the typical use case where one side initiates the close and the other signs. Some explanation somewhere is needed:

• Is the idea here that both sides can contribute to the closing fee if they want?
• Does B's closing_complete override A's? And in that case, why does B send closing_sig?
• Did both A and B send closing_complete at the same time, so we have 2 flows happening at once?

[t-bast]

There is one flow happening in each direction, where the sender of closing_complete is paying the fee.
Let me know if the proposed diagram here makes it easier to reason about.

[morehouse]

There is one flow happening in each direction, where the sender of closing_complete is paying the fee.

Do we really expect dual flows to happen like that in practice? If the other party offers to pay for the closing transaction, why would we want to send closing_complete at all?

Let me know if the proposed diagram here makes it easier to reason about.

Your diagram helps to clarify corner cases but probably still needs some high-level explanation in the spec.

[t-bast]

Do we really expect dual flows to happen like that in practice?

I think we do, because each side should independently decide what fee they'd like to pay and immediately send closing_complete (at least that's how I implemented it). Since the channel has already stopped relaying HTLCs at that point, both peers are likely eager to get their funds back asap to reinvest them into another channel.

[morehouse]

We could also mention RBF of the closing transaction here.

[ProofOfKeags]

Also I fail to understand how we could ever have a transaction be urgent enough to push for confirmation if it involved burning all of the outputs in an OP_RETURN.

[t-bast]

Done in 8cfc619

[rustyrussell]

Agreed at meeting that we should simply insist the larger output (or lower key as tiebreak) MUST have an output. I guess if both are dust we cannot close, then.

[ajtowns]

Agreed at meeting that we should simply insist the larger output (or lower key as tiebreak) MUST have an output. I guess if both are dust we cannot close, then.

You could still do this in the implementation potentially -- allow OP_RETURN spks in shutdown(), require setting has_closee_output to false in closing_sig if you chose an OP_RETURN spk (and allow signature_with_closee_output to be invalid?). If you choose OP_RETURN and the other doesn't, wait for them to initiate the close; if you both choose OP_RETURN, wait a random amount of time, and send closing_complete with fee_satoshis equal to your entire balance.

(If you did this, could you move the "has_closee_output" decision to be whether or not the closee set the sPK to OP_RETURN[something]? If you change your mind, you can always just send shutdown again, aiui? Must set the sPK to OP_RETURN if balance is dust or below?)

In general, though, for a sufficiently small channel, you can't guarantee closure: a 100k sat channel closed to a single bare OP_RETURN via a musig2 signature would give you a fee rate of about 1300sat/vb; if miners have a backlog of txs above that fee rate (ie, making >13BTC per block in fees), your close tx won't confirm.

[rustyrussell]

EDIT: This doesn't work well, since you don't know fee yet, you don't know whether you should discard: IGNORE

Maybe this whole situation is simpler if we move "do I want an output?" negotiation to shutdown. This means we allow two new forms of scriptpubkey:

1. An empty scriptpubkey to indicate "I don't want my output at all".
2. OP_RETURN <6+ bytes of padding>.

And now, you SHOULD only set an empty scriptpubkey if you have the lesser output (if both are equal, neither can). Note that the initial shutdown can happen while htlcs are still unsettled, but this will work itself out if you re-xmit shutdown afterwards, and have a rule not to send closing_complete if both sides opt for no output.

[rustyrussell]

OK, this time for sure!

We only let one side (the "lesser in msats", if any) discard their output, so we always have one. We make the signature fields a tlv, so we can simply share them across the two messages, and carefully dictate which one to use.

[ajtowns]

(Avoid @'s in commit msgs to reduce github notification spam? cf https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md )

[Roasbeef]

So I'm finally ready for interop here @t-bast!

One thing that came up during my last debugging session was the fact that we allow the Sequence field to now be set to anything by either side.

I'm not sure how y'all handle this in other implementations, but in lnd we use the sequence to decide if a funding output spend is either a: breach, local force close, remote force close, coop close, or an unknown state (SCB flow).

Before this new flow, the sequence was always set to the max possible sequence, which also disabled the RBF (which was want to enable) and also sequence locks semantics. However, if we allow a peer to set an arbitrary sequence, then they can set a sequence that looks like a past breached state (due to extraction of the state hint) or even a future unknown state. So I ended up modifying our implementation to only set/allow a sequence of mempool.MaxRBFSequence (0xfffffffd). This is the largest sequence that still allows opting into the RBF rules. With this change, we're now able to deterministically detect type of spend we see on chain based on the sequence.

IMO, there's no good reason to set your sequence to anything other than 0xfffffffd for the co-op close transaction. Activating the BIP 112 semantics isn't very useful here as why would you want to delay your co-op close with a relative time lock? If we allow peers to set arbitrary values, then this also weakens the anonymity set of a coop close from a taproot channel as you'll stand out if you use anything other than 0xfffffffd.

So what do y'all think about dropping the sequence field from the new messages? We'd then always restrict it to be 0xfffffffd, rejecting a sig if we get any other value.

[t-bast]

So I'm finally ready for interop here @t-bast!

🥳

I'm not sure how y'all handle this in other implementations, but in lnd we use the sequence to decide if a funding output spend is either a: breach, local force close, remote force close, coop close, or an unknown state (SCB flow).

We don't rely too much on the nSequence field for that in eclair. When a transaction spends the funding transaction, we match it by txid against:

• splice transactions we signed
• the current local or remote commit
• the next remote commit if we sent commit_sig and didn't receive revoke_and_ack
• any mutual close transaction we signed
• otherwise, we check if it's a revoked commit by checking nSequence and nLockTime and the revocation secrets we have
• if that doesn't match, it's an unknown state (we lost data)

You always sign mutual close transactions, so you can remember the corresponding txids and shouldn't need to rely on the nSequence field for that?

With this change, we're now able to deterministically detect type of spend we see on chain based on the sequence.

I think this is dangerous to rely on nSequence only for that, it's much safer to rely on txids IMO.

IMO, there's no good reason to set your sequence to anything other than 0xfffffffd for the co-op close transaction.

Why restrict it? There are a lot of discussions of leveraging nSequence for future bitcoin capabilities, so we should keep that door open and allow anything IMO. This is the mutual close transaction paid for by our peer, so we should let them use whatever they want?

If we allow peers to set arbitrary values, then this also weakens the anonymity set of a coop close from a taproot channel

I don't think this argument really holds: if your peer wants to do something funny that stands out, they have many other ways to do this anyway, restricting the nSequence of the mutual close tx is not going to fix that.

[Roasbeef]

I think this is dangerous to rely on nSequence only for that, it's much safer to rely on txids IMO.

We don't only rely on it, but you might not always have the txid (eg: restore from seed, no state, partial data loss, etc). The sequence effectively serves as an authenticated sequence number since only in collaboration with the other party can a new one be signed. All implementations also must use the sequence to detect which date was broadcast assuming they aren't storing the txid of every commitment txn ever signed.

In the past, we also based things purely on txid, but discovered some edge cases, causing us to re-write that logic entirely all together.

we match it by txid against any mutual close transaction we signed
You always sign mutual close transactions, so you can remember the corresponding txids and shouldn't need to rely on the nSequence field for that?

How would you then handle the case of partial or complete data loss, but before that you signed a co-op close transaction, and that hit the chain?

With the new protocol, do y'all also store very RBF'd close created by either party? Right now we only remember the very last one, as with the new flow, the other party can just continue to sign effectively useless updates to make the peer retain more state (soft DoS).

Why restrict it? There are a lot of discussions of leveraging nSequence for future bitcoin capabilities, so we should keep that door open and allow anything IMO. This is the mutual close transaction paid for by our peer, so we should let them use whatever they want?

Can think of two reasons not to:

1. You're spending the funding output for a co-op close, meaning you want to exit the channel, so why use a relative time lock?
2. If every implementation picks its own value, then it's easier to finger print. Today everyone uses the max sequence for co-op close. In a post taproot chans everywhere world, using a standard value can help co-op closes blend in with the broader transaction anonymity set.
3. One less free parameter.

Any future protocol updates that repurpose the sequence for consensus rules would need to bump the transaction version, just like we did originally for the CSV package. If we follow the line of reasoning for allowing a custom sequence, then should we also allow a custom txn version?

[t-bast]

How would you then handle the case of partial or complete data loss, but before that you signed a co-op close transaction, and that hit the chain?

It simply falls into the "unknown transaction" case, where you cannot do anything and just let it confirm (which sends funds back to your wallet). I believe there is nothing to do in that case and it is harmless?

With the new protocol, do y'all also store very RBF'd close created by either party?

Yes, we store the txid of every closing candidate, as it's quite useful for monitoring (it's also a good way to see how well RBF works in practice).

Right now we only remember the very last one, as with the new flow, the other party can just continue to sign effectively useless updates to make the peer retain more state (soft DoS).

Since they have to increase the fees every time, that doesn't sound like a dangerous DoS (especially since you only need to store 32 bytes per attempt).

Can think of two reasons not to:

Those are reasonable reasons, I don't feel very strongly either way (especially since right now, we'll always set the sequence to 0xfffffffd). Let's discuss that during today's spec meeting!

[t-bast]

This was discussed during yesterday's spec meeting, and there was rough consensus that the sequence should always be set to 0xfffffffd. Then I believe we should update the closing_signed message to be:

1. type: 40 (`closing_complete`)
2. data:
   * [`channel_id`:`channel_id`]
   * [`u64`:`fee_satoshis`]
   * [`u32`:`locktime`]

Since we want the ability to set the nLockTime for anti-fee sniping. We can add the ability to set a different nSequence in the future with a new TLV and a feature bit, if it becomes necessary.

@rustyrussell @Roasbeef does that sound good to you?

[rustyrussell]

This was discussed during yesterday's spec meeting, and there was rough consensus that the sequence should always be set to 0xfffffffd. Then I believe we should update the closing_signed message to be:

1. type: 40 (`closing_complete`)
2. data:
   * [`channel_id`:`channel_id`]
   * [`u64`:`fee_satoshis`]
   * [`u32`:`locktime`]

Since we want the ability to set the nLockTime for anti-fee sniping. We can add the ability to set a different nSequence in the future with a new TLV and a feature bit, if it becomes necessary.

@rustyrussell @Roasbeef does that sound good to you?

Yes, this looks good...

[Crypt-iQ]

so you can negotiate again (even changing the output address) if you want

doesn't this override the safety feature of upfront shutdown?

[t-bast]

This is of course if you don't have upfront_shutdown activated, we can make that clearer.

[t-bast]

Done in 8cfc619

[t-bast]

@Roasbeef whether shutdown always needs to be re-sent in response when receiving shutdown was explored in #1096 (comment)

I think the behavior described in that comment made sense: when you receive a new shutdown, you aren't forced to send another shutdown if you don't want to change anything about your version of the transaction, because you won't be sending closing_complete to update your version of the transaction. It would be nice to update your version of the closing transactions if your peer changes their output address, but if they're only changing the feerate then there's nothing to change on the local transactions.

However, I didn't consider taproot channels at all at that point: I'm wondering if the stricter requirement of renewing both local and remote shutdown messages that you implemented was to guarantee that we correctly exchange fresh nonces to sign the musig2 channel output? If that's the case, it's a compelling reason to be strict here because we want this protocol to work for taproot channels. Have you specified somewhere how nonces are exchanged with option_simple_close? If we require shutdown to be sent and received before issuing new signatures, I think it should be:

• include a fresh nonce in shutdown
• when sending closing_complete (signing our versions of the closing txs):
  • create fresh local nonces and include them in closing_complete with our partial sigs
  • use the remote nonce from their latest shutdown message
• when sending closing_sig:
  • use the local nonce from our latest shutdown message
  • use remote nonces from the closing_complete we're responding to
• once we've sent shutdown and are waiting for a remote shutdown:
  • ignore closing_complete (it uses our previous nonce)
  • ignore closing_sig if it doesn't match our last sent closing_complete (race with shutdown)

@rustyrussell there are a lot of pending comments on the PR that make it harder to make progress on testing cross-compatibility: if you don't have time to address those right now, do you want me to prepare a commit on top of your branch that tries to address these comments? This way you only have to review it instead of making the changes yourself.

[t-bast]

Closing in favor of #1205, as @rustyrussell asked me to take over that PR.