sweep: prepare for sweeper #1978

joostjager · 2018-09-26T16:23:31Z

This PR makes changes that prepare for the new Sweeper struct.

Create new subsystem for sweeping
Move sweep tx generation code to sweep subsystem.
Clean up SpendableOutput and CsvSpendableOutput structs.
Make Cltv expiry part of a sweep input.
Remove duplicated sweep tx generation code from Success and Commit resolvers

A follow PR is #1960, which builds the sweeper around the sweep tx generation and uses that from utxonursery.

Roasbeef

Solid PR! The commit structure makes the set of changes very easy to review. The only comment re the commits, is that we typically like to add additional context to the commits themselves using the message body provided. We find that this helps to make the git history more detailed as each commit has a small body that explains the changes/rationale within it, and may also extend to the commits proceeding it as well (assuming we always work to retain commit order).

Roasbeef · 2018-10-03T01:09:57Z

breacharbiter.go

@@ -849,10 +849,6 @@ func (bo *breachedOutput) BuildWitness(signer lnwallet.Signer, txn *wire.MsgTx,
 	return bo.witnessFunc(txn, hashCache, txinIdx)
 }

-// Add compile-time constraint ensuring breachedOutput implements


What's the rationale behind this change? Seems better from the PoV of unit tests, that we can feed in this interface into various methods in his file. After all, we may eventually extract this into the contractcourt package.

I removed the interface because it was not used anywhere. But indeed, I can also see the breacharbiter use an (its own) instance of the sweeper. Converted to Input types.

Roasbeef · 2018-10-03T01:10:49Z

sweep/sweeper.go

+	Signer lnwallet.Signer
+}
+
+// NewSweeper returns a new Sweeper instance.


Can instead be sweep.New as it's the primary external facing struct within the package (atm).

Changed. Not completely sure if I like it, because it'll be another rename when we found out we need another important external facing struct.

sweep/sweeper.go

Roasbeef · 2018-10-03T01:12:15Z

sweep/sweeper.go

+	cfg *SweeperConfig
+}
+
+// SweeperConfig contains dependencies of Sweeper


Missing a period at the end of the sentence.

Roasbeef · 2018-10-03T01:31:32Z

utxonursery.go

 	// Store provides access to and modification of the persistent state
 	// maintained about the utxo nursery's incubating outputs.
 	Store NurseryStore
+
+	// Sweeper manages the sweeping of unlocked outputs for nursery.


Well at this point it actually just constructs the sweeping transactions.

Roasbeef · 2018-10-03T01:52:07Z

contractcourt/contract_resolvers.go

-			return nil, err
-		}
-
-		log.Debugf("%T(%v): using %v sat/kw for sweep tx", c,


With this change, we've lost the debug logging here. Also just realized that we don't have a way currently to signal to the sweeper what conf timing, or fee rate that we'd like when sweeping.

Moved logging to sweeper.

Added TODO for configurable fee rate.

Before we merge we should be able to specify the fee rate. In distinct areas in the codebase, we use 3 or 6 depending on how quickly we want the transaction to enter the chain. Some cases (CLTV timeouts) are more time sensitive than others.

Roasbeef · 2018-10-03T01:52:20Z

contractcourt/contract_resolvers.go

-				return nil, err
-			}
-
-			log.Debugf("%T(%x): using %v sat/kw to sweep htlc"+


Similar comment here re losing the debug logging.

Roasbeef · 2018-10-03T01:52:58Z

sweep/input.go

-// BaseInput contains all the information needed to sweep a output.
-type BaseInput struct {
+// InputKit contains the base information needed to sweep an output.
+type InputKit struct {


Does this need to be public seeing as it's mostly used as a mix-in struct (via composition) now?

Roasbeef · 2018-10-03T01:53:18Z

sweep/input.go

@@ -72,28 +70,61 @@ func MakeBaseInput(outpoint *wire.OutPoint,
 }

 // Amount returns the number of satoshis contained in the breached output.
-func (bi *BaseInput) Amount() btcutil.Amount {
+func (bi *InputKit) Amount() btcutil.Amount {


I think this might cause vet on Go 1.11 to complain re the name of the method target...

Roasbeef · 2018-10-03T01:54:52Z

sweep/input.go

 	return &bi.signDesc
 }

+// AbsoluteMaturity returns the absolute timelock.


Do the absolute and relative maturity need to be a part of this struct? With the changes in this commit, it now serves more or less as the base class (via composition tho), and most "base" inputs don't require a CSV or CLTV semantics at all. Also if we go with my suggestion to always use non-final sequence locks inputs (or even randomize them as eventually we can make all transactions look like commitment transactions being broadcast) and use the next block height as the lock time value.

So you propose:

Always set tx locktime to current height + 1

Remove AbsoluteMaturity() from the interface completely, as it is no longer needed on the input level

Remove relativeMaturity from BaseInput

Implement BlockToMaturity of BaseInput as always returning 0

Create a new struct CsvInput that allows setting the relative lock time
?

Remove them only on this new baseInput struct. Yes to the rest!

Okay, got it. Didn't create CsvInput yet, as it is not needed in this PR.

joostjager · 2018-10-03T09:12:12Z

Commit message bodies added

joostjager

.

halseth · 2018-10-11T11:02:40Z

sweep/input.go

+
+// SpendableOutput an interface which can be used by the breach arbiter to
+// construct a transaction spending from outputs we control.
+type SpendableOutput interface {


Nice commit structure 👍

breacharbiter.go

server.go

sweep/log.go

halseth · 2018-10-11T11:29:15Z

sweep/input.go

-// MakeBaseInput assembles a new BaseInput that can be used to construct a
-// sweep transaction.
-func MakeBaseInput(outpoint *wire.OutPoint,
+func makeInputKit(outpoint *wire.OutPoint,


nit: can remove makeInputKit and just populate the struct manually where needed?

First I didn't like the suggestion, because I'd need to duplicate the amount assignment in makeInputKit. But then I realized that whole field is redundant. Removed it completely.

breacharbiter.go

halseth · 2018-10-11T11:36:08Z

sweep/sweeper.go

+// spends from them. This method also makes an accurate fee estimate before
+// generating the required witnesses.
+//
+// The value of currentBlockHeight argument will be set as the tx locktime. This


Should the parameter be called locktime instead? Since we are currently not passing the correct block height.

The expected value is currentBlockHeight, otherwise the comments would not be valid. Because we are currently not passing the block height in the contract resolvers, I added todo comments there.

halseth · 2018-10-11T12:10:54Z

sweep/sweeper.go

 		sweepTx.AddTxIn(&wire.TxIn{
 			PreviousOutPoint: *input.OutPoint(),
 			Sequence:         input.BlocksToMaturity(),
 		})
 	}
-	for _, input := range cltvInputs {
-		sweepTx.AddTxIn(&wire.TxIn{
-			PreviousOutPoint: *input.OutPoint(),


I think it is only set FF if NewTxIn is used, so I don't think this will change the existing behavior.

joostjager · 2018-10-11T17:30:33Z

Comments addressed.

sweep/input.go

cfromknecht · 2018-10-16T23:22:55Z

sweep/sweeper.go

+		return nil, err
+	}
+
+	log.Debugf("Using %v sat/kw for sweep tx", int64(feePerKw))


should we consider making this info?

I added a commit that restructures the functions in a imo more logical way and unites the log statements.

This commit moves the output structs to a new package as a preparation for moving more logic into that package.

Sweep txes are currently generated in multiple locations. Moving the sweep tx generation into a shared package will allow us to reuse that logic.

This commit introduces a common interface for sweep inputs. It eliminates the type checking from UtxoSweeper. Also the formerly present Amount() getter is removed. It was redundant because this value is present in SignDesc().Output.Value as well.

Removes duplicate sweep tx code from commit resolver and delegates generation to UtxoSweeper in the sweep package.

joostjager · 2018-10-17T10:46:32Z

Comments addressed

Roasbeef

LGTM 🦎

cfromknecht · 2018-10-17T23:08:35Z

sweep/sweeper.go

+			)
+
+		default:
+			log.Warnf("kindergarten output in nursery store "+


i think this is consistent with the current behavior, but currently we won't add any weight estimate for unknown witness types

i take that back, the prior version would properly filter unknown output types and not include them in the transaction. the new version will include them

cfromknecht · 2018-10-17T23:10:47Z

sweep/sweeper.go

+			weightEstimate.AddWitnessInput(
+				lnwallet.OfferedHtlcSuccessWitnessSize,
+			)
+


missing cltvCount++

ah wait, this is a new type that wasn't included before, nvm

cfromknecht · 2018-10-17T23:12:36Z

sweep/sweeper.go

+	}
+
+	log.Infof("Creating sweep transaction for %v inputs (%v CSV, %v CLTV) "+
+		"using %v sat/kw", len(inputs), csvCount, cltvCount,


first argument should be len(inputs)-csvCount-cltvCount, or len(inputs)-len(csvInputs)-len(cltvOutputs) if getWeightEstimate is modified to return slices of filtered inputs.

cfromknecht · 2018-10-17T23:26:13Z

Added a small commit to restore prior behavior that filters unknown witness types #2062

joostjager · 2018-10-18T05:53:51Z

Oh, good catch. I introduced that already in a commit before that last refactor commit.

joostjager force-pushed the sweeper-prep branch 3 times, most recently from 49685bf to b569192 Compare September 26, 2018 16:58

joostjager changed the title ~~Sweeper prep [DO NOT REVIEW]~~ sweep: prepare for sweeper [DO NOT REVIEW] Sep 26, 2018

joostjager force-pushed the sweeper-prep branch 11 times, most recently from 9c1cca0 to 158b97a Compare September 27, 2018 01:00

joostjager changed the title ~~sweep: prepare for sweeper [DO NOT REVIEW]~~ sweep: prepare for sweeper Sep 27, 2018

joostjager force-pushed the sweeper-prep branch 3 times, most recently from a168546 to 58c8813 Compare September 28, 2018 11:23

joostjager mentioned this pull request Sep 28, 2018

conctractcourt: check sweep sanity + reliable publication #1936

Merged

Roasbeef requested changes Oct 3, 2018

View reviewed changes

joostjager force-pushed the sweeper-prep branch 9 times, most recently from 22e8545 to d42c27a Compare October 3, 2018 08:26

joostjager commented Oct 3, 2018

View reviewed changes

joostjager mentioned this pull request Oct 5, 2018

lnwallet: add basic transaction rebroadcaster implementation #1704

Closed

Roasbeef requested a review from cfromknecht October 8, 2018 05:40

joostjager force-pushed the sweeper-prep branch 5 times, most recently from 19deaee to 608a9db Compare October 8, 2018 08:30

halseth suggested changes Oct 11, 2018

View reviewed changes

joostjager force-pushed the sweeper-prep branch 2 times, most recently from d0aebc4 to 121d030 Compare October 11, 2018 17:29

cfromknecht reviewed Oct 17, 2018

View reviewed changes

joostjager added 6 commits October 17, 2018 12:44

utxonursery: move spendable output structs to sweep package

9fcc7ee

This commit moves the output structs to a new package as a preparation for moving more logic into that package.

sweep: move sweep tx generation into sweep package

4dab405

Sweep txes are currently generated in multiple locations. Moving the sweep tx generation into a shared package will allow us to reuse that logic.

sweep: create new Input interface

7d69df7

This commit introduces a common interface for sweep inputs. It eliminates the type checking from UtxoSweeper. Also the formerly present Amount() getter is removed. It was redundant because this value is present in SignDesc().Output.Value as well.

cnct: reuse sweep tx logic for commit resolver

c1d845a

Removes duplicate sweep tx code from commit resolver and delegates generation to UtxoSweeper in the sweep package.

cnct: reuse sweep tx logic for success resolver

6977d59

sweep: refactor functions and unify tx info logging

95bf858

joostjager force-pushed the sweeper-prep branch from c431e4b to 95bf858 Compare October 17, 2018 10:44

Roasbeef approved these changes Oct 17, 2018

View reviewed changes

Roasbeef merged commit c9e42a6 into lightningnetwork:master Oct 17, 2018

cfromknecht reviewed Oct 17, 2018

View reviewed changes

cfromknecht mentioned this pull request Oct 17, 2018

sweep/sweeper: ignore unknown witness types #2062

Merged

sweep: prepare for sweeper #1978

sweep: prepare for sweeper #1978

Conversation

joostjager commented Sep 26, 2018 • edited Loading

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joostjager commented Oct 3, 2018

joostjager left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joostjager commented Oct 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joostjager commented Oct 17, 2018

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfromknecht commented Oct 17, 2018

joostjager commented Oct 18, 2018

joostjager commented Sep 26, 2018 •

edited

Loading

joostjager left a comment •

edited

Loading