Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signing support for Miniscript Descriptors #24149

Merged
merged 12 commits into from
Feb 16, 2023

Conversation

darosior
Copy link
Member

@darosior darosior commented Jan 25, 2022

This makes the Miniscript descriptors solvable.

Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 26, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK sipa, achow101
Stale ACK instagibbs

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #26756 (wallet: Replace GetBalance() logic with AvailableCoins() by w0xlt)
  • #26627 (wallet: Migrate non-HD keys with single combo containing a list of keys by achow101)
  • #26626 (descriptors: Add a KEY expression representing a list of individual keys by achow101)
  • #26573 (Wallet: don't underestimate the fees when spending a Taproot output by darosior)
  • #26567 (Wallet: estimate the size of signed inputs using descriptors by darosior)
  • #26101 (script: create V1SigVersion for functions which should only accept taproot/tapscript by theuni)
  • #26076 (Switch hardened derivation marker to h (in normalized descriptors and new wallets) by Sjors)
  • #22838 (descriptors: Be able to specify change and receiving in a single descriptor string by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Member

@achow101 achow101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Particularly, the PSBT<->Miniscript integration isn't part of this PR.

I don't think it would be particularly hard considering that you're already using SignatureData for non-wallet data lookups. I think all that would really need to be done is to update FillSignatureData to include the preimages from the PSBT.

Edit: This diff does the trick

diff --git a/src/psbt.cpp b/src/psbt.cpp
index 8248609ba6..9e09ccf15d 100644
--- a/src/psbt.cpp
+++ b/src/psbt.cpp
@@ -113,6 +113,18 @@ void PSBTInput::FillSignatureData(SignatureData& sigdata) const
     for (const auto& key_pair : hd_keypaths) {
         sigdata.misc_pubkeys.emplace(key_pair.first.GetID(), key_pair);
     }
+    for (const auto& [hash, preimage] : ripemd160_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
+    for (const auto& [hash, preimage] : sha256_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
+    for (const auto& [hash, preimage] : hash160_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
+    for (const auto& [hash, preimage] : hash256_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
 }
 
 void PSBTInput::FromSignatureData(const SignatureData& sigdata)

src/script/miniscript.h Outdated Show resolved Hide resolved
src/script/miniscript.cpp Outdated Show resolved Hide resolved
src/script/miniscript.h Outdated Show resolved Hide resolved
src/test/descriptor_tests.cpp Outdated Show resolved Hide resolved
@darosior
Copy link
Member Author

Rebased, addressed Andrew's comments, added some cleanups from @sipa, and added a miniscript_random fuzz target which generates a random Miniscript node based on a binary encoding. We might eventually add another one which tries to be smart, either here or in a follow-up.

I don't think it would be particularly hard considering that you're already using SignatureData for non-wallet data lookups. I think all that would really need to be done is to update FillSignatureData to include the preimages from the PSBT.

Yeah, but i figured it would be cleaner to have this as part of a PR which'd also update the various PSBT RPCs and test it using those.

darosior and others added 2 commits February 11, 2023 16:51
This is a "dumb" way of randomly generating a Miniscript node from
fuzzer input. It defines a strict binary encoding and will always generate
a node defined from the encoding without "helping" to create valid nodes.
It will cut through as soon as it encounters an invalid fragment so
hopefully the fuzzer can tend to learn the encoding and generate valid
nodes with a higher probability.

On a valid generated node a number of invariants are checked, especially
around the satisfactions and testing them against the Script
interpreter.

The node generation and testing is modular in order to later introduce
other ways to generate nodes from fuzzer inputs with minimal code.

Co-Authored-By: Pieter Wuille <[email protected]>
At the expense of more complexity, this target generates a valid
Miniscript node at every iteration.

This target will at first run populate a list of recipe (a map from
desired type to possible ways of creating such type) and curate it
(remove the unavailable or redundant recipes).
Then, at each iteration it will pick a type, choose a manner to create a
node of such type from the available recipes, and then
pseudo-recursively do the same for the type constraints of the picked
recipe.

For instance, if it is instructed based on the fuzzer output to create a
Miniscript node of type 'Bd', it could choose to create an 'or_i(subA, subB)'
nodes with type constraints 'B' for subA and 'Bd' for subB. It then
consults the recipes for creating subA and subB, etc...

Here is the list of all the existing recipes, by type constraint:

B: 0()
B: 1()
B: older()
B: after()
B: sha256()
B: hash256()
B: ripemd160()
B: hash160()
B: c:(K)
B: d:(Vz)
B: j:(Bn)
B: n:(B)
B: and_v(V,B)
B: and_b(B,W)
B: or_b(Bd,Wd)
B: or_d(Bdu,B)
B: or_i(B,B)
B: andor(Bdu,B,B)
B: thresh(Bdu)
B: thresh(Bdu,Wdu)
B: thresh(Bdu,Wdu,Wdu)
B: multi()

V: v:(B)
V: and_v(V,V)
V: or_c(Bdu,V)
V: or_i(V,V)
V: andor(Bdu,V,V)

K: pk_k()
K: pk_h()
K: and_v(V,K)
K: or_i(K,K)
K: andor(Bdu,K,K)

W: a:(B)
W: s:(Bo)

Bz: 0()
Bz: 1()
Bz: older()
Bz: after()
Bz: n:(Bz)
Bz: and_v(Vz,Bz)
Bz: or_d(Bzdu,Bz)
Bz: andor(Bzdu,Bz,Bz)
Bz: thresh(Bzdu)

Vz: v:(Bz)
Vz: and_v(Vz,Vz)
Vz: or_c(Bzdu,Vz)
Vz: andor(Bzdu,Vz,Vz)

Bo: sha256()
Bo: hash256()
Bo: ripemd160()
Bo: hash160()
Bo: c:(Ko)
Bo: d:(Vz)
Bo: j:(Bon)
Bo: n:(Bo)
Bo: and_v(Vz,Bo)
Bo: and_v(Vo,Bz)
Bo: or_d(Bodu,Bz)
Bo: or_i(Bz,Bz)
Bo: andor(Bzdu,Bo,Bo)
Bo: andor(Bodu,Bz,Bz)
Bo: thresh(Bodu)

Vo: v:(Bo)
Vo: and_v(Vz,Vo)
Vo: and_v(Vo,Vz)
Vo: or_c(Bodu,Vz)
Vo: or_i(Vz,Vz)
Vo: andor(Bzdu,Vo,Vo)
Vo: andor(Bodu,Vz,Vz)

Ko: pk_k()
Ko: and_v(Vz,Ko)
Ko: andor(Bzdu,Ko,Ko)

Bn: sha256()
Bn: hash256()
Bn: ripemd160()
Bn: hash160()
Bn: c:(Kn)
Bn: d:(Vz)
Bn: j:(Bn)
Bn: n:(Bn)
Bn: and_v(Vz,Bn)
Bn: and_v(Vn,B)
Bn: and_b(Bn,W)
Bn: multi()

Vn: v:(Bn)
Vn: and_v(Vz,Vn)
Vn: and_v(Vn,V)

Kn: pk_k()
Kn: pk_h()
Kn: and_v(Vz,Kn)
Kn: and_v(Vn,K)

Bon: sha256()
Bon: hash256()
Bon: ripemd160()
Bon: hash160()
Bon: c:(Kon)
Bon: d:(Vz)
Bon: j:(Bon)
Bon: n:(Bon)
Bon: and_v(Vz,Bon)
Bon: and_v(Von,Bz)

Von: v:(Bon)
Von: and_v(Vz,Von)
Von: and_v(Von,Vz)

Kon: pk_k()
Kon: and_v(Vz,Kon)

Bd: 0()
Bd: sha256()
Bd: hash256()
Bd: ripemd160()
Bd: hash160()
Bd: c:(Kd)
Bd: d:(Vz)
Bd: j:(Bn)
Bd: n:(Bd)
Bd: and_b(Bd,Wd)
Bd: or_b(Bd,Wd)
Bd: or_d(Bdu,Bd)
Bd: or_i(B,Bd)
Bd: or_i(Bd,B)
Bd: andor(Bdu,B,Bd)
Bd: thresh(Bdu)
Bd: thresh(Bdu,Wdu)
Bd: thresh(Bdu,Wdu,Wdu)
Bd: multi()

Kd: pk_k()
Kd: pk_h()
Kd: or_i(K,Kd)
Kd: or_i(Kd,K)
Kd: andor(Bdu,K,Kd)

Wd: a:(Bd)
Wd: s:(Bod)

Bzd: 0()
Bzd: n:(Bzd)
Bzd: or_d(Bzdu,Bzd)
Bzd: andor(Bzdu,Bz,Bzd)
Bzd: thresh(Bzdu)

Bod: sha256()
Bod: hash256()
Bod: ripemd160()
Bod: hash160()
Bod: c:(Kod)
Bod: d:(Vz)
Bod: j:(Bon)
Bod: n:(Bod)
Bod: or_d(Bodu,Bzd)
Bod: or_i(Bz,Bzd)
Bod: or_i(Bzd,Bz)
Bod: andor(Bzdu,Bo,Bod)
Bod: andor(Bodu,Bz,Bzd)
Bod: thresh(Bodu)

Kod: pk_k()
Kod: andor(Bzdu,Ko,Kod)

Bu: 0()
Bu: 1()
Bu: sha256()
Bu: hash256()
Bu: ripemd160()
Bu: hash160()
Bu: c:(K)
Bu: d:(Vz)
Bu: j:(Bnu)
Bu: n:(B)
Bu: and_v(V,Bu)
Bu: and_b(B,W)
Bu: or_b(Bd,Wd)
Bu: or_d(Bdu,Bu)
Bu: or_i(Bu,Bu)
Bu: andor(Bdu,Bu,Bu)
Bu: thresh(Bdu)
Bu: thresh(Bdu,Wdu)
Bu: thresh(Bdu,Wdu,Wdu)
Bu: multi()

Bzu: 0()
Bzu: 1()
Bzu: n:(Bz)
Bzu: and_v(Vz,Bzu)
Bzu: or_d(Bzdu,Bzu)
Bzu: andor(Bzdu,Bzu,Bzu)
Bzu: thresh(Bzdu)

Bou: sha256()
Bou: hash256()
Bou: ripemd160()
Bou: hash160()
Bou: c:(Ko)
Bou: d:(Vz)
Bou: j:(Bonu)
Bou: n:(Bo)
Bou: and_v(Vz,Bou)
Bou: and_v(Vo,Bzu)
Bou: or_d(Bodu,Bzu)
Bou: or_i(Bzu,Bzu)
Bou: andor(Bzdu,Bou,Bou)
Bou: andor(Bodu,Bzu,Bzu)
Bou: thresh(Bodu)

Bnu: sha256()
Bnu: hash256()
Bnu: ripemd160()
Bnu: hash160()
Bnu: c:(Kn)
Bnu: d:(Vz)
Bnu: j:(Bnu)
Bnu: n:(Bn)
Bnu: and_v(Vz,Bnu)
Bnu: and_v(Vn,Bu)
Bnu: and_b(Bn,W)
Bnu: multi()

Bonu: sha256()
Bonu: hash256()
Bonu: ripemd160()
Bonu: hash160()
Bonu: c:(Kon)
Bonu: d:(Vz)
Bonu: j:(Bonu)
Bonu: n:(Bon)
Bonu: and_v(Vz,Bonu)
Bonu: and_v(Von,Bzu)

Bdu: 0()
Bdu: sha256()
Bdu: hash256()
Bdu: ripemd160()
Bdu: hash160()
Bdu: c:(Kd)
Bdu: d:(Vz)
Bdu: j:(Bnu)
Bdu: n:(Bd)
Bdu: and_b(Bd,Wd)
Bdu: or_b(Bd,Wd)
Bdu: or_d(Bdu,Bdu)
Bdu: or_i(Bu,Bdu)
Bdu: or_i(Bdu,Bu)
Bdu: andor(Bdu,Bu,Bdu)
Bdu: thresh(Bdu)
Bdu: thresh(Bdu,Wdu)
Bdu: thresh(Bdu,Wdu,Wdu)
Bdu: multi()

Wdu: a:(Bdu)
Wdu: s:(Bodu)

Bzdu: 0()
Bzdu: n:(Bzd)
Bzdu: or_d(Bzdu,Bzdu)
Bzdu: andor(Bzdu,Bzu,Bzdu)
Bzdu: thresh(Bzdu)

Bodu: sha256()
Bodu: hash256()
Bodu: ripemd160()
Bodu: hash160()
Bodu: c:(Kod)
Bodu: d:(Vz)
Bodu: j:(Bonu)
Bodu: n:(Bod)
Bodu: or_d(Bodu,Bzdu)
Bodu: or_i(Bzu,Bzdu)
Bodu: or_i(Bzdu,Bzu)
Bodu: andor(Bzdu,Bou,Bodu)
Bodu: andor(Bodu,Bzu,Bzdu)
Bodu: thresh(Bodu)

Co-authored-by: Pieter Wuille <[email protected]>
Copy link
Member

@instagibbs instagibbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 36478ac

reviewed tests only

test/functional/wallet_miniscript.py Show resolved Hide resolved
Copy link
Member

@sipa sipa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 6c7a17a (to the extent that it's not my own code).

@@ -108,7 +109,8 @@ BOOST_AUTO_TEST_CASE(DoS_mapOrphans)
tx.vin[j].prevout.n = j;
tx.vin[j].prevout.hash = txPrev->GetHash();
}
BOOST_CHECK(SignSignature(keystore, *txPrev, tx, 0, SIGHASH_ALL));
SignatureData empty;
BOOST_CHECK(SignSignature(keystore, *txPrev, tx, 0, SIGHASH_ALL, empty));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the need for all these dummy SignatureData objects that need to be created in calls, would it make sense to add another overload of SignSignature that has no SignaturaData& argument?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do if i need to retouch.

@fanquake fanquake added this to the 25.0 milestone Feb 14, 2023
@achow101
Copy link
Member

ACK 6c7a17a

Reviewed code and non-fuzz tests.

@Sjors
Copy link
Member

Sjors commented Feb 15, 2023

@darosior do I understand correctly that you need #26567 in order for PSBT stuff to work? Can I just combine both PR's to continue the test I was doing, or are there more changes required?

@darosior
Copy link
Member Author

@Sjors see #24149 (comment). A minimal PSBT integration is part of this PR now. We treat Miniscript-related data when we are given a PSBT. However to fill Miniscript-related data to a PSBT from our wallet will need more work than just #26567, and also some design decisions to be taken.

@fanquake fanquake merged commit fb82d91 into bitcoin:master Feb 16, 2023
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 16, 2023
sipa added a commit to sipa/miniscript that referenced this pull request Feb 16, 2023
930e2f2 Update the Miniscript fuzz tests from Bitcoin Core master (Antoine Poinsot)
f62abcf Update the Miniscript unit tests from Bitcoin master (Antoine Poinsot)
fd9bed0 miniscript: update the source from Bitcoin Core master branch (Antoine Poinsot)

Pull request description:

  The PRs introducing Miniscript to Bitcoin Core contained the most up to date version of the code (bitcoin/bitcoin#24148 and bitcoin/bitcoin#24149).

  Now they are both merged, update the code here to reflect the updates made in these PRs.

ACKs for top commit:
  sipa:
    ACK 930e2f2. Matches the Bitcoin Core master branch code, and compiles.

Tree-SHA512: 0ef587d005dc472cada86b99e62f37aa6bbfa09a8d1db95726a0103c0e827880d557ebce39c87eeb281c952b77fcf462c9a779008eed5b2d0f2d63478f4fab4c
using miniscript::operator"" _mst;

//! Construct a miniscript node as a shared_ptr.
template<typename... Args> NodeRef MakeNodeRef(Args&&... args) { return miniscript::MakeNodeRef<CPubKey>(KEY_COMP, std::forward<Args>(args)...); }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm looks like this checks for duplicate keys on every single node created whereas we could just check that on the top-level node in GenNode()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

domob1812 added a commit to domob1812/namecoin-core that referenced this pull request Feb 21, 2023
Some of the new miniscript signing tests in wallet_miniscript.py
are disabled for now as they fail on Namecoin, this needs debugging
and fixing in the future.  These were introduced upstream in
bitcoin/bitcoin#24149.
fanquake added a commit that referenced this pull request Feb 22, 2023
…script nodes

c1b7bd0 fuzz: avoid redundant dup key checks when creating Miniscript nodes (Antoine Poinsot)

Pull request description:

  I thought i had done that already in #24149, but it must have slipped through the rebase. It's a 2x speed improvement against the existing corpora and will probably be much more as we extend them with larger nodes.

ACKs for top commit:
  sipa:
    ACK c1b7bd0

Tree-SHA512: 9e6ceb6254183964b6c5538e21ba6321df95a68acb343a15a6ecfef5c51a1980d2627df5aeef9aef1db41656e18cc4f3bc96e6f24314d12fa60368b04a350001
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 25, 2023
…ng Miniscript nodes

c1b7bd0 fuzz: avoid redundant dup key checks when creating Miniscript nodes (Antoine Poinsot)

Pull request description:

  I thought i had done that already in bitcoin#24149, but it must have slipped through the rebase. It's a 2x speed improvement against the existing corpora and will probably be much more as we extend them with larger nodes.

ACKs for top commit:
  sipa:
    ACK c1b7bd0

Tree-SHA512: 9e6ceb6254183964b6c5538e21ba6321df95a68acb343a15a6ecfef5c51a1980d2627df5aeef9aef1db41656e18cc4f3bc96e6f24314d12fa60368b04a350001
achow101 added a commit that referenced this pull request Sep 6, 2023
…ptors

10546a5 wallet: accurately account for the size of the witness stack (Antoine Poinsot)
9b7ec39 wallet: use descriptor satisfaction size to estimate inputs size (Antoine Poinsot)
8d870a9 script/signingprovider: introduce a MultiSigningProvider (Antoine Poinsot)
fa7c46b descriptor: introduce a method to get the satisfaction size (Antoine Poinsot)
bdba766 miniscript: introduce a helper to get the maximum witness size (Antoine Poinsot)
4ab382c miniscript: make GetStackSize independent of P2WSH context (Antoine Poinsot)

Pull request description:

  The wallet currently estimates the size of a signed input by doing a dry run of the signing logic. This is unnecessary since all outputs we can sign for can be represented by a descriptor, and we can derive the size of a satisfaction ("signature") directly from the descriptor itself.
  In addition, the current approach does not generalize well: dry runs of the signing logic are only possible for the most basic scripts. See for instance the discussion in #24149 around that.

  This introduces a method to get the maximum size of a satisfaction from a descriptor, and makes the wallet use that instead of the dry-run.

ACKs for top commit:
  sipa:
    utACK 10546a5
  achow101:
    re-ACK 10546a5

Tree-SHA512: 43ed1529fbd30af709d903c8c5063235e8c6a03b500bc8f144273d6184e23a53edf0fea9ef898ed57d8a40d73208b5d935cc73b94a24fad3ad3c63b3b2027174
stickies-v pushed a commit to stickies-v/bitcoin-devwiki that referenced this pull request Nov 13, 2023
@bitcoin bitcoin locked and limited conversation to collaborators Feb 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants