chapter1-legacy/p2pkh.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a P2PKH transaction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we'll create a P2PKH transaction from scratch in python. We'll go through each part of the transaction, how it's constructed, signed, and we'll test it using the bitcoin core test framework.\n",
    "\n",
    "## Prerequisite knowledge\n",
    "- For all notebooks:\n",
    "    - A high level understanding of the bitcoin. e.g. [Mastering Bitcoin](https://github.com/bitcoinbook/bitcoinbook) by Andreas Antonopoulos UTXO model, in particular [Chapter 6](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch06.asciidoc).\n",
    "    - A conceptual understanding of [hash functions](https://www.thesslstore.com/blog/what-is-a-hash-function-in-cryptography-a-beginners-guide).\n",
    "    - [Hexadecimal notation](https://inst.eecs.berkeley.edu/~cs61bl/r//cur/bits/decimal-binary-hex.html?topic=lab28.topic&step=2&course=) and [endianness](https://www.freecodecamp.org/news/what-is-endianness-big-endian-vs-little-endian/).\n",
    "\n",
    "\n",
    "- Specific to this notebook:\n",
    "    - SHA256, HASH256, HASH160 - '[Hash Functions chapter](https://github.com/DariusParvin/bitcoin-tx-tutorial/blob/main/appendix/hash-functions.ipynb)'\n",
    "    - Base58 addresses - '[Addresses chapter](https://github.com/DariusParvin/bitcoin-tx-tutorial/blob/main/appendix/Addresses.ipynb)'\n",
    "    - Bitcoin Script basics - '[Bitcoin Script chapter](https://github.com/DariusParvin/bitcoin-tx-tutorial/blob/main/appendix/Bitcoin%20Script.ipynb)'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup \n",
    "\n",
    "### Requirements\n",
    "For this exercise we'll need Bitcoin Core. This notebook has been tested with [v24.0.1](https://github.com/bitcoin/bitcoin/releases/tag/v24.0.1).\n",
    "\n",
    "Below, set the paths for:\n",
    "1. The bitcoin core functional test framework directory.\n",
    "2. The directory containing bitcoin-tx-tutorial.\n",
    "\n",
    "**You'll need to edit these next two lines for your local setup.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "path_to_bitcoin_functional_test = \"/Users/dariuscognac/bitcoin/test/functional\"\n",
    "path_to_bitcoin_tx_tutorial = \"/Users/dariuscognac/Documents/Github/bitcoin-tx-tutorial\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup bitcoin core test framework\n",
    "Start up regtest mode, delete any regtest network history so we are starting from scratch. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "# Add the functional test framework to our PATH\n",
    "sys.path.insert(0, path_to_bitcoin_functional_test)\n",
    "from test_framework.test_shell import TestShell\n",
    "\n",
    "# Add the bitcoin-tx-tutorial functions to our PATH\n",
    "sys.path.insert(0, path_to_bitcoin_tx_tutorial)\n",
    "from functions import *\n",
    "\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "code_folding": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022-12-06T21:11:26.463000Z TestFramework (INFO): Initializing test directory /var/folders/r5/yk8yg2xs1gs8xzkn5l8vr72w0000gn/T/bitcoin_func_test_zlfn_arb\n"
     ]
    }
   ],
   "source": [
    "# Setup our regtest environment\n",
    "test = TestShell().setup(\n",
    "    num_nodes=1, \n",
    "    setup_clean_chain=True\n",
    ")\n",
    "\n",
    "node = test.nodes[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a wallet so that we can fund our first output using the `sendtoaddress` command. Mine 101 blocks so that the first block subsidy will have sufficient block maturity (100 blocks) to spend from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new wallet and address to send mining rewards so we can fund our transactions\n",
    "node.createwallet(wallet_name='mywallet')\n",
    "address = node.getnewaddress()\n",
    "\n",
    "# Generate 101 blocks so that the first block subsidy reaches maturity\n",
    "result = node.generatetoaddress(nblocks=101, address=address, invalid_call=False)\n",
    "\n",
    "# Check that we were able to mine 101 blocks\n",
    "assert(node.getblockcount() == 101)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a P2PKH UTXO\n",
    "\n",
    "In order to create a transaction spending from a P2PKH UTXO, we'll first need to create the UTXO that is locked with a p2pkh script. To do that, we'll create a P2PKH address from a private key, and fund it using the bitcoind wallet created in the setup step."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create a p2pkh address \n",
    "For more on this step, review the 'Addresses' notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sender's p2pkh address: n4XmX91N5FfccY678vaG1ELNtXh6skVES7\n"
     ]
    }
   ],
   "source": [
    "sender_privkey = bytes.fromhex(\"1111111111111111111111111111111111111111111111111111111111111111\")\n",
    "sender_pubkey = privkey_to_pubkey(sender_privkey)\n",
    "sender_p2pkh_addr = pk_to_p2pkh(sender_pubkey, network = \"regtest\")\n",
    "print(\"sender's p2pkh address: \" + sender_p2pkh_addr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fund the 'sender' with 2.001 btc (0.001 btc is for the next tx fee)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5eff481de26cfefcbb288e945fc9cf95a60dd837fda51e31fec4c5e2e20ee044\n"
     ]
    }
   ],
   "source": [
    "txid_to_spend = node.sendtoaddress(sender_p2pkh_addr, 2.001)\n",
    "print(txid_to_spend)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view the transaction using the bitcoin-cli commands `getrawtransaction` and `decoderawtransaction` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"txid\": \"5eff481de26cfefcbb288e945fc9cf95a60dd837fda51e31fec4c5e2e20ee044\",\n",
      "  \"hash\": \"c9f2deb9de45c33dce1d047776cf3e66b8371415bafaa23cee97289fe8ee5271\",\n",
      "  \"version\": 2,\n",
      "  \"size\": 228,\n",
      "  \"vsize\": 147,\n",
      "  \"weight\": 585,\n",
      "  \"locktime\": 101,\n",
      "  \"vin\": [\n",
      "    {\n",
      "      \"txid\": \"0fac9344999273a988cf86b683489543789b910624b5fdc22752bbb0a98a6d24\",\n",
      "      \"vout\": 0,\n",
      "      \"scriptSig\": {\n",
      "        \"asm\": \"\",\n",
      "        \"hex\": \"\"\n",
      "      },\n",
      "      \"txinwitness\": [\n",
      "        \"304402201835791520077ef3552179ee3c68b44180ea9ce6d7fa7ed438e15db341233c0402205c3265d27f90a0ac9ca6a342ec6e52faa38408dccb862587fed99d23d2d48a6401\",\n",
      "        \"024a2083a1938578d91173e6c44e72bd599b5b8d69fe9ce34c3e6f23ffbe4a058a\"\n",
      "      ],\n",
      "      \"sequence\": 4294967293\n",
      "    }\n",
      "  ],\n",
      "  \"vout\": [\n",
      "    {\n",
      "      \"value\": \"47.99897060\",\n",
      "      \"n\": 0,\n",
      "      \"scriptPubKey\": {\n",
      "        \"asm\": \"OP_DUP OP_HASH160 76ce045c03f7254d14ba15a9b70a8103004e05e0 OP_EQUALVERIFY OP_CHECKSIG\",\n",
      "        \"desc\": \"addr(mrM8q1CAXCQT6MDGXq6WMypkPqFCa8xvBg)#jwhsgzw6\",\n",
      "        \"hex\": \"76a91476ce045c03f7254d14ba15a9b70a8103004e05e088ac\",\n",
      "        \"address\": \"mrM8q1CAXCQT6MDGXq6WMypkPqFCa8xvBg\",\n",
      "        \"type\": \"pubkeyhash\"\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"value\": \"2.00100000\",\n",
      "      \"n\": 1,\n",
      "      \"scriptPubKey\": {\n",
      "        \"asm\": \"OP_DUP OP_HASH160 fc7250a211deddc70ee5a2738de5f07817351cef OP_EQUALVERIFY OP_CHECKSIG\",\n",
      "        \"desc\": \"addr(n4XmX91N5FfccY678vaG1ELNtXh6skVES7)#n2xufj4n\",\n",
      "        \"hex\": \"76a914fc7250a211deddc70ee5a2738de5f07817351cef88ac\",\n",
      "        \"address\": \"n4XmX91N5FfccY678vaG1ELNtXh6skVES7\",\n",
      "        \"type\": \"pubkeyhash\"\n",
      "      }\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "raw_tx = node.getrawtransaction(txid_to_spend)\n",
    "decoded = node.decoderawtransaction(raw_tx)\n",
    "print(json.dumps(decoded, indent=2, default=str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Find which output index the btc was sent to\n",
    "Since we only sent 2.001 btc of the coinbase transaction (50 btc) to our address, bitcoind creates a change output to send the rest of the btc. By looking at the outputs we can see which is the change output and which was sent to our address. To do this in python we can do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "index to spend from: 1\n"
     ]
    }
   ],
   "source": [
    "if decoded[\"vout\"][0][\"scriptPubKey\"][\"address\"] == sender_p2pkh_addr:\n",
    "    index_to_spend = 0\n",
    "elif decoded[\"vout\"][1][\"scriptPubKey\"][\"address\"] == sender_p2pkh_addr:\n",
    "    index_to_spend = 1\n",
    "else:\n",
    "    raise Exception(\"couldn't find output\")\n",
    "print(\"index to spend from: \" + str(index_to_spend))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Mine a block so that the funding tx gets confirmed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "node.generatetoaddress(1, address, invalid_call=False);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spending a p2pkh UTXO\n",
    "\n",
    "Now that we have some funds locked up in a p2pkh utxo, we can create a transaction spending from it. Let's say we want to send 1.5 btc to the address `mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE`.\n",
    "\n",
    "### Decoding a base58 address\n",
    "\n",
    "The first thing we need to do is decode the address. This lets us:  \n",
    "1 - validate the checksum to know the address was transmitted without error  \n",
    "2 - make sure we are sending btc on the correct network (testnet/mainnet)  \n",
    "3 - know what to put in the scriptPubkey\n",
    "\n",
    "For more on addresses, refer back to the '[Addresses](https://github.com/DariusParvin/bitcoin-tx-tutorial/blob/main/appendix/addresses.ipynb)' chapter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0x6f\n",
      "3bc28d6d92d9073fb5e3adf481795eaf446bceed\n",
      "ee2161b7\n"
     ]
    }
   ],
   "source": [
    "receiver_address = 'mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE'\n",
    "receiver_address_decoded = decode_base58(receiver_address)\n",
    "# TODO: create a function in the address chapter to validate and parse addresses and use here\n",
    "\n",
    "prefix = receiver_address_decoded[0]  \n",
    "pubkey_hash = receiver_address_decoded[1:-4] \n",
    "checksum = receiver_address_decoded[-4:]\n",
    "print(hex(prefix))\n",
    "print(pubkey_hash.hex())\n",
    "print(checksum.hex())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first byte , in our case `6f`, tells us that this address corresponds to a p2pkh output for testnet. For more on decoding addresses, refer back to the 'Addresses' chapter.\n",
    "\n",
    "Now we can create the receiver's output scriptPubkey:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "receiver_spk = bytes.fromhex(\"76a914\") + pubkey_hash + bytes.fromhex(\"88ac\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create an unsigned p2pkh transaction\n",
    "\n",
    "The first thing we'll do is define the inputs and outputs of our transaction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note we have already defined a few variables we need to create our transaction:\n",
    "# The input utxo txid and index: `txid_to_spend` and `index_to_spend`\n",
    "# The input private key and public key: `sender_privkey` and `sender_pubkey`\n",
    "\n",
    "# Set our outputs\n",
    "# Create a new pubkey to use as a change output.\n",
    "change_privkey = bytes.fromhex(\"2222222222222222222222222222222222222222222222222222222222222222\")\n",
    "change_pubkey = privkey_to_pubkey(change_privkey)\n",
    "\n",
    "# Determine our output scriptPubkeys and amounts (in satoshis)\n",
    "output1_value_sat = int(float(\"1.5\") * 100000000)\n",
    "output1_spk = receiver_spk\n",
    "output2_value_sat = int(float(\"0.5\") * 100000000)\n",
    "output2_spk = bytes.fromhex(\"76a914\") + hash160(change_pubkey) + bytes.fromhex(\"88ac\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've defined everything we need, we can fill in the fields we need to create our unsigned transaction. What makes a transaction 'unsigned' is that the input's scriptSig, the field where the signature goes, is empty. This first step is necessary as the signature will cover the whole transaction (using SIGHASH_ALL). In a later chapter we will cover other sighash types and how they are signed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "unsigned_tx:  020000000144e00ee2e2c5c4fe311ea5fd37d80da695cfc95f948e28bbfcfe6ce21d48ff5e0100000000ffffffff0280d1f008000000001976a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac80f0fa02000000001976a914531260aa2a199e228c537dfa42c82bea2c7c1f4d88ac00000000\n"
     ]
    }
   ],
   "source": [
    "# VERSION\n",
    "# version '2' indicates that we may use relative timelocks (BIP68)\n",
    "version = bytes.fromhex(\"0200 0000\")\n",
    "\n",
    "# INPUTS\n",
    "# We have just 1 input\n",
    "input_count = bytes.fromhex(\"01\")\n",
    "\n",
    "# Convert txid and index to bytes (little endian)\n",
    "txid = (bytes.fromhex(txid_to_spend))[::-1]\n",
    "index = index_to_spend.to_bytes(4, byteorder=\"little\", signed=False)\n",
    "\n",
    "# For the unsigned transaction we use an empty scriptSig\n",
    "scriptsig = bytes.fromhex(\"\")\n",
    "\n",
    "# use 0xffffffff unless you are using OP_CHECKSEQUENCEVERIFY, locktime, or rbf\n",
    "sequence = bytes.fromhex(\"ffff ffff\")\n",
    "\n",
    "inputs = (\n",
    "    txid\n",
    "    + index\n",
    "    + varint_len(scriptsig)\n",
    "    + scriptsig\n",
    "    + sequence\n",
    ")\n",
    "\n",
    "# OUTPUTS\n",
    "# 0x02 for out two outputs\n",
    "output_count = bytes.fromhex(\"02\")\n",
    "\n",
    "# OUTPUT 1 \n",
    "output1_value = output1_value_sat.to_bytes(8, byteorder=\"little\", signed=True)\n",
    "# 'output1_spk' already defined at the start of the script\n",
    "\n",
    "# OUTPUT 2\n",
    "output2_value = output2_value_sat.to_bytes(8, byteorder=\"little\", signed=True)\n",
    "# 'output2_spk' already defined at the start of the script\n",
    "\n",
    "outputs = (\n",
    "    output1_value\n",
    "    + varint_len(output1_spk)\n",
    "    + output1_spk\n",
    "    + output2_value\n",
    "    + varint_len(output2_spk)\n",
    "    + output2_spk\n",
    ")\n",
    "\n",
    "# LOCKTIME\n",
    "locktime = bytes.fromhex(\"0000 0000\")\n",
    "\n",
    "unsigned_tx = (\n",
    "    version\n",
    "    + input_count\n",
    "    + inputs\n",
    "    + output_count\n",
    "    + outputs\n",
    "    + locktime\n",
    ")\n",
    "print(\"unsigned_tx: \", unsigned_tx.hex())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can decode this raw transaction to inspect it and see that it has all the information we need apart from the scriptSig."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"txid\": \"ca4164fded6ead1d3bfdbba0f9fa4e0b2e27615b97905fa46195fe8c83adbe47\",\n",
      "  \"hash\": \"ca4164fded6ead1d3bfdbba0f9fa4e0b2e27615b97905fa46195fe8c83adbe47\",\n",
      "  \"version\": 2,\n",
      "  \"size\": 119,\n",
      "  \"vsize\": 119,\n",
      "  \"weight\": 476,\n",
      "  \"locktime\": 0,\n",
      "  \"vin\": [\n",
      "    {\n",
      "      \"txid\": \"5eff481de26cfefcbb288e945fc9cf95a60dd837fda51e31fec4c5e2e20ee044\",\n",
      "      \"vout\": 1,\n",
      "      \"scriptSig\": {\n",
      "        \"asm\": \"\",\n",
      "        \"hex\": \"\"\n",
      "      },\n",
      "      \"sequence\": 4294967295\n",
      "    }\n",
      "  ],\n",
      "  \"vout\": [\n",
      "    {\n",
      "      \"value\": \"1.50000000\",\n",
      "      \"n\": 0,\n",
      "      \"scriptPubKey\": {\n",
      "        \"asm\": \"OP_DUP OP_HASH160 3bc28d6d92d9073fb5e3adf481795eaf446bceed OP_EQUALVERIFY OP_CHECKSIG\",\n",
      "        \"desc\": \"addr(mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE)#xlnzfr97\",\n",
      "        \"hex\": \"76a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac\",\n",
      "        \"address\": \"mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE\",\n",
      "        \"type\": \"pubkeyhash\"\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"value\": \"0.50000000\",\n",
      "      \"n\": 1,\n",
      "      \"scriptPubKey\": {\n",
      "        \"asm\": \"OP_DUP OP_HASH160 531260aa2a199e228c537dfa42c82bea2c7c1f4d OP_EQUALVERIFY OP_CHECKSIG\",\n",
      "        \"desc\": \"addr(mo6CPsdW8EsnWdmSSCrQ6225VVDtpMBTug)#cvzzm9uf\",\n",
      "        \"hex\": \"76a914531260aa2a199e228c537dfa42c82bea2c7c1f4d88ac\",\n",
      "        \"address\": \"mo6CPsdW8EsnWdmSSCrQ6225VVDtpMBTug\",\n",
      "        \"type\": \"pubkeyhash\"\n",
      "      }\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "decoded = node.decoderawtransaction(unsigned_tx.hex())\n",
    "print(json.dumps(decoded, indent=2, default=str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can sign this transaction there is one final step we need to do. We need to replace the empty scriptSig with the scriptPubkey of the input we are signing over. If we had multiple inputs, we would need to do this step for each input. We will cover signing transactions with multiple inputs in a later chapter.\n",
    "\n",
    "Since we are spending from a p2pkh utxo, we will create the scriptPubkey in the same way as we did for the outputs, but using the sender's pubkey:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "pk_hash = hash160(sender_pubkey)\n",
    "input_spk = bytes.fromhex(\"76a914\" + pk_hash.hex() + \"88ac\")\n",
    "\n",
    "inputs = (\n",
    "    txid\n",
    "    + index\n",
    "    + varint_len(input_spk)\n",
    "    + input_spk # replace the empty scriptSig with the input scriptPubkey\n",
    "    + sequence\n",
    ")\n",
    "\n",
    "# tx hex to sign\n",
    "tx_to_sign = (\n",
    "    version\n",
    "    + input_count\n",
    "    + inputs\n",
    "    + output_count\n",
    "    + outputs\n",
    "    + locktime\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to hash this transaction and produce an ecdsa signature on it. \n",
    "\n",
    "Before hashing the transaction with hash256, we append the sighash flag. In this example we'll use the most commonly used SIGHASH_ALL flag, meaning the signature guarantees the input will only be used in a transaction with these exact inputs and outputs.\n",
    "\n",
    "Note that when we append the sighash flag to the transaction, we use 4 bytes, however when we append the sighash flag to the end of the signature itself we only use 1 byte."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "code_folding": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "signed transaction:  020000000144e00ee2e2c5c4fe311ea5fd37d80da695cfc95f948e28bbfcfe6ce21d48ff5e010000006b483045022100fdeca825ddc6c52a2ed891613f02ebe748adebf80d6d9c5593107dbcb78de27d02202ba0d862bf64d8730d396270f99162e186420ddc2cdcc9bbbf3f1812f247708e0121034f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aaffffffff0280d1f008000000001976a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac80f0fa02000000001976a914531260aa2a199e228c537dfa42c82bea2c7c1f4d88ac00000000\n"
     ]
    }
   ],
   "source": [
    "# Append the sighash flag to the transaction\n",
    "sighash_flag = bytes.fromhex(\"0100 0000\") # SIGHASH_ALL\n",
    "sighash_preimage = tx_to_sign + sighash_flag\n",
    "\n",
    "# Create sigHash to be signed\n",
    "sighash = hash256(sighash_preimage)\n",
    "\n",
    "# Sign the sigHash with the input private key\n",
    "signing_key = ecdsa.SigningKey.from_string(sender_privkey, curve=ecdsa.SECP256k1) \n",
    "signature = signing_key.sign_digest(sighash, sigencode=ecdsa.util.sigencode_der_canonize)\n",
    "\n",
    "# Append SIGHASH_ALL to the signature\n",
    "signature = signature + bytes.fromhex(\"01\")\n",
    "\n",
    "# Signature\n",
    "sig_script_signed = (\n",
    "    pushbytes(signature)\n",
    "    + pushbytes(sender_pubkey)\n",
    ")\n",
    "\n",
    "# tx_in with our new sigScript containing the signature we just created\n",
    "inputs_signed = (\n",
    "    txid\n",
    "    + index\n",
    "    + varint_len(sig_script_signed)\n",
    "    + sig_script_signed\n",
    "    + sequence\n",
    ")\n",
    "\n",
    "# the final signed transaction\n",
    "signed_tx = (\n",
    "    version\n",
    "    + input_count\n",
    "    + inputs_signed\n",
    "    + output_count\n",
    "    + outputs\n",
    "    + locktime\n",
    ")\n",
    "\n",
    "print(\"signed transaction: \",signed_tx.hex())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Broadcast the transaction (on regtest mode)\n",
    "If we get back a txid (32 byte hash), then it means the tx was successfully broadcast! If we just want to see if the transaction would have been accepted, but without broadcasting it, we can use the `testmempoolaccept` command (commented out)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "61e8d3c2e9ced2a211c76a5d4c03f4fae8402964f8af0cf530dea9eed2468f7e\n"
     ]
    }
   ],
   "source": [
    "new_tx_txid = node.sendrawtransaction(signed_tx.hex())\n",
    "# new_tx_txid = node.testmempoolaccept([signed_tx.hex()])\n",
    "\n",
    "print(new_tx_txid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can decode the serialized transaction using ```decoderawtransction```. Notice that our output addresses match the change and receiver addresses from earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "receiver's p2pkh address: mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE\n",
      "sender's change p2pkh address: mo6CPsdW8EsnWdmSSCrQ6225VVDtpMBTug\n"
     ]
    }
   ],
   "source": [
    "print(\"receiver's p2pkh address: \" + receiver_address)\n",
    "change_p2pkh_addr = pk_to_p2pkh(change_pubkey, network = \"regtest\")\n",
    "print(\"sender's change p2pkh address: \" + change_p2pkh_addr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"txid\": \"61e8d3c2e9ced2a211c76a5d4c03f4fae8402964f8af0cf530dea9eed2468f7e\",\n",
      "  \"hash\": \"61e8d3c2e9ced2a211c76a5d4c03f4fae8402964f8af0cf530dea9eed2468f7e\",\n",
      "  \"version\": 2,\n",
      "  \"size\": 226,\n",
      "  \"vsize\": 226,\n",
      "  \"weight\": 904,\n",
      "  \"locktime\": 0,\n",
      "  \"vin\": [\n",
      "    {\n",
      "      \"txid\": \"5eff481de26cfefcbb288e945fc9cf95a60dd837fda51e31fec4c5e2e20ee044\",\n",
      "      \"vout\": 1,\n",
      "      \"scriptSig\": {\n",
      "        \"asm\": \"3045022100fdeca825ddc6c52a2ed891613f02ebe748adebf80d6d9c5593107dbcb78de27d02202ba0d862bf64d8730d396270f99162e186420ddc2cdcc9bbbf3f1812f247708e[ALL] 034f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa\",\n",
      "        \"hex\": \"483045022100fdeca825ddc6c52a2ed891613f02ebe748adebf80d6d9c5593107dbcb78de27d02202ba0d862bf64d8730d396270f99162e186420ddc2cdcc9bbbf3f1812f247708e0121034f355bdcb7cc0af728ef3cceb9615d90684bb5b2ca5f859ab0f0b704075871aa\"\n",
      "      },\n",
      "      \"sequence\": 4294967295\n",
      "    }\n",
      "  ],\n",
      "  \"vout\": [\n",
      "    {\n",
      "      \"value\": \"1.50000000\",\n",
      "      \"n\": 0,\n",
      "      \"scriptPubKey\": {\n",
      "        \"asm\": \"OP_DUP OP_HASH160 3bc28d6d92d9073fb5e3adf481795eaf446bceed OP_EQUALVERIFY OP_CHECKSIG\",\n",
      "        \"desc\": \"addr(mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE)#xlnzfr97\",\n",
      "        \"hex\": \"76a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac\",\n",
      "        \"address\": \"mkxwE7XtVYJKepoD2hbHnDjftuMQ1k6deE\",\n",
      "        \"type\": \"pubkeyhash\"\n",
      "      }\n",
      "    },\n",
      "    {\n",
      "      \"value\": \"0.50000000\",\n",
      "      \"n\": 1,\n",
      "      \"scriptPubKey\": {\n",
      "        \"asm\": \"OP_DUP OP_HASH160 531260aa2a199e228c537dfa42c82bea2c7c1f4d OP_EQUALVERIFY OP_CHECKSIG\",\n",
      "        \"desc\": \"addr(mo6CPsdW8EsnWdmSSCrQ6225VVDtpMBTug)#cvzzm9uf\",\n",
      "        \"hex\": \"76a914531260aa2a199e228c537dfa42c82bea2c7c1f4d88ac\",\n",
      "        \"address\": \"mo6CPsdW8EsnWdmSSCrQ6225VVDtpMBTug\",\n",
      "        \"type\": \"pubkeyhash\"\n",
      "      }\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "decoded = node.decoderawtransaction(signed_tx.hex())\n",
    "print(json.dumps(decoded, indent=2, default=str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quiz\n",
    "1. How is the transaction ID calculated? \n",
    "2. Which field of a transaction contains the signature(s)? \n",
    "3. Bitcoin ECDSA signatures are vulnerable to third party malleation. As a result, it is possible for a third party to alter the transaction ID. If Alice is broadcasting a transaction to Bob, how might a third party manage to change the transaction ID using only publicly available information?\n",
    "4. What problems or limitations does this transaction malleability cause?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ## Answers\n",
    "1. It is the double SHA256 (HASH256) of the raw transaction. Note that the output is displayed in little endian. See '[The First Bitcoin Transaction (Pay to Pubkey)](https://github.com/DariusParvin/bitcoin-tx-tutorial/blob/main/chapter1-legacy/first-btc-tx.ipynb)' for an example.\n",
    "2. The scriptSig in the input field contains the signature(s).\n",
    "3. A third party could observe the transaction in the mempool, change the signature, then broadcast the transaction with the altered signature and transaction ID.\n",
    "4. Two issues (non exhaustive):\n",
    "  - It makes it impossible to create a chain of offline transactions as the inputs of a child may change by the time it is confirmed. This is an issue for many layer 2 protocols such as lightning payment channels.\n",
    "  - If Alice or Bob are searching for the transaction using the transaction ID, they may never find it. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise\n",
    "Now that we have the change output with a known private key `change_privkey`, create a transaction that spends from it to two outputs. The first output should to send 0.1 btc to the address `mgiS1dSDrPunE7GvXmoS4xEfmdwWsStZc7`. The second address should send the rest to a change output with the address `mz8AXDhDMhvLs7kxwfQxvcH5GoVH6AdARZ`. Set the miner fee to 0.001 btc.\n",
    "\n",
    "The first step will be to decode the output addresses we want to send btc to and convert them into their scriptPubkeys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Solution - Converting the addresses to scriptPubKeys\n",
    "receiver_address = 'mgiS1dSDrPunE7GvXmoS4xEfmdwWsStZc7'\n",
    "receiver_address_decoded = decode_base58(receiver_address)\n",
    "receiver_pubkey_hash = receiver_address_decoded[1:-4] \n",
    "output1_spk = bytes.fromhex(\"76a914\") + receiver_pubkey_hash + bytes.fromhex(\"88ac\")\n",
    "\n",
    "change_address = 'mz8AXDhDMhvLs7kxwfQxvcH5GoVH6AdARZ'\n",
    "change_address_decoded = decode_base58(change_address)\n",
    "change_pubkey_hash = change_address_decoded[1:-4] \n",
    "output2_spk = bytes.fromhex(\"76a914\") + change_pubkey_hash + bytes.fromhex(\"88ac\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create an unsigned transaction. For now we will keep the scriptSig empty (`0x00`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Solution - creating an unsigned transaction\n",
    "version = bytes.fromhex(\"0200 0000\")\n",
    "\n",
    "# INPUTS\n",
    "input_count = bytes.fromhex(\"01\")\n",
    "\n",
    "# INPUT 1\n",
    "txid_to_spend = new_tx_txid\n",
    "txid = (bytes.fromhex(txid_to_spend))[::-1]\n",
    "index_to_spend = 1\n",
    "index = index_to_spend.to_bytes(4, byteorder=\"little\", signed=False)\n",
    "scriptsig = bytes.fromhex(\"\")\n",
    "sequence = bytes.fromhex(\"ffff ffff\")\n",
    "\n",
    "inputs = (\n",
    "    txid\n",
    "    + index\n",
    "    + varint_len(scriptsig)\n",
    "    + scriptsig\n",
    "    + sequence\n",
    ")\n",
    "\n",
    "# OUTPUTS\n",
    "output_count = bytes.fromhex(\"02\")\n",
    "\n",
    "# OUTPUT 1 \n",
    "output1_value_sat = int(float(\"0.1\") * 100000000)\n",
    "output1_value = output1_value_sat.to_bytes(8, byteorder=\"little\", signed=True)\n",
    "\n",
    "# OUTPUT 2\n",
    "output2_value_sat = int(float(\"0.399\") * 100000000) # 0.5 - 0.1 - 0.001 for the miner fee\n",
    "output2_value = output2_value_sat.to_bytes(8, byteorder=\"little\", signed=True)\n",
    "\n",
    "outputs = (\n",
    "    output1_value\n",
    "    + varint_len(output1_spk)\n",
    "    + output1_spk\n",
    "    + output2_value\n",
    "    + varint_len(output2_spk)\n",
    "    + output2_spk\n",
    ")\n",
    "\n",
    "# LOCKTIME\n",
    "locktime = bytes.fromhex(\"0000 0000\")\n",
    "\n",
    "unsigned_tx = (\n",
    "    version\n",
    "    + input_count\n",
    "    + inputs\n",
    "    + output_count\n",
    "    + outputs\n",
    "    + locktime\n",
    ")\n",
    "\n",
    "# print(unsigned_tx.hex())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Replace the scriptSig with the input's scriptPubKey. You can either code this yourself or take it directly from the decoded transaction. The `\"hex\"` field of the output we are spending from can be used directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Solution - replace the scriptPubKey for creating the sigHash\n",
    "input_spk = bytes.fromhex(\"76a914531260aa2a199e228c537dfa42c82bea2c7c1f4d88ac\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Solution - creating the sigHash and signing the transaction\n",
    "inputs = (\n",
    "    txid\n",
    "    + index\n",
    "    + varint_len(input_spk)\n",
    "    + input_spk\n",
    "    + sequence\n",
    ")\n",
    "\n",
    "tx_to_sign = (\n",
    "    version\n",
    "    + input_count\n",
    "    + inputs\n",
    "    + output_count\n",
    "    + outputs\n",
    "    + locktime\n",
    ")\n",
    "\n",
    "\n",
    "# Append the sighash flag to the transaction\n",
    "sighash_flag = bytes.fromhex(\"0100 0000\") # SIGHASH_ALL\n",
    "sighash_preimage = tx_to_sign + sighash_flag\n",
    "\n",
    "# Create sigHash to be signed\n",
    "sighash = hash256(sighash_preimage)\n",
    "\n",
    "# Sign the sigHash with the input private key\n",
    "signing_key = ecdsa.SigningKey.from_string(change_privkey, curve=ecdsa.SECP256k1) \n",
    "signature = signing_key.sign_digest(sighash, sigencode=ecdsa.util.sigencode_der_canonize)\n",
    "\n",
    "# Append SIGHASH_ALL to the signature\n",
    "signature = signature + bytes.fromhex(\"01\")\n",
    "\n",
    "# Signature\n",
    "sig_script_signed = (\n",
    "    pushbytes(signature)\n",
    "    + pushbytes(change_pubkey)\n",
    ")\n",
    "\n",
    "# tx_in with our new sigScript containing the signature we just created\n",
    "inputs_signed = (\n",
    "    txid\n",
    "    + index\n",
    "    + varint_len(sig_script_signed)\n",
    "    + sig_script_signed\n",
    "    + sequence\n",
    ")\n",
    "\n",
    "# the final signed transaction\n",
    "signed_tx = (\n",
    "    version\n",
    "    + input_count\n",
    "    + inputs_signed\n",
    "    + output_count\n",
    "    + outputs\n",
    "    + locktime\n",
    ")\n",
    "\n",
    "# print(\"signed transaction: \",signed_tx.hex())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'c4ed94e06172f08e7508e419e2dfa1060b83232bdc47245d408981ea3723c96d'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Try broadcasting the transaction to see if it works!\n",
    "node.sendrawtransaction(signed_tx.hex())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022-12-06T21:11:28.582000Z TestFramework (INFO): Stopping nodes\n",
      "2022-12-06T21:11:29.059000Z TestFramework (INFO): Cleaning up /var/folders/r5/yk8yg2xs1gs8xzkn5l8vr72w0000gn/T/bitcoin_func_test_zlfn_arb on exit\n",
      "2022-12-06T21:11:29.059000Z TestFramework (INFO): Tests successful\n"
     ]
    }
   ],
   "source": [
    "# stop bitcoin core\n",
    "test.shutdown()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}