Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry untractable packages #2203

Merged
merged 2 commits into from
Apr 20, 2023

Conversation

wpaulino
Copy link
Contributor

Untractable packages are those which cannot have their fees updated once signed, hence why they weren't retried. There's no harm in retrying these packages by simply re-broadcasting them though, as the fee market could have spontaneously spiked when we first broadcast it, leading to our transaction not propagating throughout node mempools unless broadcast manually.

@wpaulino wpaulino added this to the 0.0.115 milestone Apr 19, 2023
@TheBlueMatt
Copy link
Collaborator

Given how much pressure this puts on the user's broadcaster working the first time, would be really nice to land this for 115, yea :(.

@jkczyz jkczyz self-requested a review April 19, 2023 20:11
@codecov-commenter
Copy link

codecov-commenter commented Apr 19, 2023

Codecov Report

Patch coverage: 98.31% and project coverage change: -0.02 ⚠️

Comparison is base (2e15df7) 91.37% compared to head (a3b416a) 91.35%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2203      +/-   ##
==========================================
- Coverage   91.37%   91.35%   -0.02%     
==========================================
  Files         102      102              
  Lines       50748    50832      +84     
  Branches    50748    50832      +84     
==========================================
+ Hits        46369    46440      +71     
- Misses       4379     4392      +13     
Impacted Files Coverage Δ
lightning/src/chain/onchaintx.rs 92.67% <85.71%> (-0.37%) ⬇️
lightning/src/ln/functional_tests.rs 98.25% <98.27%> (-0.02%) ⬇️
lightning/src/ln/monitor_tests.rs 97.77% <98.52%> (-0.51%) ⬇️
lightning/src/chain/package.rs 92.55% <100.00%> (+0.02%) ⬆️
lightning/src/ln/functional_test_utils.rs 92.73% <100.00%> (+0.03%) ⬆️
lightning/src/ln/payment_tests.rs 97.57% <100.00%> (+<0.01%) ⬆️
lightning/src/ln/reorg_tests.rs 100.00% <100.00%> (ø)
lightning/src/util/test_utils.rs 76.39% <100.00%> (+0.36%) ⬆️

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

TheBlueMatt
TheBlueMatt previously approved these changes Apr 19, 2023
Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

lightning/src/ln/reorg_tests.rs Outdated Show resolved Hide resolved
Untractable packages are those which cannot have their fees updated once
signed, hence why they weren't retried. There's no harm in retrying
these packages by simply re-broadcasting them though, as the fee market
could have spontaneously spiked when we first broadcast it, leading to
our transaction not propagating throughout node mempools unless
broadcast manually.
Now that we leverage a package's `height_timer` even for untractable
packages, there's no need to have it be an `Option` anymore. We aim to
not break compatibility by keeping the deserialization of such as an
`option`, and use the package's `height_original` when not present. This
allows us to retry packages from older `ChannelMonitor` versions that
have had a failed initial package broadcast.
@wpaulino
Copy link
Contributor Author

$ git diff -U1 e93a73b a3b416a3
diff --git a/lightning/src/ln/reorg_tests.rs b/lightning/src/ln/reorg_tests.rs
index 022b3da5..14b06457 100644
--- a/lightning/src/ln/reorg_tests.rs
+++ b/lightning/src/ln/reorg_tests.rs
@@ -11,3 +11,3 @@

-use crate::chain::channelmonitor::ANTI_REORG_DELAY;
+use crate::chain::channelmonitor::{ANTI_REORG_DELAY, LATENCY_GRACE_PERIOD_BLOCKS};
 use crate::chain::transaction::OutPoint;
@@ -469,4 +469,3 @@ fn test_set_outpoints_partial_claiming() {
        // Connect blocks on node B
-       mine_transaction(&nodes[1], &remote_txn[0]);
-       connect_blocks(&nodes[1], 134);
+       connect_blocks(&nodes[1], TEST_FINAL_CLTV + LATENCY_GRACE_PERIOD_BLOCKS + 1);
        check_closed_broadcast!(nodes[1], true);
@@ -477,9 +476,10 @@ fn test_set_outpoints_partial_claiming() {
                let mut node_txn = nodes[1].tx_broadcaster.unique_txn_broadcast();
-               assert_eq!(node_txn.len(), 2);
-               check_spends!(node_txn[0], remote_txn[0]);
-               check_spends!(node_txn[1], remote_txn[0]);
-               assert_eq!(node_txn[0].input.len(), 1);
+               assert_eq!(node_txn.len(), 3);
+               check_spends!(node_txn[0], chan.3);
+               check_spends!(node_txn[1], node_txn[0]);
+               check_spends!(node_txn[2], node_txn[0]);
                assert_eq!(node_txn[1].input.len(), 1);
-               assert_ne!(node_txn[0].input[0].previous_output, node_txn[1].input[0].previous_output);
-               node_txn.remove(0)
+               assert_eq!(node_txn[2].input.len(), 1);
+               assert_ne!(node_txn[1].input[0].previous_output, node_txn[2].input[0].previous_output);
+               node_txn.remove(1)
        };

@TheBlueMatt TheBlueMatt merged commit 8d50c91 into lightningdevkit:main Apr 20, 2023
@wpaulino wpaulino deleted the retry-untractable-packages branch April 20, 2023 02:23
k0k0ne pushed a commit to bitlightlabs/rust-lightning that referenced this pull request Sep 30, 2024
0.0.115 - Apr 24, 2023 - "Rebroadcast the Bugfixes"

API Updates
===========

 * The MSRV of the main LDK crates has been increased to 1.48 (lightningdevkit#2107).
 * Attempting to claim an un-expired payment on a channel which has closed no
   longer fails. The expiry time of payments is exposed via
   `PaymentClaimable::claim_deadline` (lightningdevkit#2148).
 * `payment_metadata` is now supported in `Invoice` deserialization, sending,
   and receiving (via a new `RecipientOnionFields` struct) (lightningdevkit#2139, lightningdevkit#2127).
 * `Event::PaymentFailed` now exposes a failure reason (lightningdevkit#2142).
 * BOLT12 messages now support stateless generation and validation (lightningdevkit#1989).
 * The `NetworkGraph` is now pruned of stale data after RGS processing (lightningdevkit#2161).
 * Max inbound HTLCs in-flight can be changed in the handshake config (lightningdevkit#2138).
 * `lightning-transaction-sync` feature `esplora-async-https` was added (lightningdevkit#2085).
 * A `ChannelPending` event is now emitted after the initial handshake (lightningdevkit#2098).
 * `PaymentForwarded::outbound_amount_forwarded_msat` was added (lightningdevkit#2136).
 * `ChannelManager::list_channels_by_counterparty` was added (lightningdevkit#2079).
 * `ChannelDetails::feerate_sat_per_1000_weight` was added (lightningdevkit#2094).
 * `Invoice::fallback_addresses` was added to fetch `bitcoin` types (lightningdevkit#2023).
 * The offer/refund description is now exposed in `Invoice{,Request}` (lightningdevkit#2206).

Backwards Compatibility
=======================

 * Payments sent with the legacy `*_with_route` methods on LDK 0.0.115+ will no
   longer be retryable via the LDK 0.0.114- `retry_payment` method (lightningdevkit#2139).
 * `Event::PaymentPathFailed::retry` was removed and will always be `None` for
    payments initiated on 0.0.115 which fail on an earlier version (lightningdevkit#2063).
 * `Route`s and `PaymentParameters` with blinded path information will not be
   readable on prior versions of LDK. Such objects are not currently constructed
   by LDK, but may be when processing BOLT12 data in a coming release (lightningdevkit#2146).
 * Providing `ChannelMonitorUpdate`s generated by LDK 0.0.115 to a
   `ChannelMonitor` on 0.0.114 or before may panic (lightningdevkit#2059). Note that this is
   in general unsupported, and included here only for completeness.

Bug Fixes
=========

 * Fixed a case where `process_events_async` may `poll` a `Future` which has
   already completed (lightningdevkit#2081).
 * Fixed deserialization of `u16` arrays. This bug may have previously corrupted
   the historical buckets in a `ProbabilisticScorer`. Users relying on the
   historical buckets may wish to wipe their scorer on upgrade to remove corrupt
   data rather than waiting on it to decay (lightningdevkit#2191).
 * The `process_events_async` task is now `Send` and can thus be polled on a
   multi-threaded runtime (lightningdevkit#2199).
 * Fixed a missing macro export causing
   `impl_writeable_tlv_based_enum{,_upgradable}` calls to not compile (lightningdevkit#2091).
 * Fixed compilation of `lightning-invoice` with both `no-std` and serde (lightningdevkit#2187)
 * Fix an issue where the `background-processor` would not wake when a
   `ChannelMonitorUpdate` completed asynchronously, causing delays (lightningdevkit#2090).
 * Fix an issue where `process_events_async` would exit immediately (lightningdevkit#2145).
 * `Router` calls from the `ChannelManager` now call `find_route_with_id` rather
   than `find_route`, as was intended and described in the API (lightningdevkit#2092).
 * Ensure `process_events_async` always exits if any sleep future returns true,
   not just if all sleep futures repeatedly return true (lightningdevkit#2145).
 * `channel_update` messages no longer set the disable bit unless the peer has
   been disconnected for some time. This should resolve cases where channels are
   disabled for extended periods of time (lightningdevkit#2198).
 * We no longer remove CLN nodes from the network graph for violating the BOLT
   spec in some cases after failing to pay through them (lightningdevkit#2220).
 * Fixed a debug assertion which may panic under heavy load (lightningdevkit#2172).
 * `CounterpartyForceClosed::peer_msg` is now wrapped in UntrustedString (lightningdevkit#2114)
 * Fixed a potential deadlock in `funding_transaction_generated` (lightningdevkit#2158).

Security
========

 * Transaction re-broadcasting is now substantially more aggressive, including a
   new regular rebroadcast feature called on a timer from the
   `background-processor` or from `ChainMonitor::rebroadcast_pending_claims`.
   This should substantially increase transaction confirmation reliability
   without relying on downstream `TransactionBroadcaster` implementations for
   rebroadcasting (lightningdevkit#2203, lightningdevkit#2205, lightningdevkit#2208).
 * Implemented the changes from BOLT PRs lightningdevkit#1031, lightningdevkit#1032, and lightningdevkit#1040 which resolve a
   privacy vulnerability which allows an intermediate node on the path to
   discover the final destination for a payment (lightningdevkit#2062).

In total, this release features 110 files changed, 11928 insertions, 6368
deletions in 215 commits from 21 authors, in alphabetical order:
 * Advait
 * Alan Cohen
 * Alec Chen
 * Allan Douglas R. de Oliveira
 * Arik Sosman
 * Elias Rohrer
 * Evan Feenstra
 * Jeffrey Czyz
 * John Cantrell
 * Lucas Soriano del Pino
 * Marc Tyndel
 * Matt Corallo
 * Paul Miller
 * Steven
 * Steven Williamson
 * Steven Zhao
 * Tony Giorgio
 * Valentine Wallace
 * Wilmer Paulino
 * benthecarman
 * munjesi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants