Skip to content

Commit

Permalink
Merge pull request #109 from bitromortac/2112-rebalance-overhaul
Browse files Browse the repository at this point in the history 
rebalance overhaul: internal pathfinding, economic counterparties
  • Loading branch information
@bitromortac
bitromortac authored Feb 22, 2022
2 parents 967f041 + cf9d88e commit 50a8997
Show file tree
Hide file tree
Showing 25 changed files with 1,898 additions and 1,358 deletions.
50 changes: 15 additions & 35 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ lndmanage is a command line tool for advanced channel management of an
* Rebalancing command [```rebalance```](#channel-rebalancing)
* different rebalancing strategies can be chosen
* a target 'balancedness' can be specified (e.g. to empty the channel)
* Circular self-payments ```circle```
* Recommendation of good nodes [```recommend-nodes```](#channel-opening-strategies)
* Batched channel opening [```openchannels```](#batched-channel-opening)
* Support of [```lncli```](#lncli-support)
Expand All @@ -45,7 +44,6 @@ positional arguments:
listchannels lists channels with extended information [see also subcommands with -h]
listpeers lists peers with extended information
rebalance rebalance a channel
circle circular self-payment
recommend-nodes recommends nodes [see also subcommands with -h]
report displays reports of activity on the node
info displays info on channels and nodes
Expand Down Expand Up @@ -141,45 +139,27 @@ Forwardings:
```

## Channel Rebalancing
The workflow for rebalancing a channel goes as follows:

* take a look at all your unbalanced channels with:

```$ lndmanage listchannels rebalance```

The output will look like:
```
-------- Description --------
cid channel id
ub unbalancedness [-1 ... 1] (0 is 50:50 balanced)
cap channel capacity [sat]
lb local balance [sat]
rb remote balance [sat]
pbf peer base fee [msat]
pfr peer fee rate
annotation channel annotation
alias alias
-------- Channels --------
cid ub cap lb rb pbf pfr alias
xxxxxxxxxxxxxxxxxx -0.78 1000000 888861 99480 10 0.000200 abc
xxxxxxxxxxxxxxxxxx -0.63 1000000 814537 173768 300 0.000010 def
xxxxxxxxxxxxxxxxxx 0.55 2000000 450792 1540038 35 0.000002 ghi
xxxxxxxxxxxxxxxxxx 0.59 400000 81971 306335 400 0.000101 jkl
...
```
Channels with depleted liquidity can be rebalanced with channels of saturated liquidity by circular self-payments, that is, a payment leaves from the saturated channel and arrives in the depleted channel. This operation has the cost of paying routing fees. Another often overlooked cost is *state accumulation*. Rebalancing is a time consuming process where many attemps are needed, because one is often trying to counter the natural flow within the network. This can lead to enlarged databases and therefore overall slower operations and strain on the network, which is why rebalancing should be used sparingly!

Which channels could be rebalanced? Typically, we want to rebalance channels that give us high fee earning expecations [to see those channels, see `$ lndmanage listchannels forwardings`, fields `fo/w` (fees earned per week) and `fi/w` (fees earned by other channels due to incoming payments per week)]. We want to increase local balance in channels with high `fo/w` and decrease local balance in channels with high `fi/w`. Another use case is to increase/decrease local balance for new channels to see whether there's demand for forwarding. Otherwise it is not helpful to perfectly rebalance a channel 50:50 in most cases.

The workflow to rebalance a channel is as follows:
* ideally your channels have meaningful fee rates set (see `$ lndmanage update-fees`)
* the counterparty rebalance channels are chosen according to the expected earnings calculated from the fee rate that is set for the to-be-rebalanced channel, i.e., a channel with a low fee rate will send its liquidity to channels with high fee rates, such that this capital can earn fees in the future and vice-versa
* take a look at all your channels with forwarding activity and select a channel with high `fo/w` or `fi/w`, or high absolute values of `ub`

* the ```ub``` field tells you how unbalanced your channel is
and in which direction
* take a channel_id from the list you wish
to rebalance (target is a 50:50 balance)
and in which direction (negative means that the channel has mostly local balance - you may want to decrease the local balance)
* take a `channel_id` or `node_id` you wish to rebalance
* decide on whether you want to increase (positive `--amount`) or decrease your local balance (negative `--amount`), chose small amounts (< 1000000 sat) for better success
* if `--amount` is not set, the sign of liquidity change is determined automatically
* do a dry run to see what's waiting for you

```$ lndmanage rebalance --max-fee-sat 20 --max-fee-rate 0.00001 channel_id```
```$ lndmanage rebalance channel_id/node_id```

* read the output and if everything looks well,
then run with the ```--reckless``` flag
* in order to increase the success probability of your rebalancing you
can try to do it in smaller chunks, which can be set by the flag
`--chunksize 0.5` (in this example only half the amounts are used)
* if you want to disable the fee rate selection of counterparty channels to increase success probability, run with the `--force` flag

## Forwarding Information
A more sophisticated way to see if funds have to be reallocated is to
Expand Down
24 changes: 14 additions & 10 deletions lndmanage/lib/exceptions.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,23 +22,15 @@ class RebalanceFailure(Exception):
pass


class RoutesExhausted(RebalanceFailure):
pass


class RebalanceCandidatesExhausted(RebalanceFailure):
pass


class NoRebalanceCandidates(RebalanceFailure):
pass


class RebalancingTrialsExhausted(RebalanceFailure):
class NotEconomic(RebalanceFailure):
pass


class MultichannelInboundRebalanceFailure(RebalanceFailure):
class RebalancingTrialsExhausted(RebalanceFailure):
pass


Expand Down Expand Up @@ -76,6 +68,12 @@ def __init__(self, payment):
super().__init__()


class TemporaryNodeFailure(PaymentFailure):
def __init__(self, payment):
self.payment = payment
super().__init__()


class UnknownNextPeer(PaymentFailure):
def __init__(self, payment):
self.payment = payment
Expand All @@ -88,6 +86,12 @@ def __init__(self, payment):
super().__init__()


class IncorrectCLTVExpiry(PaymentFailure):
def __init__(self, payment):
self.payment = payment
super().__init__()


class ChannelDisabled(PaymentFailure):
def __init__(self, payment):
self.payment = payment
Expand Down
10 changes: 6 additions & 4 deletions lndmanage/lib/forwardings.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

from lndmanage.lib.data_types import NodeProperties
from lndmanage.lib.node import LndNode
from lndmanage.lib.ln_utilities import channel_unbalancedness_and_commit_fee
from lndmanage.lib.ln_utilities import local_balance_to_unbalancedness
from lndmanage import settings

logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -44,7 +44,7 @@ def initialize_forwarding_stats(self, time_start: float, time_end: float):
:param time_start: time interval start, unix timestamp
:param time_end: time interval end, unix timestamp
"""
channel_id_to_node_id = self.node.get_channel_id_to_node_id()
channel_id_to_node_id = self.node.channel_id_to_node_id()
self.channel_forwarding_stats = defaultdict(ForwardingStatistics)
self.node_forwarding_stats = defaultdict(ForwardingStatistics)

Expand Down Expand Up @@ -565,7 +565,7 @@ def get_node_properites(
f"Time interval (between first and last forwarding) is "
f"{forwarding_analyzer.max_time_interval_days:6.2f} days."
)
channel_id_to_node_id = node.get_channel_id_to_node_id(open_only=True)
channel_id_to_node_id = node.channel_id_to_node_id(open_only=True)
node_ids_with_open_channels = {nid for nid in channel_id_to_node_id.values()}
open_channels = node.get_open_channels()

Expand Down Expand Up @@ -641,7 +641,7 @@ def get_node_properites(
"max_public_capacity": max(properties.public_capacities)
if properties.public_capacities
else 0,
"unbalancedness": channel_unbalancedness_and_commit_fee(
"unbalancedness": local_balance_to_unbalancedness(
local_balance, capacity, 0, False
)[0],
}
Expand Down Expand Up @@ -688,6 +688,8 @@ def get_node_properites(
except ZeroDivisionError:
node_properties_forwardings[node_id]["fees_in_per_week"] = float("nan")

# TODO: unify with information from liquidity hints

return node_properties_forwardings


Expand Down
57 changes: 13 additions & 44 deletions lndmanage/lib/info.py
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
import re
import datetime
from typing import Tuple
from typing import Dict

from lndmanage.lib.network_info import NetworkAnalysis
from lndmanage.lib import ln_utilities
from lndmanage.lib.ln_utilities import (
parse_nodeid_channelid,
convert_channel_id_to_short_channel_id,
height_to_timestamp
)
from lndmanage import settings

import logging
Expand Down Expand Up @@ -64,7 +67,7 @@ def parse_and_print(self, info):

# analyzer = NetworkAnalysis(self.node)
try:
channel_id, node_pub_key = self.parse(info)
channel_id, node_pub_key = parse_nodeid_channelid(info)
except ValueError:
logger.info("Info didn't represent neither a channel nor a node.")
return
Expand All @@ -84,9 +87,9 @@ def parse_and_print(self, info):
general_info['node2_alias'] = \
self.node.network.node_alias(general_info['node2_pub'])
general_info['blockheight'] = \
ln_utilities.convert_channel_id_to_short_channel_id(
convert_channel_id_to_short_channel_id(
channel_id)[0]
general_info['open_timestamp'] = ln_utilities.height_to_timestamp(
general_info['open_timestamp'] = height_to_timestamp(
self.node, general_info['blockheight'])

# TODO: if it's our channel, add extra info
Expand All @@ -106,46 +109,11 @@ def parse_and_print(self, info):

self.print_node_info(general_info)

def parse(self, info: str) -> Tuple[int, str]:
"""Parse whether info contains a channel id or node public key and hand
it back. If no info could be extracted, raise a ValueError.
:return: channel_id, node_pub_key
"""
exp_channel_id = re.compile("^[0-9]{13,20}$")
exp_short_channel_id = re.compile("^[0-9]{6}x[0-9]{3}x[0-9]$")
exp_chan_point = re.compile("^[a-z0-9]{64}:[0-9]$")
exp_node_id = re.compile("^[a-z0-9]{66}$")

channel_id = None
node_pub_key = None

# prepare input string info
if exp_channel_id.match(info) is not None:
logger.debug("Info represents channel id.")
channel_id = int(info)
elif exp_short_channel_id.match(info) is not None:
logger.debug("Info represents short channel id.")
# TODO: convert short channel id to channel id
channel_id = 0
elif exp_chan_point.match(info) is not None:
# TODO: convert chan point to channel id
logger.debug("Info represents short channel id.")
channel_id = 0
elif exp_node_id.match(info) is not None:
logger.debug("Info represents node public key.")
node_pub_key = info
else:
raise ValueError("Info string doesn't match any pattern.")

return channel_id, node_pub_key

def print_channel_info(self, general_info):
def print_channel_info(self, general_info: Dict):
"""
Prints the channel info with peer information.
:param general_info: information about the channel in the public graph
:type general_info: dict
"""

logger.info("-------- Channel info --------")
Expand Down Expand Up @@ -189,8 +157,9 @@ def print_channel_info(self, general_info):
logger.info(f"{general_info['node1_alias']:^{COL_WIDTH}} | "
f"{general_info['node2_alias']:^{COL_WIDTH}}")

np1 = general_info['node1_policy']
np2 = general_info['node2_policy']
policies = general_info['policies']
np1 = policies[general_info['node1_pub'] > general_info['node2_pub']]
np2 = policies[general_info['node2_pub'] > general_info['node1_pub']]
last_update_1 = np1['last_update']
last_update_2 = np2['last_update']

Expand Down
Loading

0 comments on commit 50a8997

Please sign in to comment.