From 7192c86b30621995382d6651f3d9e457351f171e Mon Sep 17 00:00:00 2001 From: Yusef Napora Date: Mon, 8 Jul 2019 10:36:03 -0400 Subject: [PATCH 01/17] draft of addressing spec --- addressing/README.md | 329 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 329 insertions(+) create mode 100644 addressing/README.md diff --git a/addressing/README.md b/addressing/README.md new file mode 100644 index 000000000..88ce1a443 --- /dev/null +++ b/addressing/README.md @@ -0,0 +1,329 @@ +# Addressing in libp2p +> How network addresses are encoded and used in libp2p + +| Lifecycle Stage | Maturity | Status | Latest Revision | +|-----------------|---------------|--------|-----------------| +| 1A | Working Draft | Active | r0, 2019-05-27 | + + +Authors: [@yusefnapora] + +Interest Group: TBD + +[@yusefnapora]: https://github.com/yusefnapora + +See the [lifecycle document][lifecycle-spec] for context about maturity level +and spec status. + +[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md + +## Table of Contents + +- [Addressing in libp2p](#addressing-in-libp2p) + - [Table of Contents](#table-of-contents) + - [Overview](#overview) + - [multiaddr in libp2p](#multiaddr-in-libp2p) + - [multiaddr basics](#multiaddr-basics) + - [Composing multiaddrs](#composing-multiaddrs) + - [Encapsulation](#encapsulation) + - [Decapsulation](#decapsulation) + - [The p2p multiaddr](#the-p2p-multiaddr) + - [Historical Note: the `ipfs` multiaddr Protocol](#historical-note-the-ipfs-multiaddr-protocol) + - [Transport multiaddrs](#transport-multiaddrs) + - [IP and Name Resolution](#ip-and-name-resolution) + - [dnsaddr Links](#dnsaddr-links) + - [TCP](#tcp) + - [WebSockets](#websockets) + - [QUIC](#quic) + - [`p2p-circuit` Relay Addresses](#p2p-circuit-relay-addresses) + +## Overview + +libp2p makes a distinction between a peer's **identity** and its **location**. +A peer's identity is stable, verifiable, and valid for the entire lifetime of +the peer (whatever that may be for a given application). Peer identities are +derived from public keys as described in the [peer id spec][peer-id-spec]. + +On a particular network, at a specific point in time, a peer may have one or +more locations, which can be represented using addresses. For example, I may be +reachable via the global IPv4 address of 7.7.7.7 on TCP port 1234. + +In a system that only supported TCP/IP or UDP over IP, we could easily write our +addresses with the familiar `:` notation and store them as tuples of +address and port. However, libp2p was designed to be transport agnostic, which +means that we can't assume that we'll even be using an IP-backed network at all. + +To support a growing set of transport protocols without special-casing each +addressing scheme, libp2p uses [multiaddr][multiaddr-repo] to encode network +addresses for all supported transport protocols. + +This document covers [how multiaddr is used in libp2p](#multiaddr-in-libp2p). +For more information on other use cases, or to find links to multiaddr +implementations in various languages, see the [mulitaddr +repository][multiaddr-repo]. + +## multiaddr in libp2p + +multiaddr is used throughout libp2p for encoding network addresses, and +addresses are generally exchanged over the wire as binary-encoded multiaddrs in +libp2p's core protocols. + +When exchanging addresses, peers send a multiaddr containing both their network +address and peer id, as described in [the section on the `p2p` +multiaddr](#the-p2p-multiaddr). + +### multiaddr basics + +A multiaddr generally represents a path through a stack of successively +"higher-level" protocols that can be traversed to some destination. + +For example, the `/ip4/7.7.7.7/tcp/1234` multiaddr starts with `ip4`, which is +the lowest-level protocol that requires an address. The `tcp` protocol is +encapsulated within `ip4`, so it comes next. + +The multiaddr above consists of two components, the `/ip4/7.7.7.7` component, +and the `/tcp/1234` component. It's not possible to split either one further; +`/ip4` alone is an invalid multiaddr, because the `ip4` protocol was defined to +require a 32 bit address. Similarly, `tcp` requires a 16 bit port number. + +Although we referred to `/ip4/7.7.7.7` and `/tcp/1234` as "components" of a +larger TCP/IP address, each is actually a valid multiaddr according to the +multiaddr spec. However, not every valid multiaddr describes a complete path +through the network. As we've seen, even a simple TCP/IP connection requires +composing two multiaddrs into one. See the section on [composing +multiaddrs](#composing-multiaddrs) for information on how multiaddrs can be +combined, and the [Transport multiaddrs section](#transport-multiaddrs) for the +combinations that describe valid transport addresses. + +The [multiaddr protocol table][multiaddr-proto-table] contains all currently +defined protocols and the length of their address components. + +Some multiaddr protocols do not require any additional addressing information. +For example, WebSockets are by definition encapsulated within TCP/IP, so the +`/ws` multiaddr protocol is encapsulated within a TCP/IP multiaddr: +`/ip4/7.7.7.7/tcp/1234/ws`. This address is composed of three distinct +multiaddrs, `/ip4/7.7.7.7`, `/tcp/1234`, and `/ws`. + +### Composing multiaddrs + +As shown above, protocol addresses can be composed within a multiaddr in a way +that mirrors the composition of protocols within a networking stack. + +The terms generally used to describe composition of multiaddrs are +"encapsulation" and "decapsulation", and they essentially refer to adding and +removing protocol components from a multiaddr, respectively. + +#### Encapsulation + +A protocol is said to be "encapsulated within" another protocol when data from +an "inner" protocol is wrapped by another "outer" protocol, often by re-framing +the data from the inner protocol into the type of packets, frames or datagrams +used by the outer protocol. + +Some examples of protocol encapsulation are HTTP requests encapsulated within +TCP/IP streams, or TCP segments themselves encapsulated within IP datagrams. + +The multiaddr format was designed so that addresses encapsulate each other in +the same manner as the protocols that they describe. The result is an address +that begins with the "outermost" layer of the network stack and works +progressively "inward". For example, in the address `/ip4/7.7.7.7/tcp/80/ws`, +the outermost protocol is IPv4, which encapsulates TCP streams, which in turn +encapsulate WebSockets. + +All multiaddr implementations provide an `Encapsulate` method, which combines +two multiaddrs into a composite. For example, `/ip4/7.7.7.7` can encapsulate +`/tcp/42` to become `/ip4/7.7.7.7/tcp/42`. + +Note that no "sanity checking" is performed when encapsulating multiaddrs, and +it is possible to create valid but useless multiaddrs like `/tcp/42/udp/42` +through encapsulation. + +#### Decapsulation + +Decapsulation takes a composite multiaddr and removes an "inner" +multiaddr from it, returning the result. + +For example, if we start with `/ip4/7.7.7.7/tcp/1234/ws` and decapsulate `/ws`, +the result is `/ip4/7.7.7.7/tcp/1234`. + +It's important to note that decapsulation returns the original multiaddr up +to the last occurrence of the decapsulated multiaddr. This may remove more +than just the decapsulated component itself if there are more protocols +encapsulated within it. Using our example above, decapsulating either +`/tcp/1234/ws` _or_ `/tcp/1234` from `/ip4/7.7.7.7/tcp/ws` will result in +`/ip4/7.7.7.7`. This is unsurprising if you consider the utility of the +`/ip4/7.7.7.7/ws` address that would result from simply removing the `tcp` +component. + +### The p2p multiaddr + +libp2p defines the `p2p` multiaddr protocol, whose address component is the +[peer id][peer-id-spec] of a libp2p peer. The text representation of a `p2p` +multiaddr looks like this: + +``` +/p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N +``` + +Where `QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N` is the base58-encoded +multihash of a peer's public key, also known as their peer id. + +By itself, a `p2p` address does not give you enough addressing information to +locate a peer on the network; it is not a transport address. However, like the +`ws` protocol for WebSockets, a `p2p` address can be [encapsulated +within](#encapsulation) another multiaddr. + +For example, the above `p2p` address can be combined with the transport address +on which the node is listening: + +``` +/ip4/7.7.7.7/tcp/1234/p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N +``` + +This combination of transport address plus `p2p` address is the format in which +peers exchange addresses over the wire in the [identify protocol][identify-spec] +and other core libp2p protocols. + +#### Historical Note: the `ipfs` multiaddr Protocol + +The `p2p` multiaddr protocol was originally named `ipfs`, and may be printed as +`/ipfs/` instead of `/p2p/` depending on the implementation in +use. Both names resolve to the same protocol code, and they are equivalent in the +binary form. + + +## Transport multiaddrs + +Because multiaddr is an open and extensible format, it's not possible to +guarantee that any valid multiaddr is semantically meaningful or usable in a +particular network. For example, the `/tcp/42` multiaddr, while valid, is not +useful on its own as a locator. + +This section covers the types of multiaddr supported by libp2p transports. It's +possible that this section will go out of date as new transport modules are +developed, at which point pull-requests to update this document will be greatly +appreciated. + +### IP and Name Resolution + +Most libp2p transports use the IP protocol as a foundational layer, and as a +result, most transport multiaddrs will begin with a component that represents an +IPv4 or IPv6 address. + +This may be an actual address, such as `/ip4/7.7.7.7` or +`/ip6/fe80::883:a581:fff1:833`, or it could be something that resolves to an IP +address, like a domain name. + +libp2p will attempt to resolve "name-based" addresses into IP addresses. The +current [multiaddr protocol table][multiaddr-proto-table] defines four +resolvable or "name-based" protocols: + +| protocol | description | +|-----------|--------------------------------------------------------------------| +| `dns` | Resolves DNS A and AAAA records into both IPv4 and IPv6 addresses. | +| `dns4` | Resolves DNS A records into IPv4 addresses. | +| `dns6` | Resolves DNS AAAA records into IPv6 addresses. | +| `dnsaddr` | Resolves multiaddrs from a special TXT record. | + + +When the `/dns` protocol is used, the lookup may result in both IPv4 and IPv6 +addresses, in which case IPv6 will be preferred. To explicitly resolve to IPv4 +or IPv6 addresses, use the `/dns4` or `/dns6` protocols, respectively. + +Note that in some restricted environments, such as inside a web browser, libp2p +may not have access to the resolved IP addresses at all, in which case the +runtime will determine what IP version is used. + +When a name-based multiaddr encapsulates another multiaddr, only the name-based +component is affected by the lookup process. For example, if `example.com` +resolves to `1.2.3.4`, libp2p will resolve the address +`/dns4/example.com/tcp/42` to `/ip4/1.2.3.4/tcp/42`. + +#### dnsaddr Links + +A libp2p-specific DNS-backed format, `/dnsaddr` resolves addresses from a `TXT` +record associated with the `_dnsaddr` subdomain of a given domain. + +For example, resolving `/dnsaddr/libp2p.io` will perform a `TXT` lookup for +`_dnsaddr.libp2p.io`. If the result contains an entry of the form +`dnsaddr=`, the embedded multiaddr will be parsed and used. + +The `dnsaddr` lookup serves a similar purpose to a standard A-record DNS lookup, +however there are differences that can be important for some use cases. The most +significant is that the `dnsaddr` entry contains a full multiaddr, which may +include a port number or other information that an A-record lacks, and it may +even specify a non-IP transport. Also, there are cases in which the A-record +already serves a useful purpose; using `dnsaddr` allows a second "namespace" for +libp2p registrations. + +### TCP + +The libp2p TCP transport is supported in all implementations and can be used +wherever TCP/IP sockets are accessible. + +Addresses for the TCP transport are of the form `/tcp/`, +where `` is a multiaddr that resolves to an IP address, as +described in the [IP and Name Resolution section](#ip-and-name-resolution). +The `` argument must be a 16-bit unsigned integer. + +### WebSockets + +WebSocket connections are encapsulated within TCP/IP sockets, and the WebSocket +multiaddr format mirrors this arrangement. + +A libp2p WebSocket multiaddr is of the form `/ws`, where +` is a valid mulitaddr for the TCP transport, as [described +above](#tcp). + +### QUIC + +QUIC sessions are encapsulated within UDP datagrams, and the libp2p QUIC +multiaddr format mirrors this arrangement. + +A libp2p QUIC multiaddr is of the form `/udp//quic`, +where `` is a multiaddr that resolves to an IP address, as +described in the [IP and Name Resolution section](#ip-and-name-resolution). +The `` argument must be a 16-bit unsigned integer. + + +### `p2p-circuit` Relay Addresses + +The libp2p [circuit relay protocol][relay-spec] allows a libp2p peer to relay +traffic between two peers who could otherwise not communicate directly. + +Once a relay connection is established, peers can accept incoming connections +through the relay, using a `p2p-circuit` address. + +Like the `ws` WebSocket multiaddr protocol the `p2p-circuit` multiaddr does not +carry any additional address information. Instead it is composed with two other +multiaddrs to describe a relay circuit. + +A full `p2p-circuit` address that describes a relay circuit is of the form: +`/p2p-circuit/`. + +`` is the full address for the peer relaying the traffic (the +"relay node"), including both the transport address and the `p2p` address +containing the relay node's peer id. + +The details of the transport connection between the relay node and the +destination peer are usually not relevant to other peers in the network, so +`` generally only contains the `p2p` address of the +destination peer. + +A full example would be: + +``` +/ip4/127.0.0.1/tcp/5002/p2p/QmdPU7PfRyKehdrP5A3WqmjyD6bhVpU1mLGKppa2FjGDjZ/p2p-circuit/p2p/QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt +``` + +Here, the destination peer has the peer id +`QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt` and is reachable through a +relay node with peer id `QmdPU7PfRyKehdrP5A3WqmjyD6bhVpU1mLGKppa2FjGDjZ` running +on TCP port 5002 of the IPv4 loopback interface. + + +[peer-id-spec]: ../peer-ids/peer-ids.md +[identify-spec]: ../identify/README.md +[multiaddr-repo]: https://github.com/multiformats/multiaddr +[multiaddr-proto-table]: https://github.com/multiformats/multiaddr/blob/master/protocols.csv +[relay-spec]: ../relay/README.md From a23b40d95215f82627b2cb5710392393ab8cc35f Mon Sep 17 00:00:00 2001 From: Yusef Napora Date: Mon, 22 Jul 2019 14:50:55 -0400 Subject: [PATCH 02/17] add addressing spec to README index --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index a9d84c529..06b8aad63 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,7 @@ expected lifecycle and document formatting. These specs define abstractions and data types that form the "core" of libp2p and are used throughout the system. +- [Addressing][spec_addressing] - Working with addresses in libp2p. - [Connections and Upgrading][spec_connections] - Establishing secure, multiplexed connections between peers, possibly over insecure, single stream transports. - [Peer Ids and Keys][spec_peerids] - Public key types & encodings, peer id calculation, and @@ -112,3 +113,4 @@ you feel an issue isn't the appropriate place for your topic, please join our [spec_peerids]: ./peer-ids/peer-ids.md [spec_connections]: ./connections/README.md [spec_plaintext]: ./plaintext/README.md +[spec_addressing]: ./addressing/README.md From b88752aa7c95d691ac766e1e1feb0512a15f6d63 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 15:49:17 +0200 Subject: [PATCH 03/17] apply suggestions from code review Co-authored-by: raulk Co-authored-by: Marcin Rataj --- addressing/README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 88ce1a443..76bdce0a2 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -55,9 +55,9 @@ means that we can't assume that we'll even be using an IP-backed network at all. To support a growing set of transport protocols without special-casing each addressing scheme, libp2p uses [multiaddr][multiaddr-repo] to encode network -addresses for all supported transport protocols. +addresses for all supported transport protocols, in a self-describing manner. -This document covers [how multiaddr is used in libp2p](#multiaddr-in-libp2p). +This document does not cover the address format ([multiaddr][multiaddr-repo]), but rather [how multiaddr is used in libp2p](#multiaddr-in-libp2p). For details on the former, visit linked spec. For more information on other use cases, or to find links to multiaddr implementations in various languages, see the [mulitaddr repository][multiaddr-repo]. @@ -65,7 +65,7 @@ repository][multiaddr-repo]. ## multiaddr in libp2p multiaddr is used throughout libp2p for encoding network addresses, and -addresses are generally exchanged over the wire as binary-encoded multiaddrs in +addresses are generally exchanged over the wire as binary-encoded multiaddrs in, when addresses need to be shared or exchanged between processes, they are encoded in the binary representation of multiaddr. libp2p's core protocols. When exchanging addresses, peers send a multiaddr containing both their network @@ -88,7 +88,7 @@ require a 32 bit address. Similarly, `tcp` requires a 16 bit port number. Although we referred to `/ip4/7.7.7.7` and `/tcp/1234` as "components" of a larger TCP/IP address, each is actually a valid multiaddr according to the -multiaddr spec. However, not every valid multiaddr describes a complete path +multiaddr spec. However, not every **syntactically valid multiaddr is a functional description of a process in the network**. through the network. As we've seen, even a simple TCP/IP connection requires composing two multiaddrs into one. See the section on [composing multiaddrs](#composing-multiaddrs) for information on how multiaddrs can be @@ -135,7 +135,7 @@ two multiaddrs into a composite. For example, `/ip4/7.7.7.7` can encapsulate `/tcp/42` to become `/ip4/7.7.7.7/tcp/42`. Note that no "sanity checking" is performed when encapsulating multiaddrs, and -it is possible to create valid but useless multiaddrs like `/tcp/42/udp/42` +it is possible to create valid but unsound/dysfunctional multiaddrs like `/tcp/42/udp/42` through encapsulation. #### Decapsulation @@ -187,7 +187,7 @@ and other core libp2p protocols. #### Historical Note: the `ipfs` multiaddr Protocol The `p2p` multiaddr protocol was originally named `ipfs`, and may be printed as -`/ipfs/` instead of `/p2p/` depending on the implementation in +`/ipfs/` instead of `/p2p/` in its string representation depending on the implementation in use. Both names resolve to the same protocol code, and they are equivalent in the binary form. @@ -271,7 +271,7 @@ The `` argument must be a 16-bit unsigned integer. WebSocket connections are encapsulated within TCP/IP sockets, and the WebSocket multiaddr format mirrors this arrangement. -A libp2p WebSocket multiaddr is of the form `/ws`, where +A libp2p WebSocket multiaddr is of the form `/ws` or `/wss` (TLS-encrypted), where ` is a valid mulitaddr for the TCP transport, as [described above](#tcp). @@ -283,7 +283,7 @@ multiaddr format mirrors this arrangement. A libp2p QUIC multiaddr is of the form `/udp//quic`, where `` is a multiaddr that resolves to an IP address, as described in the [IP and Name Resolution section](#ip-and-name-resolution). -The `` argument must be a 16-bit unsigned integer. +The `` argument must be a 16-bit unsigned integer in network byte order. ### `p2p-circuit` Relay Addresses @@ -291,7 +291,7 @@ The `` argument must be a 16-bit unsigned integer. The libp2p [circuit relay protocol][relay-spec] allows a libp2p peer to relay traffic between two peers who could otherwise not communicate directly. -Once a relay connection is established, peers can accept incoming connections +Once a connection to the relay is established, peers can accept incoming connections through the relay, using a `p2p-circuit` address. Like the `ws` WebSocket multiaddr protocol the `p2p-circuit` multiaddr does not From 41e70c28fd003397880f35eed9b6a9c9d1c9dd97 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 20:33:20 +0200 Subject: [PATCH 04/17] replace contentious use of "encapsulation" --- addressing/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 76bdce0a2..8990a84b4 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -78,8 +78,8 @@ A multiaddr generally represents a path through a stack of successively "higher-level" protocols that can be traversed to some destination. For example, the `/ip4/7.7.7.7/tcp/1234` multiaddr starts with `ip4`, which is -the lowest-level protocol that requires an address. The `tcp` protocol is -encapsulated within `ip4`, so it comes next. +the lowest-level protocol that requires an address. The `tcp` protocol runs on +top of `ip4`, so it comes next. The multiaddr above consists of two components, the `/ip4/7.7.7.7` component, and the `/tcp/1234` component. It's not possible to split either one further; From f3df677e4101fe266d1f806c58607790751aa6c9 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 20:37:04 +0200 Subject: [PATCH 05/17] don't call parts of multiaddr multiaddr --- addressing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/addressing/README.md b/addressing/README.md index 8990a84b4..b2580ee07 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -102,7 +102,7 @@ Some multiaddr protocols do not require any additional addressing information. For example, WebSockets are by definition encapsulated within TCP/IP, so the `/ws` multiaddr protocol is encapsulated within a TCP/IP multiaddr: `/ip4/7.7.7.7/tcp/1234/ws`. This address is composed of three distinct -multiaddrs, `/ip4/7.7.7.7`, `/tcp/1234`, and `/ws`. +parts, `/ip4/7.7.7.7`, `/tcp/1234`, and `/ws`. ### Composing multiaddrs From a8ac4da41e5c38fc6b7daa608ca80a4376f785dc Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 20:41:10 +0200 Subject: [PATCH 06/17] add that we're eliminating support for /ipfs/ multiaddrs --- addressing/README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index b2580ee07..b38f33ae4 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -186,10 +186,11 @@ and other core libp2p protocols. #### Historical Note: the `ipfs` multiaddr Protocol -The `p2p` multiaddr protocol was originally named `ipfs`, and may be printed as -`/ipfs/` instead of `/p2p/` in its string representation depending on the implementation in -use. Both names resolve to the same protocol code, and they are equivalent in the -binary form. +The `p2p` multiaddr protocol was originally named `ipfs`, and we've been eliminating +support for the ipfs string representation of this multiaddr component. It may be +printed as `/ipfs/` instead of `/p2p/` in its string representation +depending on the implementation in use. Both names resolve to the same protocol code, +and they are equivalent in the binary form. ## Transport multiaddrs From 31d1a50fdb1d6f106d4cc4fe9c795cf629d59a33 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 20:47:55 +0200 Subject: [PATCH 07/17] dnsaddr might return multiple multiaddrs, add example --- addressing/README.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index b38f33ae4..b123a10fb 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -246,8 +246,20 @@ A libp2p-specific DNS-backed format, `/dnsaddr` resolves addresses from a `TXT` record associated with the `_dnsaddr` subdomain of a given domain. For example, resolving `/dnsaddr/libp2p.io` will perform a `TXT` lookup for -`_dnsaddr.libp2p.io`. If the result contains an entry of the form -`dnsaddr=`, the embedded multiaddr will be parsed and used. +`_dnsaddr.libp2p.io`. If the result contains entries of the form +`dnsaddr=`, the embedded multiaddrs will be parsed and used. + +For example, asking the DNS server for the TXT records of one of the bootstrap +nodes, `ams-2.bootstrap.libp2p.io`, returns the following records: +``` +> dig +short _dnsaddr.ams-2.bootstrap.libp2p.io txt +"dnsaddr=/dns4/ams-2.bootstrap.libp2p.io/tcp/443/wss/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" +"dnsaddr=/ip6/2604:1380:2000:7a00::1/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" +"dnsaddr=/ip4/147.75.83.83/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" +"dnsaddr=/ip6/2604:1380:2000:7a00::1/udp/4001/quic/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" +"dnsaddr=/ip4/147.75.83.83/udp/4001/quic/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" +"dnsaddr=/dns6/ams-2.bootstrap.libp2p.io/tcp/443/wss/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" +``` The `dnsaddr` lookup serves a similar purpose to a standard A-record DNS lookup, however there are differences that can be important for some use cases. The most From 90e852d9aad85cebf9980aa3a543c67667017834 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 20:49:12 +0200 Subject: [PATCH 08/17] apply Raul's suggestion regarding circuit relays --- addressing/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index b123a10fb..713c1e15c 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -301,8 +301,9 @@ The `` argument must be a 16-bit unsigned integer in network byte orde ### `p2p-circuit` Relay Addresses -The libp2p [circuit relay protocol][relay-spec] allows a libp2p peer to relay -traffic between two peers who could otherwise not communicate directly. +The libp2p [circuit relay protocol][relay-spec] allows a libp2p peer A to +communicate with another peer B via a third party C. This is useful for +circumstances where A and B would be unable to communicate directly. Once a connection to the relay is established, peers can accept incoming connections through the relay, using a `p2p-circuit` address. From 0dd8c50d3181ab57894ca4a4c7d9110bbc5127cc Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 20:51:07 +0200 Subject: [PATCH 09/17] a multiaddr is just a sequence of instructions --- addressing/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 713c1e15c..497c89ed9 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -74,8 +74,7 @@ multiaddr](#the-p2p-multiaddr). ### multiaddr basics -A multiaddr generally represents a path through a stack of successively -"higher-level" protocols that can be traversed to some destination. +A multiaddr sequence of instructions that can be traversed to some destination. For example, the `/ip4/7.7.7.7/tcp/1234` multiaddr starts with `ip4`, which is the lowest-level protocol that requires an address. The `tcp` protocol runs on From 0e770a83fe83c576803e7f4ff56be5350588c6c1 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 21:01:39 +0200 Subject: [PATCH 10/17] fix stray partial sentence --- addressing/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 497c89ed9..fac59ac3f 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -87,12 +87,13 @@ require a 32 bit address. Similarly, `tcp` requires a 16 bit port number. Although we referred to `/ip4/7.7.7.7` and `/tcp/1234` as "components" of a larger TCP/IP address, each is actually a valid multiaddr according to the -multiaddr spec. However, not every **syntactically valid multiaddr is a functional description of a process in the network**. -through the network. As we've seen, even a simple TCP/IP connection requires -composing two multiaddrs into one. See the section on [composing -multiaddrs](#composing-multiaddrs) for information on how multiaddrs can be -combined, and the [Transport multiaddrs section](#transport-multiaddrs) for the -combinations that describe valid transport addresses. +multiaddr spec. However, not every **syntactically valid multiaddr is a +functional description of a process in the network**. As we've seen, even a +simple TCP/IP connection requires composing two multiaddrs into one. See the +section on [composing multiaddrs](#composing-multiaddrs) for information on how +multiaddrs can be combined, and the +[Transport multiaddrs section](#transport-multiaddrs) for the combinations that +describe valid transport addresses. The [multiaddr protocol table][multiaddr-proto-table] contains all currently defined protocols and the length of their address components. From b6f452b89dae7185beb1fc3ce34c67aee67504c5 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Wed, 21 Jul 2021 21:04:12 +0200 Subject: [PATCH 11/17] remove superflous paragraph about WebSocket addresses --- addressing/README.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index fac59ac3f..0c37cf03e 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -98,12 +98,6 @@ describe valid transport addresses. The [multiaddr protocol table][multiaddr-proto-table] contains all currently defined protocols and the length of their address components. -Some multiaddr protocols do not require any additional addressing information. -For example, WebSockets are by definition encapsulated within TCP/IP, so the -`/ws` multiaddr protocol is encapsulated within a TCP/IP multiaddr: -`/ip4/7.7.7.7/tcp/1234/ws`. This address is composed of three distinct -parts, `/ip4/7.7.7.7`, `/tcp/1234`, and `/ws`. - ### Composing multiaddrs As shown above, protocol addresses can be composed within a multiaddr in a way From 2751753e5dbc57fc11f14f1a64644c96b3dbf3f3 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Thu, 22 Jul 2021 12:14:49 +0200 Subject: [PATCH 12/17] apply @mxinden's suggestions from code review Co-authored-by: Max Inden --- addressing/README.md | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 0c37cf03e..27770ad31 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -3,14 +3,15 @@ | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|---------------|--------|-----------------| -| 1A | Working Draft | Active | r0, 2019-05-27 | +| 1A | Recommendation | Active | r0, 2021-07-222 | Authors: [@yusefnapora] -Interest Group: TBD +Interest Group: [@mxinden] [@yusefnapora]: https://github.com/yusefnapora +[@mxinden]: https://github.com/mxinden/ See the [lifecycle document][lifecycle-spec] for context about maturity level and spec status. @@ -74,7 +75,7 @@ multiaddr](#the-p2p-multiaddr). ### multiaddr basics -A multiaddr sequence of instructions that can be traversed to some destination. +A multiaddr is a sequence of instructions that can be traversed to some destination. For example, the `/ip4/7.7.7.7/tcp/1234` multiaddr starts with `ip4`, which is the lowest-level protocol that requires an address. The `tcp` protocol runs on @@ -124,14 +125,10 @@ progressively "inward". For example, in the address `/ip4/7.7.7.7/tcp/80/ws`, the outermost protocol is IPv4, which encapsulates TCP streams, which in turn encapsulate WebSockets. -All multiaddr implementations provide an `Encapsulate` method, which combines -two multiaddrs into a composite. For example, `/ip4/7.7.7.7` can encapsulate +All multiaddr implementations provide a way to _encapsulate_ two multiaddrs into a composite. +For example, `/ip4/7.7.7.7` can encapsulate `/tcp/42` to become `/ip4/7.7.7.7/tcp/42`. -Note that no "sanity checking" is performed when encapsulating multiaddrs, and -it is possible to create valid but unsound/dysfunctional multiaddrs like `/tcp/42/udp/42` -through encapsulation. - #### Decapsulation Decapsulation takes a composite multiaddr and removes an "inner" @@ -159,8 +156,8 @@ multiaddr looks like this: /p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N ``` -Where `QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N` is the base58-encoded -multihash of a peer's public key, also known as their peer id. +Where `QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N` is the string representation +of a peer's peer ID derived from its public key. By itself, a `p2p` address does not give you enough addressing information to locate a peer on the network; it is not a transport address. However, like the @@ -310,8 +307,7 @@ A full `p2p-circuit` address that describes a relay circuit is of the form: `/p2p-circuit/`. `` is the full address for the peer relaying the traffic (the -"relay node"), including both the transport address and the `p2p` address -containing the relay node's peer id. +"relay node"). The details of the transport connection between the relay node and the destination peer are usually not relevant to other peers in the network, so From a38dfcc28c640a158a28760f66be6003c4e2dc45 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Thu, 22 Jul 2021 12:18:35 +0200 Subject: [PATCH 13/17] fix multiaddr paragraph --- addressing/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 27770ad31..063db2465 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -65,9 +65,9 @@ repository][multiaddr-repo]. ## multiaddr in libp2p -multiaddr is used throughout libp2p for encoding network addresses, and -addresses are generally exchanged over the wire as binary-encoded multiaddrs in, when addresses need to be shared or exchanged between processes, they are encoded in the binary representation of multiaddr. -libp2p's core protocols. +multiaddrs are used throughout libp2p for encoding network addresses. +When addresses need to be shared or exchanged between processes, they are +encoded in the binary representation of multiaddr. When exchanging addresses, peers send a multiaddr containing both their network address and peer id, as described in [the section on the `p2p` From 0134a93277c7c85e5f50e74875dd3cbec6aafda6 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Thu, 22 Jul 2021 12:35:01 +0200 Subject: [PATCH 14/17] addressing/README.md: Wrap lines at 80 chars --- addressing/README.md | 53 ++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/addressing/README.md b/addressing/README.md index 063db2465..a072c87aa 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -58,16 +58,17 @@ To support a growing set of transport protocols without special-casing each addressing scheme, libp2p uses [multiaddr][multiaddr-repo] to encode network addresses for all supported transport protocols, in a self-describing manner. -This document does not cover the address format ([multiaddr][multiaddr-repo]), but rather [how multiaddr is used in libp2p](#multiaddr-in-libp2p). For details on the former, visit linked spec. -For more information on other use cases, or to find links to multiaddr -implementations in various languages, see the [mulitaddr +This document does not cover the address format ([multiaddr][multiaddr-repo]), +but rather [how multiaddr is used in libp2p](#multiaddr-in-libp2p). For details +on the former, visit linked spec. For more information on other use cases, or to +find links to multiaddr implementations in various languages, see the [mulitaddr repository][multiaddr-repo]. ## multiaddr in libp2p -multiaddrs are used throughout libp2p for encoding network addresses. -When addresses need to be shared or exchanged between processes, they are -encoded in the binary representation of multiaddr. +multiaddrs are used throughout libp2p for encoding network addresses. When +addresses need to be shared or exchanged between processes, they are encoded in +the binary representation of multiaddr. When exchanging addresses, peers send a multiaddr containing both their network address and peer id, as described in [the section on the `p2p` @@ -75,7 +76,8 @@ multiaddr](#the-p2p-multiaddr). ### multiaddr basics -A multiaddr is a sequence of instructions that can be traversed to some destination. +A multiaddr is a sequence of instructions that can be traversed to some +destination. For example, the `/ip4/7.7.7.7/tcp/1234` multiaddr starts with `ip4`, which is the lowest-level protocol that requires an address. The `tcp` protocol runs on @@ -125,26 +127,25 @@ progressively "inward". For example, in the address `/ip4/7.7.7.7/tcp/80/ws`, the outermost protocol is IPv4, which encapsulates TCP streams, which in turn encapsulate WebSockets. -All multiaddr implementations provide a way to _encapsulate_ two multiaddrs into a composite. -For example, `/ip4/7.7.7.7` can encapsulate -`/tcp/42` to become `/ip4/7.7.7.7/tcp/42`. +All multiaddr implementations provide a way to _encapsulate_ two multiaddrs into +a composite. For example, `/ip4/7.7.7.7` can encapsulate `/tcp/42` to become +`/ip4/7.7.7.7/tcp/42`. #### Decapsulation -Decapsulation takes a composite multiaddr and removes an "inner" -multiaddr from it, returning the result. +Decapsulation takes a composite multiaddr and removes an "inner" multiaddr from +it, returning the result. For example, if we start with `/ip4/7.7.7.7/tcp/1234/ws` and decapsulate `/ws`, -the result is `/ip4/7.7.7.7/tcp/1234`. +the result is `/ip4/7.7.7.7/tcp/1234`. -It's important to note that decapsulation returns the original multiaddr up -to the last occurrence of the decapsulated multiaddr. This may remove more -than just the decapsulated component itself if there are more protocols -encapsulated within it. Using our example above, decapsulating either -`/tcp/1234/ws` _or_ `/tcp/1234` from `/ip4/7.7.7.7/tcp/ws` will result in -`/ip4/7.7.7.7`. This is unsurprising if you consider the utility of the -`/ip4/7.7.7.7/ws` address that would result from simply removing the `tcp` -component. +It's important to note that decapsulation returns the original multiaddr up to +the last occurrence of the decapsulated multiaddr. This may remove more than +just the decapsulated component itself if there are more protocols encapsulated +within it. Using our example above, decapsulating either `/tcp/1234/ws` _or_ +`/tcp/1234` from `/ip4/7.7.7.7/tcp/ws` will result in `/ip4/7.7.7.7`. This is +unsurprising if you consider the utility of the `/ip4/7.7.7.7/ws` address that +would result from simply removing the `tcp` component. ### The p2p multiaddr @@ -275,9 +276,9 @@ The `` argument must be a 16-bit unsigned integer. WebSocket connections are encapsulated within TCP/IP sockets, and the WebSocket multiaddr format mirrors this arrangement. -A libp2p WebSocket multiaddr is of the form `/ws` or `/wss` (TLS-encrypted), where -` is a valid mulitaddr for the TCP transport, as [described -above](#tcp). +A libp2p WebSocket multiaddr is of the form `/ws` or +`/wss` (TLS-encrypted), where ` is a valid +mulitaddr for the TCP transport, as [described above](#tcp). ### QUIC @@ -296,8 +297,8 @@ The libp2p [circuit relay protocol][relay-spec] allows a libp2p peer A to communicate with another peer B via a third party C. This is useful for circumstances where A and B would be unable to communicate directly. -Once a connection to the relay is established, peers can accept incoming connections -through the relay, using a `p2p-circuit` address. +Once a connection to the relay is established, peers can accept incoming +connections through the relay, using a `p2p-circuit` address. Like the `ws` WebSocket multiaddr protocol the `p2p-circuit` multiaddr does not carry any additional address information. Instead it is composed with two other From 090cf31dca6437bcd0f3e774179f1e90b711967c Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Fri, 23 Jul 2021 18:47:29 +0200 Subject: [PATCH 15/17] add more people to the Interest Group --- addressing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/addressing/README.md b/addressing/README.md index a072c87aa..d7011ba70 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -8,7 +8,7 @@ Authors: [@yusefnapora] -Interest Group: [@mxinden] +Interest Group: [@mxinden, @Stebalien, @raulk, @marten-seemann, @vyzo] [@yusefnapora]: https://github.com/yusefnapora [@mxinden]: https://github.com/mxinden/ From f243160d1865bcb814c3dfee5341984aa9e60996 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Fri, 23 Jul 2021 18:49:46 +0200 Subject: [PATCH 16/17] fix date Co-authored-by: Steve Loeppky --- addressing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/addressing/README.md b/addressing/README.md index d7011ba70..551facc14 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -3,7 +3,7 @@ | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|---------------|--------|-----------------| -| 1A | Recommendation | Active | r0, 2021-07-222 | +| 1A | Recommendation | Active | r0, 2021-07-22 | Authors: [@yusefnapora] From 2b53d89b42ea03272363d926d856163e7b3b45c6 Mon Sep 17 00:00:00 2001 From: Marten Seemann Date: Sat, 24 Jul 2021 12:33:54 +0200 Subject: [PATCH 17/17] add note about dnslink --- addressing/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/addressing/README.md b/addressing/README.md index 551facc14..94be9c863 100644 --- a/addressing/README.md +++ b/addressing/README.md @@ -237,6 +237,9 @@ resolves to `1.2.3.4`, libp2p will resolve the address A libp2p-specific DNS-backed format, `/dnsaddr` resolves addresses from a `TXT` record associated with the `_dnsaddr` subdomain of a given domain. +Note that this is different from [dnslink](https://dnslink.io/), which uses +`TXT` records to reference content addressed objects. + For example, resolving `/dnsaddr/libp2p.io` will perform a `TXT` lookup for `_dnsaddr.libp2p.io`. If the result contains entries of the form `dnsaddr=`, the embedded multiaddrs will be parsed and used.