ROADMAP.md: Add high level project roadmap #339
Conversation
This documents the libp2p short and long term roadmap based on previous work (e.g. see [libp2p/libp2p/60], [ipfs/roadmap/18], [libp2p/specs/archive]). [libp2p/libp2p/60]: libp2p/libp2p#60 [ipfs/roadmap/18]: ipfs/roadmap#18 [libp2p/specs/archive]: https://github.com/libp2p/specs/tree/master/_archive
There was a problem hiding this comment.
Lots of good stuff here. Thanks @mxinden and @marten-seemann for the efforts so far (even if a good chunk of it was bringing forward work done by others in the past).
A couple of thoughts:
1
This document right now is a good list of themes (I think the visionary items particular fall in this camp) and items.
Using the map analogy, we ultimately need to be able to create/show our route on the map. Right now we have a list of possible destinations, but no indication of costs to get there, impact, and ordering.
We can talk offline (or here) on how we get there, but we're going to need something "spreadsheet-like" where we can order, sort, etc.
2
This document is covering scope that would expect PM's to be heavily involved in (totally fine - thanks for wading into these waters). That said, I also want to make sure we capture if there are any more tactical things that Stewards/project teams should help with in the next 6 months. This could be automating releases, establishing clearer contribution guidelines, closing out old PRs/issues, etc. I know those things aren't free, and if they help us move faster/safer in the future, I want to make sure we're accounting for them. (They don't need to be in this document, but I do want them for the "roadmap" we look at internally for where we're investing our efforts in 2021H2).
| locally, but transmit frequently to a cloud where data analysis occurs. | ||
|
|
||
| **Why?** IoT can greatly benefit from peer-to-peer networking. Supporting IoT | ||
| environments with libp2p greatly expands libp2p's userbase on the one side and |
There was a problem hiding this comment.
Are we aware of certain customers asking for this? What has held them back from developing it?
There was a problem hiding this comment.
Are we aware of certain customers asking for this?
Even better. There is https://github.com/Actyx built on top of rust-libp2p using libp2p in the IoT / factory space today.
There was a problem hiding this comment.
interesting ROADMAP. but it seems a little broad in general to be actionable and make progress. maybe picking some clear goals for the next quarter would be more productive?
| **Why?** Multistream 1.0 is chatty and naïve. Streams are essential to | ||
| libp2p, and negotiating them is currently inefficient in a number of | ||
| scenarios. Also, bootstrapping a multiplexed connection is currently |
There was a problem hiding this comment.
I assume some of this information is in the docs below, but I think it would be good to call out a couple of numbers here. For example, (this is me making something up), "The p50 to create a bootstrapped multiplex connect between IPFS nodes currently takes Xms from Y round trips. With improvements to this space, we believe we can reduce this to Ams and B round trips. This is important because IPFS nodes establish these connections each time connecting to a peer for doing a lookup or initiating a bitswap session. The p50 of time to first byte of content download for a cold query will reduce from Cms to Dms."
There was a problem hiding this comment.
Good call. I don't have these numbers (yet). See protocol/web3-dev-team#119 for a qualitative analysis.
There was a problem hiding this comment.
in case you weren't aware, at least with rust-libp2p we do have a solution for 0-rtt [0]. we send a multistream-select negotiation message with the first packet and receive confirmation with the handshake packet. this can probably be improved by sending a data payload with the multistream-select message, but since that payload wouldn't be replay protected I'm really not that concerned about it.
|
|
||
| ### 🕸 Unprecedented global connectivity | ||
|
|
||
| **What?** A DHT crawl measurements (Nov 22nd 2019) showed that out |
There was a problem hiding this comment.
Can we link to where this data was collected?
Also, can this be an automated process that runs periodically (e.g., weekly, monthly) so we can see improvements to this over time?
There was a problem hiding this comment.
Unfortunately I don't know the source of these crawl measurements. Maybe @vyzo or @aschmahmann know more?
Also, can this be an automated process that runs periodically (e.g., weekly, monthly) so we can see improvements to this over time?
I would expect there is already such tool. If not, we might be able to extend the periodically run ipfs-crawler project.
👍 on running this periodically to see our improvements over time.
| share files with each-other). [This ties into offline and decentralised | ||
| networks.](#kix.xlvu7ikvo1jg) | ||
|
|
||
| ### 🏹 RPC and other common node communication patterns |
There was a problem hiding this comment.
This one seems interesting, particularly from it's ability to unlock others.
I think it could benefit from something like:
This is what a customer (developer) has to do today to satisfy usecase Y.
...List the various steps/commands...
If we do this work, they will instead only need to do:
...List the various steps/commands...
There was a problem hiding this comment.
For now, I would prefer to keep this section on a high level, especially as each of the detailed steps needed by users today is very implementation specific.
|
|
||
| **Links:** | ||
|
|
||
| - https://github.com/libp2p/notes/issues/2 |
There was a problem hiding this comment.
Did some experiments with performant authenticated replicated streams this week. Achieving speeds of 220MB/s was fairly easy, but I think this could be pushed even further. Not sure if this is of interest here.
|
486682b reduces the #polite-peering section to a connection status protocol only. I don't think there is much use in combining identify, ping and potentially even multistream select into a single protocol. //CC @marten-seemann |
Add two metrics - number of dial failures and time to first byte - to "Reducing the dial fail rate" section.
- Remove "prerequisite" - Mention nim-libp2p
|
@BigLep given that you are still tracked as "requesting changes", would you mind taking another look? |
|
Thanks everyone for the input. I will merge here. Note to anyone out there, this is a living document, thus please feel free to suggest additions / post comments / ... going forward. Looking forward to tackling the tasks ahead of us! |
|
"Dismissing" @BigLep request for change, as discussed offline. |
This documents the libp2p short and long term roadmap based on
previous work (e.g. see libp2p/libp2p/60, ipfs/roadmap/18,
libp2p/specs/archive).
Status
Feedback from the whole community is welcome.
Fixes libp2p/libp2p#68.