[P2P] Integrate LibP2P

[jessicadaugherty]

Objective

Use the results of pokt-network/pocket-network-protocol#17 to integrate LibP2P into the p2p module.

Origin Document(s)

• [SPIKE][P2P] Configuration - Research LibP2P pocket-network-protocol#17
• LibP2P Guide

Goals

• Integrate LibP2P into Pocket Core

Deliverables

• MVP integration where LibP2P transport layer is used to send messages

Non-Goals / Deliverables

• Integrating P2P deeper into transport layer
• Implementing peer churn or discovery
• Exhaustively enabling all the features we will need from LibP2P

General issue checklist

• Update the appropriate CHANGELOG
• Update the README
• Update the source code tree explanation
• Update any relevant global documentation & references
• Document small issues / TODOs along the way

---

Creators: @jessicadaugherty @Olshansk

[Olshansk]

Per our team discussion, I believe the scope of this can be summarized to, at a very high level, to bullet point 3 from the research SPIKE:

[jessicadaugherty]

Define a Host interface in the pocket repo and embed libP2P's Host interface inside of it, implementing the underlying interface but not enabling it yet.

[Olshansk]

Synched up @bryanchriswhite on this offline and wanted to capture the potential action items for a next step.

Idea: LibP2P could be treated as another "connection type" similar to what we implemented in #268.

Next steps:

1. Add LibP2P as a connection type (alongside TCP & noop)
2. Create a new ticket titled "Consolidate LibP2P & Pocket Interface"
3. "Hack" things together (in terms of data types) to replace the transport layer in P2P
4. Add comments similar to CLEANUP(#XXX): Cleanup these interfaces wherever we kind of hacked things together

Other things discussed:

1. Lack of connection pooling in the current version was not an intentional "thought-through" deicsion - we just got things working using the standard Golang libraries and I did not document it explicitly.

EDIT: @bryanchriswhite Let me know if I missed anything or if you have additional questions and/or clarifications.

[deblasis]

@Olshansk:

In principle, I like the idea and I think it's a sensible step, in fact, it was what I initially thought was the right thing to do first when I started looking into this. I quickly developed some doubts and therefore I preferred a more targeted approach (eg: starting with primitives like IDs and moving up from there) because I didn't know how to "size" the effort, it sounded like a blanket, not a t-shirt in my head at the time 😄 I wanted to start small while learning along the way.

Maybe my pivot was due to the fact that I didn't know LibP2P deeply enough to know, top of my head, all the contact points, how they work, etc.

Some extra thoughts:

1. If we want to use LibP2P side-by-side with our other implementations, we would have to abstract with interfaces all the common parts so that everything works regardless of the implementation that's selected at runtime. It sounds like a "big change" but maybe I am wrong.
2. What is the difference between hacking things together in terms of data types and being deliberate and targeting them one by one (wherever possible) in a more controlled manner?
3. If you confidently think you can hack things together in the fastest way possible with the approach described above, and the answer to the point 2) is "because... it's faster" please IGNORE all the above and go for it. I love being wrong when the outcomes are positive! 🚀

Re: connection pooling, I think it's necessary for the long run but probably overkill until we reach networks with hundreds of nodes. LibP2P however should provide features like this in an "easy way" I believe. That's basically all the networking stuff that we won't have to implement ourselves :)

[Olshansk]

What is the difference between hacking things together in terms of data types and being deliberate and targeting them one by one (wherever possible) in a more controlled manner?

@bryanchriswhite did research on all the data types but it's still a big undertaking.

I thought that if we just wire it together (with TODOs and a separate connection type) everywhere, we'd be closer to this approach and clean it up after:

Re: connection pooling, I think it's necessary for the long...

We're on the same page. It seems like it's the default and Bryan mentioned we have to make an explicit effort to turn it off. If it's less effort to keep it (during initial integration), let's keep it.

---

@bryanchriswhite This is an integration where I haven't jumped in to get my hands dirty so I'm speaking out of my depth. However, if you think it'd be helpful for me to get a foundation PR up, lmk.

[bryanchriswhite]

@Olshansk sorry for the delay in responding, I've been heads-down in preparation for being able to participate meaningfully. 😅

In my view, it's looking like the path of least resistance is to implement a new modules.Module , types.Network, and types.Transport. I also figured that it would then be more straightforward to add a boolean config flag rather than coupling to the connection/transport type. Transport is also a degree of freedom that would be available in a libp2p-based module.

I will fill in the PR description with more details when I'm back at my desk this afternoon.

[bryanchriswhite]

FWIW, libp2p is doing almost all of the work. To get to a stdnetwork analog (i.e. fully-connected network), it's just a matter of plugging things in to the right place and/or making them the right shape. To get RainTree support working, some additional massaging and/or interface redesign will be required.

[Olshansk]

@bryanchriswhite Does autoscaling (with staking validators) using k8s LocalNet + LibP2P work? https://github.com/pokt-network/pocket/tree/main/build/localnet#scaling-actors