WIP: Persisting/seeding a routing table

[raulk]

Finally got a few spare cycles to push a sketch of what I had in mind for the snapshot/seeding functionality, mostly to drive alignment.

The solution needs to be modular to enable us to:

• fetch candidates from any source: a datastore, mDNS, an XMPP/IRC room, a Twitter account, DNS TXT records, a file somewhere, etc.
• decide which candidates we want to seed; we can pick candidates randomly, or we can choose based on latency, geographical location (e.g. to favour proximity), etc.

This proposes two interfaces:

• Snapshotter: responsible for loading and storing snapshots. I've included an implementation that uses a Datastore.
• Seeder: responsible for seeding a routing table from a set of candidates recovered from a snapshot + fallback peers.

It introduces a configuration option, and also proposes an integration point for this inside dht.go.

We should be able to call this logic whenever, not just at start. That will enable us to recover from disasters.

Note that this is heavy WIP, and I probably won't have the bandwidth to take this over the finish line. But I wanted to push these ideas out there. I really think the solution needs to be modular.

[raulk]

@anacrolix @jhiesey I'll continue working to solidify this unless you have feedback or concerns about the abstractions being introduced. If so please shout ASAP.

[anacrolix]

@raulk What issues are you targetting with this?

[raulk]

@anacrolix See explanation in description for rationale. If you mean GitHub issues, see conversations in:

#254
#295

[anacrolix]

I don't think this needs to be so complex. A callback that generates peers on demand in the config, and methods to take a snapshot of the routing table, and restore a routing table are more than enough. See #254 (comment).

[raulk]

This design provides both elements, and it does so via pluggable strategies instead of hardcoding behaviours. The composability principle is key in libp2p (we are fans of modularity in libp2p). That said, providing sensible fallbacks is reasonable. Any specific feedback on the design?

See #254 (comment) for reference.

[anacrolix]

I've commented on the respective issues.

[raulk]

@aarshkshah1992 – as per #345 (comment), would you mind commenting on this issue so GitHub will let me assign it to you?

[aarshkshah1992]

Self referential comment

[aarshkshah1992]

@raulk I think I can start looking into this now. Please can you assign it to me ?

[raulk]

This one is landing in #383 -- which takes this WIP PR and makes it mergeable! \o/