Add options to select provider preferences

[whyrusleeping]

Sending out dht provider messages is the biggest pain point in ipfs right now. I have tried brainstorming solutions: ipfs/notes#162, but that hasnt produced anything yet that i feel actually solves the problems.

So to help with the issue for now, i propose we implement four provider options as follows:

none

Provide nothing, this should do roughly what the --routing=none daemon flag does, except still allow using the dht for findpeer queries and ipns. The downside here is that other peers will not be able to discover content your peer has available (though, upon direct connect, will still be able to request it)

pins

Provide all pinned content. Only content you explicitly pin will be announced to the network, the rest will be available if directly requested, but not advertised

pinroots

Provide only the pin roots. This will be a very minimal option, only the hashes that are pinned (and not their children) will be announced.

noleaves

Provide all content that is not a 'leaf' node. Where in this case a leaf node would be anything that is not the root of a larger object. For instance, only the top node of a large VM image would be announced, while all the hundreds of gigs of data blocks would not be. This option is a bit trickier to implement

[hsanjuan]

Wondering, what are the use cases for pinroots/noleaves ?

[whyrusleeping]

@hsanjuan the idea is to still provide some information to the network, but to do much less of it. pinroots would still allow peers on the network to find your pinned content, but not find random access items below pinned things. noleaves would still allow random access of all files and directories, but wouldnt allow peers to find data blocks under a file (though if they are connected to you and are reading a file, everything would work out because we only search for providers when none of our bitswap peers have a given block)

[Stebalien]

We could also have smart strategies where we mark blocks that should be provided when we add them to IPFS. The IPFS add command could, e.g., mark the roots nodes of directories/files. However, to get this working, we'll need a nice way to attach local metadata to blocks (this would be rather nice to have anyways).

[whyrusleeping]

ref #4113

[whyrusleeping]

I think a good next step here would be to make provider strategies apply as content is added. So if your strategy is 'pinned content' and you add something with --pin=false, it shouldnt provide any of those blocks.

cc @magik6k

[magik6k]

Sounds good, will have a look at this soon.

[eingenito]

@magik6k have you started this work? I'd like to take it on if you haven't already.

[magik6k]

Yup, there is some work in on this in #4333, but since I last touched it, some modules were extracted (well, bitswap), so you'll probably need to apply this manually.

[michaelavila]

This work is being tracked here: #5774 and references this issue, so I'm going to close this issue.