[2021 Theme Proposal] IPFS ❤️ to make new friends

[RubenKelevra]

Note, this is part of the 2021 IPFS project planning process - feel free to add other potential 2021 themes for the IPFS project by opening a new issue or discuss this proposed theme in the comments, especially other example workstreams that could fit under this theme for 2021. Please also review others’ proposed themes and leave feedback here!

Theme description

Adding friends to your IPFS node would allow us to send them messages, share content with them and allow them to store stuff on your node and the other way around.

When you share content with a friend, your node could create an ACL with the friend's nodes, to avoid sharing this data with the whole network (see also #75). This would require to have private files in your IPFS node, which isn't shared with the network at all by default.

Those changes avoid, that user builds small networks with network keys since they can use the public network simultaneously. This makes the transition between private, peer-group, and public data for the users much easier and allows also some additional features and benefits for the network.

Potential benefits for the network:

• bootstrapping would start to connect to the friend's nodes first, avoid connecting to a fixed list of nodes
• querying friend's nodes first when searching for data, before doing a network-wide search. This keeps content transfers local and reduces the number of requests for data in some scenarios
• lower response times to requests, since the friends are most likely geographically local
• when a friend needs a NAT traversal, your node can offer this
• a friend list in IPFS would show up in all apps which connect to the API
• ability to share a file just with the friends without encryption or a separate private ipfs-network (using ACLs)
• ability to send notifications to all (a group of) friends
• allows e.g. for RSS like, pushed updates of new pinned peer-group/public files
• notify friend(s) of a file which is explicitly shared (easy file transfer notification)
• apps could send custom messages with this feature (e.g. for chats, social networks, calls)

Cross-Device-Signing allows coalescing multiple nodes-ids to one 'friend' entry

• select a user gives all devices of the user access (e.g. sharing a CID)
• address multiple nodes together e.g. notifications
• buffered notifications: one device of a user can hold new notifications, while another device is offline

Give friends storage on your node:

• storage sharing with friends (quota per friend; quota for all friends)
• you can give friends a limited amount of storage
• simple scrambling of data: sharding of the individual blocks of the data over all your friends with redundancies
  • no 3rd party software needed for encryption (portability)
  • no encryption layer: lower CPU requirements
  • compression of the datastore is still beneficial
  • identical files can still be identified (by the node who stores them on friend's nodes)
  • multi-device ACL allows different devices to access the same data (without exchanging a crypto key)
  • disaster recovery: all friends can manually accept a new device for a user to give him access to his data back

More robustness of the network due to a build-in Web Of Trust:

• potential a better resilience against DDoS attacks (e.g. limit new connections to unknown nodes if they are excessively established)
• creates a grid of geographically local nodes, reducing the overall latency of the network
• a friend can introduce a friend (cryptographically signed) to a user
• we might be able to skip some cryptographically checks or DoS filter on network traffic received from friends
• friends can use special network features, like relays
• the maximal processing time/bandwidth can be higher for friends
• peer-group caching of 'hot' files (with a lot of accesses)
  • a node can ask the friend's nodes to store chunks of a file in their cache
  • automatically increases the availably of files which are accessed often
  • avoids that one node has to serve many clients which don't reupload the file (fast)
  • will cascade if there's still a lot of access on the file, since the friends can ask their friends to hold the data
• user-initiated peer-group caching
  • if selected a new file will be spread to the friend's cache (if there's some space left)
  • user can specify a hold time (info for the garbage collector additionally to requests on it)
  • allows the user to upload a file to the network if he has limited bandwidth/time
  • hot chunks of the file will be cached further in the network (peer-group caching of 'hot' files)

Hypothesis

IPFS is great, but there's no way to add friends to a node or connect nodes together and have them hold the same data, like a Dropbox or a Google Drive.

Adding ACLs and friends allows both: Having data available without even have some server running since a friend's computer can hold the data and easy and secure sharing of files with friends.

Vision statement and why focus on it this year

I think that's what people are currently missing from IPFS and whats somewhat holds back the adoption in daily use.

The team worked hard to grown IPFS mature enough to be used for general purpose and I think 2021 would be a good target to get these features in to make it useful for daily life.

Example workstreams

[ ] Create the ability to store a web of trust with different levels of trust in the node.
[ ] Start bootstrapping with friend's nodes while still bootstrapping simultaneously to the bootstrap nodes to avoid netsplits
[ ] Make it configurable how many connections (in percent or as integer) should be held open to friends all the time - which are not part of the normal connection limits.
[ ] Implement messages with a topic-tag, an 'application'-tag and headline/text/respond-options etc.
[ ] Add ACLs for data stored in the client
[ ] Implement cross-signing between clients
[ ] Create a 'private files' section in the clients, which can be listed and accessed with all cross-signed clients
[ ] Allow easy push and pull of data stored in the 'private files' section of the clients to other clients, or all clients commanded via messages
[ ] Create an eventually consistent message database shared between the clients, to be able to store and share messages from all the different clients and sync them without conflicts.
[ ] Change the search for data to ask and prioritize the friends when requesting data from the network, to avoid having to ask the DHT and connect to the other side of the world to get the data. Local transfers first. 😃
[ ] Add a config option to give friends a quota in your client
[ ] Give friends (with quota) the ability to push data to your client, with an ACL
[ ] Add an option to store a copy of (private) files to a list of friends, sharded to avoid that one friend do holds all data. This gives a user a chance to access files which are on a node which is offline via fetching it from friends, while not being limited
by individual upload bandwidth limits of the friends.
[ ] Implement proxied network access for low-power nodes (which only opens one connection to the proxy and commands network operations) see for more details

Other content

Related already created tickets by me (some mentioned before)

• trusted nodes (WebOfTrust) / shared storage with friends / coalescing multiple clients notes#397
• [Meta] Trusted nodes / WoT to proxy network access ipfs-shipyard/gomobile-ipfs#31
• distributed bootstrapping with IPNS libp2p/notes#24

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.