add iris grant proposal

[driemworks]

Project Abstract

Iris is a substrate-based decentralized storage network. It builds off of work done by a team at equilibrium which laid the foundation by embedding rust-ipfs within the substrate runtime. With the ability to track the DHT status onchain, we aim to provide decentralized storage which solves some of the issues present in DSNs today: governance, availability while retaining full decentralization, security, and indexability. Further, Iris is intended to work alongside parachains which communicate with Iris via a pallet (using XCM) that we provide. Nodes in Iris-compatible parachains can create iris-backed assets within their parachains, which allow the creation of a "ticket", an asset that allows access to some item in Iris. This allows users (nodes in a parachain) to use Iris as a (private) storage layer and manage or monetize access to it within their own parachains, using their own tokens and governance (retaining sovereignty).

For which grant level are you applying?

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $50,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for > $100k Web3 Foundation Council approval)

Application Checklist

• The application template has been copied, renamed ( project_name.md) and updated.
• A BTC or Ethereum (DAI/USDT) address for the payment of the milestones is provided inside the application.
• I have read and acknowledged the terms and conditions.
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted.

[CLAassistant]

All committers have signed the CLA.

[semuelle]

Hi @driemworks, thank you for your application! We will look into it as soon as possible.

[Noc2]

Thanks for the application. Looks really interesting. So your main goal of this grant is to update and maintain: https://github.com/rs-ipfs/substrate/tree/offchain_ipfs (https://rs-ipfs.github.io/offchain-ipfs-manual/) as well as to develop a UI for it, correct? That is definitely something in which we are interested. Could you maybe try to make the deliverables in the tables a little bit more concrete? (e.g. latest FRAME version, features of the UI). These are basically the requirements of the contract.

[driemworks]

Thanks for the feedback. I updated the deliverables and modified the scope slightly, should be a little more solid now.

I would actually say that development of the user interface and maintaining the offchain::ipfs repository are more like tertiary goals. I absolutely intend to maintain my fork of it, but I would be interested in discussing with equilibrium and the uddug team (who seems to already be undertaking this as part of an ongoing grant) in terms of updating and maintaining the rust-ipfs fork and manual. As for the UI, we're planning on building two user interfaces, one for iris and one as an example/template for an "iris-compatible" parachain. The iris UI is more like an admin portal for nodes to view DHT statistics (storage capacity, pinned CIDs, etc of their embedded IPFS node), earned rewards for pinning CIDs, and in the future to configure their embedded IPFS node (could possibly be used by other chains using the offchain::ipfs runtime, but may need some tweaks to do so). The second UI is built for the parachain and will serve as an example application to showcase Iris, expressed as a decentralized, privacy-preserving, dropbox-like application.

The primary goal is to create a decentralized storage network with an access management system so that we can create "owned", private content without relying on encryption. In a way, this can be thought of as using offchain::ipfs to build an incentivized, metered, and decentralized IPFS pinning service. Secondly, it is to enable "access to content" in iris through an asset that can be bought, sold, and transferred across parachains (this is the "ticket" asset).

[Noc2]

Thanks for the update. In general, I’m happy to go ahead with this, since I think the components which you create are useful for our ecosystem. However, could you still add the overview section to your development roadmap: https://github.com/w3f/Grants-Program/blob/master/applications/application-template.md#overview-1 Additionally, I have a question regarding your data availability scheme. I’m not sure how you want to make sure that notes/storage providers actually store the data as well as share this data with the owners. Especially since you say “do not need to rely on storage proofs”.

[driemworks]

I've added the overview as well as some clarifying notes in the "storing content" section, which I'll summarize here to answer your question:

I'm claiming that we don't need storage proofs because we assume that results from queries to the DHT of the embedded IPFS instances in iris nodes can be trusted. To ensure that nodes are actually pinning a CID, we rely on the DHT. After an OCW publishes results on chain claiming it pinned a CID, we query the DHT at the end of each epoch (as defined in the storage request) to verify that a node still has it pinned.

As for verifying that the right node in iris delivers the right content to the right RPC endpoint, we partly leave this up to the implementation of the RPC endpoint and the "ticket verification" by OCWs, but it also feeds into a future use case we intend to tackle when we build a governance and moderation framework. The RPC endpoint's implementation can vary, but we will verify that the iris node who called our endpoint is correct by querying Iris' runtime storage to see which node called the RPC endpoint from Iris (which will be published on chain after the OCW calls the endpoint and submits a transaction) and comparing it with a signed message provided as a header (we could potentially encrypt the data as well, but this will be delegated to future work if needed). Further, the RPC endpoint should verify the content by comparing the expected CID to the CID generated from the provided bytes. For the time being, we intend for the RPC endpoint to just reject any invalid requests. In the future we plan to include this in our approach to governance to punish nodes who deliver incorrect data offchain through functionality that would let nodes report content/addresses. It's out of the scope of this proposal, but the future state would be to return tokens (paid as fees for the OCW) to users when bad content is delivered or if it never gets delivered, and to punish misbehaving nodes in Iris. Hope that clarifies things.

[Noc2]

Thanks for the update. I’m happy to go ahead with it and I will share it with the rest of the team.

However, regarding querying the DHT at the end of each epoch, I guess this approach is similar to Arweave and you probably need to randomize what you query storage wise otherwise this quickly becomes either too computation intensive or can be exploited (storing only the relevant information, e.g. hashes instead of the data). Also this way it’s hard to ensure redundancy, since theoretically everyone can query the data in response from somebody else to reply in the correct way. Furthermore, it might lead to a set up in which data is only shared during the query process (e.g. the correct hashes to get not slashed or get the rewards), but not with anyone else because they could also use this data to create the correct reply for the query process (regarding redundancy).

[semuelle]

Thanks again for the application. I'm happy to support it. But can you explain what you mean by ensures that calls to IPFS are made through the Substrate layer? Do the IPFS nodes communicate (with each other) through the pallet only? If I own a storage node, wouldn't I be able to query any CID in the network through the IPFS node API? Maybe with a patched one?

[driemworks]

The IPFS nodes will still communicate via the DHT, but in a private network (they would all share the same swarm key and we intend to provide a way to accept new validators/storage providers, including sharing the swarm key securely). The idea is that since the IPFS node is embedded in the runtime it won't be reachable or callable by any external origins, so the IPFS HTTP API would not be available to call even if you're running a local "iris" node (some more info on this in this issue, where someone did want to make the embedded ipfs node visible to the outside world: rs-ipfs/substrate#2). In the future we plan to implement a governance framework on top of what this proposal provides which would account for situations where the private network is compromised (e.g. the swarm.key leaks), but that was left out of scope of the current work.

[semuelle]

Thanks for the clarification. It still sounds to me like it's possible to spoof a routing/streaming request from the Content Provider to the IPFS node. Which would be hard to detect, thus hard to sanction. I won't make my approval dependent on it, though.

[mmagician]

I guess it won't hurt to have more teams working on offchain-ipfs maintenance and integrations.

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀

[driemworks]

@semuelle @Noc2 @mmagician
Based on what was learned during the first milestone delivery, there have been some alterations to the original design that was proposed here, specifically on how storage is provided to the network. For milestone 3, we would like to descope the creation of a relay chain and testnet as a PoC for apps using Iris and instead use a smart contract. I have updated the proposal already, including some modifications to the language used (e.g. using data instead of content). I opened a PR to ammend the proposal here: #812

[RouvenP]

hi @driemworks we transferred the payment for milestone 2 today. Thanks!

[driemworks]

Received, thank you!