Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIP-95 Revisit #1145

Closed
wants to merge 10 commits into from
163 changes: 163 additions & 0 deletions 95.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
NIP-95
======

Relay File Storage
------------------

`draft` `optional`

This NIP defines a way to store binary data on relays. It supports parallel uploads and resumable downloads.

## Upload

To upload a file, first client must convert its bytes to base64. It may do it in chunks which sizes are multiples of 3 bytes
(250000 bytes per chunk, for example) or in one go.
The base64 data will be used on the `.content` field of one or more `kind:1195` events to be uploaded to a relay.

The chunk events may be sent on separate websockets connections for parallel uploads.

## Event

An event of `kind:1195` represents a file chunk. A client may choose to split a file
into multiple chunks or use a single event. If using chunks, all of them MUST
have the same size, except for the last one.

The event `.pubkey` may be the uploader's one or a randomly generated pubkey.

All chunks share the same [NIP-61](61.md) unbound list `n` tag that is set to `"<toHex(sha256(full-file))>-<chunk-byte-size>"`
(here the chunk size is always the greater one, because the last chunk size may be smaller than the others).

**Important**: The chunk byte size used on the `n` tag is the binary one, before conversion to base64.

The event `.content` is the base64-encoded file chunk data. Before storing this field, relays
may convert it back to binary to reduce its size and then re-encode the field to base64
before sending the event to clients.

It has a `c` tag. Its first value is the chunk number while the second one is the total number of chunks.
The client can use the `c` tag to ask for missing chunks when resuming a download.

Example:

```js
{
"id": "<id>",
"pubkey": "<pubkey>"
"kind": 1195,
"tags": [
["n", "<toHex(sha256(full-file))>-<chunk-byte-size>"],
["c", "5", "20"] // required; this is the 5th chunk of a total of 20
],
"content": "<base64-chunk>",
"created_at": <timestamp>,
"sig": "<signature>"
}
```

## Metadata

Metadata should be added using [NIP-94](94.md) or other metadata-related NIP.
If using NIP-94, instead of `kind:1063`, use `kind:1094` with the exact same format
and instead of `url` tags(s) use NIP-61 `u` tags to reference the file chunk set,
including the uploader's pubkey as second value, as follows:

`["u", "<pubkey>:<n-tag-value>", "<uploader's-pubkey>"]`

The uploader's NIP-65 "file relays" are where clients should search for the file.

## File Relay

Client should upload to user's "file relays", which use the [NIP-65](65.md) `f` flag
and follow the behavior described here and on the [Pre-Upload section](#pre-upload).
When downloading a file uploaded with this NIP, client should search on the uploader's "file relays".

**File relays must NOT honor `kind:5` deletion events referencing file chunk events.**
This is because the same file chunk set may be in use by other "uploaders".
Deletion of all file chunks is expected to be automatic when there is no registered uploader left on the "file relay"
(see [Pre-Upload section](#pre-upload) to learn how to register an uploader).

If an user wants to make sure a file won't be deleted, it
should become an uploader of that file.

## Pre-Upload

Before uploading `kind:1195` events of a specifc file,
user MUST publish a single `kind:1095` "Uploader" event,
which is an event authored by the **user's main pubkey**,
with empty `.content` and an `u` tag referencing the NIP-61 unbound list of `kind:1195`
events.

The second part of the `u` tag contains the full file hash and the chosen chunk size
(used on the `kind:1195` event(s)) that the relay should validate after upload is finished.

There is an `x` tag set to the full file hash to help find uploaders of a file and
also, consequently, find out if there are already stored chunks of a file on a file relay.

The `kind:1095` event purpose is to register the user as an uploader of the file on a specific
file relay and to request permission for uploading the file chunks.

Example:

```js
{
"id": "<id>",
"pubkey": "<user's-main-pubkey>" // uploader's pubkey
"kind": 1095,
"tags": [
["u", "<pubkey>:<n-tag-value>"],
["x", "<toHex(sha256(full-file))>"]
],
"content": "",
"created_at": <timestamp>,
"sig": "<signature>"
}
```

When receiving the `kind:1095` event, the "file relay" may answer with one of the following messages and prefixes:

- `["OK", "<kind:1095-event-id>", false, "auth-required: ..."]`: Authentication is required to register an uploader;
- `["OK", "<kind:1095-event-id>", true, "uploaded: ..."]`: The corresponding `kind:1195` file chunks are already uploaded, trying to re-upload them will fail;
- `["OK", "<kind:1095-event-id>", true, "upload: Missing chunks 1, 2, 7, 10"]`: File isn't uploaded yet or incomplete, user is allowed to upload it on this ws connection;

Trying to send a `kind:1195` event before a `kind:1095` one should fail.
Sending `kind:1095` events with an `u` tag identifier
different from the previously sent `kind:1195`'s corresponding values (`pubkey` and `n` tag) should fail.

A `kind:5` deletion event referencing a `kind:1095` event is used to **un**register the user as an uploader.

The "file relay" may split the burden of a single file by the multiple registered uploaders. For example, when there are 2 uploaders,
a "file relay" may charge each of them half of the costs to host the file.

"File relay" should delete the `kind:1095` event if no corresponding `kind:1195` event is uploaded within a reasonable period.

Clients may download `kind:1095` events from user's "file relays" to list all user files.

## nfile Entity

A [NIP-19](19.md) `nfile` bech32-encoded entity may be embedded directly on notes using a [NIP-21](21.md) URL.

It translates to:

1) the unbound list (file chunk set) author's pubkey;
2) the unbound list name (`n` tag value);
3) and the uploader's pubkey (to locate the list using uploader's NIP-65 "file relays").

The event kind number `1195` is implicit.

It should have atleast inlined MIME type
to help clients choose to download it depending on MIME type support.

For example:

`nostr:nfile1qqstn...794234d#m=image%2Fjpeg&dim=3024x4032`

## Download

In order to download the file, client should search for `kind:1195` events
by unbound list's name and author on the uploader's "file relays".

Filter example:

`{ "authors": ["<pubkey>"], "#n": ["<toHex(sha256(full-file))>|<chunk-size>"], "kinds": [1195] }`

The client must convert each `.content` value from all the `kind:1195` events back to bytes
and concatenate all of them in the correct order to recreate the file.