update NIP-65: add DM relays, clarify when to use all vs some of a relay kind

[mikedilger]

We have talked about 'DM' relays for a long time now but we never put them in NIP-65.

Three main changes of note

1. DM relays added. Fallback goes to READ relays

2. Posting a copy of DMs that you send out to your own DM relays (instead of your own WRITE relays). Clients will have to look for them in both places due to the transition, but this is clearly more secure.

3. In general, when interacting with a user's own relays, clients should interact with all of them (write to all WRITE relays, read from all READ and DM relays). When interacting with other user's relays, clients need only interact with a subset of them (read from some of their WRITE relays, write to some of their READ and DM relays).

   I'm not sure if that is best, but it is how my client has always operated, and clients should probably have a unified understanding of what is expected. Of course relays can be down so oftentimes it is inadvertently a "subset" even when you attempt to post to "all". But we should at least be trying.

[fiatjaf]

What if we rename the DM relays tag to ["dm", "wss://..."]? No one is using it yet so it won't break anything, and we save one index entry for relays.

[mikedilger]

I think we can do that.

[staab]

I think this is an improvement, but I'm a little worried about the UX, since users need to configure yet another category of relay. Also a nitpick, what about "authenticated-read" or something to support other private read use cases? Either r or a new tag name is fine with me.

[mikedilger]

The 'r' tags were my mistake that became de-facto standard. So anything new should have a different tag. Could just be 'relay' with markers, but I'm not very opinionated on this aspect.

What is an authenticated-read relay? Why would you need to advertise it? If I put things on a relay that require authentication to read, I wouldn't list it in my 10002 event. Maybe you are saying "please subscribe to my paid service?"

[vitorpamplona]

If r tags were a mistake, lets fix this before it gets too late. We can deprecate them and move to a new tag.

[mikedilger]

If we deprecate 'r' and switch to 'relay' (for example), there will be a transition period where new clients have to write both, and old clients may or may not preserve the new tags. Then once everybody uses the new tags, all the clients have to change again to not bothering to write the old 'r' tags. I'm not against doing this.

But should they be 'relay' with marker? Or something different (perhaps without a marker)?

[staab]

Marks are evil IMO.

Maybe authenticated-read is the wrong term, I'm just thinking of use cases other than DMs (like encrypted group messages) that should be treated similarly.

[mikedilger]

Relays can be used for any combination of (inbox, outbox, dm) and maybe a fourth kind @staab is talking about. So those can't be the tags unless we are adding the same relay multiple times.

Given this example:

example1 is both 'inbox' and 'dm'
example2 is both 'inbox' and 'outbox'
example3 is just 'outbox'

So here is the option of adding relays multiple times (including the lines needed for backwards compatibility):

[
    ["inbox", "wss://example1.com"],
    ["inbox", "wss://example2.com"],
    ["outbox", "wss://example2.com"],
    ["outbox", "wss://example3.com"],
    ["dm", "wss://example1.com"],
    ["r", "wss://example1.com", "read"],
    ["r", "wss://example2.com"],
    ["r", "wss://example3.com", "write"],
]

Markers make it shorter, but we would need multiple markers per relay (or an encoded marker which I'm not showing).

[
    ["relay", "wss://example1.com", "inbox", "dm"],
    ["relay", "wss://example2.com", "inbox", "outbox"],
    ["relay", "wss://example3.com", "outbox"],
    ["r", "wss://example1.com", "read"],
    ["r", "wss://example2.com"],
    ["r", "wss://example3.com", "write"],
]

[mikedilger]

I'm not against encoding the usage as a bit vector. bit0=write, bit1=read, bit2=dm. Then there is just a single marker. The tag could be "relayusage".

[mikedilger]

Here is how it looks with bit flags (my new preferred option)

[
   ["relayusage", "wss://example1.com", "6"],
   ["relayusage", "wss://example2.com", "3"],
   ["relayusage", "wss://example3.com", "1"],
   ["r", "wss://example1.com", "read"],
   ["r", "wss://example2.com"],
   ["r", "wss://example3.com", "write"],
]

[jb55]

On Tue, Jan 16, 2024 at 01:41:49PM -0800, Michael Dilger wrote: Here is how it looks with bit flags (my new preferred option) ```` [ ["relayusage", "wss://example1.com", 6], ["relayusage", "wss://example2.com", 3], ["relayusage", "wss://example3.com", 1], ["r", "wss://example1.com", "read"], ["r", "wss://example2.com"], ["r", "wss://example3.com", "write"], ] ````

To keep it simple and more descriptive, you could use single-char flags. It could be "rwd", "rw", "d", etc. Then you wouldn't need "relayusage"

[vitorpamplona]

The last change in this PR suggests that DM is both a READ and a WRITE relay:

DMs SHOULD only be broadcasted to the recipient's and author's DM relays.

Which I think it is correct, but here it says that it only works as a READ relay, which I think it is incorrect. DM relays are indeed for both write and read.

PS: The READ base works for the current GiftWrap design (the signing key is always random and thus no WRITE is needed), but there may be other encryptions that are signed by the author.

[mikedilger]

Is there a case where somebody comes to your DM relay to read your DMs? If not, it is inbox only. Just because you write into it doesn't make it an outbox. I would leave it more flexible, except that if we make these outboxes, what was the point of special DM relays that leak your DMs?

[vitorpamplona]

Is there a case where somebody comes to your DM relay to read your DMs?

Only for those gathering metadata from DMs (which does happen today already). Do we consider a shared-secret group, like what @staab has, a "DM"? They are encrypted messages but anyone with the key can access them.

It wasn't clear to me that this PR requires DM relays to authenticate who is connecting and only provide the appropriate messages to that user. If that is the case then it makes sense to be read. Maybe this is an expected behavior of DM relays and should be added here?

[staab]

Seems to me that it should act like a read relay. Auth should be recommended but optional. This works for shared key groups too, but isn't relevant to nip 65 because the group metadata defines the relays to use, not the recipient's profile.

[mikedilger]

I think it depends on the specific DM spec. I was imaging a copy lands in my read relay (for me) and my partner's read relay (for them). But maybe this NIP should defer to the DM NIPs.

[dluvian]

I am against this.

I prefer to keep kind:10002 small. DM relays are a very specific usecase, so we should put them in their own kind. Since not all clients handle DMs, it's best not to make them fetch and deal with it.

[arthurfranca]

Imo it's better to use ru instead of relay-usage as tag key to make the event smaller (we should strive to save space cause relays aren't supposed to delete them often). Two letters is enough for relays to skip indexing.

[mikedilger]

Sounds good. The Russians will love it ;-)

[fiatjaf]

I am against this.

I prefer to keep kind:10002 small. DM relays are a very specific usecase, so we should put them in their own kind. Since not all clients handle DMs, it's best not to make them fetch and deal with it.

I agree. Let's make another kind just for the DMs.

[mikedilger]

Ok I can leave it to the DM standard to do that.

I'm kind of against fixing the 'r' tag because it blow up the size with backwards-compatible entries. I just did it to demonstrate. I think the better solution is to just say that relays don't have to index this particular 'r' tag, even though that is a bit ugly.

[staab]

Agreed, let's add a new kind for DM preferences. It will reduce contention around lists, which is always an annoying problem.

[mikedilger]

Ok I made a new kind, 10102. Thank god for nostr.

The new kind also replaces the old kind (which had multiple warts such as that "r" tag).

I also squashed everything and force-pushed.

[staab]

I don't think we should deprecate the old format, as nice as the new one is. Also, as discussed somewhere on this pr adding a general-purpose DM relay type technically applies to nip 04 DMs, which could hurt deliverability in the short term, so it should probably be scoped to new-style DMs and any other NIPs that emerge which require special handling of relay selections.

[nostr-wine]

Don't have any strong preferences on how, just want to chime in and say we definitely support some way to signal where a user wants to receive private events.

We built a relay specifically for this use case a while back (https://docs.nostr.wine/inbox/readme) but it desperately needs something like this PR. I believe the use case goes beyond just DMs and could include NWC events, private DVM results, and other private events where preventing metadata leaks is useful.

[vitorpamplona]

IF d is only for DMs, like the text is saying, then I prefer defining a new relay list kind on the DM NIP itself. #686 Other specific uses might need other types of relays and it's fine to add new relay lists for those types.

IF d includes any private direct event containing whatever is supposed to be private just for this user, then:

1. The marker should not be called DM.
2. I prefer to define it here.
3. We need a clear definition of what this relay is, so that other NIPs can re-use the infrastructure.

[mikedilger]

Staab: Are you suggesting we just put DMs on the new list, and leave read/write to continue to be defined by 10002 still?

Vitor: I think 'd' could be for any private direct event, including giftwraps etc. I'm fine with renaming. What do you suggest?

[staab]

Are you suggesting we just put DMs on the new list, and leave read/write to continue to be defined by 10002 still?

Yes, and we should define the DMs per nip. Because for compatibility legacy DMs should continue to be written to the same places, but NIP 17 DMs should be written to the nip-specific relay selections. Also,

including giftwraps etc

I don't think this is a good idea, because gift wraps aren't always addressed to a particular user (like with NIP 87 groups, which are addressed to a shared key and sent to the group's relays). Another reason why relay selections should be defined per-use-case.

[mikedilger]

Are you suggesting we just put DMs on the new list, and leave read/write to continue to be defined by 10002 still?

Yes, and we should define the DMs per nip. Because for compatibility legacy DMs should continue to be written to the same places, but NIP 17 DMs should be written to the nip-specific relay selections. Also,

All NIP-04 says is to use relays that do AUTH to restrict who can fetch your NIP-04s. How does that conflict in any way with other DM nips? I can't even imagine why some DM systems would require different relays (other than groups which I address below).

including giftwraps etc

I don't think this is a good idea, because gift wraps aren't always addressed to a particular user (like with NIP 87 groups, which are addressed to a shared key and sent to the group's relays). Another reason why relay selections should be defined per-use-case.

Nothing about a relay marked "d" means that every NIP must use it to store their messages. It is the general "DM" inbox, but NIPs that do things differently can still do things differently.

[staab]

How does that conflict in any way with other DM nips?

So currently there's no recommendation. If we add one, clients may start sending messages to different relays, which could result in backwards-incompatible behavior (conversations getting dropped). The current state is dumb anyway, so this is not a big deal, I'm just attempting to be conscientious about compatibility.

Nothing about a relay marked "d" means that every NIP must use it to store their messages. It is the general "DM" inbox, but NIPs that do things differently can still do things differently.

This is fine, but it seems like a moot point because NIP 17 is the only spec that uses encryption + relay selections right? So I'd prefer relay recommendations to be defined near where they're useful. Unless we decide it's fine to apply the same recommendation to NIP 04, in which case adding it to NIP 65 is probably better.

All fine distinctions, I'm saying basically the same thing Vitor is.

[mikedilger]

Maybe I'm not the right guy to be doing this PR, because I'm just confused at this point. Could somebody that understands these concerns take this over please?

[staab]

Got you covered, Vitor added this to a "Publishing" section in NIP 17: https://github.com/nostr-protocol/nips/pull/686/files#r1515931334

[mikedilger]

Ok I had not read all the convos staab/vitor/fiatjaf were having on this yesterday and the day before. My github notifications were flooded yesterday. I'm still working through reading them.

The idea of having a separate relay list for a separate DM system signals that the user accepts that kind of DM is not a bad idea. But that is the only positive I see from doing it that way.

I don't think there is any difference to the users. Why would I specify different relays for NIP-17 and NIP-04? I wouldn't, and that complexity just confuses people. It will confuse 90% of developers too.

I am already dreading the code bloat for managing yet another relay list event. Maybe I'll just stick to NIP-04.

[staab]

I don't think there is any difference to the users.

I agree that UX is the main sticking point. But since everything is generalizable to read/write relays, it might be pretty easy to solve by adding an "advanced" mode that people can opt into if they care about inbox privacy.

[arthurfranca]

After reading your discussions, i think this NIP could say that the "io" flags could be extended with other one-char flags defined on different NIPs. Then the DM NIP can choose a char flag such as the "d" one. Then in the future another NIP can choose another new one or reuse "i", or "o", or "d".

This is the inverse of what I previously believed was the best to do. Now I think relay specialization could be good.

That's because some NIPs require special treatment of events (new ones created by the NIP) by relays that the current users' inbox/outbox relays may not honor.

So for example a client that runs its own relay could easily add an entry to the user's kind:10102 list with its own relay it knows will be able to support a new NIP feature using the corresponding NIP flag "z", without replacing the user's inbox/outbox ones.

[arthurfranca]

Really liked the usage flag mechanics.

Here I defined a new s relay usage flag to let clients configure/use user's "scheduler" relays, needed for time-releasing "delegation events".

Such relays could also be used to enable regular post scheduling feature on clients by setting two flags: os.

I say ship it! 🚀

[dluvian]

Pls god just use a new kind for these PRIVATE INBOX relays and let this nip be a simple read-write signalling mechanism.

Overloading event kinds with features is not the way to achieve interoperability across clients. Not every client has a need for private messages and therefore will just ignore/overwrite the p usage flag.

Every time we overload things it makes the dev experience worse and clients less efficient like kind 1 being root and reply posts. Or e tags marking root, parent, quote, repost and what not.

Think of the whole ecosystem instead of just the ease of implementing it in your own clients.

[mikedilger]

I have added that clients should preserve letters they don't understand.

I have already heard about other plants to add yet more letters that I will not be supporting. But I think I can preserve them.

As for clobbering (because you didn't have the latest list), that is a general issue that is already with us irrespective of this PR and can't be fixed or avoided here. Making a new kind just for PRIVATE INBOX, you still can clobber if you didn't have the latest event and you overwrite it.

[vitorpamplona]

@mikedilger, I intend to use something like #1148 to allow our Push Notification server access to the private inbox relays defined here. Let me know if you see any problems with this.

[vitorpamplona]

FYI, yesterday's release of the Draft Events #1124 led me to imagine a private outbox relay: a relay that only allows the signer to download their own posts.

I never thought this would be useful, but it actually is.

Things like App settings, Drafts, Read Statuses #933 exist for the benefit of the author alone. And those can indeed be stored in a private "outbox". Though the "outbox" name is weird because it's not "outboxing" anything.

It doesn't need to be added to this PR. But it is something that should be on everybody's radar.

[mikedilger]

It doesn't need to be added to this PR. But it is something that should be on everybody's radar.

Since it is private, there is no reason to add it to NIP-65 which is about advertising to others. But yes I agree this is a kind of relay usage.

[fiatjaf]

@dluvian is 100% correct.

[mikedilger]

Ok, I concede. Someone else will have to do it though.