created_at time precision

[mattn]

created_at is second-precision. So if you comment without a reply within 1 second, the post and the response may be interchanged. So I suggest that it make created_at be possible to be high precision. If the created_at is greater than 2147483647 (max value of int32), it SHOULD be treated as 64bit value. i.e. it is millisecond unit.
This change must be Independent NIP since this should be exported via NIP-11 relay information.

[alexgleason]

I completely agree, but it would be chaos. I'm not sure we can change it at this point.

Maybe relays could add a event.received_at ms timestamp to events instead. It would be set by the relay instead of the client. That could potentially solve multiple issues without impacting current functionality.

[AsaiToshiya]

Related issue: #626

[vitorpamplona]

Don't sort posts in the same thread by created_at, it's never going to work no matter how precise you make it be.

Nostr's created_at is not a time chain. It's not a good anchor for sorting. But the e-tag graph is. If somebody replies (e tag to a post), you know it happened after the post and before the first reply to that reply. That's all you need to know to assemble the correct order.

[mattn]

Yes, but I'm talking about non-mentioned comment as you know Japanese Nostr people doing.

[vitorpamplona]

I am not sure what you mean. Replies must be tagged otherwise nothing works. Are you thinking about other event kinds?

[alexgleason]

To me this is about feeds, not threads. It matters a lot for pagination. #620

[mattn]

I am not sure what you mean. Replies must be tagged otherwise nothing works. Are you thinking about other event kinds?

Like subtweeting. Japanese people often post notes without replies.
See note1k8f2rjd2vsewhgkr9zlnv87h6cycyh4w4lvf7weafvf7mwqtayusffnm09

[vitorpamplona]

Pagination doesn't need time accuracy. It just needs a fixed order. If you sort by created_at, then id, it should solve your problem.

[mattn]

@vitorpamplona FYI, some people report that Amethyst sometimes displayed with flickering for same timestamp posts.

[alexgleason]

@vitorpamplona We must be talking about different things. I'm using since, until, and limit filters to paginate data from relays. The limit is 20, I get the first page, and then to get the next page I use until: oldestEvent.created_at from the last page. I repeat this process.

But with the current behavior, any other event posted during the same second is potentially gone. I'm not sure whether relays use > or >= for the until filter, and it's not specified in NIP-01. To work around this, I can subtract 1 second from the oldestEvent.created_at. But doing so means each page will always have some events duplicated from the page before it.

[vitorpamplona]

@vitorpamplona FYI, some people report that Amethyst sometimes displayed with flickering for same timestamp posts.

Yep, it's because I still find some parts of the app that only use .created_at to sort posts. Since we added the .id as well 3-4 months ago, most of that flickering has stopped.

But with the current behavior, any other event posted during the same second is potentially gone. I'm not sure whether relays use > or >= for the until filter, and it's not specified in NIP-01. To work around this, I can subtract 1 second from the oldestEvent.created_at. But doing so means each page will always have some events duplicated from the page before it.

Yep, you gotta fix all of these in the client. Nothing guarantees that the pagination content is unique. You are going to have even more headaches when the list includes replaceable events. Some relays garbage collect older versions. So, between receiving a new event and running garbage collection, each version will show up in the list.

[vitorpamplona]

Japanese people often post notes without replies

That will be messy no matter what we do with created_at. If people's clocks are slightly off (most of the time, they are), sorting by created_at will be wrong in high-speed messaging, even with millisecond precision.

[mattn]

That will be messy no matter what we do with created_at. If people's clocks are slightly off (most of the time, they are), sorting by created_at will be wrong in high-speed messaging, even with millisecond precision.

Yes, for example, nostr bot which subtweeing against a posts must set +1 for original post's created_at always to avoid the crossed.

https://github.com/mattn/cloudflare-nostr-nullpoga/blob/10fed3beef8f21608cb949f5c6cbaefc3d819833/src/index.ts#L117

[fiatjaf]

Maybe this is a good idea, maybe not, but we can't change it.

[alexgleason]

Btw, I was wrong that NIP-01 isn't clear about since and until comparisons. Both use >=/<= and not >/<

The since and until properties can be used to specify the time range of events returned in the subscription. If a filter includes the since property, events with created_at greater than or equal to since are considered to match the filter. The until property is similar except that created_at must be less than or equal to until. In short, an event matches a filter if since <= created_at <= until holds.

So you should not have to add or subtract any value from the last created_at to do pagination. The next page should always return the last event from the previous page. For better or worse.

[vitorpamplona]

But can you trust every relay implements this correctly? :)

[mikedilger]

1. We can't change created_at without breaking everything.
2. NIP-11 relay support doesn't help this. created_at is generated by the clients, not the relays.
3. Even precisely timestamped events will show up out of order because people's clocks are not all synchronized.

However,

4. Relays could have a received_at and offer queries by that. I think this has been suggested multiple times.
5. If we do a relay timestamp solution, it should be more precise to solve the pagination problem.

I see that as a possible path forward.

[arthurfranca]

Relays could have a received_at and offer queries by that. I think this has been suggested multiple times.

@mikedilger Have you seen #579? It currently asks relays to use event.nip34seen = Math.floor(Date.now() / 1000) to store a new field but I could remove the / 1000 part. Then clients can filter using { kinds: [...], until: ..., limit: ..., nip34: "seen" }

But this doesn't seem to solve the problem because the relay may receive an old event and set a new received date (nip34seen).