Spec lazy_load_members and include_redundant_members

[KitsuneRal]

This is an implementation of #1227 - that better (than #1287) matches the current Synapse implementation.

[KitsuneRal]

libQMatrixClient master uses this reading in its lazy-loading implementation.

[turt2live]

Ideally someone like @ara4n who knows what the implementation actually does would be best to review this.

[ara4n]

oh, awesome - thanks @KitsuneRal. will review

[KitsuneRal]

It just occurred to me that we probably will need a separate section in the spec that explains how lazy-loading works from clients' standpoint (mostly reworded sections of https://docs.google.com/document/d/11yn-mAkYll10RJpN0mkYEVqraTbU3U4eQx9MNrzqX1U/edit#heading=h.q5mz9uiufb2g). This also covers MSC688 and MSC1337 though. I see 3 ways to do it:

1. Put this textual explanation to either of the MSCs we already have here, assuming that room summaries and extension of /members will shortly land in spec too.
2. Turn, e.g., this PR into a self-contained spec PR on lazy-loading functionality, including all parts (note that we still don't have a spec PR on MSC688).
3. After MSC1227, MSC688 and MSC1337 land in the repo as YAML files, submit one more spec PR with the text.
   Option 2 look best to me (because of being self-contained) but it may turn a bit too bulky and harder to review in one go (and also MSC1337 brings its own value regardless of lazy-loading) . The next preferable option is probably 3 because then we can land things in the spec in a fine-grained, easier consumed and probably better controlled manner.

[richvdh]

This generally LGTM, though it seems we've lost a paragraph from https://github.com/matrix-org/matrix-doc/pull/1287/files#diff-0365f42e14699ef8c27554ce3860f18fR137 which seemed useful?

[richvdh]

It just occurred to me that we probably will need a separate section in the spec

Yes we do, though now that we have this PR (and #1287...), let's get it landed. #1287 is a good demonstration of how PRs can get stuck for 6 months waiting for the last 10%.

Also, as I note on #1337, that's still in proposal stage, so I certainly wouldn't advise rolling that in here right now.

[KitsuneRal]

@richvdh, thanks for spotting, I will add the missing NB here shortly. So are you after option 1 or option 3 in your second comment?

[richvdh]

As I see it the remaining work here is:

1. document room summaries MSC688
2. write some words giving an overview of how lazy-loading works

I don't have strong feelings about whether that work is done as one or two PRs.

(separately we need to deal with #1337)

[richvdh]

lgtm

[ara4n]

So, i only just got to review this (sorry for lag). Whilst it correctly describes the original intention of LL, it should probably be reworded slightly to reflect the current reality, which is that LL only kicks in for initial sync if enabled. Gappy incremental syncs currently do include membership changes which happened during the gap, otherwise a client who wished to track the full membership of the room (for tab-complete, typing notifs etc) would have to do a full /members sync after every incremental sync... which ends up being worse than non-LL behaviour. The 'correct' solution is to provide an incremental dialect of /members, but that doesn't exist yet. See element-hq/element-web#7211 (comment) for more details.

Also, we need to be explicit that you see yourself in LL members (even if you're not speaking in the room).

I guess these should go in a separate PR?

[KitsuneRal]

Given that this is already merged, any amendment can only happen in a separate PR. Frankly, I consider the above-mentioned a feature rather than a problem. I agree it should be explicitly noted though; will come back with another PR soonish.