Add forgotten status to Room Details API

[dklimpel]

Relates to:

• Purge room when all users have forgotten it #12170

Add requested tests:

• Purge room when all users have forgotten it #12170 (review)

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file.
• Pull request includes a sign off
• Code style is correct
  (run the linters)

Signed-off-by: Dirk Klimpel [email protected]

[DMRobertson]

Drive-by comment: it might(?) be more performant to SELECT count(*) > 0 here. That gives the database a chance to stop the query as soon as it finds the first membership that hasn't been forgotten.

(There's also an index "room_memberships_user_room_forgotten" btree (user_id, room_id) WHERE forgotten = 1 but I'm not sure if there's an easy way to use that here.)

[dklimpel]

synapse/synapse/storage/databases/main/roommember.py

Lines 1188 to 1207 in 8fa7eb8

	def _get_forgotten_rooms_for_user_txn(txn: LoggingTransaction) -> Set[str]:
	# This is a slightly convoluted query that first looks up all rooms
	# that the user has forgotten in the past, then rechecks that list
	# to see if any have subsequently been updated. This is done so that
	# we can use a partial index on `forgotten = 1` on the assumption
	# that few users will actually forget many rooms.
	#
	# Note that a room is considered "forgotten" if *all* membership
	# events for that user and room have the forgotten field set (as
	# when a user forgets a room we update all rows for that user and
	# room, not just the current one).
	sql = """
	SELECT room_id, (
	SELECT count(*) FROM room_memberships
	WHERE room_id = m.room_id AND user_id = m.user_id AND forgotten = 0
	) AS count
	FROM room_memberships AS m
	WHERE user_id = ? AND forgotten = 1
	GROUP BY room_id, user_id;
	"""

This looks similar, but is very complicated.

[dklimpel]

I try to create a better sql query. I am feeling in a rabbit hole. It seems that Synapse is not possible to forget a room, at the moment. I think there is a bug in:

synapse/synapse/handlers/room_member.py

Lines 1912 to 1927 in e825f73

	async def forget(self, user: UserID, room_id: str) -> None:
	user_id = user.to_string()
	member = await self._storage_controllers.state.get_current_state_event(
	room_id=room_id, event_type=EventTypes.Member, state_key=user_id
	)
	membership = member.membership if member else None
	if membership is not None and membership not in [
	Membership.LEAVE,
	Membership.BAN,
	]:
	raise SynapseError(400, "User %s in room %s" % (user_id, room_id))
	if membership:
	await self.store.forget(user_id, room_id)

There is a gap of unit tests.

[dklimpel]

Proposal for an other query:

SELECT room_id, ( 
 SELECT count(*) > 0 FROM room_memberships 
 WHERE room_id = m.room_id AND forgotten = 0 
) AS count 
FROM room_memberships AS m 
WHERE room_id = ? AND forgotten = 1 
GROUP BY room_id;

Steps

1. Query if the room is at least one time forgotten - uses partial index on forgotten = 1
2. Query this rooms (result of no. 1) that there is no user which has not forgotten (or rejoin) this room

Results

• count == 0 -> room is forgotten
• count > 0 -> room is not fully forgotten but at least one time
• no result / count is None -> room is not in list (unknown) or room is not at least one time forgotten

[erikjohnston]

The query plan looks sensible to me

[DMRobertson]

I think we should note that this was added in Synapse 1.66. I'll add something to this file now while preparing the release.