presence processing pins the cpu for hours after joining a large room

[richvdh]

for literally hours after joining a large room like HQ, we get stuck in a loop that looks like this:

2019-05-24 12:46:37,301 - synapse.handlers.presence - 826 - DEBUG - presence.notify_new_event-91- Handling: 'm.room.member' '@freenode_aOssed-:matrix.org', $1534927445346266LDGwg:matrix.org
2019-05-24 12:46:37,301 - synapse.handlers.presence - 826 - DEBUG - presence.notify_new_event-91- Handling: 'm.room.member' '@ziccardi:matrix.org', $1542290119314550ekjkP:matrix.org
2019-05-24 12:46:37,301 - synapse.state - 178 - DEBUG - presence.notify_new_event-91- calling resolve_state_groups from get_current_users_in_room
2019-05-24 12:46:37,301 - synapse.state - 358 - DEBUG - presence.notify_new_event-91- resolve_state_groups event_ids ['$155869549715439HHnCI:matrix.org']
2019-05-24 12:46:37,315 - synapse.federation.sender.per_destination_queue - 169 - DEBUG - presence.notify_new_event-91- TX [matrix.org] Transaction already in progress
2019-05-24 12:46:37,315 - synapse.handlers.presence - 826 - DEBUG - presence.notify_new_event-91- Handling: 'm.room.member' '@debirs:matrix.org', $15535182641013390zENdq:matrix.org
2019-05-24 12:46:37,315 - synapse.state - 178 - DEBUG - presence.notify_new_event-91- calling resolve_state_groups from get_current_users_in_room
2019-05-24 12:46:37,316 - synapse.state - 358 - DEBUG - presence.notify_new_event-91- resolve_state_groups event_ids ['$155869549715439HHnCI:matrix.org']
2019-05-24 12:46:37,330 - synapse.federation.sender.per_destination_queue - 169 - DEBUG - presence.notify_new_event-91- TX [matrix.org] Transaction already in progress
2019-05-24 12:46:37,330 - synapse.handlers.presence - 826 - DEBUG - presence.notify_new_event-91- Handling: 'm.room.member' '@root:matrix.silverlane.dotorg.org', $15461114867NGgsb:matrix.silverlane.dotorg.org
2019-05-24 12:46:37,330 - synapse.handlers.presence - 826 - DEBUG - presence.notify_new_event-91- Handling: 'm.room.member' '@freenode_rubidious:matrix.org', $15308846451400755JLpKP:matrix.org
2019-05-24 12:46:37,330 - synapse.handlers.presence - 826 - DEBUG - presence.notify_new_event-91- Handling: 'm.room.member' '@pecan:atlantisnet.bg', $153494369824evggm:atlantisnet.bg

It makes the server unusable after joining a large room.

[richvdh]

this contributes to the general joining-rooms-is-awful of element-hq/element-web#1211 / https://github.com/vector-im/riot-web/issues/2076

[richvdh]

oh, I think this might be a dup of element-hq/element-web#5157, since I'm seeing that error repeatedly too. I'd be inclined to fix that and see if this goes away.

[richvdh]

well, I think it's better than it used to be. there's still lots of presence; cf element-hq/element-web#3120.