Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Keep track when we try to backfill an event #13621

Closed
Tracked by #14284
MadLittleMods opened this issue Aug 25, 2022 · 0 comments · Fixed by #13589
Closed
Tracked by #14284

Keep track when we try to backfill an event #13621

MadLittleMods opened this issue Aug 25, 2022 · 0 comments · Fixed by #13589
Assignees
@MadLittleMods
Labels
A-Messages-Endpoint /messages client API endpoint (`RoomMessageListRestServlet`) (which also triggers /backfill) A-Performance Performance, both client-facing and admin-facing O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Minor Blocks non-critical functionality, workarounds exist. T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Milestone
Faster /messages

Comments

@MadLittleMods
Copy link
Contributor

MadLittleMods commented Aug 25, 2022 •
edited
Loading

Mentioned in internal doc about speeding up /messages. Related to #13356

Keep track when we try to backfill an event. In other words, when an event fails to backfill, we can record it in the database.

This is a building block so we can later:

  1. Only try to backfill from an event if we haven't tried recently -> Only try to backfill event if we haven't tried before recently #13622
  2. When we decide to backfill that event again, process it in the background so it doesn't block and make /messages slow when we know it will probably fail again -> Process previously failed backfill events in the background #13623
@MadLittleMods MadLittleMods added the A-Messages-Endpoint /messages client API endpoint (`RoomMessageListRestServlet`) (which also triggers /backfill) label Aug 25, 2022
This was referenced Aug 25, 2022
Merged
Closed
Closed
@MadLittleMods MadLittleMods self-assigned this Aug 25, 2022
@DMRobertson DMRobertson added S-Minor Blocks non-critical functionality, workarounds exist. T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. A-Performance O-Uncommon Most users are unlikely to come across this or unexpected workflow A-Performance Performance, both client-facing and admin-facing and removed A-Performance labels Aug 25, 2022
@MadLittleMods MadLittleMods mentioned this issue Sep 1, 2022
Open
9 tasks
MadLittleMods added a commit that referenced this issue Sep 14, 2022
@MadLittleMods
Keep track when we try and fail to process a pulled event (#13589)
957e3d7 
We can follow-up this PR with:

 1. Only try to backfill from an event if we haven't tried recently -> #13622
 1. When we decide to backfill that event again, process it in the background so it doesn't block and make `/messages` slow when we know it will probably fail again -> #13623
 1. Generally track failures everywhere we try and fail to pull an event over federation -> #13700

Fix #13621

Part of #13356

Mentioned in [internal doc](https://docs.google.com/document/d/1lvUoVfYUiy6UaHB6Rb4HicjaJAU40-APue9Q4vzuW3c/edit#bookmark=id.qv7cj51sv9i5)
@kittykat kittykat mentioned this issue Oct 24, 2022
Closed
7 tasks
MadLittleMods added a commit that referenced this issue May 12, 2023
@MadLittleMods
Process previously failed backfill events in the background
fd26164 
Fix #13623

Follow-up to #13621 and #13622

Part of making `/messages` faster: #13356
@MadLittleMods MadLittleMods mentioned this issue May 12, 2023
Merged
5 tasks
@MadLittleMods MadLittleMods mentioned this issue May 23, 2023
Open
MadLittleMods added a commit that referenced this issue May 25, 2023
@MadLittleMods
Process previously failed backfill events in the background (#15585)
77156a4 
Process previously failed backfill events in the background because they are bound to fail again and we don't need to waste time holding up the request for something that is bound to fail again.

Fix #13623

Follow-up to #13621 and #13622

Part of making `/messages` faster: #13356
@matrixbot matrixbot mentioned this issue Dec 21, 2023
Open
9 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@MadLittleMods MadLittleMods

Labels
A-Messages-Endpoint /messages client API endpoint (`RoomMessageListRestServlet`) (which also triggers /backfill) A-Performance Performance, both client-facing and admin-facing O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Minor Blocks non-critical functionality, workarounds exist. T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Projects
None yet
Milestone
Faster /messages
Development

Successfully merging a pull request may close this issue.

Keep track when we try and fail to process a pulled event matrix-org/synapse
2 participants
@MadLittleMods @DMRobertson