Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Optimise backfill calculation #12522

Merged
merged 16 commits into from
Apr 26, 2022
Merged

Optimise backfill calculation #12522

merged 16 commits into from
Apr 26, 2022

Commits on Apr 25, 2022

  1. skip a dict construction 

    we may as well just chain together the two inputs
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    cd4d26f View commit details
    Browse the repository at this point in the history

  2. Move some filtering and sorting logic earlier 

    We can potentially skip some expensive db work by moving this synchronous code
earlier. The `sorted` might be expensive, but nowhere near as expensive as the
db lookups.
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    9a68a56 View commit details
    Browse the repository at this point in the history

  3. update some comments and logs

    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    8843069 View commit details
    Browse the repository at this point in the history

  4. Use lists of tuples instead of Dicts 

    Both of these queries `GROUP BY b.event_id`, so there's no deduplication being
done, and the dict has no value for us.
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    8feaece View commit details
    Browse the repository at this point in the history

  5. Use an attrs to track backfill points 

    Tuples are annoying.
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    b945de6 View commit details
    Browse the repository at this point in the history

  6. Bail out when we find a visible extremity 

    Rather than checking all of the backwards extremities (which may be legion), we
check one at a time and bail out as soon as we find a visible one
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    a1bd48f View commit details
    Browse the repository at this point in the history

  7. Use get_events_as_list 

    We don't need the dict here.
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    cb88081 View commit details
    Browse the repository at this point in the history

  8. Include insertion events in the filtered_extremities checks

    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    1f0a63e View commit details
    Browse the repository at this point in the history

  9. Only request extremities which we think will be visible

    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    d285af2 View commit details
    Browse the repository at this point in the history

  10. add a TODO comment

    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    4ac692f View commit details
    Browse the repository at this point in the history

  11. Make get_successor_events take a single event 

    ... since all its callers now only pass a single event
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    bdfa5a7 View commit details
    Browse the repository at this point in the history

  12. changelog

    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    a23e5a1 View commit details
    Browse the repository at this point in the history

  13. Update synapse/handlers/federation.py 

    Co-authored-by: David Robertson <[email protected]>
    @richvdh
    richvdh and David Robertson authored Apr 25, 2022
    Configuration menu
    Copy the full SHA
    881d794 View commit details
    Browse the repository at this point in the history

  14. More comment updates, per review

    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    8487963 View commit details
    Browse the repository at this point in the history

  15. Use a list rather than a set 

    It's impossible for us to have duplicates here, so we may as well use a boring
list.
    @richvdh
    richvdh committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    b75b3f5 View commit details
    Browse the repository at this point in the history

  16. Update and rename 12522.misc to 12522.bugfix

    @richvdh
    richvdh authored Apr 25, 2022
    Configuration menu
    Copy the full SHA
    1dfe084 View commit details
    Browse the repository at this point in the history