-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Don't fetch state for missing events that we fetched #2170
Conversation
For the record, this was introduced in synapse 0.20, in 3406333 @erikjohnston: the intention was that |
I think its more explicit to have it done in the outer function. WIll remove from _get_missing |
) | ||
else: | ||
logger.info("Found all missing prev events for %s", pdu.event_id) | ||
defer.returnValue(have_seen) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there are other calls to defer.returnValue
in this function; please update them too
also please update the docstring.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
while you're in there, the docstring claims that prevs
should be a list, which looks like a lie. my fault, but could you fix it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
synapse/handlers/federation.py
Outdated
[ev for ev, _ in pdu.prev_events] | ||
) | ||
|
||
seen = set(have_seen.keys()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
haven't we already got all this at https://github.com/matrix-org/synapse/pull/2170/files#diff-3f6de39b112e0c501c3c2925a571b908R199 ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
well, it looks very similar to me. what's the difference?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
have_seen
has been updated the line above, and have_seen
needs to be rechecked as we just fetched some missing events. Is that what you mean? Or do you mean we should wrap the call to have_seen
and seen
in a function?
synapse/handlers/federation.py
Outdated
# Update the set of things we've seen after trying to | ||
# fetch the missing stuff | ||
have_seen = yield self.store.have_events( | ||
[ev for ev, _ in pdu.prev_events] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use prevs
here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still unclear why it looks like we're duplicating code here
I am completely failing to understand what you mean I'm afraid :( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a horrible, horrible bit of code
but I think it looks ok now
Changes in synapse v0.21.0 (2017-05-18) ======================================= No changes since v0.21.0-rc3 Changes in synapse v0.21.0-rc3 (2017-05-17) =========================================== Features: * Add per user rate-limiting overrides (PR matrix-org#2208) * Add config option to limit maximum number of events requested by ``/sync`` and ``/messages`` (PR matrix-org#2221) Thanks to @psaavedra! Changes: * Various small performance fixes (PR matrix-org#2201, matrix-org#2202, matrix-org#2224, matrix-org#2226, matrix-org#2227, matrix-org#2228, matrix-org#2229) * Update username availability checker API (PR matrix-org#2209, matrix-org#2213) * When purging, don't de-delta state groups we're about to delete (PR matrix-org#2214) * Documentation to check synapse version (PR matrix-org#2215) Thanks to @hamber-dick! * Add an index to event_search to speed up purge history API (PR matrix-org#2218) Bug fixes: * Fix API to allow clients to upload one-time-keys with new sigs (PR matrix-org#2206) Changes in synapse v0.21.0-rc2 (2017-05-08) =========================================== Changes: * Always mark remotes as up if we receive a signed request from them (PR matrix-org#2190) Bug fixes: * Fix bug where users got pushed for rooms they had muted (PR matrix-org#2200) Changes in synapse v0.21.0-rc1 (2017-05-08) =========================================== Features: * Add username availability checker API (PR matrix-org#2183) * Add read marker API (PR matrix-org#2120) Changes: * Enable guest access for the 3pl/3pid APIs (PR matrix-org#1986) * Add setting to support TURN for guests (PR matrix-org#2011) * Various performance improvements (PR matrix-org#2075, matrix-org#2076, matrix-org#2080, matrix-org#2083, matrix-org#2108, matrix-org#2158, matrix-org#2176, matrix-org#2185) * Make synctl a bit more user friendly (PR matrix-org#2078, matrix-org#2127) Thanks @APwhitehat! * Replace HTTP replication with TCP replication (PR matrix-org#2082, matrix-org#2097, matrix-org#2098, matrix-org#2099, matrix-org#2103, matrix-org#2014, matrix-org#2016, matrix-org#2115, matrix-org#2116, matrix-org#2117) * Support authenticated SMTP (PR matrix-org#2102) Thanks @DanielDent! * Add a counter metric for successfully-sent transactions (PR matrix-org#2121) * Propagate errors sensibly from proxied IS requests (PR matrix-org#2147) * Add more granular event send metrics (PR matrix-org#2178) Bug fixes: * Fix nuke-room script to work with current schema (PR matrix-org#1927) Thanks @zuckschwerdt! * Fix db port script to not assume postgres tables are in the public schema (PR matrix-org#2024) Thanks @jerrykan! * Fix getting latest device IP for user with no devices (PR matrix-org#2118) * Fix rejection of invites to unreachable servers (PR matrix-org#2145) * Fix code for reporting old verify keys in synapse (PR matrix-org#2156) * Fix invite state to always include all events (PR matrix-org#2163) * Fix bug where synapse would always fetch state for any missing event (PR matrix-org#2170) * Fix a leak with timed out HTTP connections (PR matrix-org#2180) * Fix bug where we didn't time out HTTP requests to ASes (PR matrix-org#2192) Docs: * Clarify doc for SQLite to PostgreSQL port (PR matrix-org#1961) Thanks @benhylau! * Fix typo in synctl help (PR matrix-org#2107) Thanks @HarHarLinks! * ``web_client_location`` documentation fix (PR matrix-org#2131) Thanks @matthewjwolff! * Update README.rst with FreeBSD changes (PR matrix-org#2132) Thanks @feld! * Clarify setting up metrics (PR matrix-org#2149) Thanks @encks!
After fetching missing events we didn't update the list of events that were missing, so while we did successfully fetch the missing events it still thought it had a hole. This meant that it fetched the state, which adds delays, CPU time and increased DB usage (since we don't delta encode those state sets)