v0.98.1

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync Performance release in preparation for v1.

This version is not compatible with previous versions due to changes in the SQL schema on syncv3_snapshots and syncv3_account_data.

Features:

• to_device messages, OTK counts and device list changes now go through to cilents immediately when they change upstream. Previously they were piggybacked off other data.

Bug fixes:

• When a range is shrinked from both ends, we would incorrectly send back
  INVALIDATE twice for the same range (the higher end). This was caused by
  improper usage of Go for .. range loops.
• When a range is changed to a window completely outside the room list
  (e.g 5 rooms, range=[10,15]) we would incorrectly send back SYNC 10,15
  with a single room ID, the last room in the list. This was caused by
  boundary check code in SliceInto capping ranges to the last element if
  the index positions are > room list. However in this case, this causes
  the last room to be returned.

Improvements:

• The proxy now logs content["org.matrix.msgid"] fields if they are present on to-device messages at DEBUG level.
• The proxy now warns when clients fail to increase their to-device since token.
• Proxy startup times have been improved with optimised SQL queries.
• The to_device table has had an extra index added to improve performance.
• When >1 poller exists for the same user, the 2nd+ poller wil do their initial sync request filtering out all rooms in an effort to get to_device msgs more rapidly.
• BREAKING: Add stream IDs to account data table.
• BREAKING: Split the event nid array for snapshots into two: membership events and other events. This is used to massively improve performance of looking up required_state when the filter does not include any m.room.member events.

v0.98.0

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync Major release preparing for an official v1.

Features:

• Add typing extension.
• Add receipts extension.
• Add comprehensive prometheus /metrics activated via SYNCV3_PROM.
• Add SYNCV3_PPROF support.
• Add by_notification_level sort order.
• Add include_old_rooms support.
• Add support for $ME and $LAZY.
• Add correct filtering when *,* is used as required_state.
• Add num_live to each room response to indicate how many timeline entries are live.

Bug fixes:

• Use a stricter comparison function on ranges: fixes an issue whereby UTs fail on go1.19 due to change in sorting algorithm.
• Send back an errcode on HTTP errors (e.g expired sessions).
• Remove unsigned.txn_id on insertion into the DB. Otherwise other users would see other users txn IDs :(
• Improve range delta algorithm: previously it didn't handle cases like [0,20] -> [20,30] and would panic.
• Send HTTP 400 for invalid range requests.
• Don't publish no-op unread counts which just adds extra noise.
• Fix leaking DB connections which could eventually consume all available connections.
• Ensure we always unblock WaitUntilInitialSync even on invalid access tokens. Other code relies on WaitUntilInitialSync() actually returning at some point e.g on startup we have N workers which bound the number of concurrent pollers made at any one time, we need to not just hog a worker forever.

Improvements:

• Greatly improve startup times of sync3 handlers by improving JoinedRoomsTracker: a modest amount of data would take ~28s to create the handler, now it takes 4s.
• Massively improve initial initial v3 sync times, by refactoring JoinedRoomsTracker, from ~47s to <1s.
• Add SlidingSyncUntil... in tests to reduce races.
• Tweak the API shape of JoinedUsersForRoom to reduce state block processing time for large rooms from 63s to 39s.
• Add trace task for initial syncs.
• Include the proxy version in UA strings.
• HTTP errors now wait 1s before returning to stop clients tight-looping on error.
• Pending event buffer is now 2000.
• Index the room ID first to cull the most events when returning timeline entries. Speeds up SelectLatestEventsBetween by a factor of 8.
• Remove cancelled m.room_key_requests from the to-device inbox. Cuts down the amount of events in the inbox by ~94% for very large (20k+) inboxes, ~50% for moderate sized (200 events) inboxes. Adds book-keeping to remember the unacked to-device position for each client.

v0.4.1

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Features:

• Added invited_count and joined_count to the room JSON response, to allow accurate push rule calculations. This should fix an issue whereby rooms could emit bing noises even when there were no new bingable messages.

Bugfixes:

• Fixed a bug whereby a DELETE could specify an index of -1 when a 1-length window was used e.g [[0,0]].
• Fixed numerous issues where move operations could be incorrect.

Improvements:

• Memory usage has been cut significantly when a connection makes use of multiple lists. Previously, the proxy stored independent copies of room metadata per list, whereas now the proxy maintains a single copy per connection.
• Refactored the list delta algorithm which generates DELETE/INSERT operations, with a significantly increased set of unit tests, including deterministically randomised smoke tests.

v0.4.0

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Features:

• Implements a newer version of MSC3575.
• Support tags and not_tags.

Bugfixes:

• Fixed an issue whereby a full connection buffer may not result in a connection being cleaned up, which could cause messages to not be delivered.

v0.3.3

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Bugfixes:

• Fixed an issue where global account data updates were not sent to clients in a timely manner.
• Fixed an issue where sorting by room name did not work correctly when name updates were live-streamed.

v0.3.2

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Bugfixes:

• Fixed a bug which could cause stuck invites.
• Fix a bug which could cause duplicate rooms or missing rooms.

v0.3.1

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Features:

• E2EE extension: Added support for device_unused_fallback_key_types.

Improvements:

• Logging has been updated to include more useful data: the user is logged in more places and the "response line" now includes summary information of the form 13:48:40 INF duration=16783.549798 p=43 q=44 r=1 size=272 status=200 u=@kegan:matrix.org - where p is the client-sent position, q is the new position and r are the total number of rooms returned in rooms.

Bugfixes:

• Fixed a bug which could cause /sync to return HTTP 500 when a live-streamed room join for that user arrived.
• Fixed a bug which could cause invites to be missing invite_state.
• Ensure the name field is kept up-to-date as the room state changes.
• Fixed a potential duplicate event bug which was caused by the dispatcher notifying twice for events if Synapse sent a duplicate join event.

v0.3.0

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Features:

• Implements a newer version of MSC3575.
• Add support for txn_id.

Improvements:

• Buffer server responses to ensure no requests which make it to the server are lost.

Bugfixes:

• When space children are added/removed from the space, correctly update lists tracking that space.
• Fixed a bug where joining a room raced with creating a room subscription.

v0.2.1

This version is backwards incompatible with previous database schemas.

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Features:

• Add support for room_types and not_room_types filter. Breaks the database schema by adding a column to the rooms table to track the room type.
• Add spaces filter.

v0.2.0

This version is backwards incompatible with previous database schemas. In addition, this version has changed the CLI args in a backwards incompatible way.

Docker images available at: https://github.com/matrix-org/sliding-sync/pkgs/container/sliding-sync

Features:

• Implements a newer version of MSC3575.

Improvements:

• The CLI no longer takes flags, and now expects env vars instead for consistency with running in Docker.
• Sync v2 access tokens are now stored encrypted in the database. This improves reliability by allowing the proxy to start poll loops on restarts, rather than waiting for a client.
• The proxy will no longer treat state events (from the v2 response) as live events, as they should never appear in the timeline. This improves performance when performing an initial sync.

Bugfixes:

• The room delta algorithm now correctly handles complex swap operations down a list, not just up.
• The room delta algorithm now correctly returns multiple swap operations when a room moves over multiple window ranges.
• Several issues which could cause corrupt state snapshots has been fixed.
• Fixed an issue when dynamically changing room list filters where room filters would use the previous request data not the current request data to calculate a response.
• Fixed a bug where room history (as provided by timeline_limit) would be incorrect and include events in the state section of the v2 sync response.
• Fixed a bug where room history (as provided by timeline_limit) would not always return a prev_batch token. The proxy will always include one now if there is history, but as a result the proxy may not always return timeline_limit events if there are that many.