MSC2918 Refresh tokens implementation #9450

sandhose · 2021-02-19T15:58:54Z

This implements refresh tokens, as defined by MSC2918

This MSC has been implemented client side in Hydrogen Web: element-hq/hydrogen-web#235

The basics of the MSC works: requesting refresh tokens on login, having the access tokens expire, and using the refresh token to get a new one.
Here are the remaining things to do:

~~write tests (and ensure the current tests aren't broken)~~ done
ensure it works for other login types (incl. app services logins)
make it works for puppeted users
~~write the schema migration for postgres (sqlite only ATM)~~ done
~~also implement it when registering users~~ done
~~make the access token duration configurable (and maybe also allow to disable the rotation of refresh tokens?)~~ done
add a background update that deletes old refresh tokens?

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
Pull request includes a sign off
Code style is correct (run the linters)

clokep · 2021-03-12T13:16:22Z

From conversation with @sandhose today this could use an design review (not necessarily a code review) with an eye toward gotchas and things that might completely break with this approach.

erikjohnston

I think this mostly makes sense, beyond needing to make the requests idempotent

synapse/handlers/auth.py

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.sqlite

Signed-off-by: Quentin Gliech <[email protected]>

This checks for child token usage to validate the refresh token validity. This means that a token can be refreshed multiple times until one of the child tokens gets used. Signed-off-by: Quentin Gliech <[email protected]>

Signed-off-by: Quentin Gliech <[email protected]>

sandhose · 2021-04-12T15:06:18Z

I fixed the existing tests. The portdb script seems to hang though, not sure why?

erikjohnston · 2021-04-26T14:20:35Z

I fixed the existing tests. The portdb script seems to hang though, not sure why?

FTR I tried running locally at it doesn't hang. The logs on CI make it look like it hangs after its completed all the work.

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.postgres

richvdh

looks generally good to me, modulo a few tweaks!

at some point in the near future (not necessarily part of this PR) we could do with a job to remove the expired access tokens from the database.

docs/sample_config.yaml

synapse/config/registration.py

synapse/rest/client/v1/login.py

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.postgres

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.sqlite

synapse/storage/prepare_database.py

synapse/storage/databases/main/registration.py

This also rolls back the SCHEMA_VERSION to 59 since this does not introduce any breaking database change. Signed-off-by: Quentin Gliech <[email protected]>

Signed-off-by: Quentin Gliech <[email protected]>

sandhose

Thanks @richvdh for the review. Applied a bunch of your suggestions, but I'm still working on it

docs/sample_config.yaml

synapse/config/registration.py

synapse/handlers/auth.py

synapse/rest/client/v2_alpha/register.py

synapse/storage/databases/main/registration.py

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.postgres

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.sqlite

synapse/storage/prepare_database.py

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.postgres

This reverts commit 6963fe0.

richvdh

modulo a few tiny niggles, looks great!

synapse/handlers/auth.py

synapse/config/registration.py

synapse/storage/databases/main/registration.py

richvdh · 2021-06-11T10:04:24Z

There was an issue with sqlite returning 'FALSE' on BOOLEAN columns in the py3-old environment (and ofc, bool('FALSE') == True). I could not reproduce it locally, until I tried that in a docker container with the ubuntu:bionic image (same as in CI).

I first tried fixing it by casting to INTEGER (cdfd871), but the real issue was the default value of the used column: FALSE was interpreted as 'FALSE' by sqlite, so the default inserted something that was then interpreted as truthy.

Yeah, I think we've had this nonsense before, though I couldn't quite remember the details. Well done for tracking it down.

In the past it looks like we've adopted either of two solutions:

the same as you've done - see for example local_media_repository.safe_from_quarantine in delta/58/08_media_safe_from_quarantine.sql.{sqlite,postgres}.
Avoid SELECTing from the column (ie, just use it in WHERE clauses), so that the odd behaviour in old sqlite doesn't happen.

In short, +1 to your solution here.

I should really write this down somewhere.

richvdh · 2021-06-11T12:09:16Z

I should really write this down somewhere.

I made #10164 to cover this.

erikjohnston · 2021-06-16T11:12:48Z

I think this is blocked on the MSC progressing further before we want to merge?

Signed-off-by: Quentin Gliech <[email protected]>

synapse/storage/databases/main/schema/delta/59/12refresh_tokens.sql

synapse/storage/databases/main/registration.py

synapse/config/registration.py

synapse/handlers/auth.py

sandhose · 2021-06-17T11:00:27Z

I think this is blocked on the MSC progressing further before we want to merge?

@erikjohnston I just took the MSC out of "draft" state to have it properly reviewed. Note that this implementation is behind the unstable prefix defined in the MSC and can be completely disabled through config (although enabled by default)

synapse/config/registration.py

richvdh · 2021-06-17T13:42:32Z

synapse/storage/schema/main/delta/59/14refresh_tokens.sql.postgres

+ALTER TABLE "access_tokens"
+  ADD COLUMN used BOOLEAN NOT NULL DEFAULT FALSE;


since github is obstinately refusing to make my reply visible, I'll write it again:

Seems like we already did a lot of ADD COLUMN ... NOT NULL DEFAULT ... in the past:

Many of those are small tables, so can be rewritten during Synapse restart without major problems. I'm surprised to see a couple of quite large tables in that list, and I wonder if they caused us trouble at the time, but I don't think that changes my position on it here: I'd like us to avoid rewriting the access_tokens table if possible.

Looking at the existing columns with defaults, seems like we don't have that many nullable columns with defaults:

To be clear, to avoid having to rewrite the table, you need to make it nullable with no default.

done in 18628fc

This avoids rewriting the whole table on disk on Postgres < 11 Signed-off-by: Quentin Gliech <[email protected]>

This could help differenciate errors where the refresh token was never valid from errors where it is not valid anymore Signed-off-by: Quentin Gliech <[email protected]>

…time interaction in config Co-authored-by: Richard van der Hoff <[email protected]>

Signed-off-by: Quentin Gliech <[email protected]>

richvdh

lgtm

@deepbluev7

Synapse 1.38.0rc1 (2021-07-06) ============================== This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information. Features -------- - Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\#9450](#9450)) - Add support for evicting cache entries based on last access time. ([\#10205](#10205)) - Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\#10214](#10214)) - Improve validation on federation `send_{join,leave,knock}` endpoints. ([\#10225](#10225), [\#10243](#10243)) - Add SSO `external_ids` to the Query User Account admin API. ([\#10261](#10261)) - Mark events received over federation which fail a spam check as "soft-failed". ([\#10263](#10263)) - Add metrics for new inbound federation staging area. ([\#10284](#10284)) - Add script to print information about recently registered users. ([\#10290](#10290)) Bugfixes -------- - Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\#10223](#10223)) - Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\#10252](#10252)) - Fix a long-standing bug where Synapse would return errors after 231 events were handled by the server. ([\#10264](#10264), [\#10267](#10267), [\#10282](#10282), [\#10286](#10286), [\#10291](#10291), [\#10314](#10314), [\#10326](#10326)) - Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\#10279](#10279)) - Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\#10303](#10303)) Improved Documentation ---------------------- - Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\#10166](#10166)) - Choose Welcome & Overview as the default page for synapse documentation website. ([\#10242](#10242)) - Adjust the URL in the README.rst file to point to irc.libera.chat. ([\#10258](#10258)) - Fix homeserver config option name in presence router documentation. ([\#10288](#10288)) - Fix link pointing at the wrong section in the modules documentation page. ([\#10302](#10302)) Internal Changes ---------------- - Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\#10114](#10114)) - Add type hints to the federation servlets. ([\#10213](#10213)) - Improve the reliability of auto-joining remote rooms. ([\#10237](#10237)) - Update the release script to use the semver terminology and determine the release branch based on the next version. ([\#10239](#10239)) - Fix type hints for computing auth events. ([\#10253](#10253)) - Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\#10256](#10256)) - Move event authentication methods from `Auth` to `EventAuthHandler`. ([\#10268](#10268)) - Re-enable a SyTest after it has been fixed. ([\#10292](#10292))

@dklimpel

Synapse 1.38.0 (2021-07-13) =========================== This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information. No significant changes since 1.38.0rc3. Synapse 1.38.0rc3 (2021-07-13) ============================== Internal Changes ---------------- - Build the Debian packages in CI. ([\matrix-org#10247](matrix-org#10247), [\matrix-org#10379](matrix-org#10379)) Synapse 1.38.0rc2 (2021-07-09) ============================== Bugfixes -------- - Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\matrix-org#10336](matrix-org#10336)) Improved Documentation ---------------------- - Update links to documentation in the sample config. Contributed by @dklimpel. ([\matrix-org#10287](matrix-org#10287)) - Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\matrix-org#10331](matrix-org#10331)) Synapse 1.38.0rc1 (2021-07-06) ============================== Features -------- - Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\matrix-org#9450](matrix-org#9450)) - Add support for evicting cache entries based on last access time. ([\matrix-org#10205](matrix-org#10205)) - Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\matrix-org#10214](matrix-org#10214)) - Improve validation on federation `send_{join,leave,knock}` endpoints. ([\matrix-org#10225](matrix-org#10225), [\matrix-org#10243](matrix-org#10243)) - Add SSO `external_ids` to the Query User Account admin API. ([\matrix-org#10261](matrix-org#10261)) - Mark events received over federation which fail a spam check as "soft-failed". ([\matrix-org#10263](matrix-org#10263)) - Add metrics for new inbound federation staging area. ([\matrix-org#10284](matrix-org#10284)) - Add script to print information about recently registered users. ([\matrix-org#10290](matrix-org#10290)) Bugfixes -------- - Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\matrix-org#10223](matrix-org#10223)) - Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\matrix-org#10252](matrix-org#10252)) - Fix a long-standing bug where Synapse would return errors after 231 events were handled by the server. ([\matrix-org#10264](matrix-org#10264), [\matrix-org#10267](matrix-org#10267), [\matrix-org#10282](matrix-org#10282), [\matrix-org#10286](matrix-org#10286), [\matrix-org#10291](matrix-org#10291), [\matrix-org#10314](matrix-org#10314), [\matrix-org#10326](matrix-org#10326)) - Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\matrix-org#10279](matrix-org#10279)) - Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\matrix-org#10303](matrix-org#10303)) Improved Documentation ---------------------- - Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\matrix-org#10166](matrix-org#10166)) - Choose Welcome & Overview as the default page for synapse documentation website. ([\matrix-org#10242](matrix-org#10242)) - Adjust the URL in the README.rst file to point to irc.libera.chat. ([\matrix-org#10258](matrix-org#10258)) - Fix homeserver config option name in presence router documentation. ([\matrix-org#10288](matrix-org#10288)) - Fix link pointing at the wrong section in the modules documentation page. ([\matrix-org#10302](matrix-org#10302)) Internal Changes ---------------- - Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\matrix-org#10114](matrix-org#10114)) - Add type hints to the federation servlets. ([\matrix-org#10213](matrix-org#10213)) - Improve the reliability of auto-joining remote rooms. ([\matrix-org#10237](matrix-org#10237)) - Update the release script to use the semver terminology and determine the release branch based on the next version. ([\matrix-org#10239](matrix-org#10239)) - Fix type hints for computing auth events. ([\matrix-org#10253](matrix-org#10253)) - Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\matrix-org#10256](matrix-org#10256)) - Move event authentication methods from `Auth` to `EventAuthHandler`. ([\matrix-org#10268](matrix-org#10268)) - Re-enable a SyTest after it has been fixed. ([\matrix-org#10292](matrix-org#10292))

@dklimpel

Synapse 1.38.0 (2021-07-13) =========================== This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information. No significant changes since 1.38.0rc3. Synapse 1.38.0rc3 (2021-07-13) ============================== Internal Changes ---------------- - Build the Debian packages in CI. ([\#10247](matrix-org/synapse#10247), [\#10379](matrix-org/synapse#10379)) Synapse 1.38.0rc2 (2021-07-09) ============================== Bugfixes -------- - Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](matrix-org/synapse#10336)) Improved Documentation ---------------------- - Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](matrix-org/synapse#10287)) - Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](matrix-org/synapse#10331)) Synapse 1.38.0rc1 (2021-07-06) ============================== Features -------- - Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\#9450](matrix-org/synapse#9450)) - Add support for evicting cache entries based on last access time. ([\#10205](matrix-org/synapse#10205)) - Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\#10214](matrix-org/synapse#10214)) - Improve validation on federation `send_{join,leave,knock}` endpoints. ([\#10225](matrix-org/synapse#10225), [\#10243](matrix-org/synapse#10243)) - Add SSO `external_ids` to the Query User Account admin API. ([\#10261](matrix-org/synapse#10261)) - Mark events received over federation which fail a spam check as "soft-failed". ([\#10263](matrix-org/synapse#10263)) - Add metrics for new inbound federation staging area. ([\#10284](matrix-org/synapse#10284)) - Add script to print information about recently registered users. ([\#10290](matrix-org/synapse#10290)) Bugfixes -------- - Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\#10223](matrix-org/synapse#10223)) - Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\#10252](matrix-org/synapse#10252)) - Fix a long-standing bug where Synapse would return errors after 231 events were handled by the server. ([\#10264](matrix-org/synapse#10264), [\#10267](matrix-org/synapse#10267), [\#10282](matrix-org/synapse#10282), [\#10286](matrix-org/synapse#10286), [\#10291](matrix-org/synapse#10291), [\#10314](matrix-org/synapse#10314), [\#10326](matrix-org/synapse#10326)) - Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\#10279](matrix-org/synapse#10279)) - Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\#10303](matrix-org/synapse#10303)) Improved Documentation ---------------------- - Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\#10166](matrix-org/synapse#10166)) - Choose Welcome & Overview as the default page for synapse documentation website. ([\#10242](matrix-org/synapse#10242)) - Adjust the URL in the README.rst file to point to irc.libera.chat. ([\#10258](matrix-org/synapse#10258)) - Fix homeserver config option name in presence router documentation. ([\#10288](matrix-org/synapse#10288)) - Fix link pointing at the wrong section in the modules documentation page. ([\#10302](matrix-org/synapse#10302)) Internal Changes ---------------- - Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\#10114](matrix-org/synapse#10114)) - Add type hints to the federation servlets. ([\#10213](matrix-org/synapse#10213)) - Improve the reliability of auto-joining remote rooms. ([\#10237](matrix-org/synapse#10237)) - Update the release script to use the semver terminology and determine the release branch based on the next version. ([\#10239](matrix-org/synapse#10239)) - Fix type hints for computing auth events. ([\#10253](matrix-org/synapse#10253)) - Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\#10256](matrix-org/synapse#10256)) - Move event authentication methods from `Auth` to `EventAuthHandler`. ([\#10268](matrix-org/synapse#10268)) - Re-enable a SyTest after it has been fixed. ([\#10292](matrix-org/synapse#10292))

@dklimpel

Synapse 1.38.0 (2021-07-13) =========================== This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information. No significant changes since 1.38.0rc3. Synapse 1.38.0rc3 (2021-07-13) ============================== Internal Changes ---------------- - Build the Debian packages in CI. ([\#10247](matrix-org/synapse#10247), [\#10379](matrix-org/synapse#10379)) Synapse 1.38.0rc2 (2021-07-09) ============================== Bugfixes -------- - Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](matrix-org/synapse#10336)) Improved Documentation ---------------------- - Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](matrix-org/synapse#10287)) - Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](matrix-org/synapse#10331)) Synapse 1.38.0rc1 (2021-07-06) ============================== Features -------- - Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\#9450](matrix-org/synapse#9450)) - Add support for evicting cache entries based on last access time. ([\#10205](matrix-org/synapse#10205)) - Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\#10214](matrix-org/synapse#10214)) - Improve validation on federation `send_{join,leave,knock}` endpoints. ([\#10225](matrix-org/synapse#10225), [\#10243](matrix-org/synapse#10243)) - Add SSO `external_ids` to the Query User Account admin API. ([\#10261](matrix-org/synapse#10261)) - Mark events received over federation which fail a spam check as "soft-failed". ([\#10263](matrix-org/synapse#10263)) - Add metrics for new inbound federation staging area. ([\#10284](matrix-org/synapse#10284)) - Add script to print information about recently registered users. ([\#10290](matrix-org/synapse#10290)) Bugfixes -------- - Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\#10223](matrix-org/synapse#10223)) - Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\#10252](matrix-org/synapse#10252)) - Fix a long-standing bug where Synapse would return errors after 231 events were handled by the server. ([\#10264](matrix-org/synapse#10264), [\#10267](matrix-org/synapse#10267), [\#10282](matrix-org/synapse#10282), [\#10286](matrix-org/synapse#10286), [\#10291](matrix-org/synapse#10291), [\#10314](matrix-org/synapse#10314), [\#10326](matrix-org/synapse#10326)) - Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\#10279](matrix-org/synapse#10279)) - Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\#10303](matrix-org/synapse#10303)) Improved Documentation ---------------------- - Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\#10166](matrix-org/synapse#10166)) - Choose Welcome & Overview as the default page for synapse documentation website. ([\#10242](matrix-org/synapse#10242)) - Adjust the URL in the README.rst file to point to irc.libera.chat. ([\#10258](matrix-org/synapse#10258)) - Fix homeserver config option name in presence router documentation. ([\#10288](matrix-org/synapse#10288)) - Fix link pointing at the wrong section in the modules documentation page. ([\#10302](matrix-org/synapse#10302)) Internal Changes ---------------- - Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\#10114](matrix-org/synapse#10114)) - Add type hints to the federation servlets. ([\#10213](matrix-org/synapse#10213)) - Improve the reliability of auto-joining remote rooms. ([\#10237](matrix-org/synapse#10237)) - Update the release script to use the semver terminology and determine the release branch based on the next version. ([\#10239](matrix-org/synapse#10239)) - Fix type hints for computing auth events. ([\#10253](matrix-org/synapse#10253)) - Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\#10256](matrix-org/synapse#10256)) - Move event authentication methods from `Auth` to `EventAuthHandler`. ([\#10268](matrix-org/synapse#10268)) - Re-enable a SyTest after it has been fixed. ([\#10292](matrix-org/synapse#10292))

@dklimpel

Synapse 1.38.0 (2021-07-13) =========================== This release includes a database schema update which could result in elevated disk usage. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1380) for more information. No significant changes since 1.38.0rc3. Synapse 1.38.0rc3 (2021-07-13) ============================== Internal Changes ---------------- - Build the Debian packages in CI. ([\matrix-org#10247](matrix-org#10247), [\matrix-org#10379](matrix-org#10379)) Synapse 1.38.0rc2 (2021-07-09) ============================== Bugfixes -------- - Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\matrix-org#10336](matrix-org#10336)) Improved Documentation ---------------------- - Update links to documentation in the sample config. Contributed by @dklimpel. ([\matrix-org#10287](matrix-org#10287)) - Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\matrix-org#10331](matrix-org#10331)) Synapse 1.38.0rc1 (2021-07-06) ============================== Features -------- - Implement refresh tokens as specified by [MSC2918](matrix-org/matrix-spec-proposals#2918). ([\matrix-org#9450](matrix-org#9450)) - Add support for evicting cache entries based on last access time. ([\matrix-org#10205](matrix-org#10205)) - Omit empty fields from the `/sync` response. Contributed by @deepbluev7. ([\matrix-org#10214](matrix-org#10214)) - Improve validation on federation `send_{join,leave,knock}` endpoints. ([\matrix-org#10225](matrix-org#10225), [\matrix-org#10243](matrix-org#10243)) - Add SSO `external_ids` to the Query User Account admin API. ([\matrix-org#10261](matrix-org#10261)) - Mark events received over federation which fail a spam check as "soft-failed". ([\matrix-org#10263](matrix-org#10263)) - Add metrics for new inbound federation staging area. ([\matrix-org#10284](matrix-org#10284)) - Add script to print information about recently registered users. ([\matrix-org#10290](matrix-org#10290)) Bugfixes -------- - Fix a long-standing bug which meant that invite rejections and knocks were not sent out over federation in a timely manner. ([\matrix-org#10223](matrix-org#10223)) - Fix a bug introduced in v1.26.0 where only users who have set profile information could be deactivated with erasure enabled. ([\matrix-org#10252](matrix-org#10252)) - Fix a long-standing bug where Synapse would return errors after 231 events were handled by the server. ([\matrix-org#10264](matrix-org#10264), [\matrix-org#10267](matrix-org#10267), [\matrix-org#10282](matrix-org#10282), [\matrix-org#10286](matrix-org#10286), [\matrix-org#10291](matrix-org#10291), [\matrix-org#10314](matrix-org#10314), [\matrix-org#10326](matrix-org#10326)) - Fix the prometheus `synapse_federation_server_pdu_process_time` metric. Broke in v1.37.1. ([\matrix-org#10279](matrix-org#10279)) - Ensure that inbound events from federation that were being processed when Synapse was restarted get promptly processed on start up. ([\matrix-org#10303](matrix-org#10303)) Improved Documentation ---------------------- - Move the upgrade notes to [docs/upgrade.md](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md) and convert them to markdown. ([\matrix-org#10166](matrix-org#10166)) - Choose Welcome & Overview as the default page for synapse documentation website. ([\matrix-org#10242](matrix-org#10242)) - Adjust the URL in the README.rst file to point to irc.libera.chat. ([\matrix-org#10258](matrix-org#10258)) - Fix homeserver config option name in presence router documentation. ([\matrix-org#10288](matrix-org#10288)) - Fix link pointing at the wrong section in the modules documentation page. ([\matrix-org#10302](matrix-org#10302)) Internal Changes ---------------- - Drop `Origin` and `Accept` from the value of the `Access-Control-Allow-Headers` response header. ([\matrix-org#10114](matrix-org#10114)) - Add type hints to the federation servlets. ([\matrix-org#10213](matrix-org#10213)) - Improve the reliability of auto-joining remote rooms. ([\matrix-org#10237](matrix-org#10237)) - Update the release script to use the semver terminology and determine the release branch based on the next version. ([\matrix-org#10239](matrix-org#10239)) - Fix type hints for computing auth events. ([\matrix-org#10253](matrix-org#10253)) - Improve the performance of the spaces summary endpoint by only recursing into spaces (and not rooms in general). ([\matrix-org#10256](matrix-org#10256)) - Move event authentication methods from `Auth` to `EventAuthHandler`. ([\matrix-org#10268](matrix-org#10268)) - Re-enable a SyTest after it has been fixed. ([\matrix-org#10292](matrix-org#10292))

sandhose mentioned this pull request Feb 19, 2021

WIP: MSC2918 initial implementation element-hq/hydrogen-web#235

Draft

sandhose marked this pull request as draft February 19, 2021 16:02

clokep changed the title ~~WIP: MSC2918 implementation~~ WIP: MSC2918 Refresh tokens implementation Mar 12, 2021

clokep requested a review from a team March 12, 2021 13:15

erikjohnston reviewed Mar 16, 2021

View reviewed changes

synapse/handlers/auth.py Outdated Show resolved Hide resolved

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.sqlite Outdated Show resolved Hide resolved

sandhose added 7 commits April 9, 2021 21:12

WIP: MSC2918

fe80ef5

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: implement refresh tokens

523d8cf

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: Changelog

358da22

MSC2918: fix mypy and lint errors

f53466e

MSC2918: add PostgreSQL schema

324d7bf

MSC2918: do not invalidate refresh token immediately & fix tests

450a962

This checks for child token usage to validate the refresh token validity. This means that a token can be refreshed multiple times until one of the child tokens gets used. Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: lint fixes

022485e

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: also delete refresh tokens when logging out

51ba1c3

richvdh requested a review from a team April 22, 2021 13:29

MSC2918: fix field name in migrations

d281f7e

richvdh self-assigned this May 4, 2021

richvdh reviewed May 4, 2021

View reviewed changes

synapse/storage/databases/main/schema/delta/60/01refresh_tokens.sql.postgres Outdated Show resolved Hide resolved

richvdh reviewed May 4, 2021

View reviewed changes

sandhose added 6 commits May 5, 2021 10:49

MSC2918: merge SQLite and PostgreSQL schema deltas

f499d63

This also rolls back the SCHEMA_VERSION to 59 since this does not introduce any breaking database change. Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: fix sample config

e402a07

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: use parse_boolean to get query parameter value

adc6eab

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: use attr.s instead of TypedDict

6963fe0

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: remove unused sequence in refresh_tokens

318b74c

Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: try fixing port_db script when a table references itself

29806b4

Signed-off-by: Quentin Gliech <[email protected]>

sandhose commented May 5, 2021

View reviewed changes

sandhose added 2 commits May 5, 2021 15:42

MSC2918: lint

72e5c25

Revert "MSC2918: use attr.s instead of TypedDict"

eb9f680

This reverts commit 6963fe0.

sandhose added 2 commits June 4, 2021 13:55

Merge remote-tracking branch 'upstream/develop' into sandhose/msc2918

4cf49a6

MSC2918: properly fix "mark_access_token_as_used" by caching it

ef0e051

richvdh removed their assignment Jun 10, 2021

Merge remote-tracking branch 'upstream/develop' into sandhose/msc2918

7adfe0c

sandhose requested a review from richvdh June 10, 2021 13:49

richvdh approved these changes Jun 11, 2021

View reviewed changes

synapse/handlers/auth.py Show resolved Hide resolved

synapse/config/registration.py Show resolved Hide resolved

synapse/storage/databases/main/registration.py Show resolved Hide resolved

erikjohnston added the z-blocked (Deprecated Label) label Jun 16, 2021

MSC2918: add comments as suggested by richvdh

ab443a3

Signed-off-by: Quentin Gliech <[email protected]>

sandhose commented Jun 17, 2021

View reviewed changes

Merge remote-tracking branch 'upstream/develop' into sandhose/msc2918

0060bc9

richvdh reviewed Jun 17, 2021

View reviewed changes

synapse/config/registration.py Outdated Show resolved Hide resolved

richvdh reviewed Jun 17, 2021

View reviewed changes

sandhose and others added 5 commits June 18, 2021 10:05

MSC2918: make access_tokens.used nullable

18628fc

This avoids rewriting the whole table on disk on Postgres < 11 Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: 403 when using a refresh token twice

bcc33e2

This could help differenciate errors where the refresh token was never valid from errors where it is not valid anymore Signed-off-by: Quentin Gliech <[email protected]>

MSC2918: clarify comment about access_token_lifetime and session_life…

ddfc2a4

…time interaction in config Co-authored-by: Richard van der Hoff <[email protected]>

Merge remote-tracking branch 'upstream/develop' into sandhose/msc2918

a013064

MSC2918: fix refresh token invalidation test

9fe5556

Signed-off-by: Quentin Gliech <[email protected]>

richvdh mentioned this pull request Jun 24, 2021

MSC2918: Refresh tokens matrix-org/matrix-spec-proposals#2918

Merged

richvdh approved these changes Jun 24, 2021

View reviewed changes

richvdh merged commit bd4919f into matrix-org:develop Jun 24, 2021

clokep mentioned this pull request Sep 29, 2022

Support Matrix 1.3 #13953

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC2918 Refresh tokens implementation #9450

MSC2918 Refresh tokens implementation #9450

sandhose commented Feb 19, 2021 •

edited

Loading

clokep commented Mar 12, 2021

erikjohnston left a comment

sandhose commented Apr 12, 2021

erikjohnston commented Apr 26, 2021

richvdh left a comment

sandhose left a comment

richvdh left a comment

richvdh commented Jun 11, 2021

richvdh commented Jun 11, 2021

erikjohnston commented Jun 16, 2021

sandhose commented Jun 17, 2021

richvdh Jun 17, 2021

sandhose Jun 18, 2021

richvdh left a comment

		ALTER TABLE "access_tokens"
		ADD COLUMN used BOOLEAN NOT NULL DEFAULT FALSE;

MSC2918 Refresh tokens implementation #9450

MSC2918 Refresh tokens implementation #9450

Conversation

sandhose commented Feb 19, 2021 • edited Loading

Pull Request Checklist

clokep commented Mar 12, 2021

erikjohnston left a comment

Choose a reason for hiding this comment

sandhose commented Apr 12, 2021

erikjohnston commented Apr 26, 2021

richvdh left a comment

Choose a reason for hiding this comment

sandhose left a comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

richvdh commented Jun 11, 2021

richvdh commented Jun 11, 2021

erikjohnston commented Jun 16, 2021

sandhose commented Jun 17, 2021

richvdh Jun 17, 2021

Choose a reason for hiding this comment

sandhose Jun 18, 2021

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

sandhose commented Feb 19, 2021 •

edited

Loading