Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring auto-accept invite logic into Synapse #17147

Merged
merged 18 commits into from
May 21, 2024
Merged

Conversation

devonh
Copy link
Member

@devonh devonh commented May 3, 2024

This PR ports the logic from the synapse_auto_accept_invite module into synapse.

I went with the naive approach of injecting the "module" next to where third party modules are currently loaded. If there is a better/preferred way to handle this, I'm all ears. It wasn't obvious to me if there was a better location to add this logic that would cleanly apply to all incoming invite events.

Relies on #17166 to fix linter errors.

@devonh devonh requested a review from a team as a code owner May 3, 2024 00:11
@github-actions github-actions bot deployed to PR Documentation Preview May 3, 2024 00:13 Active
@github-actions github-actions bot deployed to PR Documentation Preview May 3, 2024 00:32 Active
@github-actions github-actions bot deployed to PR Documentation Preview May 3, 2024 00:46 Active
@github-actions github-actions bot deployed to PR Documentation Preview May 3, 2024 21:03 Active
synapse/handlers/profile.py Outdated Show resolved Hide resolved
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good on the whole! A few minor changes but nothing fundamental.

docs/usage/configuration/config_documentation.md Outdated Show resolved Hide resolved
synapse/config/auto_accept_invites.py Outdated Show resolved Hide resolved
synapse/handlers/profile.py Outdated Show resolved Hide resolved
docs/usage/configuration/config_documentation.md Outdated Show resolved Hide resolved
docs/usage/configuration/config_documentation.md Outdated Show resolved Hide resolved
synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
tests/events/test_auto_accept_invites.py Outdated Show resolved Hide resolved
tests/events/test_auto_accept_invites.py Outdated Show resolved Hide resolved
tests/events/test_auto_accept_invites.py Outdated Show resolved Hide resolved
@anoadragon453
Copy link
Member

I think it would make sense to archive https://github.com/matrix-org/synapse-auto-accept-invite upon merging this PR, with a note in the README to inform folks that it has been moved in-tree?

@github-actions github-actions bot deployed to PR Documentation Preview May 7, 2024 20:19 Active
@github-actions github-actions bot deployed to PR Documentation Preview May 7, 2024 21:25 Active
@devonh
Copy link
Member Author

devonh commented May 7, 2024

Relies on #17166

@github-actions github-actions bot deployed to PR Documentation Preview May 7, 2024 22:06 Active
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some very minor requested changes now, plus a small q:

synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, that is one of the downsides. While this code doesn't actually need any private APIs, it is inevitably handy.

I can actually add another downside. While we wouldn't end up adding to Synapse's config if we made this a module... it would beg the question of how we'd actually document the config of this module. We wouldn't be able to put it in https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html, unless we added a new section for each in-tree module... and then you've ended up making the user-visible config larger anyhow.

That leads me to think that the only benefit of keeping the modules separate would be if we ever wanted to move them out-of-tree again in future. But I think the times we'll actually do that are minimal. And if we really need to do so, then untangling it from deep within Synapse isn't impossible, just slightly more fiddly.

The initial reason for me suggesting that we keep this code separate is that internal code using the module API felt weird. But after reflection I don't think it's really an issue. It doesn't block us from modifying the API since the code is internal and can change. I also don't believe we have any assumptions in the code that all consumers of the API are external.

So all in all, I'm OK with leaving this code how it is and where it is.

synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
@@ -817,7 +817,7 @@ def is_allowed_mime_type(content_type: str) -> bool:
server_name = profile["avatar_url"].split("/")[-2]
media_id = profile["avatar_url"].split("/")[-1]
if self._is_mine_server_name(server_name):
media = await self._media_repo.store.get_local_media(media_id)
media = await self._media_repo.store.get_local_media(media_id) # type: ignore[has-type]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this not go in #17166 instead? How come it was moved back to this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha because when I put it in that PR, the linter complained about an unnecessary ignore....
So rather than fight a circle of nonsense, I feel like it's easier just to keep it here :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit cautious about introducing more ignore for types without trying to spend some time in understanding what has happened. By the looks of it the type inference has failed somehow, so am wondering if we've managed to introduce a loop in the typing maybe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this parcitular lint is related to this issue: matrix-org/synapse#11165
Which seems like quite the can of worms.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The particular error is:
synapse/handlers/sso.py:820: error: Cannot determine type of "store" [has-type]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For further reference, this is the only instance of using self._media_repo.store outside of the media_repo source.
That probably has something to do with the linter error only showing up here.

Since this code is completely unrelated to the changes in this PR, I suggest we allow the addition of the ignore here until the larger issue of #11165 is brought to a resolution. Adding an ignore keeps things in effectively the same state as before since mypy should have been treating this as an error, but it was silently ignoring it. At least with the ignore added, now the issue is explicit and visible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synapse/handlers/sso.py:820: error: Cannot determine type of "store" [has-type] this error sounds like one where we could hopefully add a type annotation instead?

Copy link
Member Author

@devonh devonh May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type is generic & is specified at the following locations:

DATASTORE_CLASS = DataStore # type: ignore

DATASTORE_CLASS = GenericWorkerStore # type: ignore

DATASTORE_CLASS = AdminCmdStore # type: ignore

All of which also have an explicit type ignore on them.
I can't seem to come up with any annotation that would make mypy happy - maybe someone else knows a way, but this seems like a much deeper issue around DataStore typing.

synapse/events/auto_accept_invites.py Outdated Show resolved Hide resolved
devonh added a commit that referenced this pull request May 8, 2024
Linter errors are showing up in #17147 that are unrelated to that PR.
The errors do not currently show up on develop.

This PR aims to resolve the linter errors separately from #17147.
@github-actions github-actions bot deployed to PR Documentation Preview May 8, 2024 16:18 Active
@github-actions github-actions bot deployed to PR Documentation Preview May 17, 2024 15:02 Active
@github-actions github-actions bot deployed to PR Documentation Preview May 17, 2024 15:42 Active
@MatMaul
Copy link
Contributor

MatMaul commented May 17, 2024

What is the rationale for moving a feature that is easily concealed in a module, and seems to work fine that way, into synapse?

Is it mainly because of the deployment burden, with docker for example?

@devonh
Copy link
Member Author

devonh commented May 17, 2024

What is the rationale for moving a feature that is easily concealed in a module, and seems to work fine that way, into synapse?

Is it mainly because of the deployment burden, with docker for example?

That is definitely part of the reason.
The overall rationale is to bring something that provides generally useful functionality into synapse proper.

This allows us to:

  • make deployment easier (as you mentioned)
  • make maintenance & testing easier

It's a very fine line as to what should be in a module vs. in synapse itself.
We made the decision that this functionality has become generally useful enough that it should now be moved directly within synapse.

@devonh devonh requested a review from anoadragon453 May 17, 2024 20:35
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, thanks!

changelog.d/17147.feature Outdated Show resolved Hide resolved
@github-actions github-actions bot deployed to PR Documentation Preview May 21, 2024 19:43 Active
@devonh devonh merged commit 6a9a641 into develop May 21, 2024
40 checks passed
@devonh devonh deleted the devon/auto-accept-invite branch May 21, 2024 20:09
Mic92 pushed a commit to Mic92/synapse that referenced this pull request Jun 14, 2024
This PR ports the logic from the
[synapse_auto_accept_invite](https://github.com/matrix-org/synapse-auto-accept-invite)
module into synapse.

I went with the naive approach of injecting the "module" next to where
third party modules are currently loaded. If there is a better/preferred
way to handle this, I'm all ears. It wasn't obvious to me if there was a
better location to add this logic that would cleanly apply to all
incoming invite events.

Relies on element-hq#17166 to fix linter
errors.
spantaleev added a commit to spantaleev/matrix-docker-ansible-deploy that referenced this pull request Jun 18, 2024
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jun 18, 2024
# Synapse 1.109.0 (2024-06-18)

- Add the ability to auto-accept invites on the behalf of users. See
  the
  [`auto_accept_invites`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#auto-accept-invites)
  config option for
  details. ([\#17147](element-hq/synapse#17147))

- Add experimental
  [MSC3575](matrix-org/matrix-spec-proposals#3575)
  Sliding Sync `/sync/e2ee` endpoint for to-device messages and device
  encryption
  info. ([\#17167](element-hq/synapse#17167))

- Support
  [MSC3916](matrix-org/matrix-spec-proposals#3916)
  by adding unstable media endpoints to
  `/_matrix/client`. ([\#17213](element-hq/synapse#17213))

- Add logging to tasks managed by the task scheduler, showing CPU and
  database
  usage. ([\#17219](element-hq/synapse#17219))


# Synapse 1.108.0 (2024-05-28)

- Add a feature that allows clients to query the configured federation
  whitelist. Disabled by
  default. ([\#16848](element-hq/synapse#16848),
  [\#17199](element-hq/synapse#17199))

- Add the ability to allow numeric user IDs with a specific prefix
  when in the CAS flow. Contributed by Aurélien
  Grimpard. ([\#17098](element-hq/synapse#17098))


Synapse 1.107.0 (2024-05-14)

- Add preliminary support for [MSC3823: Account
  Suspension](matrix-org/matrix-spec-proposals#3823).
  ([\#17051](element-hq/synapse#17051))

- Declare support for [Matrix
  v1.10](https://matrix.org/blog/2024/03/22/matrix-v1.10-release/). Contributed
  by
  @clokep. ([\#17082](element-hq/synapse#17082))

- Add support for [MSC4115: membership metadata on
  events](matrix-org/matrix-spec-proposals#4115).
  ([\#17104](element-hq/synapse#17104),
  [\#17137](element-hq/synapse#17137))


# Synapse 1.106.0 (2024-04-30)

- Send an email if the address is already bound to an user
  account. ([\#16819](element-hq/synapse#16819))

- Implement the rendezvous mechanism described by
  [MSC4108](matrix-org/matrix-spec-proposals#4108).
  ([\#17056](element-hq/synapse#17056))

- Support delegating the rendezvous mechanism described
  [MSC4108](matrix-org/matrix-spec-proposals#4108)
  to an external
  implementation. ([\#17086](element-hq/synapse#17086))
yingziwu added a commit to yingziwu/synapse that referenced this pull request Jun 26, 2024
- Fix the building of binary wheels for macOS by switching to macOS 12 CI runners. ([\#17319](element-hq/synapse#17319))

- When rolling back to a previous Synapse version and then forwards again to this release, don't require server operators to manually run SQL. ([\#17305](element-hq/synapse#17305), [\#17309](element-hq/synapse#17309))

- Use the release branch for sytest in release-branch PRs. ([\#17306](element-hq/synapse#17306))

- Fix bug where one-time-keys were not always included in `/sync` response when using workers. Introduced in v1.109.0rc1. ([\#17275](element-hq/synapse#17275))
- Fix bug where `/sync` could get stuck due to edge case in device lists handling. Introduced in v1.109.0rc1. ([\#17292](element-hq/synapse#17292))

- Add the ability to auto-accept invites on the behalf of users. See the [`auto_accept_invites`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#auto-accept-invites) config option for details. ([\#17147](element-hq/synapse#17147))
- Add experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync/e2ee` endpoint for to-device messages and device encryption info. ([\#17167](element-hq/synapse#17167))
- Support [MSC3916](matrix-org/matrix-spec-proposals#3916) by adding unstable media endpoints to `/_matrix/client`. ([\#17213](element-hq/synapse#17213))
- Add logging to tasks managed by the task scheduler, showing CPU and database usage. ([\#17219](element-hq/synapse#17219))

- Fix deduplicating of membership events to not create unused state groups. ([\#17164](element-hq/synapse#17164))
- Fix bug where duplicate events could be sent down sync when using workers that are overloaded. ([\#17215](element-hq/synapse#17215))
- Ignore attempts to send to-device messages to bad users, to avoid log spam when we try to connect to the bad server. ([\#17240](element-hq/synapse#17240))
- Fix handling of duplicate concurrent uploading of device one-time-keys. ([\#17241](element-hq/synapse#17241))
- Fix reporting of default tags to Sentry, such as worker name. Broke in v1.108.0. ([\#17251](element-hq/synapse#17251))
- Fix bug where typing updates would not be sent when using workers after a restart. ([\#17252](element-hq/synapse#17252))

- Update the LemonLDAP documentation to say that claims should be explicitly included in the returned `id_token`, as Synapse won't request them. ([\#17204](element-hq/synapse#17204))

- Improve DB usage when fetching related events. ([\#17083](element-hq/synapse#17083))
- Log exceptions when failing to auto-join new user according to the `auto_join_rooms` option. ([\#17176](element-hq/synapse#17176))
- Reduce work of calculating outbound device lists updates. ([\#17211](element-hq/synapse#17211))
- Improve performance of calculating device lists changes in `/sync`. ([\#17216](element-hq/synapse#17216))
- Move towards using `MultiWriterIdGenerator` everywhere. ([\#17226](element-hq/synapse#17226))
- Replaces all usages of `StreamIdGenerator` with `MultiWriterIdGenerator`. ([\#17229](element-hq/synapse#17229))
- Change the `allow_unsafe_locale` config option to also apply when setting up new databases. ([\#17238](element-hq/synapse#17238))
- Fix errors in logs about closing incorrect logging contexts when media gets rejected by a module. ([\#17239](element-hq/synapse#17239), [\#17246](element-hq/synapse#17246))
- Clean out invalid destinations from `device_federation_outbox` table. ([\#17242](element-hq/synapse#17242))
- Stop logging errors when receiving invalid User IDs in key querys requests. ([\#17250](element-hq/synapse#17250))

* Bump anyhow from 1.0.83 to 1.0.86. ([\#17220](element-hq/synapse#17220))
* Bump bcrypt from 4.1.2 to 4.1.3. ([\#17224](element-hq/synapse#17224))
* Bump lxml from 5.2.1 to 5.2.2. ([\#17261](element-hq/synapse#17261))
* Bump mypy-zope from 1.0.3 to 1.0.4. ([\#17262](element-hq/synapse#17262))
* Bump phonenumbers from 8.13.35 to 8.13.37. ([\#17235](element-hq/synapse#17235))
* Bump prometheus-client from 0.19.0 to 0.20.0. ([\#17233](element-hq/synapse#17233))
* Bump pyasn1 from 0.5.1 to 0.6.0. ([\#17223](element-hq/synapse#17223))
* Bump pyicu from 2.13 to 2.13.1. ([\#17236](element-hq/synapse#17236))
* Bump pyopenssl from 24.0.0 to 24.1.0. ([\#17234](element-hq/synapse#17234))
* Bump serde from 1.0.201 to 1.0.202. ([\#17221](element-hq/synapse#17221))
* Bump serde from 1.0.202 to 1.0.203. ([\#17232](element-hq/synapse#17232))
* Bump twine from 5.0.0 to 5.1.0. ([\#17225](element-hq/synapse#17225))
* Bump types-psycopg2 from 2.9.21.20240311 to 2.9.21.20240417. ([\#17222](element-hq/synapse#17222))
* Bump types-pyopenssl from 24.0.0.20240311 to 24.1.0.20240425. ([\#17260](element-hq/synapse#17260))
venimus pushed a commit to superhero-com/matrix-docker-ansible-deploy that referenced this pull request Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants