Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import Reorg Plan #4

Merged
merged 4 commits into from
Aug 31, 2022
Merged

Import Reorg Plan #4

merged 4 commits into from
Aug 31, 2022

Conversation

mitsuhiko
Copy link
Member

This RFC proposes a preliminary refactoring of the Sentry monolith in order
to support later compartmentalization into services by eliminating mass
re-exports in sentry.app and sentry.models. The goal is to make the
dependency tree easier to understand in order to break it into separate
services by making modules the boundary of our dependencies.

In other words we want to be able to say that if module A depends on module B
it depends on the entirey of it. Today this is violated by a sentry.app
and sentry.models.

Rendered RFC

@mitsuhiko mitsuhiko changed the title Import Reorg Plan 0004 - Import Reorg Plan Aug 18, 2022
@mitsuhiko mitsuhiko changed the title 0004 - Import Reorg Plan Import Reorg Plan Aug 18, 2022
Copy link
Member

@asottile-sentry asottile-sentry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, I think this is a very useful building block towards the goals

Comment on lines +26 to +30
This also in parts shows up in import times. To run a single test file for a
utils module (`test_rust.py`) pytest will spend 0.2 seconds in test execution vs
2 seconds in import time. While imports are so far acceptable, the bigger issue
caused by it is that it increases the total amount of surface that we need to
consider for test execution.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I did my quick "what tests what" demo using coverage I completely ignored the module scope because of the difficulties here. this work would also better enable us to do some automated selective test execution (internal notion doc)

text/0004-import-reorg.md Outdated Show resolved Hide resolved

# Drawbacks

The drawback of this change is that imports become more verbose:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personally I find the more verbose imports to be better -- it's much easier to know where something comes from rather than having to git grep '^class OrganizationMember\b' and such

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m with you there. I see it as advantage :P

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally agree. The only time it's useful to be able to import everything at once is when I'm in the shell. Not a good enough reason to not do this though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in the shell. Not a good enough reason to not do this though.

Could we add shell startup code that generates more ergonomic glob modules that make it possible to do from sentry.models import User only in the console?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I think we can and probably should add some lazy import code there afterwards for shell usage.

Co-authored-by: anthony sottile <[email protected]>
Comment on lines +61 to +64
As such all code related to the processing pipeline should be moved into a clear
structure and have a largely independent test setup (think moving all of processing
related logic to `sentry.services.processing` or `sentry_processing` for better
enforcability).
Copy link
Member

@markstory markstory Aug 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should put thought into how services overlap & intersect as they will. Core concepts like Organizations and Projects are everywhere, and as we deepen connections between issues, performance, profiling and replays we create tension with the independence of each service.

In a previous role we had a 'service API' which was the public interface that other services inside the application could use. A similar approach has some documentation at shopify.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I would like to handle that as a separate independent proposal though.

@mitsuhiko mitsuhiko merged commit d1a10d3 into main Aug 31, 2022
@mitsuhiko mitsuhiko deleted the feature/import-reorg branch August 31, 2022 11:33
@mitsuhiko
Copy link
Member Author

I merged this as a tentative decision. Before doing something beyond what is done to date I will revisit this with a few folks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants