Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(github-invite): use sql for missing members API to utilize db indices #56911

Closed
wants to merge 3 commits into from

Conversation

cathteng
Copy link
Member

@cathteng cathteng commented Sep 26, 2023

Use SQL to better utilize db indices to improve the performance of the API. Attempts to fix SENTRY-161D.

Existing indices

  • Commit: ("repository_id", "date_added")

CommitAuthor doesn't have an index on external_id, but if we filter on it after using a subquery, it's already filtered.

Also removes querying for commit author external_id and/or email because we don't surface this anywhere on the FE.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 26, 2023
@cathteng cathteng requested a review from a team September 26, 2023 13:07
@cathteng cathteng marked this pull request as ready for review September 26, 2023 13:07
@cathteng cathteng requested a review from a team as a code owner September 26, 2023 13:07
for filtered_email in filtered_email_domains:
additional_query += f"""and email not like '%{filtered_email}' """

query = f"""
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: this query doesn't include the relation to the org member set, but somehow tests are all passing

.annotate(Count("commit"))
.order_by("-commit__count")
)
def _get_missing_organization_members_query(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth looking into if its better to write raw SQL or trying to use SubQuery django stuff etc. e.g.

@getsantry
Copy link
Contributor

getsantry bot commented Oct 19, 2023

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

@getsantry getsantry bot added the Stale label Oct 19, 2023
@getsantry getsantry bot closed this Oct 27, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Nov 11, 2023
@cathteng cathteng deleted the cathy/github-growth/use-sql-query branch December 8, 2023 23:49
@cathteng cathteng changed the title fix(github-growth): use sql for missing members API to utilize db indices fix(github-invite): use sql for missing members API to utilize db indices Dec 14, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants