Skip to content

Commit

Permalink
fix(releases): increase lock to 10 minutes on set_commits (#77660)
Browse files Browse the repository at this point in the history
we observed SENTRY-3F2A during the rollout of
#77453.

This didn't make sense until i realized our redis lock is only 10
seconds right now, and this function can take longer than that. We
increase the max timeout to 10 minutes, which should be plenty for
99.99+% of `set_commits` calls.
  • Loading branch information
JoshFerge committed Sep 17, 2024
1 parent 79d75b6 commit c605825
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions src/sentry/models/releases/set_commits.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from sentry.models.commitfilechange import CommitFileChange
from sentry.models.grouphistory import GroupHistoryStatus, record_group_history
from sentry.models.groupinbox import GroupInbox, GroupInboxRemoveAction, remove_group_from_inbox
from sentry.models.release import Release
from sentry.models.releases.exceptions import ReleaseCommitError
from sentry.signals import issue_resolved
from sentry.users.services.user import RpcUser
Expand Down Expand Up @@ -50,8 +51,9 @@ def set_commits(release, commit_list):
commit_list = [
c for c in commit_list if not RepositoryProvider.should_ignore_commit(c.get("message", ""))
]
lock_key = type(release).get_lock_key(release.organization_id, release.id)
lock = locks.get(lock_key, duration=10, name="release_set_commits")
lock_key = Release.get_lock_key(release.organization_id, release.id)
# Acquire the lock for a maximum of 10 minutes
lock = locks.get(lock_key, duration=10 * 60, name="release_set_commits")
if lock.locked():
# Signal failure to the consumer rapidly. This aims to prevent the number
# of timeouts and prevent web worker exhaustion when customers create
Expand Down

0 comments on commit c605825

Please sign in to comment.