Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid deadlock in gevent/urllib3 connection pool (fixes occasional worker heartbeat timeouts) #2813

Conversation

tdadela
Copy link
Contributor

@tdadela tdadela commented Jul 22, 2024

Fixes #2812
I tested it only with the newest versions of locust and its dependencies.

Currently, I don't know use cases of os.getenv("LOCUST_SKIP_URLLIB3_PATCH", None).
However, this PR touches implementation details of urllib3 and gevent, so I want to provide possibility to opt-out.
(For example, it may fail with future release of urllib3.)

PR is based on this answer.

@cyberw cyberw changed the title Avoid deadlock in gevent/urllib3 connection pool (fixes worker heartbeat timeouts) Avoid deadlock in gevent/urllib3 connection pool (fixes occasional worker heartbeat timeouts) Jul 22, 2024
@cyberw cyberw merged commit 49002b6 into locustio:master Jul 22, 2024
14 checks passed
@tdadela tdadela deleted the fix_worker_disconnection_deadlock_urlib3_gevent branch July 22, 2024 20:42
apereocas-bot referenced this pull request in apereo/cas Aug 1, 2024
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [locust](https://locust.io/) ([source](https://togithub.com/locustio/locust)) | `==2.29.1` -> `==2.30.0` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/locust/2.30.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/locust/2.30.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/locust/2.29.1/2.30.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/locust/2.29.1/2.30.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>locustio/locust (locust)</summary>

### [`v2.30.0`](https://togithub.com/locustio/locust/releases/tag/2.30.0)

[Compare Source](https://togithub.com/locustio/locust/compare/2.29.1...2.30.0)

#### What's Changed

-   FastHttpSession: Enable passing json as a positional argument for post() and stop converting response times to int by [@&#8203;tdadela](https://togithub.com/tdadela) in [https://github.com/locustio/locust/pull/2772](https://togithub.com/locustio/locust/pull/2772)
-   Remove Line Chart Default Zoom by [@&#8203;andrewbaldwin44](https://togithub.com/andrewbaldwin44) in [https://github.com/locustio/locust/pull/2774](https://togithub.com/locustio/locust/pull/2774)
-   FastHttpSession requests typing  by [@&#8203;tdadela](https://togithub.com/tdadela) in [https://github.com/locustio/locust/pull/2775](https://togithub.com/locustio/locust/pull/2775)
-   new events for heartbeat and usage monitor by [@&#8203;mgor](https://togithub.com/mgor) in [https://github.com/locustio/locust/pull/2777](https://togithub.com/locustio/locust/pull/2777)
-   dispatch benchmark test improvements by [@&#8203;tdadela](https://togithub.com/tdadela) in [https://github.com/locustio/locust/pull/2780](https://togithub.com/locustio/locust/pull/2780)
-   SequentialTaskSet: Allow weighted tasks and dict in .tasks by [@&#8203;bakhtos](https://togithub.com/bakhtos) in [https://github.com/locustio/locust/pull/2742](https://togithub.com/locustio/locust/pull/2742)
-   Fix StatsEntry docstring by [@&#8203;tdadela](https://togithub.com/tdadela) in [https://github.com/locustio/locust/pull/2784](https://togithub.com/locustio/locust/pull/2784)
-   Implement Poetry build system (mainly so we don't have to commit dynamically generated front end bundles to git) by [@&#8203;mquinnfd](https://togithub.com/mquinnfd) in [https://github.com/locustio/locust/pull/2725](https://togithub.com/locustio/locust/pull/2725)
-   Typing: strict optional in dispatch.py by [@&#8203;tdadela](https://togithub.com/tdadela) in [https://github.com/locustio/locust/pull/2779](https://togithub.com/locustio/locust/pull/2779)
-   Correctly set version from Poetry in published builds by [@&#8203;mquinnfd](https://togithub.com/mquinnfd) in [https://github.com/locustio/locust/pull/2791](https://togithub.com/locustio/locust/pull/2791)
-   Fix Extend Webui Example by [@&#8203;andrewbaldwin44](https://togithub.com/andrewbaldwin44) in [https://github.com/locustio/locust/pull/2800](https://togithub.com/locustio/locust/pull/2800)
-   Provide warning for local installs where yarn is not present by [@&#8203;mquinnfd](https://togithub.com/mquinnfd) in [https://github.com/locustio/locust/pull/2801](https://togithub.com/locustio/locust/pull/2801)
-   Fix tests on windows by [@&#8203;mquinnfd](https://togithub.com/mquinnfd) in [https://github.com/locustio/locust/pull/2803](https://togithub.com/locustio/locust/pull/2803)
-   Add example of a bottlenecked server and use that test to make a new graph for the docs by [@&#8203;cyberw](https://togithub.com/cyberw) in [https://github.com/locustio/locust/pull/2805](https://togithub.com/locustio/locust/pull/2805)
-   Replace total avg response time with 50 percentile (avg was broken) by [@&#8203;andrewbaldwin44](https://togithub.com/andrewbaldwin44) in [https://github.com/locustio/locust/pull/2806](https://togithub.com/locustio/locust/pull/2806)
-   Avoid deadlock in gevent/urllib3 connection pool (fixes occasional worker heartbeat timeouts) by [@&#8203;tdadela](https://togithub.com/tdadela) in [https://github.com/locustio/locust/pull/2813](https://togithub.com/locustio/locust/pull/2813)
-   Fix Dockerfile style warning by [@&#8203;mehrdadbn9](https://togithub.com/mehrdadbn9) in [https://github.com/locustio/locust/pull/2814](https://togithub.com/locustio/locust/pull/2814)
-   Fix pypy gc.freeze() AttributeError by [@&#8203;jimoleary](https://togithub.com/jimoleary) in [https://github.com/locustio/locust/pull/2819](https://togithub.com/locustio/locust/pull/2819)
-   Update poetry windows tests by [@&#8203;mquinnfd](https://togithub.com/mquinnfd) in [https://github.com/locustio/locust/pull/2821](https://togithub.com/locustio/locust/pull/2821)

#### New Contributors

-   [@&#8203;bakhtos](https://togithub.com/bakhtos) made their first contribution in [https://github.com/locustio/locust/pull/2742](https://togithub.com/locustio/locust/pull/2742)
-   [@&#8203;mquinnfd](https://togithub.com/mquinnfd) made their first contribution in [https://github.com/locustio/locust/pull/2725](https://togithub.com/locustio/locust/pull/2725)
-   [@&#8203;mehrdadbn9](https://togithub.com/mehrdadbn9) made their first contribution in [https://github.com/locustio/locust/pull/2814](https://togithub.com/locustio/locust/pull/2814)
-   [@&#8203;jimoleary](https://togithub.com/jimoleary) made their first contribution in [https://github.com/locustio/locust/pull/2819](https://togithub.com/locustio/locust/pull/2819)

**Full Changelog**: locustio/locust@2.29.1...2.30.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "after 10pm every weekday,before 6am every weekday" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR was generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View the [repository job log](https://developer.mend.io/github/apereo/cas).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbIkJvdCIsIlJlbm92YXRlIl19-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Worker sometimes fails to send heartbeat after upgrade to urllib3>=1.26.16
2 participants