You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Dec 25, 2023. It is now read-only.
We use python threading's locks a lot for ensuring code correctness in multithreaded contexts. However, we've recently run into a class of issues where those locks don't really play well with ndb tasklets. In particular, if we try to acquire a threading.Lock currently held by another tasklet, we'll deadlock, because we're waiting on a lock held by our own thread, and we don't give ndb's event loop an opportunity to switch back to the tasklet holding the lock. (If, instead, we use an RLock, then we fail the other direction -- we end up with both tasklets having the lock.) The only way to avoid this in our code is to yield in a tasklet while holding a lock (and thus never give ndb a chance to switch tasklets).
The ideal way for us to handle this would be for ndb to provide a lock class designed for tasklets in the same event loop. It could presumably work somewhat similarly to asyncio's locks. In principle this could be implemented in user-land, but at a minimum it'd require something making a fake RPC (which would resolve when we can get the lock), which seems pretty ugly even if it can be made to work without private APIs, so it'd be much better for ndb to implement it.
Note that implementing #243 probably resolves this, too, but I assume that's a bit farther down the road.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
We use python
threading
's locks a lot for ensuring code correctness in multithreaded contexts. However, we've recently run into a class of issues where those locks don't really play well with ndb tasklets. In particular, if we try to acquire athreading.Lock
currently held by another tasklet, we'll deadlock, because we're waiting on a lock held by our own thread, and we don't give ndb's event loop an opportunity to switch back to the tasklet holding the lock. (If, instead, we use anRLock
, then we fail the other direction -- we end up with both tasklets having the lock.) The only way to avoid this in our code is to yield in a tasklet while holding a lock (and thus never give ndb a chance to switch tasklets).The ideal way for us to handle this would be for ndb to provide a lock class designed for tasklets in the same event loop. It could presumably work somewhat similarly to asyncio's locks. In principle this could be implemented in user-land, but at a minimum it'd require something making a fake RPC (which would resolve when we can get the lock), which seems pretty ugly even if it can be made to work without private APIs, so it'd be much better for ndb to implement it.
Note that implementing #243 probably resolves this, too, but I assume that's a bit farther down the road.
The text was updated successfully, but these errors were encountered: