Why is calling sync from a running loop a bad idea?
#1570
-
|
Hi all! I just got this error: filesystem_spec/fsspec/asyn.py Line 80 in 05e7d80 I'm wondering why that exception is there in the first place? Thank you |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
|
Unfortunately, the event loop in asyncio is non-reentrant. That means, you cannot say "hey loop, run this" from within the loop. The only way to do that is
|
Beta Was this translation helpful? Give feedback.
-
|
Wee also https://github.com/erdewit/nest_asyncio , which was one attempt to solve the problem, but a low-level library like fsspec shouldn't alter the runtime for all other possible users. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the explanation @martindurant ! |
Beta Was this translation helpful? Give feedback.
-
|
I also go this error when using fsspec in HoloViz Panel. I'll probably have to switch to async interface. |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, I don't think fsspec can call nest_syncio, since that changes session-wide definitions of builtin behaviour - so you are stuck with choosing the right API. |
Beta Was this translation helpful? Give feedback.
-
|
Figured out the problem is with adfls and not fsspec as such. See holoviz/panel#6934 (comment). |
Beta Was this translation helpful? Give feedback.
Unfortunately, the event loop in asyncio is non-reentrant. That means, you cannot say "hey loop, run this" from within the loop. The only way to do that is
await, but you can't do that in a non-async function. So you must do one or the other: usesync(), orawaiton async methods.sync()works by using a dedicated IO thread, since we can wait on thread signaling primitives liveeventin a normal function. If you are running your whole application as async, you don't need another thread.