Default event loop in Notebook 6.1.6 on Windows is not ProactorEventLoop

[omasoud]

Description
Recently, via PR5907, a workaround, that was put in due to a pre-6.1.0 tornado limitation, was removed (because in 6.1.0, tornado became compatible with the proactor event loop). The workaround was doing this: asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy()).
However, it seems that even after this removal, the even loop policy is still SelectorEventLoop even though on Windows, the default (python 3.8+) asyncio even loop is ProactorEventLoop. This causes running into limitations when attempting to run some code inside notebook (because it would need to run within notebook's event loop). Specifically, anything that uses asyncio.create_subprocess_shell() and asyncio.create_subprocess_exec() will not work.

To Reproduce
The following snippet, if you paste it in a jupyter notebook cell, works and prints hello:

import asyncio
async def foo():
    await asyncio.sleep(1)
    print('hello')
await foo()

But the following one does not work and raises NotImplementedError:

import asyncio
async def foo():
    p=await asyncio.create_subprocess_shell('echo hello',stdout=asyncio.subprocess.PIPE)
    print((await p.stdout.read()).decode())
await foo()

The same code works if run in python.exe (asyncio.run() is used here because there is no already-running event loop):

import asyncio
async def foo():
    p=await asyncio.create_subprocess_shell('echo hello',stdout=asyncio.subprocess.PIPE)
    print((await p.stdout.read()).decode())
asyncio.run(foo())

Inside notebook,

print(asyncio.get_event_loop())
print(asyncio.get_event_loop_policy())

prints:

<_WindowsSelectorEventLoop running=True closed=False debug=False>
<asyncio.windows_events.WindowsSelectorEventLoopPolicy object at 0x000002C6D3C27640>

If you do this:

asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())

the policy changes (note difference in prints below) but the code above still fails.

<_WindowsSelectorEventLoop running=True closed=False debug=False>
<asyncio.windows_events.WindowsProactorEventLoopPolicy object at 0x000002C6D885A520>

When you run these prints inside python.exe you get:

<ProactorEventLoop running=False closed=False debug=False>
<asyncio.windows_events.WindowsProactorEventLoopPolicy object at 0x000001DDB6217940>

Expected Behavior
The event loop used by notebook would ideally be ProactorEventLoop, which is the Windows default asyncio event loop beginning python 3.8, unless there are other reasons not to.
(It is possible of course this is a problem with my own setup and not a real issue; but that can be confirmed if others try the example code I provided)

Versions:
OS: Windows 10

(base) C:\>python -c "import tornado;print(tornado.version_info)"
(6, 1, 0, 0)

(base) C:\>jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.1.6
qtconsole        : 4.7.7
ipython          : 7.19.0
ipykernel        : 5.3.4
jupyter client   : 6.1.7
jupyter lab      : 2.2.6
nbconvert        : 6.0.7
ipywidgets       : 7.5.1
nbformat         : 5.0.8
traitlets        : 5.0.5

(base) C:\>python -V
Python 3.8.5

Additional Notes
It is possible to work around this issue via module nest_asyncio by creating another nested loop (asyncio by design does not allow nesting of event loops):

import asyncio
import nest_asyncio
from contextlib import closing

async def foo():
    p=await asyncio.create_subprocess_shell('echo hello',stdout=asyncio.subprocess.PIPE)
    print((await p.stdout.read()).decode())

loop = asyncio.ProactorEventLoop()
nest_asyncio.apply(loop)

with closing(loop):
    loop.run_until_complete(foo())

[kevin-bates]

Hi @omasoud - I believe this discussion is relative to a similar change made in ipykernel and available in its 5.4.x releases. Since it appears you're still running 5.3.4, I recommend upgrading your ipykernel to the latest version (5.4.2).

[omasoud]

That was it! Thanks you @kevin-bates
Closing the issue.