NotImplementedError raised when sphinx-autobuild tries to start tornado server on Python 3.8

[bskinn]

Transferred from tornadoweb/tornado#2756 based on discussion there.

---

I just upgraded my Windows Python to 3.8.0 amd64, and now sphinx-autobuild is failing when it tries to bring up the web server. Based on the traceback, it looks to be a problem in tornado, trying to use a non-implemented (no-longer-implemented-in-3.8-on-Windows?) asyncio feature.

This comment gives a recommended solution:

Do you have any sense of where the right place is to file a bug report? Looks like either livereload or sphinx-autobuild?

Probably sphinx-autobuild based on a quick glance at what's going on there. They would just need to add a line that sets the correct event loop policy when on Windows.

Traceback:

>make livehtml

+--------- manually triggered build ---------------------------------------------
| Running Sphinx v2.2.0
| loading pickled environment... done
| loading intersphinx inventory from isphx/objects_python.inv...
| loading intersphinx inventory from isphx/objects_numpy.inv...
| encountered some issues with some of the inventories, but they had working alternatives:
| intersphinx inventory 'isphx/objects_numpy.inv' not fetchable due to <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'C:\\...\\doc\\source\\isphx/objects_numpy.inv'
| building [mo]: targets for 0 po files that are out of date
| building [html]: targets for 0 source files that are out of date
| updating environment: 0 added, 0 changed, 0 removed
| looking for now-outdated files... none found
| no targets are out of date.
| build succeeded.
|
| The HTML pages are in build.
+--------------------------------------------------------------------------------

[I 191017 14:36:13 server:296] Serving on http://127.0.0.1:8000
Traceback (most recent call last):
  File "c:\...\python38\Lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\...\python38\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\...\env\Scripts\sphinx-autobuild.exe\__main__.py", line 7, in <module>
  File "c:\...\env\lib\site-packages\sphinx_autobuild\__init__.py", line 318, in main
    server.serve(port=portn, host=args.host, root=outdir)
  File "c:\...\env\lib\site-packages\livereload\server.py", line 298, in serve
    self.application(
  File "c:\...\env\lib\site-packages\livereload\server.py", line 253, in application
    app.listen(port, address=host)
  File "c:\...\env\lib\site-packages\tornado\web.py", line 2112, in listen
    server.listen(port, address)
  File "c:\...\env\lib\site-packages\tornado\tcpserver.py", line 152, in listen
    self.add_sockets(sockets)
  File "c:\...\env\lib\site-packages\tornado\tcpserver.py", line 165, in add_sockets
    self._handlers[sock.fileno()] = add_accept_handler(
  File "c:\...\env\lib\site-packages\tornado\netutil.py", line 279, in add_accept_handler
    io_loop.add_handler(sock, accept_handler, IOLoop.READ)
  File "c:\...\env\lib\site-packages\tornado\platform\asyncio.py", line 99, in add_handler
    self.asyncio_loop.add_reader(fd, self._handle_events, fd, IOLoop.READ)
  File "c:\...\python38\Lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError
NotImplementedError

Packages in the environment:

>pip list
Package                       Version
----------------------------- ----------
alabaster                     0.7.12
appdirs                       1.4.3
argh                          0.26.2
attrs                         17.4.0
Babel                         2.7.0
black                         19.3b0
bleach                        3.1.0
certifi                       2019.9.11
chardet                       3.0.4
Click                         7.0
colorama                      0.4.1
coverage                      4.5.4
docutils                      0.15.2
entrypoints                   0.3
filelock                      3.0.12
flake8                        3.7.8
flake8-docstrings             1.5.0
idna                          2.8
imagesize                     1.1.0
Jinja2                        2.10.3
livereload                    2.6.1
MarkupSafe                    1.1.1
mccabe                        0.6.1
numpy                         1.17.3+mkl
packaging                     19.2
pathtools                     0.1.2
pip                           19.3
pkginfo                       1.5.0.1
pluggy                        0.13.0
port-for                      0.3.1
py                            1.8.0
pycodestyle                   2.5.0
pydocstyle                    4.0.1
pyflakes                      2.1.1
Pygments                      2.4.2
pyparsing                     2.4.2
pytz                          2019.3
PyYAML                        5.1.2
readme-renderer               24.0
requests                      2.22.0
requests-toolbelt             0.9.1
restview                      2.9.2
setuptools                    41.4.0
six                           1.12.0
snowballstemmer               2.0.0
Sphinx                        2.2.0
sphinx-autobuild              0.7.1
sphinx-issues                 1.2.0
sphinx-rtd-theme              0.4.3
sphinxcontrib-applehelp       1.0.1
sphinxcontrib-devhelp         1.0.1
sphinxcontrib-htmlhelp        1.0.2
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.2
sphinxcontrib-serializinghtml 1.1.3
toml                          0.10.0
tornado                       6.0.3
tox                           3.14.0
tqdm                          4.36.1
twine                         2.0.0
urllib3                       1.25.6
virtualenv                    16.7.6
watchdog                      0.9.0
webencodings                  0.5.1
wheel                         0.33.6

[eight04]

lepture/python-livereload#209

[pradyunsg]

Whee. Looks like a bug to be fixed.

PRs welcome.

[bskinn]

Actually, it looks like it was something that needed a fix in a dependency, either livereload or watchdog, I suspect.

I tried out sphinx-autobuild again just now on Win, 3.8, and it worked fine.

Honestly, from a quick look at the code here in the repo, I don't know if it even could have been fixed from within sphinx-autobuild -- I don't think the various objects exposed by the dependencies even provide ways to control how the async event loop is instantiated...watchdog might? <shrug> Doesn't really matter!

For reference, the working environment (Python 3.8.1, Win10):

>pip list
Package                       Version   Location
----------------------------- --------- --------------------
alabaster                     0.7.12
appdirs                       1.4.4
argh                          0.26.2
atomicwrites                  1.4.0
attrs                         20.1.0
Babel                         2.8.0
bandit                        1.6.2
black                         19.10b0
bleach                        3.1.5
certifi                       2020.6.20
chardet                       3.0.4
click                         7.1.2
colorama                      0.4.3
coverage                      5.2.1
distlib                       0.3.1
docutils                      0.16
eradicate                     1.0
filelock                      3.0.12
flake8                        3.8.3
flake8-2020                   1.6.0
flake8-absolute-import        1.0
flake8-bandit                 2.1.2
flake8-black                  0.2.1
flake8-bugbear                20.1.4
flake8-builtins               1.5.3
flake8-colors                 0.1.6
flake8-comprehensions         3.2.3
flake8-docstrings             1.5.0
flake8-eradicate              0.4.0
flake8-implicit-str-concat    0.1.0
flake8-import-order           0.18.1
flake8-pie                    0.5.0
flake8-polyfill               1.0.2
flake8-rst-docstrings         0.0.13
fuzzywuzzy                    0.18.0
gitdb                         4.0.5
GitPython                     3.1.7
idna                          2.10
imagesize                     1.2.0
iniconfig                     1.0.1
Jinja2                        2.11.2
jsonschema                    3.2.0
keyring                       21.3.0
livereload                    2.6.2
MarkupSafe                    1.1.1
mccabe                        0.6.1
more-itertools                8.4.0
packaging                     20.4
pathspec                      0.8.0
pathtools                     0.1.2
pbr                           5.4.5
pep517                        0.8.2
pep8-naming                   0.11.1
pip                           20.2.2
pkginfo                       1.5.0.1
pluggy                        0.13.1
port-for                      0.3.1
py                            1.9.0
pycodestyle                   2.6.0
pydocstyle                    5.0.2
pyflakes                      2.2.0
Pygments                      2.6.1
pyparsing                     2.4.7
pyrsistent                    0.16.0
pytest                        6.0.1
pytest-cov                    2.10.1
pytest-ordering               0.6
pytest-subtests               0.3.2
pytest-timeout                1.4.2
pytz                          2020.1
pywin32-ctypes                0.2.0
PyYAML                        5.3.1
readme-renderer               26.0
regex                         2020.7.14
requests                      2.24.0
requests-toolbelt             0.9.1
restructuredtext-lint         1.3.1
restview                      2.9.2
rfc3986                       1.4.0
setuptools                    49.6.0
six                           1.15.0
smmap                         3.0.4
snowballstemmer               2.0.0
Sphinx                        2.3.1
sphinx-autobuild              0.7.1
sphinx-issues                 1.2.0
sphinx-rtd-theme              0.5.0
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        1.0.3
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-programoutput   0.16
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.4
sphobjinv                     2.1.dev0  c:\git\sphobjinv\src
stdio-mgr                     1.0.1
stevedore                     3.2.0
toml                          0.10.1
tornado                       6.0.4
tox                           3.19.0
tqdm                          4.48.2
twine                         3.2.0
typed-ast                     1.4.1
urllib3                       1.25.10
virtualenv                    20.0.31
watchdog                      0.10.3
webencodings                  0.5.1
wget                          3.2
wheel                         0.35.1

[pradyunsg]

This has been fixed in livereload: lepture/python-livereload#209 (comment)