Correct the docker setup for data retrieval and post processing

[akshaygpt]

No description provided.

[akshaygpt]

ERROR: Command errored out with exit status 1:
command: /usr/bin/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3kdreow6/matplotlib/setup.py'"'"'; file='"'"'/tmp/pip-install-3kdreow6/matplotlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-3kdreow6/matplotlib/pip-egg-info
cwd: /tmp/pip-install-3kdreow6/matplotlib/
Complete output (117 lines):
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_common.pyx
Traceback (most recent call last):
File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/tools/cythonize.py", line 61, in process_pyx
from Cython.Compiler.Version import version as cython_version
ModuleNotFoundError: No module named 'Cython'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/tools/cythonize.py", line 238, in <module>
    main()
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/tools/cythonize.py", line 234, in main
    find_process_files(root_dir)
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/tools/cythonize.py", line 225, in find_process_files
    process(root_dir, fromfile, tofile, function, hash_db)
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/tools/cythonize.py", line 191, in process
    processor_function(fromfile, tofile)
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/tools/cythonize.py", line 66, in process_pyx
    raise OSError('Cython needs to be installed in Python as a module')
OSError: Cython needs to be installed in Python as a module
Running from numpy source directory.
/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
  run_build = parse_setuppy_commands()
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py", line 488, in <module>
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py", line 469, in setup_package
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py", line 275, in generate_cython
    version=__version__,
RuntimeError: Running cythonize failed!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-3kdreow6/matplotlib/setup.py", line 311, in <module>
    cmdclass=cmdclass,
  File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 142, in setup
    _install_setup_requires(attrs)
  File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 137, in _install_setup_requires
    dist.fetch_build_eggs(dist.setup_requires)
  File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 586, in fetch_build_eggs
    replace_conflicting=True,
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 780, in resolve
    replace_conflicting=replace_conflicting
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1063, in best_match
    return self.obtain(req, installer)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1075, in obtain
    return installer(requirement)
  File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 653, in fetch_build_egg
    return cmd.easy_install(req)
  File "/usr/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/usr/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 705, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/usr/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/usr/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/usr/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
    run_setup(setup_script, args)
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 253, in run_setup
    raise
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 166, in save_modules
    saved_exc.resume()
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/usr/lib/python3.6/site-packages/setuptools/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/usr/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py", line 488, in <module>
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py", line 469, in setup_package
  File "/tmp/easy_install-49l6hbcn/numpy-1.18.1/setup.py", line 275, in generate_cython
    version=__version__,
RuntimeError: Running cythonize failed!
================================================================================
Edit setup.cfg to change the build options

BUILDING MATPLOTLIB
  matplotlib: yes [3.1.3]
      python: yes [3.6.9 (default, Oct 17 2019, 12:14:22)  [GCC 6.4.0]]
    platform: yes [linux]

OPTIONAL SUBPACKAGES
 sample_data: yes [installing]
       tests: no  [skipping due to configuration]

OPTIONAL BACKEND EXTENSIONS
         agg: yes [installing]
       tkagg: yes [installing; run-time loading from Python Tcl/Tk]
      macosx: no  [Mac OS-X only]

OPTIONAL PACKAGE DATA
        dlls: no  [skipping due to configuration]

Cythonizing sources
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The command '/bin/sh -c pip3 --no-cache-dir install -r requirements.txt' returned a non-zero code: 1

[sanyam1997]

Add this is in the setuptools file to handle the file

try:
from Cython.Build import cythonize
except ImportError:
# create closure for deferred import
def cythonize (*args, ** kwargs ):
from Cython.Build import cythonize
return cythonize(*args, ** kwargs)

[sanyam1997]

You can try to upgrade the numpy version

sudo pip3 install --upgrade numpy

[sanyam1997]

pip install --upgrade cython

[akshaygpt]

Updating the cython and numpy doesn't work. It appears that this is a bug in pip itself.

Pip issues page

[sanyam1997]

SciTools/cartopy#1132

[akshaygpt]

Closing this issue for now since a working solution has been found. May have to update later with a solution that works with alpine image.