Octant on Windows

[ghost]

I'd like to be able to install Octant on windows. Is there a way to do so using its current install.py? Is it possible to install just the pure python parts to avoid incompatibilities?

[rsignell-usgs]

@hetland, I thought you separated the binary grid generation stuff out so that folks like Charles could just use the 'other' stuff? Is there some trick?

[ghost]

We have attempted to run setup.py on windows using the --compiler option to tell it to use mingw, the command line was this:
python setup.py install build --compiler=mingw32

This was mainly based on the optimistic assumption that the c code might be portable enough to run on Windows. The source compiled, but in the linking stage, there were numerous link errors concerning undefined references to stderr in triangle.c.

[rsignell-usgs]

As I understand it, the setup.py will not work on windows as it currently stands because the code being compiled has already been run through the C-preprocessor on Linux. So it's really not set up for a multi-platform build process. But you don't need that C stuff anyway for basic use -- I think that is only for grid generation.

[hetland]

The pre-processor issue is fixed, but I think it is still not working on PCs.

-r

On Mar 11, 2014, at 11:32 AM, Rich Signell [email protected] wrote:

As I understand it, the setup.py will not work on windows as it currently stands because the code being compiled has already been run through the C-preprocessor on Linux. So it's really not set up for a multi-platform build process. But you don't need that C stuff anyway for basic use -- I think that is only for grid generation.

—
Reply to this email directly or view it on GitHub.

---

Robert Hetland, Professor
Department of Oceanography
Texas A&M University
http://pong.tamu.edu/~rob

[ghost]

I noticed that in addition to gridgen, there are extensions 'iso', which uses a fortran source file, and 'csa', which uses c source files. are these just things that gridgen relies on, or are they needed by other modules?

I tried running setup.py using only the 'iso' and 'csa' extensions and the install ran without error, however, when I try to import octant from within Python, I get this error:
Traceback (most recent call last):
File "", line 1, in
File "octant__init__.py", line 22, in
import csa
File "octant\csa.py", line 9, in
class CSA(object):
File "octant\csa.py", line 61, in CSA
_csa = np.ctypeslib.load_library('_csa', octant.path[0])
File "C:\Python27\lib\site-packages\numpy\ctypeslib.py", line 131, in load_library
raise exc
WindowsError: [Error 126] The specified module could not be found

[hetland]

CSA is certainly a stand-alone utility. It is a wrapper of Pavel Sakov's utility for fast interpolation. It is useful for interpolating many many points, like interpolating millions of soundings onto a grid.

The iso stuff is for finding properties along an isosurface. The most obvious use case is to find isosurfaces in z. I.e., get the salinity at 50 m depth. There is no fast way to do this in pure python, so the grunt work is done by FORTRAN.

You should be able to use essentially all of octant without either of these things. It would be nice to have an option to install or not to install compiled modules, but I am not that good at distutils. For now, it is easy just to comment those things out. You should get something that works just with those things missing.

[ghost]

We did get Octant to install with the extensions turned off after we realized that it was dependent on Matplotlib's Basemap library. I haven't done much testing, but it's now possible to import octant without errors.

[hetland]

Awesome. Perhaps we could make a setup-basic.py file that could be used instead that would not use any compiled code. I still need to refactor to make explicit all the dependencies in the code. I think basemap is used for the finding the great circle distance in the grid generation software, so it might not be necessary for you either.

[ghost]

Having Octant be dependent on Basemap isn't a problem since we use basemap anyway. Since CSA is a fast-resampling program, it would be very beneficial to us to have that working under Windows. Even though we could probably write something without it, there's a good chance that it could make our application faster if we used it.

[ghost]

I'm trying to install this on a new computer, which is still Windows. It's
a home/telecommuting computer, so I don't have any IT help. I'm using the
latest version of octant, which only has 'iso' as an extension. When I try
to build without it, then use any Octant imports, I get errors like this:

import octant.depths as odep
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\octant__init__.py", line 24, in

import tools
File "C:\Python27\lib\site-packages\octant\tools.py", line 13, in
import _iso
ImportError: No module named _iso

I have included the setup.py file that I used.

On Wed, Mar 19, 2014 at 2:04 PM, Rob Hetland [email protected]
wrote:

Awesome. Perhaps we could make a setup-basic.py file that could be used
instead that would not use any compiled code. I still need to refactor to
make explicit all the dependencies in the code. I think basemap is used for
the finding the great circle distance in the grid generation software, so
it might not be necessary for you either.

—
Reply to this email directly or view it on GitHub
#1 (comment).

"""Octant is a suite of tools for working with ocean models.

Requires:
NumPy (http://numpy.scipy.org)
matplotlib with the Basemap toolkit (http://matplotlib.sourceforge.net)
netCDF4 (http://www.cdc.noaa.gov/people/jeffrey.s.whitaker/python/netCDF4.html)

Contains:
grid - Tools for dealing with curvilinear grids
gridgen
focus
boundaryclick
Grid
gshhs
greatcircle

tools -  Generic tools.
  polygeom
  polyclick
  delaunay

io
  pupynere
  Dataset

ocean
  depths
  velocity
  time

roms -  Tools specific to the Regional Ocean Modeling System
  nc_variables
  write_roms_grid

getm - Tools specific to the General Estuarine Transport Model
  nc_variables
  write_getm_grid

"""

classifiers = """
Development Status :: beta
Environment :: Console
Intended Audience :: Science/Research
Intended Audience :: Developers
License :: MIT
Operating System :: OS Independent
Programming Language :: Python
Topic :: Scientific/Engineering
Topic :: Software Development :: Libraries :: Python Modules
"""

#from numpy.distutils.core import Extension

#iso = Extension(name = '_iso',

sources = ['octant/src/iso.f'])

csa = Extension(name = '_csa',

sources=["octant/src/csa/csa.c",

"octant/src/csa/svd.c"])

doclines = doc.split("\n")

if name == 'main':
from numpy.distutils.core import setup
setup(name = "octant",
version = '0.1.0',
description = doclines[0],
long_description = "\n".join(doclines[2:]),
author = "Robert Hetland",
author_email = "[email protected]",
url = "http://octant.googlecode.com/",
packages = ['octant',
'octant.extern',
'octant.ocean'],
license = 'BSD',
platforms = ["any"],
ext_package='octant',
#ext_modules = [iso],
# ext_modules = [iso, csa],
classifiers = filter(None, classifiers.split("\n")),
)

[hetland]

Does your windows machine have a fortran compiler? That would be required to build the iso module. Perhaps the import should be wrapped in a try/except, and then it will at least import correctly.

On Jul 31, 2014, at 1:42 PM, ccarleton-noaa [email protected] wrote:

I'm trying to install this on a new computer, which is still Windows. It's
a home/telecommuting computer, so I don't have any IT help. I'm using the
latest version of octant, which only has 'iso' as an extension. When I try
to build without it, then use any Octant imports, I get errors like this:

import octant.depths as odep
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\octant__init__.py", line 24, in

import tools
File "C:\Python27\lib\site-packages\octant\tools.py", line 13, in
import _iso
ImportError: No module named _iso

I have included the setup.py file that I used.

On Wed, Mar 19, 2014 at 2:04 PM, Rob Hetland [email protected]
wrote:

Awesome. Perhaps we could make a setup-basic.py file that could be used
instead that would not use any compiled code. I still need to refactor to
make explicit all the dependencies in the code. I think basemap is used for
the finding the great circle distance in the grid generation software, so
it might not be necessary for you either.

—
Reply to this email directly or view it on GitHub
#1 (comment).

"""Octant is a suite of tools for working with ocean models.

Requires:
NumPy (http://numpy.scipy.org)
matplotlib with the Basemap toolkit (http://matplotlib.sourceforge.net)
netCDF4 (http://www.cdc.noaa.gov/people/jeffrey.s.whitaker/python/netCDF4.html)

Contains:
grid - Tools for dealing with curvilinear grids
gridgen
focus
boundaryclick
Grid
gshhs
greatcircle

tools - Generic tools.
polygeom
polyclick
delaunay

io
pupynere
Dataset

ocean
depths
velocity
time

roms - Tools specific to the Regional Ocean Modeling System
nc_variables
write_roms_grid

getm - Tools specific to the General Estuarine Transport Model
nc_variables
write_getm_grid
"""

classifiers = """
Development Status :: beta
Environment :: Console
Intended Audience :: Science/Research
Intended Audience :: Developers
License :: MIT
Operating System :: OS Independent
Programming Language :: Python
Topic :: Scientific/Engineering
Topic :: Software Development :: Libraries :: Python Modules
"""

#from numpy.distutils.core import Extension

#iso = Extension(name = '_iso',

sources = ['octant/src/iso.f'])

csa = Extension(name = '_csa',

sources=["octant/src/csa/csa.c",

"octant/src/csa/svd.c"])

doclines = doc.split("\n")

if name == 'main':
from numpy.distutils.core import setup
setup(name = "octant",
version = '0.1.0',
description = doclines[0],
long_description = "\n".join(doclines[2:]),
author = "Robert Hetland",
author_email = "[email protected]",
url = "http://octant.googlecode.com/",
packages = ['octant',
'octant.extern',
'octant.ocean'],
license = 'BSD',
platforms = ["any"],
ext_package='octant',
#ext_modules = [iso],

ext_modules = [iso, csa],

classifiers = filter(None, classifiers.split("\n")),
)

—
Reply to this email directly or view it on GitHub.

---

Robert Hetland, Professor
Department of Oceanography
Texas A&M University
http://pong.tamu.edu/~rob

[rsignell-usgs]

@pelson, we are struggling a bit (okay, a lot!) here to support windows users. Octant has both C and Fortran wrapped python modules. Could we use something like the approach here:
https://github.com/rmcgibbo/python-appveyor-conda-example
and that you apparently used to build cartopy for conda?
https://binstar.org/pelson/cartopy

It would be so great if windows users could just do "conda install octant"
And also so great if we didn't have to install 32 and 64 windows build environments!

[pelson]

@pelson, we are struggling a bit (okay, a lot!) here to support windows users.
And also so great if we didn't have to install 32 and 64 windows build environments!

Supreme timing. We are just about to embark (tomorrow) on a fortnight of effort pulling together our building and uploading of the scitools dependencies on linux-64, osx-64 (via travis-ci) and Windows (32 bit initially, but the approach will be applicable 64 bit Windows) via Appveyor.

Could we use something like the approach here: https://github.com/rmcgibbo/python-appveyor-conda-example

Yes, the approach is effectively that which we will be taking. In fact, I've already got a proof of concept package (https://github.com/pelson/Obvious-CI) which should encapsulate all of the hard work (like setting up the dev environment and installing miniconda) and allow the user to focus on the actual build steps.

Please don't copy it just yet (as it will change significantly in the next few days/weeks), but my current progress can be seen in https://github.com/pelson/conda-recipes-scitools/compare/SciTools:master...appveyor - with the key file being https://github.com/pelson/conda-recipes-scitools/blob/da95f054f114232038cf9a11c7fe80921c5b7242/appveyor.yml.

I have a sum total of about 2 weeks worth of time building software on Windows, so don't see me as an expert, but I'm pretty hopeful that the approach we are going to take can be repeated by other packages for automated building of distributions for multiple OSes.

[rsignell-usgs]

@pelson , that sounds awesome! We can definitely wait a few weeks until you guys have finished your work on this -- we've waited years already. You may have seen that I posted a "how-do-I-build-on-windows" question on the conda google group here https://groups.google.com/a/continuum.io/forum/#!topic/conda/yHevuIBaqtI
but don't know whether you want to tell folks about what you are doing there. I think they would appreciate it and perhaps help others like me from wasting effort over the next few weeks, but maybe you don't want to publicize yet either.

In any case, thank you from the community for doing this!

[hetland]

Yes, indeed, this is very welcome. I have exactly zero weeks of experience
building software on windows, so this is quite useful. Note that we have
also had issues with various C compilers on OS X and Linux in the past, so
that there may be issues beyond just getting the windows environment right,
and uploaded to binstar.

On Mon, Dec 8, 2014 at 10:46 AM, Rich Signell [email protected]
wrote:

@pelson https://github.com/pelson , that sounds awesome! We can
definitely wait a few weeks until you guys have finished your work on this
-- we've waited years already. You may have seen that I posted a
"how-do-I-build-on-windows" question on the conda google group here
https://groups.google.com/a/continuum.io/forum/#!topic/conda/yHevuIBaqtI
but don't know whether you want to tell folks about what you are doing
there. I think they would appreciate it and perhaps help others like me
from wasting effort over the next few weeks, but maybe you don't want to
publicize yet either.

In any case, thank you from the community for doing this!

—
Reply to this email directly or view it on GitHub
#1 (comment).

Prof. Rob Hetland
Texas A&M Univ. – Dept. of Oceanography
http://pong.tamu.edu/~rob

[phobson]

@pelson (et al) just chiming in to let you know that this is a library I use -- specifically just to generate grids for 3-D hydrodynamic models. So if you need any testing on Windows or any other system, do give me a shout.

Presently I have to do everything in a Linux VM, which is fine, but conda install octant on any system would be A+.

[nikhil003]

In order to install and use octant on windows with cygwin, two simple fixes

1. run cpp on the triangle.c script in /external/gridgen/triangle.c
2. apply the fix mentioned on https://github.com/hetland/pygridgen/issues/21

and hopefully it should compile and run as desired.