-
Notifications
You must be signed in to change notification settings - Fork 48
/
INSTALL
267 lines (202 loc) · 10.6 KB
/
INSTALL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rayleigh Installation Instructions
Throughout this Document, <rayleigh root> refers to the directory in which this
file (named 'INSTALL') is located. If this file is not located in <rayleigh
root>, then you should identify the location of <rayleigh root> before
proceeding.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I. Overview of the Installation Process
Compilation and installation of Rayleigh is accomplished using the "configure /
make / make install " process. Building Rayleigh requires the execution of
three commands from within <rayleigh root>:
1. ./configure --OPTION1=... -OPTION2=...
2. make
3. make install
(4.) Running 'make distclean' will revert <rayleigh root> to it's
pre-configured state.
The configure script automatically supports the GNU, Intel, AOCC, and Cray
compilers. An IBM-compiler example is provided at the end of this document.
On many Intel-based machines, running ./configure or ./configure -mkl will
suffice. In other circumstances, it may be necessary to specify a subset of
configure's options. Some possible variations are illustrated in section III.
To see a detailed description of all available options, run ./configure -h.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
II. Dependencies
Rayleigh requires the the following compilers and libraries:
1. Fortran-MPI compiler that supports the Fortran 2003 standard.
2. C++ compiler (does not need to be MPI-capable)
3. Intel's Math Kernel Library (MKL)
OR
3a. BLAS
3b. LAPack
3c. FFTW (version 3.x)
MKL provides an interface to BLAS, LAPack, and FFTW routines.
We strongly recommend that you link against the MKL library
when running Rayleigh on Intel processors. This is true even if
you are not compiling with the Intel compiler.
If you do not have MKL installed, then you will need to know the location
of your BLAS, LAPACK, and FFTW installations. In particular, the configure
script requires that you know the "root" directory for these installations.
By "root," we mean the directory that contains subdirectories named "lib,"
"include," etc. Rayleigh expects to find the following four files when
building without MKL:
<BLAS ROOT>/lib/libblas.so OR <BLAS ROOT>/lib/libopenblas.so
<LAPACK ROOT>/lib/liblapack.so
<FFTW ROOT>/lib/libfftw3.so
<FFTW ROOT>/include/fftw3.f
These libraries may be linked statically, as discussed in section III.
In that case, similarly named library files with a '.a' extension will
be required.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
III. Configuration
III.a Overview
Rayleigh's configure script currently supports the Intel and GNU compilers.
For other compilers, the optimization, include, and library flags
will need to be set manually by passing approriate flags to configure.
NOTE: If your system uses a module system to manage software, be sure to load
all relevant modules (e.g., intel, impi, mkl, etc.) before running
configure.
The configure process also determines where the rayleigh executables will
be placed. By default, executables are placed in <rayleigh root>/bin.
The 'bin' subdirectory is created when "make install" is run. If you
would like to install Rayleigh to another directory, use the prefix
keyword when running configure. E.g.,
./configure --prefix=<rayleigh install>
The compilation process generates two executables:
<rayleigh install>/bin/rayleigh.opt
and
<rayleigh install>/bin/rayleigh.dbg
The rayleigh.opt executable is compiled with optimization flags appropriate
for your compiler type and should be used for production runs.
The rayleigh.dbg executable is compiled with debugging flags and should
not be used for production runs. Instead, if Rayleigh crashes unexpectedly,
it is a good idea to re-run the same job with rayleigh.dbg and report
the error message(s) to the development team.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
III.b Configuration using the Intel compiler and MKL.
Running ./configure will work "out of the box" on many Intel systems,
provided the following environment variables are defined:
1. FC (the name of your Fortran-MPI compiler; e.g., mpiifort, mpif90 etc.)
2. CC (the name of your C-compiler; e.g., icc or icpc)
3. MKLROOT (the location of your MKL installation)
If these environment variables are undefined, or if they are set to something
that you do not wish to override, invoke configure in a fashion similar to:
./configure --with-mkl=<MKL root> --CC=icc --FC=mpiifort (or mpifc, mpif90, etc.)
NOTE: When compiling with Intel, you will be prompted to specify the level of
vectorization. Alternatively, you may compile for all levels of vectorization
supported by Intel processors. If you choose to do so, no rayleigh.opt will be
produced. Instead, rayleigh.avx, rayleigh.sse, and rayleigh.avx2 will be
generated (in addition to rayleigh.dbg).
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
III.c Configuration using the GNU compiler and MKL
MKL can also be linked to code compiled using GFortran. The configuration
process in this situation is similar to that used for Intel compilers, but
configure is called with the -mkl flag. If the MKLROOT, FC, and CC environment
variables are set, run the following:
./configure -mkl
If they are not set, run:
./configure --with-mkl=<mkl root> --CC=gcc --FC=mpif90 (or mpifc, etc.)
As with Intel compilation, FC should refer to your Fortran-MPI compiler.
Be sure to set FC to mpif90 or mpifc, and not to gfortran.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
III.d Configuration using non-MKL libraries
Rayleigh can be configured to use your preferred FFTW, BLAS, and LAPACK
libraries. These directories are specified using the --with-blas,
--with-lapack, and --with-fftw flags. By default, Raleigh links dynamically
to shared libraries. To alter this behavior, use the --static-blas,
--static-lapack, and or --static-fftw flags. If using OpenBLAS instead of
BLAS, run configure with the -openblas flag.
The following example illustrates how to link with shared OpenBLAS and FFTW libraries,
and a static LAPack library:
BLASROOT=/software/openblas/0.2.20
LAPACKROOT=/software/lapack/3.8.0
FFTWROOT=/software/fftw/3.3.4
./configure \
--FC=mpif90 \
--CC=gcc \
--with-blas=$BLASROOT \
--with-lapack=$LAPACKROOT \
--with-fftw=$FFTWROOT \
--static-lapack \
--openblas
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
III.e Configuration using Alternative Compilers / Customized Compiler Flags
For compilers other than GNU, Intel, AOCC, or Cray, you will need to manually
specify the optimization, library, and include flags. You may also wish to do
this if you are experimenting with new compilation flags that are not
automatically provided by configure for known compilers. These flags can be set
using --LIBFLAGS, --FFLAGS_OPT, --FFLAGS_DBG, and --INCLUDE. FFLAGS_OPT and
FFLAGS_DBG define the optimization flags used for rayleigh.opt and rayleigh.dbg
respectively.
Alternatively, the --FFLAGS can be set to use the same optimization flags for
rayleigh.opt and rayleigh.dbg.
NOTE: the -nodirs (or --nodirs) flag can be set on Machines where linking
C-code code to Fortran code via Rayleigh's Makefile produces errors. In that
case, users will need to create their subdirectories for each Rayleigh run
manually. This can be done by running the script rayleigh/etc/make_dirs from
within a run directory.
The example below shows how to configure the code for Mira, an IBM Blue Gene/Q.
export BLAS_LIB=/soft/libraries/alcf/current/xl/BLAS/lib
export LAPACK_LIB=/soft/libraries/alcf/current/xl/LAPACK/lib
export FFTW_LIB=/soft/libraries/alcf/current/xl/FFTW3/lib
export FFTW_INC=/soft/libraries/alcf/current/xl/FFTW3/include
./configure \
--FC=mpixlf2003_r \
--CC=gcc \
--LIBFLAGS="-L$FFTW_LIB -lfftw3 -L$LAPACK_LIB -llapack -L$BLAS_LIB -lblas" \
--INCLUDE="-I . -I$FFTW_INC" \
--FFLAGS_OPT="-qfree=f90 -O2" \
--FFLAGS_DBG="-qfree=f90 -O0 -g -traceback -qcheck -C" \
--nodirs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
III.f Configuration on a Mac using brew package manager
Perhaps the simplest way to compile Rayleigh on a Mac is to begin by installing
the required compilers and libraries with Homebrew. Homebrew installs all the
stuff you need that Apple didn't. To install Homebrew on your system, follow
the instructions on this website: https://brew.sh/
Next, install the GNU, MPI, and FFTW compilers and libraries with Homebrew as
below.
brew install gcc open-mpi fftw
Working in a bash shell, the example below shows how to configure the code. To
install Rayleigh, change the <install prefix> to an appropriate location on
your machine.
export FC=mpif90
export CC=gcc
export FFTWROOT=/usr/local
export LAPACKROOT=/usr
export BLASROOT=/usr
export PREFIX=<installation prefix>
./configure --FC=$FC --CC=$CC --with-fftw=$FFTWROOT --with-lapack=$LAPACKROOT --with-blas=$BLASROOT --prefix=$PREFIX
make
make install
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
IV
Ultimately, the role of 'configure' is to generate a definitions file that is
included by 'Make' at compilation time. This file is named
"Machine_Definitions," and it is placed in <rayleigh root>/src/build by
configure. As a last resort, you may edit this file directly in order to
customize the compiler flags generated by running configure. If the configure
process fails to create Machine_Definitions, your build is a challening one.
Run './configure --blank' to generate a template that you may edit as needed.
IMPORTANT: Ensure that whitespace is removed at the end of each line in this
file.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
V Building the documentation
If you plan to modify the documentation you may want to build it locally to
check your changes before submission. To do so you need a number of additional
dependencies that are not required to build the Rayleigh executable. In
particular you need the following packages:
sphinx (including its dependencies)
pandoc
python
numpy
scipy
sphinxcontrib-bibtex
nbsphinx
jupyter-client
ipykernel
In order to build the documentation navigate to the doc/ directory, and start
the sphinx build with 'make html' to build the html documentation, or 'make
latexpdf' to build the pdf manual. The generated files will be placed in
'doc/build/html' or 'doc/build/latex' respectively. Note that the design of the
html documentation will deviate slightly from the online version.