Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update installation guide and have docs point to it #93

Merged
merged 7 commits into from
Jun 4, 2024
Merged

Update installation guide and have docs point to it #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a16a27a
Update installation guide (INSTALL.md) and reference it in the README…
rileyjmurray Jun 4, 2024
20b541e
Update INSTALL.md
rileyjmurray Jun 4, 2024
58ee13e
update README
rileyjmurray Jun 4, 2024
7263dde
accidentally left out
rileyjmurray Jun 4, 2024
49d2ecc
update DevNotes
rileyjmurray Jun 4, 2024
865bb68
tweak
rileyjmurray Jun 4, 2024
4fa95bb
web docs
rileyjmurray Jun 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 55 additions & 116 deletions INSTALL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,39 @@

# Installing and using RandBLAS

Sections 1 through 3 of this file describe how to perform a *basic* installation
of RandBLAS and its dependencies.
This guide has five sections.

Section 4 explains how RandBLAS can be used in other CMake projects.
Sections 1 through 3 describe how to build and install RandBLAS using CMake.

Section 5 gives detailed recommendations on configuring BLAS++ and LAPACK++
for use with RandBLAS.
Its installation instructions for BLAS++ can be used in place
of the BLAS++ instructions in Section 1.
Section 4 explains how to use RandBLAS in other CMake projects.

*We recommend that you not bother with Section 5 the first time you build RandBLAS.*
Section 5 concludes with extra tips.

If you want a TL;DR version of this guide, refer to one of the following.
* Our GitHub Actions to [workflow files](https://github.com/BallisticLA/RandBLAS/tree/main/.github/workflows).
* The [examples folder](https://github.com/BallisticLA/RandBLAS/tree/main/examples).

## 1. Required Dependencies: BLAS++ and Random123

## 1. Required dependencies: a C++20 compatible compiler, BLAS++, and Random123

RandBLAS uses C++20 [concepts](https://en.cppreference.com/w/cpp/language/constraints).
Make sure your compiler supports these. Some compilers (like gcc 8.5) might need to be
invoked with an additional flag (``-fconcepts``) in order to support this aspect of the
C++20 standard. See [this issue](https://github.com/BallisticLA/RandBLAS/issues/90) for more info.

BLAS++ is a C++ API for the Basic Linear Algebra Subroutines.
BLAS++ can be installed with GNU make or CMake;
It can be installed with GNU make or CMake;
RandBLAS requires the CMake install of BLAS++.

Random123 is a collection of counter-based random number generators.

We give recipes for installing BLAS++ and Random123 below.
Later on, we'll assume these recipes were executed from a directory
that contains (or will contain) the ``RandBLAS`` project directory as a subdirectory.

One can compile and install BLAS++ from
[source](https://bitbucket.org/icl/blaspp/src/master/) using CMake by running
[source](https://bitbucket.org/icl/blaspp/src/master/) using CMake by running the following.
Note that all CMake-related terms for BLAS++ use the name ``blaspp`` instead of ``BLAS++``.
```shell
git clone https://github.com/icl-utk-edu/blaspp.git
mkdir blaspp-build
Expand All @@ -49,28 +56,33 @@ make prefix=`pwd`/../random123-install install-include

## 2. Optional dependencies: GTest and OpenMP

GoogleTest is Google’s C++ testing and mocking framework. GTest is an optional
dependency without which RandBLAS regression tests will not be available. GTest
GoogleTest (aka *GTest*) is Google’s C++ testing and mocking framework. It is an optional
dependency, without which RandBLAS regression tests will not be available. It
can be installed with your favorite package manager.

OpemMP is an open standard that enables code to be parallelized as it is
compiled. RandBLAS detects the presence of OpenMP automatically and makes use of
it if it's found.
OpenMP is a standard that enables code to be parallelized as it is compiled.
RandBLAS does not strictly require OpenMP, but it needs OpenMP to quickly
sample dense sketching operators and to quickly perform any sparse matrix computations.

## 3. Building and installing RandBLAS
RandBLAS' CMake configuration step should automatically detect if OpenMP is available.
Sometimes the CMake configuration will fail to recognize OpenMP even if it's
on your system. This is especially common with the default system compilers on macOS
(you can execute ``gcc`` or ``g++`` on macOS, but those are just aliased to
limited versions of ``clang`` and ``clang++``). See [this GitHub issue comment](https://github.com/BallisticLA/RandBLAS/issues/86#issue-2248281376)
for more info.

RandBLAS is configured with CMake and built with GNU make.
The configuration and build processes are simple once RandBLAS' dependencies are in place.

## 3. Building and installing RandBLAS

Assuming you used the recipes from Section 1 to get RandBLAS' dependencies,
you can build download, build, and install RandBLAS as follows:
you can download, build, and install RandBLAS as follows:

```shell
git clone [email protected]:BallisticLA/RandBLAS.git
mkdir RandBLAS-build
cd RandBLAS-build
cmake -DCMAKE_BUILD_TYPE=Release \
-Dblaspp_DIR=`pwd`/../blaspp-install/lib/blaspp/ \
-Dblaspp_DIR=`pwd`/../blaspp-install/lib/cmake/blaspp/ \
-DRandom123_DIR=`pwd`/../random123-install/include/ \
-DCMAKE_BINARY_DIR=`pwd` \
-DCMAKE_INSTALL_PREFIX=`pwd`/../RandBLAS-install \
Expand All @@ -90,11 +102,6 @@ functions in "random_gen.hh".
Here are the conceptual meanings of the recipe's other build flags:

* `-Dblaspp_DIR=X` means `X` is the directory containing the file `blasppConfig.cmake`.

If you follow BLAS++ installation instructions from Section 5 instead of
Section 1, then you'd set ``-Dblaspp_DIR=/opt/mklpp/lib/blaspp``.
Recall that we do not recommend that you follow Section 5 the first time you
build RandBLAS.

* `-DRandom123_DIR=Y` means `Y` is the directory containing the Random123
header files.
Expand All @@ -112,109 +119,41 @@ For instance, the following CMakeLists.txt demonstrates how an executable can
be linked to the RandBLAS library:

```cmake
cmake_minimum_required(VERSION 3.0)
project(myexec)

cmake_minimum_required(VERSION 3.11)
find_package(RandBLAS REQUIRED)

add_executable(myexec ...)
target_link_libraries(myexec RandBLAS ...)
```
In order to build that CMake project you'd need to specify a build flag ``-DRandBLAS_DIR=X``, where ``X`` is a directory that contains ``RandBLAS.cmake``.

The vast majority of projects that use RandBLAS will also use BLAS++ and LAPACK++.
In Section 5 give recommendations on how to configure BLAS++ and LAPACK++.
If you follow those recommendations, then you'd use build flags
``-Dblaspp_DIR=/opt/mklpp/lib/blaspp`` and ``-Dlapackpp_DIR=/opt/mklpp/lib/lapackpp``
when running CMake for your project.
Here is example CMake code for such a project. Note that it references BLAS++ in the final line (as ``blaspp``),
but it doesn't have a ``find_package`` command for BLAS++. That's because when CMake is told to find RandBLAS,
the RandBLAS installation will tell CMake where to find blaspp as a dependency.
Note also that LAPACK++ is referenced as ``lapackpp``.
```cmake
cmake_minimum_required(VERSION 3.11)
project(my_randblas_project)
# ^ The project name can be whatever you want.
find_package(RandBLAS REQUIRED)
find_package(lapackpp REQUIRED)

## 5. Tips
set(myproject_cxx_source my_project.cc)
add_executable(my_project ${myproject_cxx_source})
target_include_directories(myproject PUBLIC ${Random123_DIR})
target_link_libraries(myproject PUBLIC RandBLAS blaspp lapackpp)
```

### Pay attention to the BLAS++ configuration
## 5. Tips

The performance of RandBLAS depends heavily on how BLAS++ is configured.
If performance matters to you then you should inspect the
information that's printed to screen when you run ``cmake`` for the BLAS++ installation.
Save that information somewhere while you're setting up your RandBLAS
development environment.

### Recommended BLAS++ and LAPACK++ configuration

LAPACK++ is a C++ API for a wide range of possible LAPACK implementations.
While LAPACK++ is not required for RandBLAS, it is required in
RandLAPACK, and so it's prudent to configure BLAS++ and LAPACK++ at the same time.

We recommend you install BLAS++ and LAPACK++ so they link to Intel MKL
version 2022 or higher.
That version of MKL will come with CMake configuration files.
Those configuration files are extremely useful if
you want to make a project that connects RandBLAS and Intel MKL.
Such a situation might arise if you want to use RandBLAS together with
MKL's sparse linear algebra functionality.

One of the RandBLAS developers (Riley) has run into trouble
getting BLAS++ to link to MKL as intended.
Here's how Riley configured his BLAS++ and LAPACK++ installations:

0. Install and configure MKL. You can get MKL [here](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html?operatingsystem=linux&distributions=webdownload&options=online).
Once you've installed it you need to edit your `.bashrc` file.
Riley's bashrc file was updated to contain the line
```
export MAIN_MKL_LIBS="/home/riley/intel/oneapi/mkl/latest/lib/intel64"
export LD_LIBRARY_PATH="${MAIN_MKL_LIBS}:${LD_LIBRARY_PATH}"
export LIBRARY_PATH="${MAIN_MKL_LIBS}:${LIBRARY_PATH}"
```

1. Download BLAS++ source, create a new folder called ``build``
at the top level of the BLAS++ project directory, and ``cd`` into that
folder.

2. Run ``export CXX=g++`` so that ``gcc`` is the default compiler for
the current bash session.

3. Decide a common prefix for where you'll put BLAS++ and LAPACK++
installation files. We recommend ``/opt/mklpp``.

4. Run the following CMake command
```
cmake -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/mklpp \
-Dblas=mkl \
-Dblas_int=int64 \
-Dbuild_tests=OFF ..
```
Save the output of that command somewhere. It contains information
on the precise BLAS libraries linked to BLAS++.

5. Run ``cmake --build .``

6. Run ``sudo make install``

7. Download LAPACK++ source, create a new folder called ``build`` at the top level
of the LAPACK++ project directory, and ``cd`` into that folder.

8. Run the following CMake command
```
cmake -DCMAKE_BUILD_TYPE=Release \
-Dblaspp_DIR=/opt/mklpp/lib/blaspp \
-DCMAKE_INSTALL_PREFIX=/opt/mklpp \
-DCMAKE_BINARY_DIR=`pwd` \
-Dbuild_tests=OFF ..
make -j2 install
```

You can then link to BLAS++ and LAPACK++ in other CMake projects
just by including ``find_package(blaspp REQUIRED)`` and ``find_package(lapackpp REQUIRED)``
in your ``CMakeLists.txt`` file, and then passing build flags
```
-Dblaspp_DIR=/opt/mklpp/lib/blaspp -Dlapackpp_DIR=/opt/mklpp/lib/lapackpp
```
when running ``cmake``.

### Installation trouble

RandBLAS has a GitHub Actions workflow to install it from scratch and run its suite of unit tests.
If you're having trouble installing RandBLAS, you can always refer to [that workflow file](https://github.com/BallisticLA/RandBLAS/tree/main/.github/workflows).
The workflow includes statements which print the working directory
and list the contents of that directory at various points in the installation.
We do that so that it's easier to infer a valid choice of directory structure for building RandBLAS.
[An earlier version](https://github.com/BallisticLA/RandBLAS/blob/9d0a03fa41fd7c126b252002a54c2f2562fae31a/INSTALL.md#5-tips)
of this installation guide had specific recommendations for configuring BLAS++ and LAPACK++ on Intel machines.
Those recommendations may be useful to you if you're having a hard time getting these libraries setup correctly.
We removed those recommendations from this guide, since they encouraged a somewhat bad practice of installing BLAS++
and LAPACK++ to a system-wide location (under ``/opt/``) instead of a location that's used for one project.
29 changes: 19 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,29 @@ RandBLAS supports high-level randomized linear algebra algorithms (like randomiz
Our goal is for RandBLAS to become a standard like the BLAS, in that hardware vendors might
release their own optimized implementations of algorithms which confirm to the RandBLAS API.

For those who are new to randomized linear algebra, we recommend you check out [this superb 35-minute YouTube video](https://www.youtube.com/watch?v=6htbyY3rH1w) on the subject.
For those who are new to randomized linear algebra, we recommend you check out [this 35-minute YouTube video](https://www.youtube.com/watch?v=6htbyY3rH1w) on the subject.

### Source Code
### Documentation

The source code can be obtained at the [RandBLAS github repository](https://github.com/BallisticLA/RandBLAS).
We have three types of documentation.
1. Traditional source code comments.
2. Web documentation, split into a [tutorial](https://randblas.readthedocs.io/en/latest/tutorial/index.html) and an [API reference](https://randblas.readthedocs.io/en/latest/api_reference/index.html).
3. Developer notes; [one](RandBLAS/DevNotes.md) for RandBLAS as a whole and [another](RandBLAS/sparse_data/DevNotes.md) for our sparse matrix functionality.

### Documentation
Detailed installation instructions are in [INSTALL.md](INSTALL.md).

### Continuous integration builds

![Latest Ubuntu (OpenMP)](https://github.com/BallisticLA/RandBLAS/actions/workflows/core-linux.yaml/badge.svg)
![Latest macOS (serial)](https://github.com/BallisticLA/RandBLAS/actions/workflows/core-macos.yml/badge.svg)
![Latest macOS (OpenMP)](https://github.com/BallisticLA/RandBLAS/actions/workflows/openmp-macos.yaml/badge.svg)
![Old macOS (OpenMP)](https://github.com/BallisticLA/RandBLAS/actions/workflows/openmp-macos-13.yaml/badge.svg)

See [RandBLAS User's Guide](https://randblas.readthedocs.io/en/latest/) for a terse but thorough overview of RandBLAS' functionality.
### Copyright and license

### CI
![Ubuntu Latest](https://github.com/BallisticLA/RandBLAS/actions/workflows/core-linux.yaml/badge.svg)
![Macos Latest](https://github.com/BallisticLA/RandBLAS/actions/workflows/core-macos.yml/badge.svg)
RandBLAS is licensed under the BSD 3-Clause License.
See [LICENSE](LICENSE) for information and copyright assertions.

### Copyright and License
### Source code

RandBLAS is licensed under the BSD 3-Clause License. See [license](LICENSE) for more information.
The source code can be found at the [RandBLAS github repository](https://github.com/BallisticLA/RandBLAS).
11 changes: 5 additions & 6 deletions RandBLAS/DevNotes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
# Developer Notes for RandBLAS

This file has discussions of RandBLAS' implementation that aren't (currently) suitable
for the RandBLAS User Guide.
This file reviews aspects of RandBLAS' implementation that aren't (currently) suitable
for our user guide.

## What's where?

* Our basic random number generation is handled by [Random123](https://github.com/DEShawResearch/random123).
We have small wrappers around Random123 code in ``RandBLAS/base.hh`` and ``RandBLAS/random_gen.hh``.

* ``RandBLAS/dense_skops.hh`` has code for representing and sampling dense sketching operators.
The sampling code is complicated because it supports multi-threaded (yet threading invariant!)
random (sub)matrix generation.
The sampling code is complicated because it supports multi-threaded random (sub)matrix generation, and yet the generated (sub)matrices are the same no matter how many threads
you're using.

* ``RandBLAS/sparse_skops.hh`` has code for representing and sampling sparse sketching operators.
The sampling code has a customized method for repeatedly sampling from an index set without
Expand All @@ -36,4 +35,4 @@ for the RandBLAS User Guide.
The abstractions can either own their associated data or just wrap existing data (say, data
attached to a sparse matrix in Eigen). RandBLAS has reasonably flexible and high-performance code
for multiplying a sparse matrix and a dense matrix. All code related to sparse matrices is in
``RandBLAS/sparse_data``. See that folder's ``DevNotes.md`` file for details.
``RandBLAS/sparse_data``. See that folder's [``DevNotes.md``](sparse_data/DevNotes.md) file for details.
11 changes: 6 additions & 5 deletions rtd/source/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,6 @@ RandBLAS can be used in distributed environments through its ability to (reprodu

Learn more by reading our `Tutorial <tutorial/index.html>`_ or our `API Reference <api_reference/index.html>`_.

Source Code
-----------
Source code can be obtained at the `RandBLAS github repository <https://github.com/BallisticLA/RandBLAS>`_.


Build and Install
-----------------
RandBLAS is configured with CMake. The following CMake variables influence the build.
Expand All @@ -42,3 +37,9 @@ RandBLAS is configured with CMake. The following CMake variables influence the b
| Random123_DIR | The path to your local random123 install |
+-------------------------+----------------------------------------------------+

See `INSTALL.md <https://github.com/BallisticLA/RandBLAS/blob/main/INSTALL.md>`_ for the most up-to-date
installation instructions.

Source Code
-----------
Source code can be obtained at our `github repository <https://github.com/BallisticLA/RandBLAS>`_.
Loading