Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/branch-23.08' into cuda-major-…
Browse files Browse the repository at this point in the history
…minor
  • Loading branch information
raydouglass committed Aug 3, 2023
2 parents 25e2850 + d3ec7de commit fe7003e
Show file tree
Hide file tree
Showing 4 changed files with 167 additions and 39 deletions.
39 changes: 0 additions & 39 deletions .github/workflows/dockerhub-readme.yml

This file was deleted.

20 changes: 20 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,23 @@ jobs:
with:
build_type: branch
secrets: inherit
readme:
runs-on: ubuntu-latest
needs: docker
if: startsWith(github.ref, 'refs/tags/v')
strategy:
matrix:
repo_name:
- rapidsai/base
- rapidsai/notebooks
steps:
- name: checkout code
uses: actions/checkout@v3

- name: Update DockerHub README for ${{ matrix.repo_name }}
uses: peter-evans/dockerhub-description@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
repository: ${{ matrix.repo_name }}
readme-filepath: dockerhub-readme.md
4 changes: 4 additions & 0 deletions ci/release/update-version.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,7 @@ sed_runner "s/ARG RAPIDS_VER=.*/ARG RAPIDS_VER=${NEXT_SHORT_TAG}/g" Dockerfile
for FILE in .github/workflows/*.yml; do
sed_runner "/shared-action-workflows/ s/@.*/@branch-${NEXT_SHORT_TAG}/g" "${FILE}"
done

sed_runner "s/v[[:digit:]]\+\.[[:digit:]]\+/v${NEXT_SHORT_TAG}/g" dockerhub-readme.md
sed_runner "s/[[:digit:]]\+\.[[:digit:]]\+-cuda/${NEXT_SHORT_TAG}-cuda/g" dockerhub-readme.md
sed_runner "s/[[:digit:]]\+\.[[:digit:]]\+a-cuda/${NEXT_SHORT_TAG}a-cuda/g" dockerhub-readme.md
143 changes: 143 additions & 0 deletions dockerhub-readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# RAPIDS - Open GPU Data Science

## What is RAPIDS?

Visit [rapids.ai](https://rapids.ai) for more information.

The RAPIDS suite of software libraries gives you the freedom to execute end-to-end data science and analytics pipelines entirely on GPUs. It relies on NVIDIA® CUDA® primitives for low-level compute optimization, but exposes GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.

**NOTE:** Review our [system requirements](https://docs.rapids.ai/install#system-req) to ensure you have a compatible system!


### Current Version - RAPIDS v23.08

RAPIDS Libraries included in the images:
- `cuDF`
- `cuML`
- `cuGraph`
- `RMM`
- `RAFT`
- `cuSpatial`
- `cuSignal`
- `cuxfilter`
- `cuCIM`
- `xgboost`
- `dask-sql`


### Image Types

The RAPIDS images are based on [`nvidia/cuda`](https://hub.docker.com/r/nvidia/cuda) and [`rapidsai/mambaforge-cuda`](https://hub.docker.com/r/rapidsai/mambaforge-cuda). The RAPIDS images provide `amd64` & `arm64` architectures [where supported](https://docs.rapids.ai/install#system-req).


There are two types:
- `rapidsai/base` - contains a RAPIDS environment ready for use.
- **TIP: Use this image if you want to use RAPIDS as a part of your pipeline.**
- `rapidsai/notebooks` - extends the `rapidsai/base` image by adding a [`jupyterlab` server](https://jupyter.org/), example notebooks, and dependencies.
- **TIP: Use this image if you want to explore RAPIDS through notebooks and examples.**

### Image Tag Naming Scheme

The tag naming scheme for RAPIDS images incorporates key platform details into the tag as shown below:
```
23.08-cuda11.8-py3.10
^ ^ ^
| | Python version
| |
| CUDA version
|
RAPIDS version
```

**Note: Nightly builds of the images have the RAPIDS version appended with an `a` (ie `23.08a-cuda11.8-py3.10`)**

## Usage

The `rapidsai/base` image starts with an [`ipython` shell](https://ipython.org/) by default.

The `rapidsai/notebooks` image starts with the [JupyterLab notebook server](https://jupyter.org/) by default.

### Container Ports

`rapidsai/notebooks` exposes port `8888` for the [JupyterLab notebook server](https://jupyter.org/).

### Environment Variables

The following environment variables can be passed to the `docker run` commands:

- `EXTRA_CONDA_PACKAGES` - used to install additional `conda` packages in the container. Use a space separated list of values
- `CONDA_TIMEOUT` - how long (in seconds) the `conda` command should wait before exiting
- `EXTRA_PIP_PACKAGES` - used to install additional `pip` packages in the container. Use a space separated list of values
- `PIP_TIMEOUT` - how long (in seconds) the `pip` command should wait before exiting

Example:

```sh
$ docker run \
--rm \
-it \
--pull always \
--gpus all \
-shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
-e EXTRA_CONDA_PACKAGES="jq" \
-e EXTRA_PIP_PACKAGES="beautifulsoup4" \
-p 8888:8888 \
rapidsai/notebooks:23.08-cuda11.8-py3.10
```

### Bind Mounts

Mounting files/folders to the locations specified below provide additional functionality for the images.

- `/home/rapids/environment.yml` - a YAML file that contains a list of dependencies that will be installed by `conda`. The file should look like:

```yml
dependencies:
- beautifulsoup4
- jq
```
Example:
```sh
$ docker run \
--rm \
-it \
--pull always \
--gpus all \
-shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
-v $(pwd)/environment.yml:/home/rapids/environment.yml \
rapidsai/base:23.08-cuda11.8-py3.10
```

### Use JupyterLab to Explore the Notebooks

The `rapidsai/notebooks` container has notebooks for the RAPIDS libraries in `/home/rapids/notebooks`.

### Extending RAPIDS Images

All RAPIDS images use `conda` as their package manager, and all RAPIDS packages are available in the `base` conda environment. These image run as the `rapids` user.

### Access Documentation within Notebooks

You can check the documentation for RAPIDS APIs inside the JupyterLab notebook using a `?` command, like this:
```
[1] ?cudf.read_csv
```
This prints the function signature and its usage documentation. If this is not enough, you can see the full code for the function using `??`:
```
[1] ??cudf.read_csv
```
Check out the RAPIDS [documentation](https://docs.rapids.ai/) for more detailed information.

## More Information

Check out the [RAPIDS User Guides](https://docs.rapids.ai/user-guide) and [XGBoost](https://xgboost.readthedocs.io/en/latest/) API docs.

## Where can I get help or file bugs/requests?

Please submit issues with the container to this GitHub repository: [https://github.com/rapidsai/docker](https://github.com/rapidsai/docker/issues/new)

For issues with RAPIDS libraries like cuDF, cuML, RMM, or others file an issue in the related GitHub project.

Additional help can be found on [Stack Overflow](https://stackoverflow.com/tags/rapids).

0 comments on commit fe7003e

Please sign in to comment.