C++ Implementation of PyTorch Tutorials for Everyone

OS (Compiler)\LibTorch	2.3.0
macOS (clang 11, 12, 13)
Linux (gcc 9, 10, 11)
Windows (msvc 2019, 2022)

Table of Contents

This repository provides tutorial code in C++ for deep learning researchers to learn PyTorch (i.e. Section 1 to 3)
Python Tutorial: https://github.com/yunjey/pytorch-tutorial

1. Basics

• PyTorch Basics
• Linear Regression
• Logistic Regression
• Feedforward Neural Network

2. Intermediate

• Convolutional Neural Network
• Deep Residual Network
• Recurrent Neural Network
• Bidirectional Recurrent Neural Network
• Language Model (RNN-LM)

3. Advanced

• Generative Adversarial Networks
• Variational Auto-Encoder
• Neural Style Transfer
• Image Captioning (CNN-AttentionRNN)

4. Interactive Tutorials

• Tensor Slicing

5. Other Popular Tutorials

• Deep Learning with PyTorch: A 60 Minute Blitz

Getting Started

Requirements

1. C++-17 compatible compiler
2. CMake (minimum version 3.19)
3. LibTorch version >= 1.12.0 and <= 2.3.0
4. Conda

For Interactive Tutorials

Note: Interactive Tutorials are currently running on LibTorch Nightly Version.
So there are some tutorials which can break when working with nightly version.

conda create --name pytorch-cpp
conda activate pytorch-cpp
conda install xeus-cling notebook -c conda-forge

Clone, build and run tutorials

In Google Colab

On Local Machine

git clone https://github.com/prabhuomkar/pytorch-cpp.git
cd pytorch-cpp

Generate build system

cmake -B build #<options>

Note for Windows users:
Libtorch only supports 64bit Windows and an x64 generator needs to be specified. For Visual Studio this can be done by appending -A x64 to the above command.

Some useful options:

Option	Default	Description
-D CUDA_V=(11.8|12.1|none)	none	Download LibTorch for a CUDA version (none = download CPU version).
-D LIBTORCH_DOWNLOAD_BUILD_TYPE=(Release|Debug)	Release	Determines which libtorch build type version to download (only relevant on Windows).
-D DOWNLOAD_DATASETS=(OFF|ON)	ON	Download required datasets during build (only if they do not already exist in pytorch-cpp/data).
-D CREATE_SCRIPTMODULES=(OFF|ON)	OFF	Create all required scriptmodule files for prelearned models / weights during build. Requires installed python3 with pytorch and torchvision.
-D CMAKE_PREFIX_PATH=path/to/libtorch/share/cmake/Torch	<empty>	Skip the downloading of LibTorch and use your own local version (see Requirements) instead.
-D CMAKE_BUILD_TYPE=(Release|Debug|...)	<empty>	Determines the CMake build-type for single-configuration generators (see CMake docs).

Example Linux

Aim

• Use existing Python, PyTorch (see Requirements) and torchvision installation.
• Download all datasets and create all necessary scriptmodule files.

Command

cmake -B build \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_PREFIX_PATH=/path/to/libtorch/share/cmake/Torch \
-D CREATE_SCRIPTMODULES=ON 

Example Windows

Aim

• Automatically download LibTorch for CUDA 11.8 (Release version) and all necessary datasets.
• Do not create scriptmodule files.

Command

cmake -B build \
-A x64 \
-D CUDA_V=11.8

Build

Note for Windows (Visual Studio) users:
The CMake script downloads the Release version of LibTorch, so --config Release has to be appended to the build command.

How dataset download and scriptmodule creation work:

• If DOWNLOAD_DATASETS is ON, the datasets required by the tutorials you choose to build will be downloaded to pytorch-cpp/data (if they do not already exist there).
• If CREATE_SCRIPTMODULES is ON, the scriptmodule files for the prelearned models / weights required by the tutorials you choose to build will be created in the model folder of the respective tutorial's source folder (if they do not already exist).

All tutorials

To build all tutorials use

cmake --build build

All tutorials in a category

You can choose to only build tutorials in one of the categories basics, intermediate, advanced or popular. For example, if you are only interested in the basics tutorials:

cmake --build build --target basics
# In general: cmake --build build --target {category}

Single tutorial

You can also choose to only build a single tutorial. For example to build the language model tutorial only:

cmake --build build --target language-model
# In general: cmake --build build --target {tutorial-name}

Note:
The target argument is the tutorial's foldername with all underscores replaced by hyphens.

Tip for users of CMake version >= 3.15:
You can specify several targets separated by spaces, for example:

cmake --build build --target language-model image-captioning

Run Tutorials

1. (IMPORTANT!) First change into the tutorial's directory within build/tutorials. For example, assuming you are in the pytorch-cpp directory and want to change to the pytorch basics tutorial folder:

   cd build/tutorials/basics/pytorch_basics
   # In general: cd build/tutorials/{basics|intermediate|advanced|popular/blitz}/{tutorial_name}

2. Run the executable. Note that the executable's name is the tutorial's foldername with all underscores replaced with hyphens (e.g. for tutorial folder: pytorch_basics -> executable name: pytorch-basics (or pytorch-basics.exe on Windows)). For example, to run the pytorch basics tutorial:
   
   Linux/Mac

   ./pytorch-basics
   # In general: ./{tutorial-name}

   Windows

   .\pytorch-basics.exe
   # In general: .\{tutorial-name}.exe

Using Docker

Find the latest and previous version images on Docker Hub.

You can build and run the tutorials (on CPU) in a Docker container using the provided Dockerfile and docker-compose.yml files:

1. From the root directory of the cloned repo build the image:

   docker-compose build --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g)

   Note:
   When you run the Docker container, the host repo directory is mounted as a volume in the Docker container in order to cache build and downloaded dependency files so that it is not necessary to rebuild or redownload everything when a container is restarted. In order to have correct file permissions it is necessary to provide your user and group ids as build arguments when building the image on Linux.

2. Now start the container and build the tutorials using:

   docker-compose run --rm pytorch-cpp

   This fetches all necessary dependencies and builds all tutorials. After the build is done, by default the container starts bash in interactive mode in the build/tutorials folder.
   As with the local build, you can choose to only build tutorials of a category (basics, intermediate, advanced, popular):

   docker-compose run --rm pytorch-cpp {category}

   In this case the container is started in the chosen category's base build directory.
   Alternatively, you can also directly run a tutorial by instead invoking the run command with a tutorial name as additional argument, for example:

   docker-compose run --rm pytorch-cpp pytorch-basics
   # In general: docker-compose run --rm pytorch-cpp {tutorial-name} 

   This will - if necessary - build the pytorch-basics tutorial and then start the executable in a container.

License

This repository is licensed under MIT as given in LICENSE.