feat: add ansible setup scripts (tier4#8)

* ci: add build CI

Signed-off-by: Kenji Miyake <[email protected]>

* feat: add files for Ansible Galaxy

Signed-off-by: Kenji Miyake <[email protected]>

* feat: copy ansible files from tier4's proposal

Remove cache_valid_time

Signed-off-by: Kenji Miyake <[email protected]>

v0.4.0

Signed-off-by: mitsudome-r <[email protected]>

copy setup

Signed-off-by: Takamasa Horibe <[email protected]>

add ros2 ansible roles

Signed-off-by: Takamasa Horibe <[email protected]>

ansible add ubuntu20 setup

Signed-off-by: Takamasa Horibe <[email protected]>

ansible fix pacmod

Signed-off-by: Takamasa Horibe <[email protected]>

temp disable cuda & tensorrt

Signed-off-by: Takamasa Horibe <[email protected]>

ansible fix kvaser library repo

Signed-off-by: Takamasa Horibe <[email protected]>

temporally comment out for unused modules

Signed-off-by: Takamasa Horibe <[email protected]>

remove unused codes

Signed-off-by: Takamasa Horibe <[email protected]>

Install osqp only once (tier4#95)

* update readme and fix osqp_vendor tag

* Remove ansible osqp role

fix: ansbile role

Closes tier4#140

add ansible for cuda 11.1 & tensorrt 7.2.1 & cudnn 8.0.5

Signed-off-by: Takamasa Horibe <[email protected]>

add script for install livox sdk

Signed-off-by: kosuke murakami <[email protected]>

add ansible sdk task

Signed-off-by: kosuke murakami <[email protected]>

Use Livox vendor package (tier4#156)

* Use Livox vendor package

* Removed Livox SDK from ansible

* Use upstream Livox ROS2 driver

Install kvaser library

Signed-off-by: wep21 <[email protected]>

Update ansible/localhost-setup-ubuntu20.04-devpc.yml

Install kvaser library from origin

Signed-off-by: wep21 <[email protected]>

Update ansible/localhost-setup-ubuntu20.04-devpc.yml

install vcstool and do vcs import in ansible setup (tier4#170)

Update documents and fix typos

* update install instructions
* remove redundant statements
* fix typos

Separate Ansible tasks related to vcs import to new role (tier4#180)

* Separate Ansible tasks related to vcs import to new role

* Move the Ansible task to install vcstool to the ROS2 role

Add rmw-cyclonedds-cpp to ansible (tier4#67)

* Add rmw-cyclonedds-cpp to ansible

Signed-off-by: Kenji Miyake <[email protected]>

* Add settings to .bashrc

Signed-off-by: Kenji Miyake <[email protected]>

Fix setup script typos (tier4#202) (tier4#73)

Signed-off-by: Lalith Vipulananthan <[email protected]>
Co-authored-by: Kazuki Miyahara <[email protected]>

Revive pacmod role (tier4#103)

* Revert "Install kvaser library from origin"

This reverts commit 3859fffd60164febbaacc2f68b045d3b0d07f3d6.

Signed-off-by: Kenji Miyake <[email protected]>

* Remove ROS role

Signed-off-by: Kenji Miyake <[email protected]>

install pip before rosdep (tier4#72)

Fix: Update GPG key download task for ROS2 (tier4#125)

* Fix: Update GPG key download task for ROS2

* Fix: Update comments, source list

Add pre-commit CI (tier4#167)

* Add pre-commit CI

Signed-off-by: Kenji Miyake <[email protected]>

* Add .prettierrc.yaml

Signed-off-by: Kenji Miyake <[email protected]>

* Ignore some markdownlint rules

Signed-off-by: Kenji Miyake <[email protected]>

* Apply format

Signed-off-by: Kenji Miyake <[email protected]>

* Fix typo

Signed-off-by: Kenji Miyake <[email protected]>

* Fix typo

Signed-off-by: Kenji Miyake <[email protected]>

Move to ROS2 Galactic (tier4#107)

* Fix rosdistro in ansible

Fix rosdistro other than autoware in ansible (tier4#146)

* Use upstream grid_map

Signed-off-by: Kenji Miyake <[email protected]>

* Remove rosbag2

Signed-off-by: kosuke55 <[email protected]>

* Change position of ecu_system_setup section

Signed-off-by: Kenji Miyake <[email protected]>

* Add missing_packages in galactic

Signed-off-by: Kenji Miyake <[email protected]>

* Add ros2/rosbag2 galactic to repos

Signed-off-by: kosuke55 <[email protected]>

* Purge installed rosbag2 to build it from source

Signed-off-by: kosuke55 <[email protected]>

* Add TODO comment to repos

Co-authored-by: tkimura4 <[email protected]>
Co-authored-by: kosuke55 <[email protected]>

Install ros-galactic-ros2cli-common-extensions in ansible (tier4#183)

* Signed-off-by: kosuke55 <[email protected]>

Install ros-galactic-ros2cli-common-extensions in ansible

Signed-off-by: kosuke55 <[email protected]>

* Add descriptions of rosbag2 purge

Use apt installed rosbag2 (tier4#189)

Signed-off-by: kosuke55 <[email protected]>

Remove vcs import in ansible (tier4#198)

* Remove vcs import in ansible

Signed-off-by: kosuke55 <[email protected]>

* Update installation steps

Signed-off-by: Kenji Miyake <[email protected]>

Co-authored-by: Kenji Miyake <[email protected]>

fix gdown install command (tier4#214)

Simplify ansible files (tier4#122)

* Simplify ansible files

Signed-off-by: Kenji Miyake <[email protected]>

* Add build-pr.yml

Signed-off-by: Kenji Miyake <[email protected]>

add libtorch (tier4#131) (tier4#191)

* add libtorch (tier4#131)

* add libtorch

* update libtorch 1.6.0 -> 1.8.1

* libtorch for CUDA 11.1

* use the same command

* libtorch 1.8.1 -> 1.6.0

* add LibTorch for CUDA 11.1

* change install order

* Deprecate 18.04 support for libtorch

Signed-off-by: wep21 <[email protected]>

* pre-commit fixes

Signed-off-by: wep21 <[email protected]>

Co-authored-by: Yusuke Muramatsu <[email protected]>

fix libtorch install (tier4#196)

Feature/remove pacmod3 (tier4#349)

* remove pacmod3

* Remove kvaser_interface from repos

Signed-off-by: wep21 <[email protected]>

* Fix ansible

Signed-off-by: wep21 <[email protected]>

* Fix vcs import action

Signed-off-by: wep21 <[email protected]>

* Run pre-commit

Signed-off-by: wep21 <[email protected]>

Co-authored-by: wep21 <[email protected]>

add go dependency for pre-commit (tier4#388)

* add go dependency by snap install --classic

* move install-go from autoware/ to pre-commit/

* update playbook.yml

* use apt without snap to install golang

* chore: clean up files

Signed-off-by: Kenji Miyake <[email protected]>

* fix: specify pip3 executable to prevent pipx to be used

Signed-off-by: Kenji Miyake <[email protected]>

* feat: add docker role

Signed-off-by: Kenji Miyake <[email protected]>

* fix: use ansible_user_id instead of ansible_env.USER

root user doesn't have ansible_env.USER.

Signed-off-by: Kenji Miyake <[email protected]>

* chore: fix verbose option

Signed-off-by: Kenji Miyake <[email protected]>

* feat: remove libtorch

Signed-off-by: Kenji Miyake <[email protected]>

* feat: add --no-nvidia option

Signed-off-by: Kenji Miyake <[email protected]>

* style: rename ansible_options to ansible_args

Signed-off-by: Kenji Miyake <[email protected]>

* feat: update CUDA versions

Signed-off-by: Kenji Miyake <[email protected]>

* chore: fix scripts around sudo

Signed-off-by: Kenji Miyake <[email protected]>

* feat: support arm64-sbsa in CUDA role

Signed-off-by: Kenji Miyake <[email protected]>

* ci: add ARM build

Signed-off-by: Kenji Miyake <[email protected]>

* chore: comment out simulator repositories

autowarefoundation/autoware#21
Signed-off-by: Kenji Miyake <[email protected]>

Co-authored-by: Yukihiro Saito <[email protected]>

.github/workflows/build.yaml

@@ -0,0 +1,38 @@
+name: build
+
+on:
+  schedule:
+    - cron: 0 19 * * 0 # run at 4 AM JST on Sundays
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ${{ matrix.os }}
+    container: ubuntu:20.04
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, ARM64]
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+
+      - name: Run setup script
+        run: |
+          ./setup-dev-env.sh -y universe
+
+      - name: Run vcs import
+        run: |
+          mkdir src
+          vcs import src < autoware.repos
+
+      - name: Run rosdep install
+        run: |
+          sudo apt-get -y update
+          rosdep update
+          DEBIAN_FRONTEND=noninteractive rosdep install -y --from-paths src --ignore-src --rosdistro galactic
+
+      - name: Build
+        run: |
+          . /opt/ros/galactic/setup.sh
+          colcon build --event-handlers console_cohesion+ --cmake-args -DCMAKE_BUILD_TYPE=Release

README.md

@@ -31,6 +31,20 @@ If you have any questions or ideas, please feel free to start a discussion on [G
 
 > Note: Detailed setup documents will be put into [autowarefoundation/autoware-documentation](https://github.com/autowarefoundation/autoware-documentation) soon.
 
+## How to setup development environment
+
+```bash
+./setup-dev-env.sh
+```
+
+This script will install the development environment for Autoware, which cannot be installed by `rosdep`.
+
+> Note: Please confirm the licenses before installing NVIDIA libraries.
+
+- [CUDA](https://docs.nvidia.com/cuda/eula/index.html)
+- [cuDNN](https://docs.nvidia.com/deeplearning/cudnn/sla/index.html)
+- [TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/sla/index.html)
+
 ## How to setup workspace
 
 ```bash

ansible-galaxy-requirements.yaml

@@ -1 +1,3 @@
 collections:
+  - source: ./ansible
+    type: dir

ansible/.gitignore

@@ -0,0 +1 @@
+*.retry

ansible/README.md

@@ -0,0 +1 @@
+# Ansible Collection - autoware.dev_env

ansible/galaxy.yml

@@ -0,0 +1,20 @@
+# Required
+namespace: autoware
+name: dev_env
+version: 0.1.0
+readme: README.md
+authors:
+  - Kenji Miyake <[email protected]>
+
+# Optional
+description: Set up development environment for Autoware
+license:
+  - Apache-2.0
+tags:
+  - autoware
+dependencies: {}
+repository: https://github.com/autowarefoundation/autoware
+documentation: https://autowarefoundation.github.io/autoware-documentation
+homepage: https://www.autoware.org/
+issues: https://github.com/autowarefoundation/autoware/issues
+build_ignore: []

ansible/playbooks/core.yaml

@@ -0,0 +1,12 @@
+- hosts: localhost
+  connection: local
+  vars:
+    - rosdistro: galactic
+    - rmw_implementation: rmw_cyclonedds_cpp
+  roles:
+    - role: autoware.dev_env.autoware_core
+    - role: autoware.dev_env.docker
+    - role: autoware.dev_env.pre_commit
+    - role: autoware.dev_env.ros2
+    - role: autoware.dev_env.ros2_dev_tools
+    - role: autoware.dev_env.rmw_implementation

ansible/playbooks/universe.yaml

@@ -0,0 +1,37 @@
+- hosts: localhost
+  connection: local
+  vars:
+    - rosdistro: galactic
+    - rmw_implementation: rmw_cyclonedds_cpp
+    - cuda_version: 11-4
+    - cudnn_version: 8.2.2.26-1+cuda11.4
+    - tensorrt_version: 8.2.2-1+cuda11.4
+  vars_prompt:
+    - name: install_nvidia
+      prompt: |-
+        [Warning] Some components depend on NVIDIA libraries (CUDA, cuDNN, TensorRT), which require license agreements.
+        Did you confirm the licenses and are you sure to to install them? [y/N]
+      private: false
+  pre_tasks:
+    - name: Show warnings that NVIDIA libraries will not be installed
+      ansible.builtin.pause:
+        seconds: 10
+        prompt: |
+          [Warning] Skipping installation of NVIDIA libraries. Please manually install them if you use the depending components.
+      when: install_nvidia != 'y'
+  roles:
+    # Core
+    - role: autoware.dev_env.autoware_core
+    - role: autoware.dev_env.docker
+    - role: autoware.dev_env.pre_commit
+    - role: autoware.dev_env.ros2
+    - role: autoware.dev_env.ros2_dev_tools
+    - role: autoware.dev_env.rmw_implementation
+
+    # Universe
+    - role: autoware.dev_env.autoware_universe
+    - role: autoware.dev_env.cuda
+      when: install_nvidia == 'y'
+    - role: autoware.dev_env.pacmod
+    - role: autoware.dev_env.tensorrt
+      when: install_nvidia == 'y'

ansible/roles/autoware_core/README.md

@@ -0,0 +1,7 @@
+# autoware_core
+
+This role installs development/runtime dependencies for Autoware Core.
+
+## Inputs
+
+None.

ansible/roles/autoware_core/tasks/main.yaml

@@ -0,0 +1,7 @@
+- name: Install gdown to download files from CMakeLists.txt
+  become: true
+  ansible.builtin.pip:
+    name:
+      - gdown
+    state: latest
+    executable: pip3

ansible/roles/autoware_universe/README.md

@@ -0,0 +1,7 @@
+# autoware_universe
+
+This role installs development/runtime dependencies for Autoware Universe.
+
+## Inputs
+
+None.

ansible/roles/autoware_universe/meta/main.yaml

@@ -0,0 +1,2 @@
+dependencies:
+  - autoware_core

ansible/roles/autoware_universe/tasks/main.yaml

@@ -0,0 +1,12 @@
+- name: Install geographiclib-tools
+  become: true
+  ansible.builtin.apt:
+    name: geographiclib-tools
+    state: latest
+    update_cache: true
+
+- name: Install egm2008-1
+  become: true
+  ansible.builtin.command:
+    cmd: geographiclib-get-geoids egm2008-1
+    creates: /usr/share/GeographicLib/geoids/egm2008-1.pgm

ansible/roles/cuda/README.md

@@ -0,0 +1,9 @@
+# cuda
+
+This role installs [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) following [this page](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=deb_network) and [this page](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions).
+
+## Inputs
+
+| Name         | Required | Description                  |
+| ------------ | -------- | ---------------------------- |
+| cuda_version | true     | The version of CUDA Toolkit. |

ansible/roles/cuda/tasks/main.yaml

@@ -0,0 +1,46 @@
+- name: Get CUDA architecture name
+  ansible.builtin.shell: |
+    if [ "$(uname -m)" = "x86_64" ]; then
+      echo "x86_64"
+    else
+      echo "sbsa"
+    fi
+  register: cuda_architecture
+  changed_when: false
+
+- name: Download pin file
+  become: true
+  ansible.builtin.get_url:
+    url: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/{{ cuda_architecture.stdout }}/cuda-ubuntu2004.pin
+    dest: /etc/apt/preferences.d/cuda-repository-pin-600
+
+- name: Add NVIDIA apt key
+  become: true
+  ansible.builtin.apt_key:
+    url: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/{{ cuda_architecture.stdout }}/7fa2af80.pub
+
+- name: Add CUDA repository into sources.list
+  become: true
+  ansible.builtin.apt_repository:
+    repo: deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/{{ cuda_architecture.stdout }}/ /
+    state: present
+
+- name: Install cuda-{{ cuda_version }}
+  become: true
+  ansible.builtin.apt:
+    name: cuda-{{ cuda_version }}
+    update_cache: true
+
+- name: Add PATH to .bashrc
+  ansible.builtin.lineinfile:
+    dest: ~/.bashrc
+    line: export PATH="/usr/local/cuda/bin:$PATH"
+    state: present
+    create: true
+
+- name: Add LD_LIBRARY_PATH to bashrc
+  ansible.builtin.lineinfile:
+    dest: ~/.bashrc
+    line: export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
+    state: present
+    create: true

ansible/roles/docker/README.md

@@ -0,0 +1,11 @@
+# pre_commit
+
+This role installs Docker environment following [this page](https://docs.docker.com/engine/install/ubuntu/) and sets up rootless execution following [this page](https://docs.docker.com/engine/install/linux-postinstall/).
+
+Also, it installs [rocker](https://github.com/osrf/rocker) for easy NVIDIA support.
+
+## Inputs
+
+| Name                   | Required | Description                    |
+| ---------------------- | -------- | ------------------------------ |
+| docker_compose_version | false    | The version of Docker Compose. |

ansible/roles/docker/defaults/main.yaml

@@ -0,0 +1 @@
+docker_compose_version: v2.2.2

ansible/roles/docker/tasks/main.yaml

@@ -0,0 +1,86 @@
+- name: Uninstall old versions
+  become: true
+  ansible.builtin.apt:
+    name:
+      - docker
+      - docker-engine
+      - docker.io
+      - containerd
+      - runc
+    state: absent
+    update_cache: true
+
+- name: Install dependencies for setting up apt sources
+  become: true
+  ansible.builtin.apt:
+    name:
+      - ca-certificates
+      - curl
+      - gnupg
+      - lsb-release
+    update_cache: true
+
+# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
+- name: Authorize Docker GPG key
+  become: true
+  ansible.builtin.apt_key:
+    url: https://download.docker.com/linux/ubuntu/gpg
+    keyring: /usr/share/keyrings/docker-archive-keyring.gpg
+
+- name: Save result of 'dpkg --print-architecture'
+  ansible.builtin.command: dpkg --print-architecture
+  register: deb_architecture
+  changed_when: false
+
+- name: Save result of 'lsb_release -cs'
+  ansible.builtin.command: lsb_release -cs
+  register: lsb_release
+  changed_when: false
+
+# echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+- name: Add Docker apt repository to source list
+  become: true
+  ansible.builtin.apt_repository:
+    repo: deb [arch={{ deb_architecture.stdout }} signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu {{ lsb_release.stdout }} stable
+    filename: docker
+    state: present
+    update_cache: true
+
+- name: Install Docker Engine
+  become: true
+  ansible.builtin.apt:
+    name:
+      - docker-ce
+      - docker-ce-cli
+      - containerd.io
+    update_cache: true
+
+# sudo groupadd docker
+- name: Add docker group
+  become: true
+  ansible.builtin.group:
+    name: docker
+    state: present
+
+# sudo usermod -aG docker $USER
+- name: Add user to docker group
+  become: true
+  ansible.builtin.user:
+    name: "{{ ansible_user_id }}"
+    groups: docker
+    append: true
+
+- name: Install Docker Compose
+  become: true
+  ansible.builtin.get_url:
+    url: https://github.com/docker/compose/releases/download/{{ docker_compose_version }}/docker-compose-{{ ansible_system }}-{{ ansible_architecture }}
+    dest: /usr/local/bin/docker-compose
+    force: true
+    mode: 0775
+
+- name: Install rocker
+  become: true
+  ansible.builtin.pip:
+    name: rocker
+    state: latest
+    executable: pip3

ansible/roles/pacmod/README.md

@@ -0,0 +1,9 @@
+# pacmod
+
+This role sets up the prerequisites to install pacmod3_msgs following [this page](https://github.com/astuff/pacmod3_msgs#installation) and [this page](https://github.com/astuff/docker-builds/blob/e9f563ef04b0578ba0b545711ea3e41b20a51d27/ros2/foxy/ros-core/Dockerfile).
+
+## Inputs
+
+| Name      | Required | Description |
+| --------- | -------- | ----------- |
+| rosdistro | true     | ROS distro. |

ansible/roles/pacmod/meta/main.yaml

@@ -0,0 +1,2 @@
+dependencies:
+  - ros2

ansible/roles/pacmod/tasks/main.yaml

@@ -0,0 +1,29 @@
+- name: Install apt-transport-https
+  become: true
+  ansible.builtin.apt:
+    name: apt-transport-https
+    update_cache: true
+
+- name: Save result of 'lsb_release -sc'
+  ansible.builtin.command: lsb_release -sc
+  register: lsb_release
+  changed_when: false
+
+# echo "deb [trusted=yes] https://s3.amazonaws.com/autonomoustuff-repo/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/autonomoustuff-public.list
+- name: Add AutonomouStuff apt repository to source list
+  become: true
+  ansible.builtin.apt_repository:
+    repo: deb [trusted=yes] https://s3.amazonaws.com/autonomoustuff-repo/ {{ lsb_release.stdout }} main
+    filename: autonomoustuff-public
+    state: present
+    update_cache: true
+
+# echo "yaml https://s3.amazonaws.com/autonomoustuff-repo/autonomoustuff-public-$ROS_DISTRO.yaml $ROS_DISTRO" > /etc/ros/rosdep/sources.list.d/40-autonomoustuff-public-'$ROS_DISTRO'.list
+- name: Add AutonomouStuff rosdep yaml
+  become: true
+  ansible.builtin.lineinfile:
+    path: /etc/ros/rosdep/sources.list.d/40-autonomoustuff-public-{{ rosdistro }}.list
+    line: yaml https://s3.amazonaws.com/autonomoustuff-repo/autonomoustuff-public-{{ rosdistro }}.yaml
+    state: present
+    create: true
+    mode: 0644

ansible/roles/pre_commit/README.md

@@ -0,0 +1,9 @@
+# pre_commit
+
+This role installs dependent tools for [pre-commit](https://pre-commit.com/).
+
+## Inputs
+
+| Name                 | Required | Description                 |
+| -------------------- | -------- | --------------------------- |
+| clang_format_version | false    | The version of ClangFormat. |

ansible/roles/pre_commit/defaults/main.yaml

@@ -0,0 +1 @@
+clang_format_version: 13.0.0