-
Notifications
You must be signed in to change notification settings - Fork 56
Building Envoy
linuxonz edited this page Jun 21, 2024
·
4 revisions
The instructions provided below specify the steps to build Envoy version 1.29.2 on Linux on IBM Z for the following distributions:
- RHEL (8.8, 8.9, 9.2, 9.3)
- SLES (12 SP5, 15 SP5)
- Ubuntu (20.04, 22.04, 24.04)
General Notes:
- When following the steps below please use a standard permission user unless otherwise specified.
- A directory
/<source_root>/will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.
If you want to build Envoy using manual steps, go to Step 2.
Use the following commands to build Envoy using the build script. Please make sure you have wget installed.
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Envoy/1.29.2/build_envoy.sh
# Build Envoy
bash build_envoy.sh [Provide -t option for executing build with tests]In case of error, check logs for more details or go to STEP 2 to follow manual build steps.
export SOURCE_ROOT=/<source_root>/
export PREFIX=/usr/local
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Envoy/1.29.2/patch/"-
RHEL (8.8, 8.9)
sudo yum install -y autoconf curl libtool patch python3-pip pkg-config vim wget curl file git gzip make python2 python3 java-11-openjdk-devel unzip zip tar openssl-devel xz expat-devel glib2-devel gcc gcc-c++ #set JAVA_HOME location export JAVA_HOME=/usr/lib/jvm/java-11-openjdk export PATH=$JAVA_HOME/bin:$PATH
-
RHEL (9.2, 9.3)
sudo yum install -y wget curl bison ncurses ncurses-devel pkgconfig net-tools gcc-toolset-12-gcc gcc-toolset-12-gcc-c++ gcc-toolset-12-binutils-devel gcc-toolset-12-annobin-plugin-gcc gcc-toolset-12-libstdc++-devel gcc-toolset-12-binutils-gold gcc-toolset-12-libatomic-devel patch unzip openssl-devel java-11-openjdk-devel python3 zlib-devel diffutils libtool libatomic libarchive zip file #set gcc 12 as default source /opt/rh/gcc-toolset-12/enable
-
SLES 12 SP5
sudo zypper install -y autoconf curl libtool patch python3-pip pkg-config vim wget git gzip make cmake python python3 python36 java-11-openjdk-devel unzip zip tar xz which gawk glib2-devel libexpat-devel bind-chrootenv coreutils ed expect file iproute2 iputils less libopenssl-devel python2 python3 python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel unzip zlib-devel python3-PyYAML gcc gcc-c++ gcc12-c++ git-core automake autoconf libtool libelf-devel gawk glibc-devel-static kmod binutils-gold #set gcc 12 as default sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-12 12 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 12 sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-12 12 #set JAVA_HOME location export JAVA_HOME=/usr/lib64/jvm/java-11-openjdk export PATH=$JAVA_HOME/bin:$PATH
- Install Openssl
cd $SOURCE_ROOT wget https://www.openssl.org/source/openssl-1.1.1w.tar.gz --no-check-certificate tar -xzf openssl-1.1.1w.tar.gz cd openssl-1.1.1w ./config --prefix=/usr/local --openssldir=/usr/local make && sudo make install sudo mkdir -p /usr/local/etc/openssl sudo wget https://curl.se/ca/cacert.pem --no-check-certificate -P /usr/local/etc/openssl export LDFLAGS="-L/usr/local/lib/ -L/usr/local/lib64/" export LD_LIBRARY_PATH=/usr/local/lib/:/usr/local/lib64/${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}} export LD_RUN_PATH=/usr/local/lib:/usr/local/lib64${LD_RUN_PATH:+:${LD_RUN_PATH}} export CPPFLAGS="-I/usr/local/include/ -I/usr/local/include/openssl" export SSL_CERT_FILE=/usr/local/etc/openssl/cacert.pem sudo ldconfig /usr/local/lib64
- Install CMake
cd $SOURCE_ROOT wget https://github.com/Kitware/CMake/releases/download/v3.22.5/cmake-3.22.5.tar.gz tar -xf cmake-3.22.5.tar.gz cd cmake-3.22.5 ./bootstrap -- -DCMAKE_BUILD_TYPE:STRING=Release make && sudo make install sudo ln -sf /usr/local/bin/cmake /usr/bin/cmake
- Install Openssl
-
SLES 15 SP5
sudo zypper install -y autoconf curl libtool patch python3-pip pkg-config vim wget git gcc12-c++ gzip make cmake python python3 java-11-openjdk-devel unzip zip tar xz which gawk glib2-devel libexpat-devel meson ninja gobject-introspection-devel python3-bind bind-chrootenv coreutils ed expect file iproute2 iputils lcov less libopenssl-devel netcat python2 python2-devel python3 python3-devel python3-pip python3-requests python3-setuptools python3-six python3-wheel unzip zlib-devel python3-python-gnupg python3-PyYAML #set gcc 12 as default sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-12 12 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 12 sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-12 12 #set JAVA_HOME location export JAVA_HOME=/usr/lib64/jvm/java-11-openjdk export PATH=$JAVA_HOME/bin:$PATH
-
Ubuntu 20.04
sudo apt-get update sudo apt-get install -y autoconf curl git libtool patch python3-pip virtualenv pkg-config gcc g++ locales build-essential openjdk-11-jdk python2 python2-dev python-is-python3 python3 python3-dev zip unzip libssl-dev #set JAVA_HOME location export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-s390x export PATH=$JAVA_HOME/bin:$PATH
-
Ubuntu 22.04
sudo apt-get update sudo apt-get install -y autoconf curl git libtool patch python3-pip unzip virtualenv pkg-config locales libssl-dev build-essential openjdk-11-jdk python2 python2-dev python3 python3-dev zip # Install GCC 12 from repo sudo apt-get install -y gcc-12 g++-12 sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-12 12 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 12 sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-12 12 #set JAVA_HOME location export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-s390x export PATH=$JAVA_HOME/bin:$PATH
-
Ubuntu 24.04
sudo apt-get update sudo apt-get install -y autoconf curl git libtool patch python3-pip unzip virtualenv pkg-config locales gcc g++ openssl libssl-dev build-essential openjdk-11-jdk python3 gcc-12 g++-12 zip unzip #set JAVA_HOME location export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-s390x export PATH=$JAVA_HOME/bin:$PATH # use gcc-12 to build bazel due to error compiling with gcc-13: https://github.com/bazelbuild/bazel/issues/18642 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 20 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 20 sudo update-alternatives --install /usr/bin/gcov gcov /usr/bin/gcov-12 20 sudo update-alternatives --install /usr/bin/gcov-tool gcov-tool /usr/bin/gcov-tool-12 20 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 13 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 13 sudo update-alternatives --install /usr/bin/gcov gcov /usr/bin/gcov-13 13 sudo update-alternatives --install /usr/bin/gcov-tool gcov-tool /usr/bin/gcov-tool-13 13
cd $SOURCE_ROOT
mkdir bazel && cd bazel
wget https://github.com/bazelbuild/bazel/releases/download/6.3.2/bazel-6.3.2-dist.zip
unzip -q bazel-6.3.2-dist.zip
chmod -R +w .
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/6.3.2/patch/bazel.patch | patch -p1
bash ./compile.sh
export PATH=$PATH:${SOURCE_ROOT}/bazel/output/
-
RHEL 8.x
sudo yum install -y gcc-toolset-12-gcc-c++ gcc-toolset-12-libstdc++-devel gcc-toolset-12-binutils-devel gcc-toolset-12-binutils-gold gcc-toolset-12-libatomic-devel rust-toolset python3.11 pkgconf-pkg-config #Set gcc 12 as default source /opt/rh/gcc-toolset-12/enable
-
Ubuntu 20.04
cd "$SOURCE_ROOT" URL=https://ftp.gnu.org/gnu/gcc/gcc-12.3.0/gcc-12.3.0.tar.gz curl -sSL $URL | tar xzf - || error "GCC 12.3.0" cd gcc-12.3.0 ./contrib/download_prerequisites mkdir objdir && cd objdir ../configure --enable-languages=c,c++ --prefix=${PREFIX} \ --enable-shared --enable-threads=posix \ --disable-multilib --disable-libmpx \ --with-system-zlib --with-long-double-128 --with-arch=zEC12 \ --disable-libphobos --disable-werror \ --build=s390x-linux-gnu --host=s390x-linux-gnu --target=s390x-linux-gnu make -j 8 bootstrap sudo make install export PATH=${PREFIX}/bin${PATH:+:${PATH}} LD_LIBRARY_PATH=${PREFIX}/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} LD_LIBRARY_PATH+=:${PREFIX}/lib LD_LIBRARY_PATH+=:/usr/lib64 export LD_LIBRARY_PATH LD_RUN_PATH=${PREFIX}/lib64${LD_RUN_PATH:+:${LD_RUN_PATH}} LD_RUN_PATH+=:${PREFIX}/lib LD_RUN_PATH+=:/usr/lib64 export LD_RUN_PATH
-
Ubuntu 24.04
sudo update-alternatives --remove-all gcc sudo update-alternatives --remove-all g++ sudo update-alternatives --remove-all gcov sudo update-alternatives --remove-all gcov-tool sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-13 12 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 12 sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 12 sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-13 12
-
RHEL (8.x, 9.x)
sudo yum install -y clang-14.0.6 export CC=clang export CXX=clang++
-
SLES 12 SP5
cd $SOURCE_ROOT URL=https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-14.0.6.tar.gz curl -sSL $URL | tar xzf - || error "Clang 14.0.6" cd llvm-project-llvmorg-14.0.6 mkdir build && cd build cmake -DLLVM_ENABLE_PROJECTS=clang -DCMAKE_C_COMPILER="/usr/bin/gcc" -DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_BUILD_TYPE="Release" -G "Unix Makefiles" ../llvm make clang -j8 clangbuild=${SOURCE_ROOT}/llvm-project-llvmorg-14.0.6/build export PATH=$clangbuild/bin:$PATH export LD_LIBRARY_PATH=$clangbuild/lib:$LD_LIBRARY_PATH cd $clangbuild/bin sudo ln -sf clang++ clang++-14 export CC=${SOURCE_ROOT}/llvm-project-llvmorg-14.0.6/build/bin/clang export CXX=${SOURCE_ROOT}/llvm-project-llvmorg-14.0.6/build/bin/clang++-14
-
SLES 15 SP5
sudo zypper install -y clang14 llvm14 bpftool libclang-cpp14 libunwind llvm14-gold libLLVM14 binutils-gold export CC=clang export CXX=clang++
-
Ubuntu 20.04
sudo apt-get install -y clang-12 export CC=clang-12 export CXX=clang++-12 sudo ln -sf /usr/bin/clang-12 /usr/bin/clang sudo ln -sf /usr/bin/clang++-12 /usr/bin/clang++
-
Ubuntu 22.04
cd $SOURCE_ROOT sudo apt-get update sudo apt install -y lsb-release wget software-properties-common gnupg wget https://apt.llvm.org/llvm.sh sed -i 's,add-apt-repository "${REPO_NAME}",add-apt-repository "${REPO_NAME}" -y,g' llvm.sh chmod +x llvm.sh sudo ./llvm.sh 14 rm ./llvm.sh export CC=clang-14 export CXX=clang++-14 sudo ln -sf /usr/bin/clang-14 /usr/bin/clang sudo ln -sf /usr/bin/clang++-14 /usr/bin/clang++
-
Ubuntu 24.04
sudo apt-get install -y clang-14 export CC=clang-14 export CXX=clang++-14 sudo ln -sf /usr/bin/clang-14 /usr/bin/clang sudo ln -sf /usr/bin/clang++-14 /usr/bin/clang++
cd $SOURCE_ROOT
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh /dev/stdin -y
export PATH="$HOME/.cargo/bin:$PATH"
sudo locale-gen en_US.UTF-8
export LC_ALL=C
unset LANGUAGE
export GOPATH=$SOURCE_ROOT
cd $GOPATH
GO_VERSION=1.21.5
wget -q https://golang.org/dl/go"${GO_VERSION}".linux-s390x.tar.gz
chmod ugo+r go"${GO_VERSION}".linux-s390x.tar.gz
sudo rm -rf /usr/local/go /usr/bin/go
sudo tar -C /usr/local -xzf go"${GO_VERSION}".linux-s390x.tar.gz
sudo ln -sf /usr/local/go/bin/go /usr/bin/
sudo ln -sf /usr/local/go/bin/gofmt /usr/bin/
go version
export PATH=$PATH:$GOPATH/bin
cd $SOURCE_ROOT
git clone -b v6.3.2 https://github.com/bazelbuild/buildtools.git
#Build buildifer
cd $SOURCE_ROOT/buildtools/buildifier
bazel build //buildifier
export BUILDIFIER_BIN=$GOPATH/bin/buildifier
#Build buildozer
cd $SOURCE_ROOT/buildtools/buildozer
bazel build //buildozer
export BUILDOZER_BIN=$GOPATH/bin/buildozer
cd $SOURCE_ROOT
git clone --depth 1 -b 0.10.1 https://github.com/bazelbuild/rules_foreign_cc.git
cd rules_foreign_cc/
curl -sSL $PATCH_URL/rules_foreign_cc.patch | git apply -
wget -O $SOURCE_ROOT/rules_foreign_cc/toolchains/pkgconfig-valgrind.patch $PATCH_URL/pkgconfig-valgrind.patch
-
Download Envoy and Apply Patches
cd $SOURCE_ROOT git clone --depth 1 -b v1.29.2 https://github.com/envoyproxy/envoy.git cd envoy
# Apply patch- curl -sSL $PATCH_URL/envoy_patch.diff | git apply - #Apply patch to update certificates- curl -sSL https://github.com/phlax/envoy/commit/c84d38dbc13982c899b9bedc290525938c92fd16.patch | git apply - # Patch for failing tests- curl -sSL $PATCH_URL/envoy-test.patch | git apply - # Copy patch files to envoy/bazel which will be applied to external packages while building envoy wget -O $SOURCE_ROOT/envoy/bazel/boringssl-s390x.patch $PATCH_URL/boringssl-s390x.patch wget -O $SOURCE_ROOT/envoy/bazel/cel-cpp-memory.patch $PATCH_URL/cel-cpp-memory.patch wget -O $SOURCE_ROOT/envoy/bazel/grpc-s390x.patch $PATCH_URL/grpc-s390x.patch wget -O $SOURCE_ROOT/envoy/bazel/foreign_cc/luajit-s390x.patch $PATCH_URL/luajit-s390x.patch wget -O $SOURCE_ROOT/envoy/bazel/quiche-s390x.patch $PATCH_URL/quiche-s390x.patch
- Only on SLES 12 SP5
#Apply distro specific patch curl -sSL $PATCH_URL/envoy-sl12.patch | git apply - wget -O $SOURCE_ROOT/envoy/bazel/io_uring.patch $PATCH_URL/io_uring.patch
-
Build Envoy
bazel build envoy -c opt --override_repository=rules_foreign_cc=${SOURCE_ROOT}/rules_foreign_cc --config=clangThe binary will be generated in
$SOURCE_ROOT/envoy/bazel-bin/source/exe/envoy-static. -
Verify the version of Envoy
$SOURCE_ROOT/envoy/bazel-bin/source/exe/envoy-static --versionOutput should be similar to:
bazel-bin/source/exe/envoy-static version: 2092d65bd4d476be8235ea541e5d25c096b513e6/1.29.2/Modified/RELEASE/BoringSSL
cd $SOURCE_ROOT/envoy
# Run tests-
bazel test //test/... -c opt --override_repository=rules_foreign_cc=${SOURCE_ROOT}/rules_foreign_cc --config=clang --keep_going
Notes:
-
Below mentioned test failures are observed on all distros on Intel and s390x:
-
//test/extensions/filters/listener/original_dst:original_dst_integration_test: To fix the test, follow official doc to set up the original_dst filter for a quick manual test. To run this test after applying this fix, the following command can be used.bazel test //test/extensions/filters/listener/original_dst:original_dst_integration_test -c opt --override_repository=rules_foreign_cc=${SOURCE_ROOT}/rules_foreign_cc --config=clang
-
//test/extensions/tracers/dynamic_ot:dynamic_opentracing_driver_impl_test: The test can pass by adding --linkopt=-lstdc++ to bazel as discussed here .
-
-
Below mentioned test failure is observed on all RHEL and SLES distros on Intel and s390x:
//test/config_test:example_configs_test: This test fails because the file "/etc/ssl/certs/ca-certificates.crt" only exists on debian like systems. -
Below mentioned test failures are observed on SLES 12 SP5 on Intel and s390x:
//test/common/common:packed_struct_test//test/common/http:header_utility_test//test/common/network:utility_test//test/extensions/transport_sockets/alts:alts_integration_test -
Some tests might timeout as they are being run with heap checker enabled in "normal" mode. Heap checker can be disabled using below test command:
bazel test //test/... -c opt --override_repository=rules_foreign_cc=${SOURCE_ROOT}/rules_foreign_cc --config=clang --keep_going --test_output=all --test_env=HEAPCHECK=
The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.