-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Amazon Linux:2 Docker- After install While using the sdk - Could NOT find crypto (missing: crypto_LIBRARY) #1910
Comments
Hello,
Best regards, |
Thanks for response I did have this statement in the Docker as specified on instructions here but don't have "openssl-static" |
We have updated our CRT dependency which brings quite some changes to our dependency tree. Thanks for bringing up the documentation page, we will follow up with updating it. |
Also noticed that docker file listed here in your repo is also not updated (https://github.com/aws/aws-sdk-cpp/blob/master/CI/docker-file/AmazonLinux2/Dockerfile) After the update it worked thank you |
|
I've got a project that builds the AWS SDK on Alpine 3.16.2 that started having an issue finding libcrypto while building the SDK precisely beginning with AWS SDK 1.9.235, which is right after fdf5a3c added cmake/Findcrypto.cmake to the AWS SDK project. Then beginning with AWS SDK 1.9.255, it became exactly this issue, where the SDK is built but then when used by a project it couldn't find libcrypto as above. That's right after e5cda3f (which also amends the erroneous commit 62c00f7 right before it). Interestingly, the Alpine system where this is occurring,
So, it looks like it is failing because it cannot find the static libraries if they are not installed. This is even though I'm building the SDK with Following the advice above, I did:
This brings in:
Now it works. The SDK builds, and the project builds. So, it works because the static library is installed, even though the project I'm building ends up linking with the So, perhaps it's a bug that the static library files must be installed even when only the shared library will be used. |
In anyone is interested, this is the dockerfile that I'm using to build the build environment which works with the latest AWS SDK: #
# Reference:
# - https://phoenixnap.com/kb/docker-image-size
# - https://docs.aws.amazon.com/lambda/latest/dg/foundation-arch.html
#
FROM alpine:3.16.2
ENV AWS_SDK_VERSION=1.9.328
ENV AWS_SDK_COMPONENTS="s3"
ENV VIPS_VERSION=8.13.0
LABEL \
aws-sdk="${AWS_SDK_VERSION} ${AWS_SDK_COMPONENTS}" \
lambda-runtime="latest" \
vips="${VIPS_VERSION}"
#
# Install the AWS SDK and lambda runtime library into /opt/aws
#
# Alpine packages:
#
# - `zip` is needed for the AWS Lambda Runtime packager to package the lambda together with its shared libraries
# for deployment
#
# - `libexecinfo-dev` is necessary for backtrace support, which is missing in the `musl` C library
# that Alpine linux ships with (and the lambda will link against)
#
# See: https://github.com/awslabs/aws-lambda-cpp/issues/124
#
# - `openssl-libs-static` seems unnecessary, but currently the AWS SDK won't let the project build unless the
# static objects are installed
#
# See: https://github.com/aws/aws-sdk-cpp/issues/1910
#
# CMake parameters:
#
# - https://github.com/aws/aws-sdk-cpp/blob/main/Docs/CMake_Parameters.md
WORKDIR /tmp/work
RUN apk add --progress --no-cache cmake make g++ git bash zip curl-dev zlib-dev libexecinfo-dev openssl-libs-static
#
# Install the AWS C++ SDK
#
RUN \
git clone --branch $AWS_SDK_VERSION --depth 1 --recurse-submodules https://github.com/aws/aws-sdk-cpp.git && \
cd aws-sdk-cpp && \
mkdir build && cd build && \
cmake .. \
-DCMAKE_BUILD_TYPE=Debug \
-DBUILD_ONLY=$AWS_SDK_COMPONENTS \
-DBUILD_SHARED_LIBS=OFF \
-DENABLE_TESTING=OFF \
-DCMAKE_INSTALL_PREFIX=/opt/aws && \
make && make install && \
cd ../../ && \
rm -rf aws-sdk-cpp
#
# Install the AWS Lambda Runtime C++ library
#
RUN \
git clone --depth 1 https://github.com/awslabs/aws-lambda-cpp.git && \
cd aws-lambda-cpp && \
mkdir build && cd build && \
cmake .. \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/aws && \
make && make install && \
cd ../../ && \
rm -rf aws-lambda-cpp
#
# Install libvips and its required and carefully selected optional dependencies
#
RUN apk add --progress --no-cache glib-dev expat-dev libpng-dev libwebp-dev
RUN \
wget https://github.com/libvips/libvips/releases/download/v$VIPS_VERSION/vips-$VIPS_VERSION.tar.gz -O vips.tar.gz && \
tar xvzf vips.tar.gz && rm -v vips.tar.gz && \
cd vips-* && \
./configure \
--without-rsvg \
--without-openslide \
--without-nsgit \
--without-ppm \
--without-analyze \
--without-radiance \
--without-pangocairo \
--without-fontconfig \
--without-libexif && \
make && make install && \
cd ../ && \
rm -rf vips-* It is used to build lambda functions that interact with S3 and do some processing on image files stored there. |
Hi Ildar, Thanks a lot for deep investigation of the linking issue.
Unfortunately, static libcrypto is required by one of our dependencies: AWS CPP SDK dependency tree is complicated, we use AWS CRT libraries which includes s2n and (optionally) aws-lc libraries. Thanks again for your dive deep into the linking issue. Best regards, |
I'm quite excited about #1888. I'll be watching the progress on that issue. Starting with AWS SDK 1.9.255, I'm able to build the SDK without
Or: set(crypto_LIBRARY "/usr/lib/libcrypto.so") |
I have an idea of how to maybe approach this nuanced situation. Here's what I did: diff --git a/cmake/Findcrypto.cmake b/cmake/Findcrypto.cmake
index 5425280d74..8b2014a718 100644
--- a/cmake/Findcrypto.cmake
+++ b/cmake/Findcrypto.cmake
@@ -55,7 +55,7 @@ else()
PATH_SUFFIXES build/crypto build lib64 lib
)
- if (NOT crypto_LIBRARY)
+ if (NOT EXISTS crypto_LIBRARY)
if (BUILD_SHARED_LIBS)
set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
else() This carries the benefit that However, the build crashes not while executing cmake/Findcrypto.cmake which is installed under
What I discovered is that there are actually three of them:
The other two aren't in this repository. They come from here and here during the AWS SDK installation process. These three files look like slightly different versions of essentially the same thing. So, I amended the first one in the source code and patched the other two post-install like this: --- Findcrypto-s2n-orig.cmake 2022-08-22 06:38:54.986125580 -0400
+++ Findcrypto-s2n.cmake 2022-08-22 06:55:21.691155692 -0400
@@ -70,10 +70,16 @@
${CMAKE_INSTALL_PREFIX}/lib
)
- if (BUILD_SHARED_LIBS)
- set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
- else()
- set(crypto_LIBRARY ${crypto_STATIC_LIBRARY})
+ if(NOT crypto_LIBRARY)
+ if (BUILD_SHARED_LIBS)
+ set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
+ else()
+ if (crypto_STATIC_LIBRARY)
+ set(crypto_LIBRARY ${crypto_STATIC_LIBRARY})
+ else()
+ set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
+ endif()
+ endif()
endif()
include(FindPackageHandleStandardArgs) --- Findcrypto-c-cal-orig.cmake 2022-08-22 06:38:54.986125580 -0400
+++ Findcrypto-c-cal.cmake 2022-08-22 06:54:58.318154978 -0400
@@ -70,10 +70,16 @@
${CMAKE_INSTALL_PREFIX}/lib
)
- if (BUILD_SHARED_LIBS)
- set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
- else()
- set(crypto_LIBRARY ${crypto_STATIC_LIBRARY})
+ if(NOT crypto_LIBRARY)
+ if (BUILD_SHARED_LIBS)
+ set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
+ else()
+ if (crypto_STATIC_LIBRARY)
+ set(crypto_LIBRARY ${crypto_STATIC_LIBRARY})
+ else()
+ set(crypto_LIBRARY ${crypto_SHARED_LIBRARY})
+ endif()
+ endif()
endif()
include(FindPackageHandleStandardArgs) Now the project builds as expected:
|
Describe the bug
While using the aws sdk in my code using Cmake, I get Could NOT find crypto (missing: crypto_LIBRARY) error since last 8 days, suddenly in Docker.
I dont get this error while building my code on Mac Os or Ubuntu Linux but see this error in Docker using Amazon Linux 2
Expected Behavior
cmake ../../
-- ###############################################
-- CMAKE_HOST_SYSTEM_NAME: Linux
-- ###############################################
LINUX OS IS = Ubuntu
-- Found AWS SDK for C++, Version: 1.9.192, Install Root:/usr/local, Platform Prefix:, Platform Dependent Libraries: pthread;crypto;ssl;z;curl
-- Components specified for AWSSDK: s3, application will be depending on libs: aws-cpp-sdk-s3;aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-core
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- LibCrypto Static Lib: /usr/lib/x86_64-linux-gnu/libcrypto.so
-- Found aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-s3
-- Found aws-cpp-sdk-s3
-- Configuring done
-- Generating done
-- Build files have been written to:
Current Behavior
Step 38/40 : RUN cmake ../../
---> Running in f6107dab4a33
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- ###############################################
-- CMAKE_HOST_SYSTEM_NAME: Linux
-- ###############################################
LINUX OS IS AMZON LINUX
-- Found AWS SDK for C++, Version: 1.9.238, Install Root:/usr/local, Platform Prefix:, Platform Dependent Libraries: pthread;crypto;ssl;z;curl
-- Components specified for AWSSDK: s3, application will be depending on libs: aws-cpp-sdk-s3;aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-core
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
CMake Error at /opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find crypto (missing: crypto_LIBRARY)
Call Stack (most recent call first):
/opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
/usr/local/lib64/s2n/cmake/modules/Findcrypto.cmake:67 (find_package_handle_standard_args)
/opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
/usr/local/lib64/s2n/cmake/s2n-config.cmake:9 (find_dependency)
/opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
/usr/local/lib64/aws-c-io/cmake/aws-c-io-config.cmake:4 (find_dependency)
/opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
/usr/local/lib64/aws-c-http/cmake/aws-c-http-config.cmake:3 (find_dependency)
/opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
/usr/local/lib64/aws-crt-cpp/cmake/aws-crt-cpp-config.cmake:3 (find_dependency)
/opt/cmake-3.13.3-Linux-x86_64/share/cmake-3.13/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
/usr/local/lib64/cmake/aws-cpp-sdk-core/aws-cpp-sdk-core-config.cmake:13 (find_dependency)
/usr/local/lib64/cmake/AWSSDK/AWSSDKConfig.cmake:307 (find_package)
CMakeLists.txt:55 (find_package)
Reproduction Steps
Docker FIle
FROM public.ecr.aws/amazonlinux/amazonlinux:2
RUN yum -y install unzip aws-cli
RUN amazon-linux-extras enable corretto8
RUN yum install java-1.8.0-amazon-corretto-devel -y
install boost
RUN yum install boost-devel -y
install dev tools complier etc
RUN yum groupinstall "Development Tools" -y
RUN curl https://cmake.org/files/v3.13/cmake-3.13.3-Linux-x86_64.tar.gz --output cmake-3.13.3-Linux-x86_64.tar.gz &&
tar -xvzf cmake-3.13.3-Linux-x86_64.tar.gz &&
mv cmake-3.13.3-Linux-x86_64 /opt &&
rm cmake-3.13.3-Linux-x86_64.tar.gz &&
ln -s /opt/cmake-3.13.3-Linux-x86_64/bin/cmake /usr/local/bin/cmake
#Install curl and openssl
RUN yum install curl-devel -y &&
yum install openssl-devel -y &&
yum install ninja-build -y
###################################################################
install AWS SDK s3 C++
###################################################################
WORKDIR /usr/
copy the aws sdk to Docker
RUN git clone --recurse-submodules https://github.com/aws/aws-sdk-cpp
change to this directory
WORKDIR /usr/aws-sdk-cpp/
RUN mkdir sdk_build
WORKDIR /usr/aws-sdk-cpp/sdk_build
run cmake to install its
#cmake ../aws-sdk-cpp/ -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/usr/local/include -DBUILD_ONLY="s3"
RUN cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/usr/local/include -DBUILD_ONLY="s3"
RUN make
RUN make install
The Aws sdk gets built nicely but when I use in my code
The following folder will be included
include_directories("${PROJECT_SOURCE_DIR}")
#Load required services/packages: This basic example uses S3.
find_package(AWSSDK REQUIRED COMPONENTS s3)
Possible Solution
No response
Additional Information/Context
No response
AWS CPP SDK version used
Latest
Compiler and Version used
-- The C compiler identification is GNU 7.3.1 -- The CXX compiler identification is GNU 7.3.1
Operating System and version
Amamzon Linux: latest
The text was updated successfully, but these errors were encountered: