Cross compiled OpenSSL and Curl android dependencies are not "installed"

[dschmitt007]

What platform/OS are you using?

OSX 10.13.6 (and Ubuntu 16.04)

What compiler are you using? what version?

Clang 6.0.2 via Android NDK r17c
(with some modifications to pass the right args & guess the version correctly)

What's your CMake arguments?

export BUILD_ABI=arm64-v8a
export EXTRA_AWS_OPTIONS=" -DTARGET_ARCH=ANDROID -DANDROID_ABI=$BUILD_ABI "
cmake
-DCMAKE_INSTALL_PREFIX=$INSTALL_DIR
-DCMAKE_RULE_MESSAGES:BOOL=OFF
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
-DBUILD_SHARED_LIBS:BOOL=OFF
-DFORCE_SHARED_CRT:BOOL=OFF
$EXTRA_AWS_OPTIONS
-DBUILD_ONLY='logs;core;iam;access-management;identity-management;cognito-identity;cognito-idp;s3;kinesis;sts'
$PROJECTHOME/aws-sdk-cpp "

Can you provide a TRACE level log? (sanitize any sensitive information)

After taring up the install directory, when moving to another machine linking against the installed files fails with:

/opt/work/test/../aws-sdk-cpp/lib/libaws-cpp-sdk-core.a(CryptoImpl.cpp.o): In function Aws::Utils::Crypto::OpenSSL::init_static_state()': CryptoImpl.cpp:(.text._ZN3Aws5Utils6Crypto7OpenSSL17init_static_stateEv+0xc): undefined reference to ERR_load_CRYPTO_strings'
CryptoImpl.cpp:(.text._ZN3Aws5Utils6Crypto7OpenSSL17init_static_stateEv+0x10): undefined reference to `OPENSSL_add_all_algorithms_noconf'

And a bunch more, and some curl references.

The root cause appears to be that the libcurl.a and libssl.a live in the build directory and make install doesn't put the dependencies where aws can find them.

It would be nice if make install could install the special built components required (they exist in the build directory, but would prefer not to archive that.)
aws-sdk-cpp/external/curl/lib/libcurl.a
aws-sdk-cpp/external/openssl/lib/libssl.a
aws-sdk-cpp/external/openssl/lib/libcrypto.a
aws-sdk-cpp/external/zlib/lib/libz.a

[phantom-j]

Hi @dschmitt007

Have you figured it out?

[dschmitt007]

I have a horrid hack (I tar up the results from build and dump them into install, and then have ugly ugly CMakeLists.txt files that have to have all the aws transitive dependencies listed.)

[phantom-j]

hi @dschmitt007
does it working?
explain the steps to do?

[jkoutavas]

@dschmitt007, yes. Please provide details.

[dschmitt007]

So, the horrid hack looks like a shell script for calling CMake with

    export AWS_CMAKE_COMMAND=" \
    cmake \
        -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
        -DCMAKE_RULE_MESSAGES:BOOL=OFF \
        -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
        -DBUILD_SHARED_LIBS:BOOL=OFF \
        -DFORCE_SHARED_CRT:BOOL=OFF \
        $EXTRA_AWS_OPTIONS \
        -DBUILD_ONLY='logs;core;iam;access-management;identity-management;cognito-identity;cognito-idp;s3;kinesis;sts' \
        $PROJECTHOME/aws-sdk-cpp "
    $AWS_CMAKE_COMMAND || $AWS_CMAKE_COMMAND
    # do it twice in case stand alone tool chain needs to be created for android...
    make_verbose
    make_install
    # HACK because aws build/install is borked
    tar cf - external | (cd $INSTALL_DIR ; tar xf -)

And then adding the -L parts to stuff that uses it. (The EXTRA_AWS_OPTIONS are just

    export EXTRA_AWS_OPTIONS=" -DTARGET_ARCH=ANDROID -DANDROID_ABI=$BUILD_ABI "

[jkoutavas]

@dschmitt007 Cool. That's a rather straightforward work-around.

Maybe you can help me out on a related topic... what I'm running up against is trying to use the aws-sdk-cpp stuff from an Android Studio Native C++ project. I get link errors from the native-lib link. It's not finding symbols in the external lib .a files. Here's what my cmake looks like:

# For more information about using CMake with Android Studio, read the
# documentation: https://d.android.com/studio/projects/add-native-code.html

# Sets the minimum version of CMake required to build the native library.

cmake_minimum_required(VERSION 3.4.1)

# Creates and names a library, sets it as either STATIC
# or SHARED, and provides the relative paths to its source code.
# You can define multiple libraries, and CMake builds them for you.
# Gradle automatically packages shared libraries with your APK.

add_library( # Sets the name of the library.
        native-lib

        # Sets the library as a shared library.
        SHARED

        # Provides a relative path to your source file(s).
        native-lib.cpp)

include_directories( AFTER
    "../../../../../install/ANDROID/include"
)
add_library(aws-cpp-sdk-core STATIC IMPORTED)
set_target_properties(aws-cpp-sdk-core PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/../../../../../install/ANDROID/lib/x86-api-21/libaws-cpp-sdk-core.a)

add_library(curl STATIC IMPORTED)
set_target_properties(curl PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/../../../../../build/ANDROID/external/curl/lib/libcurl.a)

add_library(crypto STATIC IMPORTED)
set_target_properties(crypto PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/../../../../../build/ANDROID/external/openssl/lib/libcrypto.a)

add_library(openssl STATIC IMPORTED)
set_target_properties(openssl PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/../../../../../build/ANDROID/external/openssl/lib/libssl.a)

add_library(zlib STATIC IMPORTED)
set_target_properties(zlib PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/../../../../../build/ANDROID/external/zlib/lib/libz.a)

# Searches for a specified prebuilt library and stores the path as a
# variable. Because CMake includes system libraries in the search path by
# default, you only need to specify the name of the public NDK library
# you want to add. CMake verifies that the library exists before
# completing its build.

find_library( # Sets the name of the path variable.
        log-lib

        # Specifies the name of the NDK library that
        # you want CMake to locate.
        log)

# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in this
# build script, prebuilt third-party libraries, or system libraries.

target_link_libraries( # Specifies the target library.
        native-lib
        # Links the target library to the log library
        # included in the NDK.
        ${log-lib}
        zlib
        curl
        crypto
        openssl
        aws-cpp-sdk-core
)

I suspect it's because native-lib is SHARED and the dependencies are STATIC? If I build aws-cpp-sdk-core as a shared lib, and change its add_library() to SHARED, aws-sdk-core.so isn't found at app run time. I guess the .so needs to be copied to somewhere?

I realize this issue isn't the ideal place to raise this question. But, I'll start here. :-) Thank you for any insights you are willing to share. (It goes without saying I'm just coming up the learning curve on using aws-cpp-sdk in an Android app.)

[dschmitt007]

I'd vote stackoverflow would be a place to refine views on shared/static libs and the NDK in general.

My short view is "Android doesn't do shared at the os level for versioning, just dynamic load" so I make all sub-projects static, link them into one big shared .so and drop it in my apk with the required C++ stdlib .so. The exception to that would be if you are integrating AARs into the same app that do native and might share the .so, but it would require coordination because the java side has to load it and the stdlib needs to be the same.

If you do want to carry a bunch of .so files around, make sure some java does the System.loadLibrary for each and making sure they get pulled into all the abi directories for the build.
I'm not sure what magic you need to fix that target_link_libraries to do shared stuff.

[jkoutavas]

Thanks, @dschmitt007.

I went with a static linking of aws-cpp-sdk-core, but am now running into these linker errors:

Build command failed.
Error while executing process /Users/jay/Library/Android/sdk/cmake/3.6.4111459/bin/cmake with arguments {--build /Users/jay/dev/testbuild/app/.externalNativeBuild/cmake/debug/x86 --target native-lib}
[1/2] Building CXX object CMakeFiles/native-lib.dir/native-lib.cpp.o
[2/2] Linking CXX shared library /Users/jay/dev/testbuild/app/build/intermediates/cmake/debug/obj/x86/libnative-lib.so
FAILED: : && /Users/jay/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++  --target=i686-none-linux-android16 --gcc-toolchain=/Users/jay/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/Users/jay/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++  -O0 -fno-limit-debug-info  -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libnative-lib.so -o /Users/jay/dev/testbuild/app/build/intermediates/cmake/debug/obj/x86/libnative-lib.so CMakeFiles/native-lib.dir/native-lib.cpp.o  -llog /Users/jay/dev/testbuild/app/src/main/cpp/../../../../../install/ANDROID/lib/x86-api-21/libaws-cpp-sdk-core.a /Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/curl/lib/libcurl.a /Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/openssl/lib/libssl.a /Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/openssl/lib/libcrypto.a /Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/zlib/lib/libz.a -latomic -lm && :
/Users/jay/dev/testbuild/app/src/main/cpp/../../../../../install/ANDROID/lib/x86-api-21/libaws-cpp-sdk-core.a(HttpClientFactory.cpp.o):/Users/jay/Library/aws-sdk-cpp/aws-cpp-sdk-core/source/http/HttpClientFactory.cpp:function Aws::Http::DefaultHttpClientFactory::InitStaticState(): error: undefined reference to 'signal'
/Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/openssl/lib/libcrypto.a(ui_openssl.c.o):/Users/jay/dev/testbuild/build/ANDROID/openssl-src/crypto/ui/ui_openssl.c:function open_console: error: undefined reference to 'tcgetattr'
/Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/openssl/lib/libcrypto.a(ui_openssl.c.o):/Users/jay/dev/testbuild/build/ANDROID/openssl-src/crypto/ui/ui_openssl.c:function pushsig: error: undefined reference to 'signal'
/Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/openssl/lib/libcrypto.a(ui_openssl.c.o):/Users/jay/dev/testbuild/build/ANDROID/openssl-src/crypto/ui/ui_openssl.c:function noecho_console: error: undefined reference to 'tcsetattr'
/Users/jay/dev/testbuild/app/src/main/cpp/../../../../../build/ANDROID/external/openssl/lib/libcrypto.a(ui_openssl.c.o):/Users/jay/dev/testbuild/build/ANDROID/openssl-src/crypto/ui/ui_openssl.c:function echo_console: error: undefined reference to 'tcsetattr'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

I suspect this is because aws-sdk-cpp for Android is dependent on NDK 12b, which at this time is quite dated. If I move the Android API version back to something from that time, the rest of my app has issues, expecting newer API features. It looks like I'm stuck. Am I?

OpenSSL was built as part of the aws-sdk-cpp build itself:
cmake -DBUILD_ONLY="cognito-identity;cognito-idp" -DSIMPLE_INSTALL=OFF -DMINIMIZE_SIZE=ON -DTARGET_ARCH=ANDROID -DCMAKE_INSTALL_PREFIX=/Users/jay/dev/testbuild/install/ANDROID -DCMAKE_PREFIX_PATH=/Users/jay/dev/testbuild/install/ANDROID -DBUILD_SHARED_LIBS=OFF -DNDK_DIR=/Users/jay/Library/Android/android-ndk-r12b -DENABLE_TESTING=OFF -DANDROID_ABI=x86 /Users/jay/Library/aws-sdk-cpp

[dschmitt007]

I'm not sure how to help. Maybe run these issues down:

https://stackoverflow.com/questions/38192458/linking-openssl-on-android-and-undefined-references

We hacked the heck out of the android tools finding cmake stuff and have it working with r18b so I can't help much.

[jkoutavas]

Thanks, I did look at that stackoverflow article before posting my last entry on this thread. What I gained from it was I'd have to munge-around with how OpenSSL got built by the aws-sdk-cpp cmake (and thus why I made a point of stating "OpenSSL was built as part of the aws-sdk-cpp build itself".)

I guess I'm headed towards either hacking aws-sdk-cpp cmake or waiting for the exciting news that the aws-sdk-cpp team has moved to a newer NDK version.

@wps132230, do you have any words of encouragement? :-)

[jmklix]

#1888