[Discussion]: The problem in TUR on Device

[licy183]

English Version

PS: English is not my native language, there may be some expression deviations. Please feel free to point out if there are any deviations in expression. Thanks!

Problem Description

The original intention of the TUR on Device repository, is to use termux-docker to provide an environment similar to running Termux on Android phones, install a set of compilers, and use the same build system as termux-packages to compile some packages. It's just like to compile some packages on device. If you use the x86 environment provided by Github Action, you can simulate running i686 and x86_64 environments. There is an issue if running the arm environment using qemu as a translation layer on x86 machines (termux/termux-docker#28), but this can be solved by using an arm server. That is exactly what the TUR on Device repository does now.

When compiling classic-flang, I found out a new problem. The libc.so in the termux-docker environment is taken from Android 9, __ANDROID__API__ is 28 (see termux/termux-docker#16 (comment)). Currently, termux packages defines __ANDROID__API__ to 24, the system version corresponds to Android 7. Some functions, such as catopen/catgets/catclose from libc, clog/clogf/cpow/cpowf from libm, are implemented in Android 9, but not in Android 7. When compiling these packages in an Android 9 environment, some functions will be automatically linked to libc, and running these executables on lower versions of Android devices will result in errors.

Possible Solutions

1. Switch termux-docker to Android 7

This is the way I initially tested, but libc of Android 7 seems to have some issues. The environment works fine onx86_64/aarch64 architecture, but doesn't work on i686/x86 architecture. The easiest way to verify is to run apt install glib after building the image and running the container, and you can find that the installation failed.

2.Provide some shared library, just like libandroid-support

This is what I've tried so far. I also implement some shared libraries in TUR repo like libandroid-mathlib (providing clog/clogf/cpow/cpowf), but this means that we should check every package added to TUR on Device manually to ensure they work fine on Android 7 devices. If all packages are tested in this way, the workload will be unimaginable.

3. Remove/Localize some symbols from libc.so

This is what I've been trying to do these days, but due to my lack of personal skills, I only know that patchelf --clear-symbol-version can be used to remove some non-default versioned symbols (such as a@LIBC_O, the symbols with default version like a@@LIBC_O cannot be removed temporarily, see NixOS/patchelf#252, and objcopy/llvm-objcopy --localize-symbol can localize some function symbols, but it doesn't do much sence because objcopy --localize-symbol doesn't change the visibility of versioned symbols.

4. Use Android Emulator

This is not very recommended. It uses qemu as the underlying layer, and it is supposed to run much more slower in Github Action (without KVM acceleration).

中文版

附言：英语不是我的母语，可能存在一些表达偏差，因此，我用中文写出来一份相同的文档，如果有表达偏差请随时指出，谢谢！

问题描述

TUR on Device 仓库的设计初衷是，使用 termux-docker 来提供一个类似在 Android 手机上运行 Termux 的环境，安装 clang 等一系列编译器，运用和 termux-packages 一样的编译系统来实现一些包的编译。如果使用 Github Action 提供的 x86 环境，可以模拟运行 i686 和 x86_64 的环境。在 x86 架构的电脑上，使用 qemu 作为兼容层运行 arm 环境存在问题 (https://github.com/termux/termux-docker/issues/28)，但可以通过使用 arm 架构的服务器来规避这个问题。这正是现在 TUR on Device 仓库的做法。

在编译 classic-flang 时，我遇到了新的问题。termux-docker 环境中的 libc.so 版本取自 Android 9 ，__ANDROID__API__ 为 28 (参见 https://github.com/termux/termux-docker/pull/16#issuecomment-959925937)，目前，Termux 中软件包的 __ANDROID__API__ 是 24，系统版本对应为 Android 7，一些函数，如来自 libc 的 catopen/catgets/catclose，来自 libm 的 clog/clogf/cpow/cpowf，在 Android 9 的 libc 中已经实现，但在 Android 7 的 libc 中并没有实现。当在 Android 9 的环境中去编译这些包，一些函数将会被自动连接到 libc，在低版本的 Android 设备中运行这些可执行文件，那么将会报错。

可能的解决方式

我觉得有以下四种解决方式，如果有别的未提到的，请随时指出。

1. 将 termux-docker 切换到 Android 7

这是我最一开始所测试的方式，但是 Android 7 的 libc 似乎存在一些问题， x86_64/aarch64 架构可以正常运行，i686/x86 架构并不能正常运行。最简单的验证方式为构建镜像运行容器后，运行 apt install glib，可以发现安装失败。

2. 提供一些共享库，就像 libandroid-support 一样

这是我目前尝试的做法，也在 TUR 中实现了一些共享库，如 libandroid-mathlib (提供了 clog/clogf/cpow/cpowf)，但是这对于每一个添加到 TUR on Device 的包都需要手动的去 Android 7 的设备上测试，来检测能否正常运行，如果所有的包都进行这样的测试，那么工作量将不堪设想。

3. 从 libc.so 中移除/不可见部分符号

这是我这几天在尝试做的事，但由于个人水平不足，目前仅知道可以使用 patchelf --clear-symbol-version 移除非默认版本的符号 (即形如 a@LIBC_O 的符号，形如 a@@LIBC_O 的默认版本的符号暂时不能移除，参见https://github.com/NixOS/patchelf/issues/252)，使用 objcopy/llvm-objcopy --localize-symbol 可以使函数符号不可见，但是这并没有太大的作用，因为 objcopy --localize-symbol 并不能改变带版本的符号。

4. 使用 Android 模拟器

这并不是十分的推荐，因为 Android 模拟器在本地运行就很慢，它使用 qemu 作为底层，估计在 Github Action 中运行会更慢 (如果没有 KVM 加速的话)。

CC: @truboxl @2096779623 @kcubeterm @T-Dynamos

[2096779623]

I think the third point is better.

我觉得第三点比较好

[T-Dynamos]

@licy183 Can this be used?https://github.com/dockcross/dockcross