Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NDK platform static libraries need to be updated #272

Closed
enh opened this issue Dec 24, 2016 · 24 comments
Closed

NDK platform static libraries need to be updated #272

enh opened this issue Dec 24, 2016 · 24 comments
Assignees
Milestone

Comments

@enh
Copy link
Contributor

enh commented Dec 24, 2016

looks like the libc.a files haven't changed since API level 21? (and before that, not since 9?)

/tmp/android-ndk-r14-beta1$ find platforms/ -name libc.a | xargs sha1sum | sort
303952da61b721445bd36783a65a12f252bb80c1  platforms/android-21/arch-mips64/usr/lib64/libc.a
303952da61b721445bd36783a65a12f252bb80c1  platforms/android-22/arch-mips64/usr/lib64/libc.a
303952da61b721445bd36783a65a12f252bb80c1  platforms/android-23/arch-mips64/usr/lib64/libc.a
303952da61b721445bd36783a65a12f252bb80c1  platforms/android-24/arch-mips64/usr/lib64/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-12/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-13/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-14/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-15/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-16/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-17/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-18/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-19/arch-arm/usr/lib/libc.a
4fb38c3f717236f53e26f25bc47cd1f4455c83c9  platforms/android-9/arch-arm/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-12/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-13/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-14/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-15/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-16/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-17/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-18/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-19/arch-x86/usr/lib/libc.a
53f7fd97112dc262479057d35de112afd001e4ed  platforms/android-9/arch-x86/usr/lib/libc.a
5981f99cea41a70401f6c87bd5d47916cea1d6ae  platforms/android-21/arch-arm/usr/lib/libc.a
5981f99cea41a70401f6c87bd5d47916cea1d6ae  platforms/android-22/arch-arm/usr/lib/libc.a
5981f99cea41a70401f6c87bd5d47916cea1d6ae  platforms/android-23/arch-arm/usr/lib/libc.a
5981f99cea41a70401f6c87bd5d47916cea1d6ae  platforms/android-24/arch-arm/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-21/arch-x86_64/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-21/arch-x86/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-22/arch-x86_64/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-22/arch-x86/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-23/arch-x86_64/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-23/arch-x86/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-24/arch-x86_64/usr/lib/libc.a
5a209620ce2d805cbe29bc3b5a122731ce930a1a  platforms/android-24/arch-x86/usr/lib/libc.a
5f17a5b10001698571ffe98e58fdfe709b7b79b2  platforms/android-21/arch-x86_64/usr/lib64/libc.a
5f17a5b10001698571ffe98e58fdfe709b7b79b2  platforms/android-22/arch-x86_64/usr/lib64/libc.a
5f17a5b10001698571ffe98e58fdfe709b7b79b2  platforms/android-23/arch-x86_64/usr/lib64/libc.a
5f17a5b10001698571ffe98e58fdfe709b7b79b2  platforms/android-24/arch-x86_64/usr/lib64/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-12/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-13/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-14/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-15/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-16/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-17/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-18/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-19/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-21/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-22/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-23/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-24/arch-mips/usr/libr6/libc.a
72c485139dd3f3ed98654050cddce0f37e8f2f0d  platforms/android-9/arch-mips/usr/libr6/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-12/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-13/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-14/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-15/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-16/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-17/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-18/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-19/arch-mips/usr/lib/libc.a
90bc2a46746e5f76bec141b6e24e4c8ef0a5ad10  platforms/android-9/arch-mips/usr/lib/libc.a
a94b8664771de0db1c0bbe1da4f54158f3562be5  platforms/android-21/arch-arm64/usr/lib/libc.a
a94b8664771de0db1c0bbe1da4f54158f3562be5  platforms/android-22/arch-arm64/usr/lib/libc.a
a94b8664771de0db1c0bbe1da4f54158f3562be5  platforms/android-23/arch-arm64/usr/lib/libc.a
a94b8664771de0db1c0bbe1da4f54158f3562be5  platforms/android-24/arch-arm64/usr/lib/libc.a
fafb113df91ed07126abfe9fbfa8cfbe0999e396  platforms/android-21/arch-mips/usr/lib/libc.a
fafb113df91ed07126abfe9fbfa8cfbe0999e396  platforms/android-22/arch-mips/usr/lib/libc.a
fafb113df91ed07126abfe9fbfa8cfbe0999e396  platforms/android-23/arch-mips/usr/lib/libc.a
fafb113df91ed07126abfe9fbfa8cfbe0999e396  platforms/android-24/arch-mips/usr/lib/libc.a
/tmp/android-ndk-r14-beta1$ 

hit while trying to build toybox with NDK r14beta1 using --unified-headers. we'd get references to stuff (even including stderr) that isn't in the libc.a file.

@yan12125
Copy link

yan12125 commented Dec 25, 2016

we'd get references to stuff (even including stderr) that isn't in the libc.a file.

Did you specify -D__ANDROID_API__=XX everywhere?

From https://android.googlesource.com/platform/ndk.git/+/master/docs/UnifiedHeaders.md:

Pass -D__ANDROID_API__=$API when compiling. This define used to be provided
by <android/api-level.h>, but with only one set of headers this is no
longer possible. In the future we will look in to adding -mandroid-version
or similar to Clang so this is automatic.

I've got the same error before. The reason is that in stdio.h, stderr have different names before and after API 23:

#if __ANDROID_API__ >= 23
extern FILE* stdin __INTRODUCED_IN(23);
extern FILE* stdout __INTRODUCED_IN(23);
extern FILE* stderr __INTRODUCED_IN(23);

/* C99 and earlier plus current C++ standards say these must be macros. */
#define stdin stdin
#define stdout stdout
#define stderr stderr
#else
/* Before M the actual symbols for stdin and friends had different names. */
extern FILE __sF[] __REMOVED_IN(23);

#define stdin (&__sF[0])
#define stdout (&__sF[1])
#define stderr (&__sF[2])
#endif

@enh
Copy link
Contributor Author

enh commented Dec 25, 2016

the point is, both __sF and stdin/stdout/stderr should be in a >= 23 libc.a, but they're not. the "new" libc.a files are just copies of the 21 libc.a files, as the hashes show. (it would perhaps have been clearer if i'd also shown an nm | grep or something too.)

@yan12125
Copy link

I got it. I was using dynamic linking and libc.so have stdin/stdout/stderr symbols. Now I got linker failures for static linking, too.

@johannkoenig
Copy link

Was this resolved? I do not have linking issues with r14b, but r15c is failing to link stderr/stdout. Using make_standalone_toolchain.py --arch arm

git clone https://chromium.googlesource.com/webm/libvpx
mkdir build

create toolchain, add to path

CROSS=arm-linux-androideabi- ../libvpx/configure --target=armv7-android-gcc

ivfdec.c.o:ivfdec.c:function file_is_ivf: error: undefined reference to 'stderr'
(among others)

@enh
Copy link
Contributor Author

enh commented Sep 8, 2017

no, no-one is working on updating the libc.a files yet.

if you're linking statically, you'll want to target API < 23. iirc API 21 was the last libc.a update.

@johannkoenig
Copy link

make_standalone_toolchains.py defaults to API 14 for arm (21 for arm64):
WARNING:main:Defaulting to target API 14 (minimum supported target for arm)

Using CC=clang CXX=clang++ the build completes successfully.

Adding the api explicitly to the gcc build does work:
CFLAGS=-D__ANDROID_API__=14 LDFLAGS=-D__ANDROID_API__=14 CXXFLAGS=-D__ANDROID_API__=14

so maybe the default API is not treated the same way between r14b and r15c?

@yan12125
Copy link

yan12125 commented Sep 9, 2017

Might be related to a known issue [1]:

  • Standalone toolchains using GCC are not supported out of the box. To use GCC, pass -D__ANDROID_API__=$API when compiling. Note: this is not something we will be fixing.

[1] https://android.googlesource.com/platform/ndk.git/+/master/docs/UnifiedHeaders.md

@nargus
Copy link

nargus commented Nov 13, 2017

If the libc.a won't be updated to match the unified header of stdio.h in r16, can we not simply "fix" the stdio.h so it define them correctly for the library?

Otherwise, we'd have to either hack stdio.h, re-define the macros manually ourselves, or be forced to link against libc.so instead.

@enh
Copy link
Contributor Author

enh commented Nov 13, 2017

i think you should just switch to clang now. (we're removing GCC in r18 anyway.)

@DanAlbert
Copy link
Member

What does clang have to do with this?

@nargus: no, we can't fix this for static libraries without breaking it for shared libraries, which are the average case by a very wide margin.

@enh
Copy link
Contributor Author

enh commented Nov 13, 2017

(there are two bugs on this bug... it's half forked into "i'm getting the wrong results because i'm using GCC and it doesn't define __ANDROID_API__". it's not clear whether nargus has a real problem or it just piling on the GCC problem.)

@DanAlbert
Copy link
Member

Ah, I see. Yeah, the GCC problem is not something we have any intention of fixing. It's been marked as such in the changelog since r14.

@nargus
Copy link

nargus commented Nov 13, 2017

I'm actually already on Clang and have been trying to get my project compiled on the new r16 (beta-2). It's now almost working, excepted for the stdin/stdout linker errors mentioned above.

@Adam-VisualVocal
Copy link

Same here. I'm trying to compile libjpeg-turbo for armeabi-v7a and I also get the same type of "undefined reference to 'stderr'" errors. I'm using clang.

@yan12125
Copy link

Looks like it's not completely fixed for clang yet: https://android-review.googlesource.com/c/platform/bionic/+/522595

@DanAlbert
Copy link
Member

That's why the bug is still open. Targeting anything newer than android-21 for a static executable is not expected to work regardless of your compiler.

@nargus
Copy link

nargus commented Nov 16, 2017

@Adam-VisualVocal I managed to "fixed" the issue by using --include in my cmake flags with my custom header that undef & define the macros back to its pre-API 23. A bit hacky, but works just fine :)

@DanAlbert
Copy link
Member

Why not just target android-21?

@nargus
Copy link

nargus commented Nov 16, 2017

Unfortunately, we actually need 24 :)

@DanAlbert
Copy link
Member

If you're statically linking libc.a, you're not getting it. That's the root cause of this bug.

nickdiego added a commit to nickdiego/openssl-tests-android that referenced this issue Nov 30, 2017
- Compiling and linking ok
- Supporting only 'armeabi' for now
- Forcing APP_PLATFORM=android-23 to avoid NDK Bugs:
  android/ndk#272
  android/ndk#410
@DanAlbert DanAlbert added this to the r18 milestone Jan 29, 2018
@DanAlbert
Copy link
Member

This may end up happening for 17 beta 2, but I suspect this is more likely for r18. There's some work that needs to be done to make sure libandroid_support isn't used with libc.a since libc.a already contains everything in libandroid_support.a and will lead to multiple symbol definition errors. Unfortunately given the number of build systems we support I think this is likely to be a fair amount of work.

@alexcohn
Copy link

@DanAlbert I believe you don't need all that many rebuilds of libc.a. You drop armeabi and both MIPS for r17, don't you? Then, you don't really care about the versions below API 21: whoever needs these can use the android-14 binary (they have already found the workarounds, anyway; unified headers hardly worsen the situation here).

@DanAlbert
Copy link
Member

Our plan is to have only one libc.a for each arch going forward (always the latest). So far we haven't been able to come up with any reasons to use an old library.

That still leaves the problem of not being able to link when using libandroid_support.

@DanAlbert
Copy link
Member

Fixed in r17.

@DanAlbert DanAlbert modified the milestones: r18, r17 Feb 14, 2018
miodragdinic pushed a commit to MIPS/external-libcxx that referenced this issue Apr 17, 2018
If we're targeting a new enough platform version, we don't actually
need to cover any gaps in libc for libc++ support. In those cases,
save size in the APK by avoiding libandroid_support.

This is also a requirement for static executables, since using
libandroid_support with a modern libc.a will result in multiple symbol
definition errors.

Test: ndk/checkbuild.py && ndk/run_tests.py
Bug: android/ndk#272
Change-Id: I4cb76272dc7271a02cba46d3cc20a369a16ff39d
(cherry picked from commit db08a420b9b812321691180e97da1a3facc7391e)
miodragdinic pushed a commit to MIPS/ndk that referenced this issue Apr 17, 2018
We now have some static libraries from the platform rather than the
old prebuilts in development/ndk. Install those when building a
standalone standalone toolchain.

Test: ./run_tests.py --filter standalone_toolchain*
Test: Build static library, ls in sysroot/usr/lib
Bug: android/ndk#272
Change-Id: I949cadbcd3ea50b58f46fb515ba8ca0684dbce31
miodragdinic pushed a commit to MIPS/ndk that referenced this issue Apr 17, 2018
If we're targeting a new enough platform version, we don't actually
need to cover any gaps in libc for libc++ support. In those cases,
save size in the APK by avoiding libandroid_support.

This is also a requirement for static executables, since using
libandroid_support with a modern libc.a will result in multiple symbol
definition errors.

Test: ./checkbuild.py && ./run_tests.py
Bug: android/ndk#272
Change-Id: Iec548d7a75cc407d091b1a2bc632df68756c717d
(cherry picked from commit cf5442c13a4cd0fe8b01e27de3850727007988f0)
miodragdinic pushed a commit to MIPS/ndk that referenced this issue Apr 17, 2018
We now have up to date static libraries coming from the platform as
prebuilts. They exist in $NDK/sysroot/usr/lib/$TRIPLE, but for the
time being we'll also install them to the old platforms directory
while we transition the build systems to the new location.

Test: ./checkbuild.py && ./run_tests.py
Bug: android/ndk#272
Change-Id: I3159ee32d3d3b02bbe44d011494a018732391a49
(cherry picked from commit 65290c0584fcfe51c1e7cae30abf26be3e990a77)
miodragdinic pushed a commit to MIPS/prebuilts-ndk that referenced this issue Apr 17, 2018
Test: ndk/checkbuild.py && ndk/validate.py
Bug: android/ndk#272
Bug: android/ndk#294
Change-Id: Iea240e8b2241576121db1acf3a1bd9cfe3426677
neolit123 added a commit to neolit123/subsurface that referenced this issue Apr 29, 2018
Attempt to fix "undefined reference to 'stdout'" by
adding a __ANDROID_API__ definition equal to the Android
API version for the toolchain.

ref:
android/ndk#272 (comment)

Signed-off-by: Lubomir I. Ivanov <[email protected]>
neolit123 added a commit to neolit123/subsurface that referenced this issue Apr 29, 2018
Attempt to fix "undefined reference to 'stdout'" by
adding a __ANDROID_API__ definition equal to the Android
API version for the toolchain.

ref:
android/ndk#272 (comment)

Signed-off-by: Lubomir I. Ivanov <[email protected]>
neolit123 added a commit to neolit123/subsurface that referenced this issue Apr 29, 2018
GCC is going to be removed in newer NDKs.

GCC has issues according to users:
android/ndk#272 (comment)

Signed-off-by: Lubomir I. Ivanov <[email protected]>
vladimiroltean added a commit to vladimiroltean/termux-packages that referenced this issue May 3, 2018
* Upgrading is needed in order to fix the stupid situation we're in
  with docker-tini:
  * Needs to have API >= 23 for sigtimedwait
    (termux#899)
  * Needs to have API <= 21 for static linking
    (android/ndk#272)
* The latter of the 2 above restrictions was removed in NDK r17, so
  that's why we're upgrading.
* The other changes (removed headers and patches) are according to
  Mr. Fornwall himself:
  termux#1905

Signed-off-by: Vladimir Oltean <[email protected]>
vladimiroltean added a commit to vladimiroltean/termux-packages that referenced this issue May 3, 2018
* Upgrading is needed in order to fix the stupid situation we're in
  with docker-tini:
  * Needs to have API >= 23 for sigtimedwait
    (termux#899)
  * Needs to have API <= 21 for static linking
    (android/ndk#272)
* The latter of the 2 above restrictions was removed in NDK r17, so
  that's why we're upgrading.
* The other changes (removed headers and patches) are according to
  Mr. Fornwall himself:
  termux#1905

Signed-off-by: Vladimir Oltean <[email protected]>
vladimiroltean added a commit to vladimiroltean/termux-packages that referenced this issue May 3, 2018
* Upgrading is needed in order to fix the stupid situation we're in
  with docker-tini:
  * Needs to have API >= 23 for sigtimedwait
    (termux#899)
  * Needs to have API <= 21 for static linking
    (android/ndk#272)
* The latter of the 2 above restrictions was removed in NDK r17, so
  that's why we're upgrading.
* The other changes (removed headers and patches) are according to
  Mr. Fornwall himself:
  termux#1905

Signed-off-by: Vladimir Oltean <[email protected]>
tito added a commit to kivy/python-for-android that referenced this issue Jan 8, 2019
… having __sF symbol not found. It make it work for API 21 (tested on Android 5.1)

Maybe android/ndk#272.

But adding -lc is enough to have numpy not complaining about __sF missing anymore (stdout/stderr/stdin)
chenyt9 pushed a commit to MotorolaMobilityLLC/external-zlib that referenced this issue Jul 1, 2022
Test: build/soong/scripts/build-ndk-prebuilts.sh
Bug: android/ndk#272
Change-Id: Icd3b737b443afa18a87558447808dceb2a401036
sepehrst pushed a commit to spsforks/android-bionic-libc that referenced this issue Apr 22, 2024
Test: build/soong/scripts/build-ndk-prebuilts.sh
Bug: android/ndk#272
Change-Id: Ibb21d799fdcf230ed45db0b809e20dd86d2d8e52
sepehrst pushed a commit to spsforks/android-bionic-libc that referenced this issue Apr 22, 2024
Test: build/soong/scripts/build-ndk-prebuilts.sh
Bug: android/ndk#272
Change-Id: Ibb21d799fdcf230ed45db0b809e20dd86d2d8e52
sepehrst pushed a commit to spsforks/android-bionic-libc that referenced this issue Apr 22, 2024
Test: build/soong/scripts/build-ndk-prebuilts.sh
Bug: android/ndk#272
Change-Id: Ibb21d799fdcf230ed45db0b809e20dd86d2d8e52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants