ARROW-11905: [C++] Fix SIMD detection on macOS

[pitrou]

No description provided.

[pitrou]

@jonkeane @uwe I don't know if you can easily test this on a x86 Mac.

[jonkeane]

👀

[github-actions]

https://issues.apache.org/jira/browse/ARROW-11905

[jonkeane]

It works! Thanks

This is with the R code from #9641, but c++ from this branch:

jkeane@het r % R -e "arrow:::runtime_info()"
> arrow:::runtime_info()
[1] "avx" "avx"
> 
jkeane@het r % ARROW_USER_SIMD_LEVEL=sse4_2 R -e "arrow:::runtime_info()"
> arrow:::runtime_info()
[1] "sse4_2" "avx"   
> 

[pitrou]

What CPU do you have, by the way?

[jonkeane]

On the system I ran that I've got an Intel Xeon CPU E5-1650 v2

[pitrou]

Indeed, it doesn't have AVX2. Thank you!

[cyb70289]

Also add M1 arm64 flags?
{"hw.optional.neon", ASIMD}
I haven't tested this flag myself, just references golang/go#42747 (comment)

[pitrou]

Can do :-) By the way, is there a difference between "Neon" and "ASIMD"?

[pitrou]

Also, here are the available values on a M1:
https://gist.github.com/pitrou/fa1527038fea2d7a4cf367ecba7a2d2c

[cyb70289]

Per my knowledge, NEON exists since 32 bit Arm, and ASIMD extends it to 64 bit system, with additional instructions and wider registers. ASIMD should be the correct term for Arm64 world, but obviously NEON is well known and widely used.

[cyb70289]

Double checked with an expert from Arm. Actually Neon == Advanced SIMD.

From Arm Architecture Reference Manual (ArmARM):
The Advanced SIMD architecture, its associated implementations, and supporting software, are commonly referred to as NEON™ technology".

Neon was the codename that became public and more commonly used than the "proper name" (Advanced SIMD).

[xhochy]

On an M1 I get the following output for a397fc3cbebf44f0e0b3bd1e3eccfbd69eb507eb

In [2]: pa.runtime_info()
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.sse4_2'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx1_0'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx2_0'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.bmi1'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.bmi2'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx512f'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx512cd'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx512dq'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx512bw'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.optional.avx512vl'. Detail: [errno 45] Operation not supported
../src/arrow/util/cpu_info.cc:92: IOError: sysctlbyname failed for 'hw.l3cachesize'. Detail: [errno 22] Invalid argument
Out[2]: RuntimeInfo(simd_level='none', detected_simd_level='none')

jonkeane uwe I don't know if you can easily test this on a x86 Mac.

Different Uwe in that handle ;)

[pitrou]

Thank you @xhochy .
The joke is that ENOTSUP isn't mentioned in Apple's own manual page...

[pitrou]

Also, can you try under Rosetta?

[xhochy]

Thank you @xhochy .
The joke is that ENOTSUP isn't mentioned in Apple's own manual page...

Also not mentioned on the manpage on the actual system. In general, I have the feeling that near to none of the internal workings / changes of macos-on-M1 vs macos-on-Intel has been published yet.

[xhochy]

For 662182a:

Rosetta 2:

In [2]: pa.runtime_info()
Out[2]: RuntimeInfo(simd_level='sse4_2', detected_simd_level='sse4_2')

Native:

In [2]: pa.runtime_info()
Out[2]: RuntimeInfo(simd_level='none', detected_simd_level='none')

[pitrou]

Ok, thank you. So it seems to work now. I'll wait for CI and then merge.

[xhochy]

Shouldn't native show neon?

[pitrou]

simd_level corresponds to the values allowed in ARROW_USER_SIMD_LEVEL. Currently we don't have dynamic dispatch for ARM64 (and I don't think we allow compiling with Neon disabled).