Compilers: CFLAGS and CLANG/LLVM options

[ptr1337]

AS very kernel version also changes the config and features, i think it should be a great idea to deploy several configs for the kernel which will be compiled.

For example:

Raspberry Pi - Zero/W/WH & 1 Model A/B/A+/B+ : -march=armv6 -mfloat-abi=hard -mfpu=vfp
Raspberry Pi - 2 & 3 Model A/B: -march=armv7-a -mfloat-abi=hard -mfpu=neon-vfpv4
Raspberry Pi - 3 & 4 Model A+/B+ & Compute 3/3-lite/3+ (32-Bit): -march=armv8-a -mfloat-abi=hard -mfpu=neon-fp-armv8
Raspberry Pi - 3 & 4 Model A+/B+ & Compute 3/3-lite/3+ (64-Bit): -march=armv8-a+fp+simd

Other example from GCC Manuals since GCC11:

`-mtune=name

This option specifies the name of the target ARM processor for which GCC should tune the performance of the code. For some ARM implementations better performance can be obtained by using this option. Permissible names are: ‘arm7tdmi’, ‘arm7tdmi-s’, ‘arm710t’, ‘arm720t’, ‘arm740t’, ‘strongarm’, ‘strongarm110’, ‘strongarm1100’, 0‘strongarm1110’, ‘arm8’, ‘arm810’, ‘arm9’, ‘arm9e’, ‘arm920’, ‘arm920t’, ‘arm922t’, ‘arm946e-s’, ‘arm966e-s’, ‘arm968e-s’, ‘arm926ej-s’, ‘arm940t’, ‘arm9tdmi’, ‘arm10tdmi’, ‘arm1020t’, ‘arm1026ej-s’, ‘arm10e’, ‘arm1020e’, ‘arm1022e’, ‘arm1136j-s’, ‘arm1136jf-s’, ‘mpcore’, ‘mpcorenovfp’, ‘arm1156t2-s’, ‘arm1156t2f-s’, ‘arm1176jz-s’, ‘arm1176jzf-s’, ‘generic-armv7-a’, ‘cortex-a5’, ‘cortex-a7’, ‘cortex-a8’, ‘cortex-a9’, ‘cortex-a12’, ‘cortex-a15’, ‘cortex-a17’, ‘cortex-a32’, ‘cortex-a35’, ‘cortex-a53’, ‘cortex-a55’, ‘cortex-a57’, ‘cortex-a72’, ‘cortex-a73’, ‘cortex-a75’, ‘cortex-a76’, ‘cortex-a76ae’, ‘cortex-a77’, ‘cortex-a78’, ‘cortex-a78ae’, ‘cortex-a78c’, ‘ares’, ‘cortex-r4’, ‘cortex-r4f’, ‘cortex-r5’, ‘cortex-r7’, ‘cortex-r8’, ‘cortex-r52’, ‘cortex-m0’, ‘cortex-m0plus’, ‘cortex-m1’, ‘cortex-m3’, ‘cortex-m4’, ‘cortex-m7’, ‘cortex-m23’, ‘cortex-m33’, ‘cortex-m35p’, ‘cortex-m55’, ‘cortex-x1’, ‘cortex-m1.small-multiply’, ‘cortex-m0.small-multiply’, ‘cortex-m0plus.small-multiply’, ‘exynos-m1’, ‘marvell-pj4’, ‘neoverse-n1’, ‘neoverse-n2’, ‘neoverse-v1’, ‘xscale’, ‘iwmmxt’, ‘iwmmxt2’, ‘ep9312’, ‘fa526’, ‘fa626’, ‘fa606te’, ‘fa626te’, ‘fmp626’, ‘fa726te’, ‘xgene1’.

Additionally, this option can specify that GCC should tune the performance of the code for a big.LITTLE system. Permissible names are: ‘cortex-a15.cortex-a7’, ‘cortex-a17.cortex-a7’, ‘cortex-a57.cortex-a53’, ‘cortex-a72.cortex-a53’, ‘cortex-a72.cortex-a35’, ‘cortex-a73.cortex-a53’, ‘cortex-a75.cortex-a55’, ‘cortex-a76.cortex-a55’. `

Link can be found here for more informtions:

https://gcc.gnu.org/onlinedocs/gcc-11.2.0/gcc/ARM-Options.html#ARM-Options

[pyavitz]

The builder is currently using the defconfig(s) provided by the foundation and I have no plans to veer from this course. There is already an option for people to use a custom defconfig, if they so chose too.

If someone would be willing to take on this task "wink wink" I'm all for them doing so.

[ptr1337]

Alright. Since using Clang is mostly no profit , if the needed values are not enabled in the config

[pyavitz]

I was under the impression when using Clang specific things got ticked off by default in the defconfig? I've also noticed that using Clang on 5.10.y is pretty much garbage and that it only shines when using 5.12.y and up.

With that said, I'm sure there are things that can improve the situation with fine tuning. I'm just saying that it's not a current priority of mine and that if someone is willing to pick up that torch and run with it, I would of course have no problem with them doing so.

[pyavitz]

git clone -b clang https://github.com/pyavitz/rpi-img-builder.git clang

make admin
make config
make mlconfig

You should find the following in the userdata.txt file. Tune as you wish and build.

### CLANG TUNING
CFLAGS=""

Example:

### CLANG TUNING
CFLAGS="-march=armv8-a+fp+simd"

When everyone is satisfied that this actual works, I will push it to master.

[ptr1337]

Also with GCC 11 possible.

[pyavitz]

Also with GCC 11 possible.

The builder doesn't support GCC-11 yet, but I don't see it being a problem when it becomes apart of the equation. We would just need to adjust the lib/function/gcc file, add the CFLAGS variable and assign it to the distro / releases that use it by default.

I'm not 100% sure, but I think the only release that currently uses it as an option is, Hirsute Hippo?

Anyway, let me know what you find in your testing. I also need to test to see if nothing is placed inside the variable, does the build care? Because if so, the whole tuning bit will need to be isolated.

[pyavitz]

We should now be able to changeLLVM=1 and LLVM_IAS=1 on the fly in the userdata.txt file.

[pyavitz]

Is this normal?

LLVM_IAS=1

Linux version 5.13.9-v8 (root@rpi-img-builder) (gcc (Ubuntu 10.3.0-1ubuntu1)
10.3.0, GNU ld (GNU Binutils for Ubuntu) 2.36.1) #1 SMP PREEMPT Wed Aug 11 22:23:22 EDT 2021

LLVM=1

Linux version 5.13.9-v8 (root@rpi-img-builder) (Ubuntu clang version 12.0.0-3ubuntu1~21.04.1, LLD 12.0.0)
#1 SMP PREEMPT Wed Aug 11 14:27:03 EDT 2021

I'm no expert, but it would appear to me that something is a miss.

This is in reference to the comment made here: 5714c1f

[ptr1337]

You should use both flags if compiling with Clang LTO.

In the example above you used only one compiler flag.

it also depends which version of Ubuntu you compile, since you force the kernel compile process depends on the version of the distro. But there should be no difference between the versions distro and the compiled kernel.

For example, if xanmod shares a kernel which is compiled with clang 12 and lto, it does not depend on the distro version. (Based on Debian )

[ptr1337]

In the userspace the is difference between using the same compiler. For example, Ubuntu 21.04 uses mostly only llvm/clang lto compiled packages/userspace.

The compile process if using THINLTO/FULLL-LTO takes around 30 percent more time then without.

[arakeen]

Actually for the LVM and LVM_IAS flags they are very specific "only" to the kernel itself.
The kernel use LLVM to specificly set the compiler with this:
ifneq ($(LLVM),)
HOSTCC = clang
HOSTCXX = clang++
else
HOSTCC = gcc
HOSTCXX = g++
endif

And with IAS.. it uses it to choose between LLVM or GNU assembly.. distro independent,.
ifeq ($(LLVM_IAS),1)
CLANG_FLAGS += -integrated-as
else
CLANG_FLAGS += -no-integrated-as

I have found a few test cases that uses it however for output code there are no output targets and none that are non kernel specific. These are also very new and untested by the way

[ptr1337]

@arakeen

Yes, thats correct. Mostly using llvm_ias for the kernel is since 5.12 a good tested solution and im also using it as daily driver on several servers and desktops.

There is definitly a benefit, i also compiled complete plasma with thinlto and -march=x86-64-v3 and there is also a big differnce to feel.

Same with applications like firefox, which is compiled with lto and pgo. The performance differnce is insane.

[arakeen]

Hey would it be possible to use https://web.libera.chat/ to communicate as well so we can have more dynamic discussion? Thank you. we use the #arm-linux-builder channel.

[pyavitz]

The following commit should resolve this: 01df896

When GCC-11 becomes a factor, feel free to reopen the issue.