Broken build on and after tag master-99009e7

[rbrisita]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Build main to run llama.cpp.

Current Behavior

No build with last line output:
make: *** [Makefile:238: ggml-quants-k.o] Error 1

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

Orange PI 5

• Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu

Architecture:           aarch64
  CPU op-mode(s):       32-bit, 64-bit
  Byte Order:           Little Endian
CPU(s):                 8
  On-line CPU(s) list:  0-7
Vendor ID:              ARM
  Model name:           Cortex-A55
    Model:              0
    Thread(s) per core: 1
    Core(s) per socket: 4
    Socket(s):          1
    Stepping:           r2p0
    CPU max MHz:        1800.0000
    CPU min MHz:        408.0000
    BogoMIPS:           48.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32 at
                        omics fphp asimdhp cpuid asimdrdm lrcpc dcpop
                         asimddp
  Model name:           Cortex-A76
    Model:              0
    Thread(s) per core: 1
    Core(s) per socket: 2
    Socket(s):          2
    Stepping:           r4p0
    CPU max MHz:        2256.0000
    CPU min MHz:        408.0000
    BogoMIPS:           48.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32 at
                        omics fphp asimdhp cpuid asimdrdm lrcpc dcpop
                         asimddp
Caches (sum of all):    
  L1d:                  384 KiB (8 instances)
  L1i:                  384 KiB (8 instances)
  L2:                   2.5 MiB (8 instances)
  L3:                   3 MiB (1 instance)
Vulnerabilities:        
  Itlb multihit:        Not affected
  L1tf:                 Not affected
  Mds:                  Not affected
  Meltdown:             Not affected
  Spec store bypass:    Mitigation; Speculative Store Bypass disabled
                         via prctl
  Spectre v1:           Mitigation; __user pointer sanitization
  Spectre v2:           Vulnerable: Unprivileged eBPF enabled
  Srbds:                Not affected
  Tsx async abort:      Not affected

• Operating System, e.g. for Linux:

$ uname -a

Linux opi-desktop 5.10.110-rockchip-rk3588 #2 SMP Wed May 24 20:44:10 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

• SDK version, e.g. for Linux:

$ python3 --version
$ make --version
$ g++ --version

Python 3.10.6

GNU Make 4.3
Built for aarch64-unknown-linux-gnu

g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0

Failure Information (for bugs)

Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

1. git clone https://github.com/ggerganov/llama.cpp.git
2. cd llama.cpp
3. make

Failure Logs

I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  aarch64
I UNAME_M:  aarch64
I CFLAGS:   -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -mcpu=native
I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -mcpu=native
I LDFLAGS:  
I CC:       cc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
I CXX:      g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0

cc  -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -mcpu=native   -c ggml-quants-k.c -o ggml-quants-k.o
ggml-quants-k.c: In function ‘ggml_vec_dot_q2_k_q8_k’:
ggml-quants-k.c:1038:36: warning: missing braces around initializer [-Wmissing-braces]
 1038 |         const int16x8x2_t mins16 = {vreinterpretq_s16_u16(vmovl_u8(vget_low_u8(mins))), vreinterpretq_s16_u16(vmovl_u8(vget_high_u8(mins)))};
      |                                    ^
      |                                     {                                                                                                      }
ggml-quants-k.c: In function ‘ggml_vec_dot_q4_k_q8_k’:
ggml-quants-k.c:1522:5: note: use ‘-flax-vector-conversions’ to permit conversions between vectors with differing element types or numbers of subparts
 1522 |     const uint32x4_t mzero = vdupq_n_s32(0);
      |     ^~~~~
ggml-quants-k.c:1522:30: error: incompatible types when initializing type ‘uint32x4_t’ using type ‘int32x4_t’
 1522 |     const uint32x4_t mzero = vdupq_n_s32(0);
      |                              ^~~~~~~~~~~
ggml-quants-k.c:1567:54: error: incompatible type for argument 1 of ‘vdotq_s32’
 1567 |         const int32x4_t p1 = vdotq_s32(vdotq_s32(mzero, q4bytes.val[0], q8bytes.val[0]), q4bytes.val[1], q8bytes.val[1]);
      |                                                  ^~~~~
      |                                                  |
      |                                                  uint32x4_t

In file included from ggml-quants-k.c:14:
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h:32134:22: note: expected ‘int32x4_t’ but argument is of type ‘uint32x4_t’
32134 | vdotq_s32 (int32x4_t __r, int8x16_t __a, int8x16_t __b)
      |            ~~~~~~~~~~^~~
ggml-quants-k.c:1574:54: error: incompatible type for argument 1 of ‘vdotq_s32’
 1574 |         const int32x4_t p2 = vdotq_s32(vdotq_s32(mzero, q4bytes.val[0], q8bytes.val[0]), q4bytes.val[1], q8bytes.val[1]);
      |                                                  ^~~~~
      |                                                  |
      |                                                  uint32x4_t

In file included from ggml-quants-k.c:14:
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h:32134:22: note: expected ‘int32x4_t’ but argument is of type ‘uint32x4_t’
32134 | vdotq_s32 (int32x4_t __r, int8x16_t __a, int8x16_t __b)
      |            ~~~~~~~~~~^~~
ggml-quants-k.c: In function ‘ggml_vec_dot_q5_k_q8_k’:
ggml-quants-k.c:1807:52: error: incompatible type for argument 1 of ‘vdotq_s32’
 1807 |           sumi += vaddvq_s32(vdotq_s32(vdotq_s32(mzero, q5bytes.val[0], q8bytes.val[0]), q5bytes.val[1], q8bytes.val[1])) * *scales++;
      |                                                  ^~~~~
      |                                                  |
      |                                                  uint32x4_t

In file included from ggml-quants-k.c:14:
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h:32134:22: note: expected ‘int32x4_t’ but argument is of type ‘uint32x4_t’
32134 | vdotq_s32 (int32x4_t __r, int8x16_t __a, int8x16_t __b)
      |            ~~~~~~~~~~^~~
ggml-quants-k.c:1808:52: error: incompatible type for argument 1 of ‘vdotq_s32’
 1808 |           sumi += vaddvq_s32(vdotq_s32(vdotq_s32(mzero, q5bytes.val[2], q8bytes.val[2]), q5bytes.val[3], q8bytes.val[3])) * *scales++;
      |                                                  ^~~~~
      |                                                  |
      |                                                  uint32x4_t

In file included from ggml-quants-k.c:14:
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h:32134:22: note: expected ‘int32x4_t’ but argument is of type ‘uint32x4_t’
32134 | vdotq_s32 (int32x4_t __r, int8x16_t __a, int8x16_t __b)
      |            ~~~~~~~~~~^~~
ggml-quants-k.c: In function ‘ggml_vec_dot_q6_k_q8_k’:
ggml-quants-k.c:2010:38: warning: missing braces around initializer [-Wmissing-braces]
 2010 |         const int16x8x2_t q6scales = {vmovl_s8(vget_low_s8(scales)), vmovl_s8(vget_high_s8(scales))};
      |                                      ^
      |                                       {                                                            }
make: *** [Makefile:238: ggml-quants-k.o] Error 1

[rbrisita]

Also breaks for ARM Debian 11.

[spuuntries]

Can confirm, having the same issue on a Neoverse N1 Oracle server.

[bipindr123]

Can confirm this on ubuntu 22 ARM64

[rbrisita]

Try LLAMA_NO_K_QUANTS=1 with make. Got this from issue #2075. There seems to be a code work around in issue #1927 here.

[ccjincong]

ubuntu18 arm64 also has this problem

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.