make is error ( k_quants.c)

[zhp8341]

llama.cpp$ make

I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: x86_64
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -std=c11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS
I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS
I LDFLAGS:
I CC: cc (Ubuntu 7.5.0-3ubuntu116.04) 7.5.0
I CXX: g++ (Ubuntu 7.5.0-3ubuntu116.04) 7.5.0

cc -I. -O3 -std=c11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS -c -o k_quants.o k_quants.c
k_quants.c: In function ‘ggml_vec_dot_q2_K_q8_K’:
k_quants.c:1121:36: warning: implicit declaration of function ‘_mm256_set_m128i’; did you mean ‘_mm256_set_epi8’? [-Wimplicit-function-declaration]
const __m256i scales[2] = {_mm256_set_m128i(l_scales, l_scales), _mm256_set_m128i(h_scales, h_scales)};
^~~~~~~~~~~~~~~~
_mm256_set_epi8
k_quants.c:1121:35: warning: missing braces around initializer [-Wmissing-braces]
const __m256i scales[2] = {_mm256_set_m128i(l_scales, l_scales), _mm256_set_m128i(h_scales, h_scales)};
^
{ }
k_quants.c: In function ‘ggml_vec_dot_q3_K_q8_K’:
k_quants.c:1361:35: warning: missing braces around initializer [-Wmissing-braces]
const __m256i scales[2] = {_mm256_set_m128i(l_scales, l_scales), _mm256_set_m128i(h_scales, h_scales)};
^
{ }
k_quants.c: In function ‘ggml_vec_dot_q4_K_q8_K’:
k_quants.c:1635:32: error: incompatible types when initializing type ‘__m256i {aka const __vector(4) long long int}’ using type ‘int’
const __m256i scales = _mm256_set_m128i(sc128, sc128);
^~~~~~~~~~~~~~~~
k_quants.c: In function ‘ggml_vec_dot_q5_K_q8_K’:
k_quants.c:1865:32: error: incompatible types when initializing type ‘__m256i {aka const __vector(4) long long int}’ using type ‘int’
const __m256i scales = _mm256_set_m128i(sc128, sc128);
^~~~~~~~~~~~~~~~
: recipe for target 'k_quants.o' failed
make: *** [k_quants.o] Error 1

[hbf731eF]

try upgrading gcc/g++
ggerganov/whisper.cpp#851 (comment)

[SlyEcho]

-march=native is not detecting the right CPU instructions to use. Use CMake for better control over the configuration or update the compiler, which may help or not, depending on the OS, etc.

[irajmoradi]

Did you manage to get it to work?

[alecGraves]

My pal GPT says the following works too if your intrinsics library does not contain _mm256_set_m128i; just add an alternative implementation to the top of the k_quants.c file and rename the references.

static inline __m256i my_mm256_set_m128i(__m128i hi, __m128i lo) {
    __m256i val = _mm256_castsi128_si256(lo);
    val = _mm256_insertf128_si256(val, hi, 1);
    return val;
}

then replace calls to _mm256_set_m128i with my_mm256_set_m128i

This let me compile and run on an old version of Debian with GCC-7.5.

[22dimensions]

My pal GPT says the following works too if your intrinsics library does not contain _mm256_set_m128i; just add an alternative implementation to the top of the k_quants.c file and rename the references.

static inline __m256i my_mm256_set_m128i(__m128i hi, __m128i lo) {
    __m256i val = _mm256_castsi128_si256(lo);
    val = _mm256_insertf128_si256(val, hi, 1);
    return val;
}

then replace calls to _mm256_set_m128i with my_mm256_set_m128i

This let me compile and run on an old version of Debian with GCC-7.5.

It works, thanks.

[alecGraves]

You could also modify the code, adding a check for gcc versions

#include <immintrin.h>
#if <gcc version lt 8?>
static inline __m256i __mm256_set_m128i(__m128i hi, __m128i lo) {
    __m256i val = _mm256_castsi128_si256(lo);
    val = _mm256_insertf128_si256(val, hi, 1);
    return val;
}
#endif

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.