ggml : add AVX support based on AVX2 code #1376
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I reopen new PR w/o unused function.
I added AVX support code from AVX2 code and modified AVX2 code
added AVX support code
static inline __m256i bytes_from_bits_32(const uint8_t * x)
static inline __m256i bytes_from_nibbles_32(const uint8_t * rsi)
static inline __m256 sum_i16_pairs_float(const __m128i xh, const __m128i xl)
static inline __m256 mul_sum_i8_pairs_float(const __m256i x, const __m256i y)
static void quantize_row_q4_1(const float * restrict x, void * restrict vy, int k)
static void dequantize_row_q4_0(const void * restrict vx, float * restrict y, int k)
static void dequantize_row_q4_1(const void * restrict vx, float * restrict y, int k)
static void ggml_vec_dot_q4_1_q8_1(const int n, float * restrict s, const void * restrict vx, const void * restrict vy)
static void ggml_vec_dot_q4_2_q8_0(const int n, float * restrict s, const void * restrict vx, const void * restrict vy)
static void ggml_vec_dot_q5_0_q8_0(const int n, float * restrict s, const void * restrict vx, const void * restrict vy)
static void ggml_vec_dot_q5_1_q8_1(const int n, float * restrict s, const void * restrict vx, const void * restrict vy)
static void ggml_vec_dot_q8_0_q8_0(const int n, float * restrict s, const void * restrict vx, const void * restrict vy)
modified AVX2 code
static void quantize_row_q4_1(const float * restrict x, void * restrict vy, int k)
static void dequantize_row_q4_1(const void * restrict vx, float * restrict y, int k)
ggerganov and sw, please confirm this PR.