Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llama : add Mixtral support #4406

Merged
merged 47 commits into from
Dec 13, 2023
Merged

llama : add Mixtral support #4406

merged 47 commits into from
Dec 13, 2023

Commits on Dec 9, 2023

  1. convert : support Mixtral as LLAMA arch

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    dff8cbe View commit details
    Browse the repository at this point in the history

  2. convert : fix n_ff typo

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    d38e41e View commit details
    Browse the repository at this point in the history

  3. llama : model loading

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    a3eefe9 View commit details
    Browse the repository at this point in the history

  4. ggml : sync latest ggml_mul_mat_id

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    861cd67 View commit details
    Browse the repository at this point in the history

  5. llama : update graph to support MoE

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    aedfad1 View commit details
    Browse the repository at this point in the history

  6. llama : fix cur -> cur_expert

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    af1a096 View commit details
    Browse the repository at this point in the history

  7. llama : first working version

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    7ea3695 View commit details
    Browse the repository at this point in the history

  8. llama : fix expert weighting in the FFN

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    8b185b7 View commit details
    Browse the repository at this point in the history

  9. ggml : ggml_get_rows support 2D indexing [n_tokens, n_experts] (cpu o… 

    …nly)
    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    7372b62 View commit details
    Browse the repository at this point in the history

  10. ggml : add n_as argument to ggml_mul_mat_id

    @slaren
    slaren committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    ee8fb39 View commit details
    Browse the repository at this point in the history

  11. ggml : fix ggml_get_rows to take into account ne02 / ne11

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    9064b1c View commit details
    Browse the repository at this point in the history

  12. metal : add more general support for ggml_get_rows + tests

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    2cbcba8 View commit details
    Browse the repository at this point in the history

  13. llama : add basic support for offloading moe with CUDA

    @slaren
    slaren committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    06dfde3 View commit details
    Browse the repository at this point in the history

  14. metal : add/mul/div use general kernel when src1 not cont

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    7e2006b View commit details
    Browse the repository at this point in the history

  15. metal : reduce the kernel launches for ggml_mul_mat_id

    @ggerganov
    ggerganov committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    8c5b66e View commit details
    Browse the repository at this point in the history

  16. ggml : get_rows : support non-contiguos tensors with gaps, generalize… 

    … up to 3D
    @slaren
    slaren committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    ac3f7d8 View commit details
    Browse the repository at this point in the history

  17. ggml : update get_rows f16 and q

    @slaren
    slaren committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    2e4db48 View commit details
    Browse the repository at this point in the history

  18. cuda : support non-contiguous src1 in get_rows

    @slaren
    slaren committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    62b95f9 View commit details
    Browse the repository at this point in the history

  19. llama : offload missing ffn_moe_silu

    @slaren
    slaren committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    0710b0f View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2023

  1. metal : fix ggml_get_rows to work with non-cont src1

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    016f9bb View commit details
    Browse the repository at this point in the history

  2. metal : add indirect mat-vec kernels for all quantization types

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    6cfb31f View commit details
    Browse the repository at this point in the history

  3. llama : do not quantize expert gating tensors

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    d1259b7 View commit details
    Browse the repository at this point in the history

  4. llama : add n_expert and n_expert_used to hparams + change quants

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    e640cbe View commit details
    Browse the repository at this point in the history

  5. test-backend-ops : add moe test

    @slaren
    slaren committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    cefebb3 View commit details
    Browse the repository at this point in the history

  6. cuda : fix get_rows when ncols is odd

    @slaren
    slaren committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    8614aa7 View commit details
    Browse the repository at this point in the history

  7. convert : determine n_ctx correctly

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    65923a8 View commit details
    Browse the repository at this point in the history

  8. metal : fix ggml_mul_mat_id for F32

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    b0b83dd View commit details
    Browse the repository at this point in the history

  9. test-backend-ops : make experts more evenly probable (test_moe)

    @ggerganov
    ggerganov committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    54ba263 View commit details
    Browse the repository at this point in the history

  10. test-backend-ops : cleanup, add moe test for batches

    @slaren
    slaren committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    54d254b View commit details
    Browse the repository at this point in the history

  11. test-backend-ops : add cpy from f32 -> all types test

    @slaren
    slaren committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    f1380d7 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. test-backend-ops : fix dequantize block offset

    @slaren
    slaren committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    b002981 View commit details
    Browse the repository at this point in the history

  2. llama : fix hard-coded number of experts

    @ggerganov
    ggerganov committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    8cbaed1 View commit details
    Browse the repository at this point in the history

  3. test-backend-ops : simplify and disable slow tests to avoid CI timeout

    @slaren
    slaren committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    ffda94c View commit details
    Browse the repository at this point in the history

  4. test-backend-ops : disable MOE test with thread sanitizer

    @slaren
    slaren committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    33e50f1 View commit details
    Browse the repository at this point in the history

  5. cuda : fix mul_mat_id with multi gpu

    @slaren
    slaren committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    296c945 View commit details
    Browse the repository at this point in the history

  6. convert : use 1e6 rope_freq_base for mixtral

    @slaren
    slaren committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    7dc75e3 View commit details
    Browse the repository at this point in the history

  7. convert : fix style

    @slaren
    slaren committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    f1cbfab View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. convert : support safetensors format

    @ggerganov
    ggerganov committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    6a419f4 View commit details
    Browse the repository at this point in the history

  2. gguf-py : bump version

    @slaren
    slaren committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    a742d9f View commit details
    Browse the repository at this point in the history

  3. metal : add cpy f16 -> f32 kernel

    @ggerganov
    ggerganov committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    08eb991 View commit details
    Browse the repository at this point in the history

  4. metal : fix binary ops for ne10 % 4 != 0

    @ggerganov
    ggerganov committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    a51bc0c View commit details
    Browse the repository at this point in the history

  5. test-backend-ops : add one more sum_rows test

    @ggerganov
    ggerganov committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    ea4402b View commit details
    Browse the repository at this point in the history

  6. ggml : do not use BLAS with ggml_mul_mat_id

    @ggerganov
    ggerganov committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    90c12e6 View commit details
    Browse the repository at this point in the history

  7. convert-hf : support for mixtral-instruct (#4428) 

    * convert : typo fix, add additional hyperparameters, use LLaMA arch for Mixtral-instruct

* convert : use sentencepiece tokenizer for Mixtral-instruct

* convert : make flake8 happy
    @Mrkvak
    Mrkvak authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    82e4f64 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. metal : fix soft_max kernels 

    ref: ggerganov/ggml@1914017
    @ggerganov
    ggerganov committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ab558ac View commit details
    Browse the repository at this point in the history

  2. metal : limit kernels to not use more than the allowed threads

    @ggerganov
    ggerganov committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    109e7aa View commit details
    Browse the repository at this point in the history

  3. metal : switch to execution barriers + fix one of the barriers

    @ggerganov
    ggerganov committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    e1241d9 View commit details
    Browse the repository at this point in the history