Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Quantizing manually #118

Draft
wants to merge 9 commits into
base: llama_fp8
Choose a base branch
from
Draft

Quantizing manually #118

wants to merge 9 commits into from

Commits on Jul 19, 2024

  1. (WIP) llama fp8 safetensor conversion

    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    03637ef View commit details
    Browse the repository at this point in the history

  2. add in changes for loading model. 

    its ugly
    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    3e40a53 View commit details
    Browse the repository at this point in the history

  3. move file

    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    ada5cac View commit details
    Browse the repository at this point in the history

  4. re-add original

    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    2e5e8f8 View commit details
    Browse the repository at this point in the history

  5. fix importer 

    TODO: split qkv prior to irpa
    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    ca28354 View commit details
    Browse the repository at this point in the history

  6. dont quantize quantized parameters facedesk

    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    98607ff View commit details
    Browse the repository at this point in the history

  7. datatype

    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    c8104e9 View commit details
    Browse the repository at this point in the history

  8. add some fixes to run

    @dan-garvey
    dan-garvey committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    4ed3c9d View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Quantizing manually

    @rohan-tan-bhowmik
    rohan-tan-bhowmik committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5ec5c0c View commit details
    Browse the repository at this point in the history