Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: enable ModelLoaderHuggerFace to support loading models in fp16 for inference #555

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

feat: enable ModelLoaderHuggerFace to support loading models in fp16 for inference #555

wants to merge 6 commits into from
+19 −3

Commits on Sep 22, 2024

  1. feat: add fp16_inference option to support fp16 infer 

    when `fp16_inference` is enabled, the model will be loaded as fp16
paramters when inference.
    @0x404
    0x404 committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    2198f6f View commit details
    Browse the repository at this point in the history

  2. solve conflicts

    @0x404
    0x404 committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    4a58ede View commit details
    Browse the repository at this point in the history

  3. update

    @fpzh2011 @0x404
    fpzh2011 authored and 0x404 committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    49fc21e View commit details
    Browse the repository at this point in the history

  4. support chatglm with fp16 inference

    @0x404
    0x404 committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    4edb33a View commit details
    Browse the repository at this point in the history

  5. Revert "update" 

    This reverts commit 49fc21e.
    @0x404
    0x404 committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    ec1a81a View commit details
    Browse the repository at this point in the history

  6. set defaults to False

    @0x404
    0x404 committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    c2a8ef6 View commit details
    Browse the repository at this point in the history