-
Notifications
You must be signed in to change notification settings - Fork 878
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Llama] Add Llama3.2 Instruct 1B 3B to preset (#575)
This PR supports Llama3.2-1B and 3B Instruct. Hence we add the following Llama3.2 models to the prebuilt list: - `Llama-3.2-1B-Instruct-q4f16_1-MLC` - `Llama-3.2-1B-Instruct-q4f32_1-MLC` - `Llama-3.2-1B-Instruct-q0f16-MLC` - `Llama-3.2-1B-Instruct-q0f32-MLC` - `Llama-3.2-3B-Instruct-q4f16_1-MLC` - `Llama-3.2-3B-Instruct-q4f32_1-MLC`
- Loading branch information
1 parent
0b5f405
commit db77ff5
Showing
1 changed file
with
79 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
db77ff5
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome.
Out of curiosity: what does Q0 mean? Unquantified?
db77ff5
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The huggingface folder is called
Llama-3.2-1B-Instruct-q4f16_0-MLC
instead of/Llama-3.2-1B-Instruct-q4f16_1-MLC
?Oh wait, no, they both exist.
What is the difference between 0 and 1 again?