gguf-split
add a default option to not include tensors data in first shard
#6463
Labels
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
split
GGUF split model sharding
Motivation
be able to make a split where the first shard is very small and contains primarily the metadata so that it can be downloaded quickly and then start the download of the other shards without waiting for the first to finish
Proposition
Add an option to not include tensor data in the first file. Maybe it should be enabled by default.
Should be well tested.
ggml_alloc
should not be called as it will complain withWARNING: Behavior may be unexpected when allocating 0 bytes for ggml_malloc!
We can add extra meta data in the first file that describes all tensors in the shards for example
References
split.no
metadata huggingface/huggingface.js#604The text was updated successfully, but these errors were encountered: