convert-hf : print output file name when completed #8181
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds the output file name to the log message when the conversion is completed. The motivation for this change is that when `--outfile` option is not specified it migth not be obvious where the output file is written. With this change the output of running the script will be something like the following: ```console INFO:hf-to-gguf:Model successfully exported to models/gemma-2-9b-it.gguf. ``` Signed-off-by: Daniel Bevenius <[email protected]>
mofosyne
added
the
Review Complexity : Low
compilade
reviewed
Jun 29, 2024
There was a problem hiding this comment.
This is nice, although since #6942, the output file names are already printed before the progress bar.
On master, after printing the tensors, their types and their shape, there's this:
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/bf16-tests/tinymistral-bf16.gguf: n_tensors = 111, total_size = 496.1M
Writing: 100%|█████████████████████████████████████████████████████████████████████████████████| 496M/496M [00:05<00:00, 94.4Mbyte/s]
INFO:hf-to-gguf:Model successfully exported.
Your proposal would change this into
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/bf16-tests/tinymistral-bf16.gguf: n_tensors = 111, total_size = 496.1M
Writing: 100%|█████████████████████████████████████████████████████████████████████████████████| 496M/496M [00:05<00:00, 94.4Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to /srv/LLMstash/tmp/bf16-tests/tinymistral-bf16.gguf.
Where the output file name is present twice.
And when using split options, the final name would not exist.
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0-00001-of-00003.gguf: n_tensors = 17, total_size = 99.5M
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0-00002-of-00003.gguf: n_tensors = 58, total_size = 99.5M
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0-00003-of-00003.gguf: n_tensors = 36, total_size = 64.7M
Shard (3/3): 100%|███████████████████████████████████████████████████████████████████████████| 64.7M/64.7M [00:00<00:00, 70.0Mbyte/s]
Writing: 100%|█████████████████████████████████████████████████████████████████████████████████| 264M/264M [00:07<00:00, 33.4Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to /srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0.gguf.
$ ls /srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/
tinymistral-q8_0-00001-of-00003.gguf tinymistral-q8_0-00002-of-00003.gguf tinymistral-q8_0-00003-of-00003.gguf
But it might still be useful when not splitting.
Updates the output of to support printing the directory if the output is split into multiple files. Also the output file name is now retrieved from the model_instance object. Signed-off-by: Daniel Bevenius <[email protected]>
Ah sorry, I see it now, for some reason I just missed it :( Now that I know where to look I'm happy with not having the output file name/path in the final output, but perhaps it might be nice to have it just the same in case others run into the same issue I did. I'll push a commit with a suggestion with the changes you suggested and also include printing the directory when the conversion performs a split. Thanks! |
compilade
reviewed
Jun 29, 2024
Use parent attribute of Path object and string interpolation. Signed-off-by: Daniel Bevenius <[email protected]>
Use os.sep instead of hardcoding the path separator. Signed-off-by: Daniel Bevenius <[email protected]>
arthw
pushed a commit
to arthw/llama.cpp
that referenced
this pull request
Jul 3, 2024
* convert-hf : print output file name when completed This commit adds the output file name to the log message when the conversion is completed. The motivation for this change is that when `--outfile` option is not specified it migth not be obvious where the output file is written. With this change the output of running the script will be something like the following: ```console INFO:hf-to-gguf:Model successfully exported to models/gemma-2-9b-it.gguf. ``` Signed-off-by: Daniel Bevenius <[email protected]> * squash! convert-hf : print output file name when completed Updates the output of to support printing the directory if the output is split into multiple files. Also the output file name is now retrieved from the model_instance object. Signed-off-by: Daniel Bevenius <[email protected]> * squash! convert-hf : print output file name when completed Use parent attribute of Path object and string interpolation. Signed-off-by: Daniel Bevenius <[email protected]> * squash! convert-hf : print output file name when completed Use os.sep instead of hardcoding the path separator. Signed-off-by: Daniel Bevenius <[email protected]> --------- Signed-off-by: Daniel Bevenius <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds the output file name to the log message when the conversion is completed.
The motivation for this change is that when
--outfileoption is not specified it migth not be obvious where the output file is written.With this change the output of running the script will be something like the following:
INFO:hf-to-gguf:Model successfully exported to models/gemma-2-9b-it.gguf.