Error: Module 'gguf' has no attribute 'TENSOR_NAMES'

[DimmNet]

While converting the model I get the following error: AttributeError: module 'gguf' has no attribute 'TENSOR_NAMES'

The source model used for the conversion:
https://huggingface.co/flozi00/Llama-2-7b-german-assistant-v3

Conversion start command:
!python3 /content/convert.py /content/new_model --outfile /content/llama-2-7b-custom_q8.gguf --outtype q8_0

Server: Google colab

Execution result:
Loading model file /content/new_model/model-00001-of-00002.safetensors
Loading model file /content/new_model/model-00001-of-00002.safetensors
Loading model file /content/new_model/model-00002-of-00002.safetensors
params = Params(n_vocab=32000, n_embd=4096, n_layer=32, n_ctx=4096, n_ff=11008, n_head=32, n_head_kv=32, f_norm_eps=1e-06, f_rope_freq_base=None, f_rope_scale=None, ftype=<GGMLFileType.MostlyF16: 1>, path_model=PosixPath('/content/new_model'))
Loading vocab file '/content/new_model/tokenizer.model', type 'spm'
Permuting layer 0
Permuting layer 1
Permuting layer 2
Permuting layer 3
Permuting layer 4
Permuting layer 5
Permuting layer 6
Permuting layer 7
Permuting layer 8
Permuting layer 9
Permuting layer 10
Permuting layer 11
Permuting layer 12
Permuting layer 13
Permuting layer 14
Permuting layer 15
Permuting layer 16
Permuting layer 17
Permuting layer 18
Permuting layer 19
Permuting layer 20
Permuting layer 21
Permuting layer 22
Permuting layer 23
Permuting layer 24
Permuting layer 25
Permuting layer 26
Permuting layer 27
Permuting layer 28
Permuting layer 29
Permuting layer 30
Permuting layer 31
model.embed_tokens.weight -> token_embd.weight | F16 | [32000, 4096]
model.layers.0.input_layernorm.weight -> blk.0.attn_norm.weight | F16 | [4096]
model.layers.0.mlp.down_proj.weight -> blk.0.ffn_down.weight | F16 | [4096, 11008]
model.layers.0.mlp.gate_proj.weight -> blk.0.ffn_gate.weight | F16 | [11008, 4096]
model.layers.0.mlp.up_proj.weight -> blk.0.ffn_up.weight | F16 | [11008, 4096]
model.layers.0.post_attention_layernorm.weight -> blk.0.ffn_norm.weight | F16 | [4096]
model.layers.0.self_attn.k_proj.weight -> blk.0.attn_k.weight | F16 | [4096, 4096]
model.layers.0.self_attn.o_proj.weight -> blk.0.attn_output.weight | F16 | [4096, 4096]
model.layers.0.self_attn.q_proj.weight -> blk.0.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.0.attn_rot_embd
model.layers.0.self_attn.v_proj.weight -> blk.0.attn_v.weight | F16 | [4096, 4096]
model.layers.1.input_layernorm.weight -> blk.1.attn_norm.weight | F16 | [4096]
model.layers.1.mlp.down_proj.weight -> blk.1.ffn_down.weight | F16 | [4096, 11008]
model.layers.1.mlp.gate_proj.weight -> blk.1.ffn_gate.weight | F16 | [11008, 4096]
model.layers.1.mlp.up_proj.weight -> blk.1.ffn_up.weight | F16 | [11008, 4096]
model.layers.1.post_attention_layernorm.weight -> blk.1.ffn_norm.weight | F16 | [4096]
model.layers.1.self_attn.k_proj.weight -> blk.1.attn_k.weight | F16 | [4096, 4096]
model.layers.1.self_attn.o_proj.weight -> blk.1.attn_output.weight | F16 | [4096, 4096]
model.layers.1.self_attn.q_proj.weight -> blk.1.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.1.attn_rot_embd
model.layers.1.self_attn.v_proj.weight -> blk.1.attn_v.weight | F16 | [4096, 4096]
model.layers.10.input_layernorm.weight -> blk.10.attn_norm.weight | F16 | [4096]
model.layers.10.mlp.down_proj.weight -> blk.10.ffn_down.weight | F16 | [4096, 11008]
model.layers.10.mlp.gate_proj.weight -> blk.10.ffn_gate.weight | F16 | [11008, 4096]
model.layers.10.mlp.up_proj.weight -> blk.10.ffn_up.weight | F16 | [11008, 4096]
model.layers.10.post_attention_layernorm.weight -> blk.10.ffn_norm.weight | F16 | [4096]
model.layers.10.self_attn.k_proj.weight -> blk.10.attn_k.weight | F16 | [4096, 4096]
model.layers.10.self_attn.o_proj.weight -> blk.10.attn_output.weight | F16 | [4096, 4096]
model.layers.10.self_attn.q_proj.weight -> blk.10.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.10.attn_rot_embd
model.layers.10.self_attn.v_proj.weight -> blk.10.attn_v.weight | F16 | [4096, 4096]
model.layers.11.input_layernorm.weight -> blk.11.attn_norm.weight | F16 | [4096]
model.layers.11.mlp.down_proj.weight -> blk.11.ffn_down.weight | F16 | [4096, 11008]
model.layers.11.mlp.gate_proj.weight -> blk.11.ffn_gate.weight | F16 | [11008, 4096]
model.layers.11.mlp.up_proj.weight -> blk.11.ffn_up.weight | F16 | [11008, 4096]
model.layers.11.post_attention_layernorm.weight -> blk.11.ffn_norm.weight | F16 | [4096]
model.layers.11.self_attn.k_proj.weight -> blk.11.attn_k.weight | F16 | [4096, 4096]
model.layers.11.self_attn.o_proj.weight -> blk.11.attn_output.weight | F16 | [4096, 4096]
model.layers.11.self_attn.q_proj.weight -> blk.11.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.11.attn_rot_embd
model.layers.11.self_attn.v_proj.weight -> blk.11.attn_v.weight | F16 | [4096, 4096]
model.layers.12.input_layernorm.weight -> blk.12.attn_norm.weight | F16 | [4096]
model.layers.12.mlp.down_proj.weight -> blk.12.ffn_down.weight | F16 | [4096, 11008]
model.layers.12.mlp.gate_proj.weight -> blk.12.ffn_gate.weight | F16 | [11008, 4096]
model.layers.12.mlp.up_proj.weight -> blk.12.ffn_up.weight | F16 | [11008, 4096]
model.layers.12.post_attention_layernorm.weight -> blk.12.ffn_norm.weight | F16 | [4096]
model.layers.12.self_attn.k_proj.weight -> blk.12.attn_k.weight | F16 | [4096, 4096]
model.layers.12.self_attn.o_proj.weight -> blk.12.attn_output.weight | F16 | [4096, 4096]
model.layers.12.self_attn.q_proj.weight -> blk.12.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.12.attn_rot_embd
model.layers.12.self_attn.v_proj.weight -> blk.12.attn_v.weight | F16 | [4096, 4096]
model.layers.13.input_layernorm.weight -> blk.13.attn_norm.weight | F16 | [4096]
model.layers.13.mlp.down_proj.weight -> blk.13.ffn_down.weight | F16 | [4096, 11008]
model.layers.13.mlp.gate_proj.weight -> blk.13.ffn_gate.weight | F16 | [11008, 4096]
model.layers.13.mlp.up_proj.weight -> blk.13.ffn_up.weight | F16 | [11008, 4096]
model.layers.13.post_attention_layernorm.weight -> blk.13.ffn_norm.weight | F16 | [4096]
model.layers.13.self_attn.k_proj.weight -> blk.13.attn_k.weight | F16 | [4096, 4096]
model.layers.13.self_attn.o_proj.weight -> blk.13.attn_output.weight | F16 | [4096, 4096]
model.layers.13.self_attn.q_proj.weight -> blk.13.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.13.attn_rot_embd
model.layers.13.self_attn.v_proj.weight -> blk.13.attn_v.weight | F16 | [4096, 4096]
model.layers.14.input_layernorm.weight -> blk.14.attn_norm.weight | F16 | [4096]
model.layers.14.mlp.down_proj.weight -> blk.14.ffn_down.weight | F16 | [4096, 11008]
model.layers.14.mlp.gate_proj.weight -> blk.14.ffn_gate.weight | F16 | [11008, 4096]
model.layers.14.mlp.up_proj.weight -> blk.14.ffn_up.weight | F16 | [11008, 4096]
model.layers.14.post_attention_layernorm.weight -> blk.14.ffn_norm.weight | F16 | [4096]
model.layers.14.self_attn.k_proj.weight -> blk.14.attn_k.weight | F16 | [4096, 4096]
model.layers.14.self_attn.o_proj.weight -> blk.14.attn_output.weight | F16 | [4096, 4096]
model.layers.14.self_attn.q_proj.weight -> blk.14.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.14.attn_rot_embd
model.layers.14.self_attn.v_proj.weight -> blk.14.attn_v.weight | F16 | [4096, 4096]
model.layers.15.input_layernorm.weight -> blk.15.attn_norm.weight | F16 | [4096]
model.layers.15.mlp.down_proj.weight -> blk.15.ffn_down.weight | F16 | [4096, 11008]
model.layers.15.mlp.gate_proj.weight -> blk.15.ffn_gate.weight | F16 | [11008, 4096]
model.layers.15.mlp.up_proj.weight -> blk.15.ffn_up.weight | F16 | [11008, 4096]
model.layers.15.post_attention_layernorm.weight -> blk.15.ffn_norm.weight | F16 | [4096]
model.layers.15.self_attn.k_proj.weight -> blk.15.attn_k.weight | F16 | [4096, 4096]
model.layers.15.self_attn.o_proj.weight -> blk.15.attn_output.weight | F16 | [4096, 4096]
model.layers.15.self_attn.q_proj.weight -> blk.15.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.15.attn_rot_embd
model.layers.15.self_attn.v_proj.weight -> blk.15.attn_v.weight | F16 | [4096, 4096]
model.layers.16.input_layernorm.weight -> blk.16.attn_norm.weight | F16 | [4096]
model.layers.16.mlp.down_proj.weight -> blk.16.ffn_down.weight | F16 | [4096, 11008]
model.layers.16.mlp.gate_proj.weight -> blk.16.ffn_gate.weight | F16 | [11008, 4096]
model.layers.16.mlp.up_proj.weight -> blk.16.ffn_up.weight | F16 | [11008, 4096]
model.layers.16.post_attention_layernorm.weight -> blk.16.ffn_norm.weight | F16 | [4096]
model.layers.16.self_attn.k_proj.weight -> blk.16.attn_k.weight | F16 | [4096, 4096]
model.layers.16.self_attn.o_proj.weight -> blk.16.attn_output.weight | F16 | [4096, 4096]
model.layers.16.self_attn.q_proj.weight -> blk.16.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.16.attn_rot_embd
model.layers.16.self_attn.v_proj.weight -> blk.16.attn_v.weight | F16 | [4096, 4096]
model.layers.17.input_layernorm.weight -> blk.17.attn_norm.weight | F16 | [4096]
model.layers.17.mlp.down_proj.weight -> blk.17.ffn_down.weight | F16 | [4096, 11008]
model.layers.17.mlp.gate_proj.weight -> blk.17.ffn_gate.weight | F16 | [11008, 4096]
model.layers.17.mlp.up_proj.weight -> blk.17.ffn_up.weight | F16 | [11008, 4096]
model.layers.17.post_attention_layernorm.weight -> blk.17.ffn_norm.weight | F16 | [4096]
model.layers.17.self_attn.k_proj.weight -> blk.17.attn_k.weight | F16 | [4096, 4096]
model.layers.17.self_attn.o_proj.weight -> blk.17.attn_output.weight | F16 | [4096, 4096]
model.layers.17.self_attn.q_proj.weight -> blk.17.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.17.attn_rot_embd
model.layers.17.self_attn.v_proj.weight -> blk.17.attn_v.weight | F16 | [4096, 4096]
model.layers.18.input_layernorm.weight -> blk.18.attn_norm.weight | F16 | [4096]
model.layers.18.mlp.down_proj.weight -> blk.18.ffn_down.weight | F16 | [4096, 11008]
model.layers.18.mlp.gate_proj.weight -> blk.18.ffn_gate.weight | F16 | [11008, 4096]
model.layers.18.mlp.up_proj.weight -> blk.18.ffn_up.weight | F16 | [11008, 4096]
model.layers.18.post_attention_layernorm.weight -> blk.18.ffn_norm.weight | F16 | [4096]
model.layers.18.self_attn.k_proj.weight -> blk.18.attn_k.weight | F16 | [4096, 4096]
model.layers.18.self_attn.o_proj.weight -> blk.18.attn_output.weight | F16 | [4096, 4096]
model.layers.18.self_attn.q_proj.weight -> blk.18.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.18.attn_rot_embd
model.layers.18.self_attn.v_proj.weight -> blk.18.attn_v.weight | F16 | [4096, 4096]
model.layers.19.input_layernorm.weight -> blk.19.attn_norm.weight | F16 | [4096]
model.layers.19.mlp.down_proj.weight -> blk.19.ffn_down.weight | F16 | [4096, 11008]
model.layers.19.mlp.gate_proj.weight -> blk.19.ffn_gate.weight | F16 | [11008, 4096]
model.layers.19.mlp.up_proj.weight -> blk.19.ffn_up.weight | F16 | [11008, 4096]
model.layers.19.post_attention_layernorm.weight -> blk.19.ffn_norm.weight | F16 | [4096]
model.layers.19.self_attn.k_proj.weight -> blk.19.attn_k.weight | F16 | [4096, 4096]
model.layers.19.self_attn.o_proj.weight -> blk.19.attn_output.weight | F16 | [4096, 4096]
model.layers.19.self_attn.q_proj.weight -> blk.19.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.19.attn_rot_embd
model.layers.19.self_attn.v_proj.weight -> blk.19.attn_v.weight | F16 | [4096, 4096]
model.layers.2.input_layernorm.weight -> blk.2.attn_norm.weight | F16 | [4096]
model.layers.2.mlp.down_proj.weight -> blk.2.ffn_down.weight | F16 | [4096, 11008]
model.layers.2.mlp.gate_proj.weight -> blk.2.ffn_gate.weight | F16 | [11008, 4096]
model.layers.2.mlp.up_proj.weight -> blk.2.ffn_up.weight | F16 | [11008, 4096]
model.layers.2.post_attention_layernorm.weight -> blk.2.ffn_norm.weight | F16 | [4096]
model.layers.2.self_attn.k_proj.weight -> blk.2.attn_k.weight | F16 | [4096, 4096]
model.layers.2.self_attn.o_proj.weight -> blk.2.attn_output.weight | F16 | [4096, 4096]
model.layers.2.self_attn.q_proj.weight -> blk.2.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.2.attn_rot_embd
model.layers.2.self_attn.v_proj.weight -> blk.2.attn_v.weight | F16 | [4096, 4096]
model.layers.20.input_layernorm.weight -> blk.20.attn_norm.weight | F16 | [4096]
model.layers.20.mlp.down_proj.weight -> blk.20.ffn_down.weight | F16 | [4096, 11008]
model.layers.20.mlp.gate_proj.weight -> blk.20.ffn_gate.weight | F16 | [11008, 4096]
model.layers.20.mlp.up_proj.weight -> blk.20.ffn_up.weight | F16 | [11008, 4096]
model.layers.20.post_attention_layernorm.weight -> blk.20.ffn_norm.weight | F16 | [4096]
model.layers.20.self_attn.k_proj.weight -> blk.20.attn_k.weight | F16 | [4096, 4096]
model.layers.20.self_attn.o_proj.weight -> blk.20.attn_output.weight | F16 | [4096, 4096]
model.layers.20.self_attn.q_proj.weight -> blk.20.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.20.attn_rot_embd
model.layers.20.self_attn.v_proj.weight -> blk.20.attn_v.weight | F16 | [4096, 4096]
model.layers.21.input_layernorm.weight -> blk.21.attn_norm.weight | F16 | [4096]
model.layers.21.mlp.down_proj.weight -> blk.21.ffn_down.weight | F16 | [4096, 11008]
model.layers.21.mlp.gate_proj.weight -> blk.21.ffn_gate.weight | F16 | [11008, 4096]
model.layers.21.mlp.up_proj.weight -> blk.21.ffn_up.weight | F16 | [11008, 4096]
model.layers.21.post_attention_layernorm.weight -> blk.21.ffn_norm.weight | F16 | [4096]
model.layers.21.self_attn.k_proj.weight -> blk.21.attn_k.weight | F16 | [4096, 4096]
model.layers.21.self_attn.o_proj.weight -> blk.21.attn_output.weight | F16 | [4096, 4096]
model.layers.21.self_attn.q_proj.weight -> blk.21.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.21.attn_rot_embd
model.layers.21.self_attn.v_proj.weight -> blk.21.attn_v.weight | F16 | [4096, 4096]
model.layers.22.input_layernorm.weight -> blk.22.attn_norm.weight | F16 | [4096]
model.layers.22.mlp.down_proj.weight -> blk.22.ffn_down.weight | F16 | [4096, 11008]
model.layers.22.mlp.gate_proj.weight -> blk.22.ffn_gate.weight | F16 | [11008, 4096]
model.layers.22.mlp.up_proj.weight -> blk.22.ffn_up.weight | F16 | [11008, 4096]
model.layers.22.post_attention_layernorm.weight -> blk.22.ffn_norm.weight | F16 | [4096]
model.layers.22.self_attn.k_proj.weight -> blk.22.attn_k.weight | F16 | [4096, 4096]
model.layers.22.self_attn.o_proj.weight -> blk.22.attn_output.weight | F16 | [4096, 4096]
model.layers.22.self_attn.q_proj.weight -> blk.22.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.22.attn_rot_embd
model.layers.22.self_attn.v_proj.weight -> blk.22.attn_v.weight | F16 | [4096, 4096]
model.layers.23.input_layernorm.weight -> blk.23.attn_norm.weight | F16 | [4096]
model.layers.23.mlp.down_proj.weight -> blk.23.ffn_down.weight | F16 | [4096, 11008]
model.layers.23.mlp.gate_proj.weight -> blk.23.ffn_gate.weight | F16 | [11008, 4096]
model.layers.23.mlp.up_proj.weight -> blk.23.ffn_up.weight | F16 | [11008, 4096]
model.layers.23.post_attention_layernorm.weight -> blk.23.ffn_norm.weight | F16 | [4096]
model.layers.23.self_attn.k_proj.weight -> blk.23.attn_k.weight | F16 | [4096, 4096]
model.layers.23.self_attn.o_proj.weight -> blk.23.attn_output.weight | F16 | [4096, 4096]
model.layers.23.self_attn.q_proj.weight -> blk.23.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.23.attn_rot_embd
model.layers.23.self_attn.v_proj.weight -> blk.23.attn_v.weight | F16 | [4096, 4096]
model.layers.3.input_layernorm.weight -> blk.3.attn_norm.weight | F16 | [4096]
model.layers.3.mlp.down_proj.weight -> blk.3.ffn_down.weight | F16 | [4096, 11008]
model.layers.3.mlp.gate_proj.weight -> blk.3.ffn_gate.weight | F16 | [11008, 4096]
model.layers.3.mlp.up_proj.weight -> blk.3.ffn_up.weight | F16 | [11008, 4096]
model.layers.3.post_attention_layernorm.weight -> blk.3.ffn_norm.weight | F16 | [4096]
model.layers.3.self_attn.k_proj.weight -> blk.3.attn_k.weight | F16 | [4096, 4096]
model.layers.3.self_attn.o_proj.weight -> blk.3.attn_output.weight | F16 | [4096, 4096]
model.layers.3.self_attn.q_proj.weight -> blk.3.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.3.attn_rot_embd
model.layers.3.self_attn.v_proj.weight -> blk.3.attn_v.weight | F16 | [4096, 4096]
model.layers.4.input_layernorm.weight -> blk.4.attn_norm.weight | F16 | [4096]
model.layers.4.mlp.down_proj.weight -> blk.4.ffn_down.weight | F16 | [4096, 11008]
model.layers.4.mlp.gate_proj.weight -> blk.4.ffn_gate.weight | F16 | [11008, 4096]
model.layers.4.mlp.up_proj.weight -> blk.4.ffn_up.weight | F16 | [11008, 4096]
model.layers.4.post_attention_layernorm.weight -> blk.4.ffn_norm.weight | F16 | [4096]
model.layers.4.self_attn.k_proj.weight -> blk.4.attn_k.weight | F16 | [4096, 4096]
model.layers.4.self_attn.o_proj.weight -> blk.4.attn_output.weight | F16 | [4096, 4096]
model.layers.4.self_attn.q_proj.weight -> blk.4.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.4.attn_rot_embd
model.layers.4.self_attn.v_proj.weight -> blk.4.attn_v.weight | F16 | [4096, 4096]
model.layers.5.input_layernorm.weight -> blk.5.attn_norm.weight | F16 | [4096]
model.layers.5.mlp.down_proj.weight -> blk.5.ffn_down.weight | F16 | [4096, 11008]
model.layers.5.mlp.gate_proj.weight -> blk.5.ffn_gate.weight | F16 | [11008, 4096]
model.layers.5.mlp.up_proj.weight -> blk.5.ffn_up.weight | F16 | [11008, 4096]
model.layers.5.post_attention_layernorm.weight -> blk.5.ffn_norm.weight | F16 | [4096]
model.layers.5.self_attn.k_proj.weight -> blk.5.attn_k.weight | F16 | [4096, 4096]
model.layers.5.self_attn.o_proj.weight -> blk.5.attn_output.weight | F16 | [4096, 4096]
model.layers.5.self_attn.q_proj.weight -> blk.5.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.5.attn_rot_embd
model.layers.5.self_attn.v_proj.weight -> blk.5.attn_v.weight | F16 | [4096, 4096]
model.layers.6.input_layernorm.weight -> blk.6.attn_norm.weight | F16 | [4096]
model.layers.6.mlp.down_proj.weight -> blk.6.ffn_down.weight | F16 | [4096, 11008]
model.layers.6.mlp.gate_proj.weight -> blk.6.ffn_gate.weight | F16 | [11008, 4096]
model.layers.6.mlp.up_proj.weight -> blk.6.ffn_up.weight | F16 | [11008, 4096]
model.layers.6.post_attention_layernorm.weight -> blk.6.ffn_norm.weight | F16 | [4096]
model.layers.6.self_attn.k_proj.weight -> blk.6.attn_k.weight | F16 | [4096, 4096]
model.layers.6.self_attn.o_proj.weight -> blk.6.attn_output.weight | F16 | [4096, 4096]
model.layers.6.self_attn.q_proj.weight -> blk.6.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.6.attn_rot_embd
model.layers.6.self_attn.v_proj.weight -> blk.6.attn_v.weight | F16 | [4096, 4096]
model.layers.7.input_layernorm.weight -> blk.7.attn_norm.weight | F16 | [4096]
model.layers.7.mlp.down_proj.weight -> blk.7.ffn_down.weight | F16 | [4096, 11008]
model.layers.7.mlp.gate_proj.weight -> blk.7.ffn_gate.weight | F16 | [11008, 4096]
model.layers.7.mlp.up_proj.weight -> blk.7.ffn_up.weight | F16 | [11008, 4096]
model.layers.7.post_attention_layernorm.weight -> blk.7.ffn_norm.weight | F16 | [4096]
model.layers.7.self_attn.k_proj.weight -> blk.7.attn_k.weight | F16 | [4096, 4096]
model.layers.7.self_attn.o_proj.weight -> blk.7.attn_output.weight | F16 | [4096, 4096]
model.layers.7.self_attn.q_proj.weight -> blk.7.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.7.attn_rot_embd
model.layers.7.self_attn.v_proj.weight -> blk.7.attn_v.weight | F16 | [4096, 4096]
model.layers.8.input_layernorm.weight -> blk.8.attn_norm.weight | F16 | [4096]
model.layers.8.mlp.down_proj.weight -> blk.8.ffn_down.weight | F16 | [4096, 11008]
model.layers.8.mlp.gate_proj.weight -> blk.8.ffn_gate.weight | F16 | [11008, 4096]
model.layers.8.mlp.up_proj.weight -> blk.8.ffn_up.weight | F16 | [11008, 4096]
model.layers.8.post_attention_layernorm.weight -> blk.8.ffn_norm.weight | F16 | [4096]
model.layers.8.self_attn.k_proj.weight -> blk.8.attn_k.weight | F16 | [4096, 4096]
model.layers.8.self_attn.o_proj.weight -> blk.8.attn_output.weight | F16 | [4096, 4096]
model.layers.8.self_attn.q_proj.weight -> blk.8.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.8.attn_rot_embd
model.layers.8.self_attn.v_proj.weight -> blk.8.attn_v.weight | F16 | [4096, 4096]
model.layers.9.input_layernorm.weight -> blk.9.attn_norm.weight | F16 | [4096]
model.layers.9.mlp.down_proj.weight -> blk.9.ffn_down.weight | F16 | [4096, 11008]
model.layers.9.mlp.gate_proj.weight -> blk.9.ffn_gate.weight | F16 | [11008, 4096]
model.layers.9.mlp.up_proj.weight -> blk.9.ffn_up.weight | F16 | [11008, 4096]
model.layers.9.post_attention_layernorm.weight -> blk.9.ffn_norm.weight | F16 | [4096]
model.layers.9.self_attn.k_proj.weight -> blk.9.attn_k.weight | F16 | [4096, 4096]
model.layers.9.self_attn.o_proj.weight -> blk.9.attn_output.weight | F16 | [4096, 4096]
model.layers.9.self_attn.q_proj.weight -> blk.9.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.9.attn_rot_embd
model.layers.9.self_attn.v_proj.weight -> blk.9.attn_v.weight | F16 | [4096, 4096]
lm_head.weight -> output.weight | F16 | [32000, 4096]
model.layers.24.input_layernorm.weight -> blk.24.attn_norm.weight | F16 | [4096]
model.layers.24.mlp.down_proj.weight -> blk.24.ffn_down.weight | F16 | [4096, 11008]
model.layers.24.mlp.gate_proj.weight -> blk.24.ffn_gate.weight | F16 | [11008, 4096]
model.layers.24.mlp.up_proj.weight -> blk.24.ffn_up.weight | F16 | [11008, 4096]
model.layers.24.post_attention_layernorm.weight -> blk.24.ffn_norm.weight | F16 | [4096]
model.layers.24.self_attn.k_proj.weight -> blk.24.attn_k.weight | F16 | [4096, 4096]
model.layers.24.self_attn.o_proj.weight -> blk.24.attn_output.weight | F16 | [4096, 4096]
model.layers.24.self_attn.q_proj.weight -> blk.24.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.24.attn_rot_embd
model.layers.24.self_attn.v_proj.weight -> blk.24.attn_v.weight | F16 | [4096, 4096]
model.layers.25.input_layernorm.weight -> blk.25.attn_norm.weight | F16 | [4096]
model.layers.25.mlp.down_proj.weight -> blk.25.ffn_down.weight | F16 | [4096, 11008]
model.layers.25.mlp.gate_proj.weight -> blk.25.ffn_gate.weight | F16 | [11008, 4096]
model.layers.25.mlp.up_proj.weight -> blk.25.ffn_up.weight | F16 | [11008, 4096]
model.layers.25.post_attention_layernorm.weight -> blk.25.ffn_norm.weight | F16 | [4096]
model.layers.25.self_attn.k_proj.weight -> blk.25.attn_k.weight | F16 | [4096, 4096]
model.layers.25.self_attn.o_proj.weight -> blk.25.attn_output.weight | F16 | [4096, 4096]
model.layers.25.self_attn.q_proj.weight -> blk.25.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.25.attn_rot_embd
model.layers.25.self_attn.v_proj.weight -> blk.25.attn_v.weight | F16 | [4096, 4096]
model.layers.26.input_layernorm.weight -> blk.26.attn_norm.weight | F16 | [4096]
model.layers.26.mlp.down_proj.weight -> blk.26.ffn_down.weight | F16 | [4096, 11008]
model.layers.26.mlp.gate_proj.weight -> blk.26.ffn_gate.weight | F16 | [11008, 4096]
model.layers.26.mlp.up_proj.weight -> blk.26.ffn_up.weight | F16 | [11008, 4096]
model.layers.26.post_attention_layernorm.weight -> blk.26.ffn_norm.weight | F16 | [4096]
model.layers.26.self_attn.k_proj.weight -> blk.26.attn_k.weight | F16 | [4096, 4096]
model.layers.26.self_attn.o_proj.weight -> blk.26.attn_output.weight | F16 | [4096, 4096]
model.layers.26.self_attn.q_proj.weight -> blk.26.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.26.attn_rot_embd
model.layers.26.self_attn.v_proj.weight -> blk.26.attn_v.weight | F16 | [4096, 4096]
model.layers.27.input_layernorm.weight -> blk.27.attn_norm.weight | F16 | [4096]
model.layers.27.mlp.down_proj.weight -> blk.27.ffn_down.weight | F16 | [4096, 11008]
model.layers.27.mlp.gate_proj.weight -> blk.27.ffn_gate.weight | F16 | [11008, 4096]
model.layers.27.mlp.up_proj.weight -> blk.27.ffn_up.weight | F16 | [11008, 4096]
model.layers.27.post_attention_layernorm.weight -> blk.27.ffn_norm.weight | F16 | [4096]
model.layers.27.self_attn.k_proj.weight -> blk.27.attn_k.weight | F16 | [4096, 4096]
model.layers.27.self_attn.o_proj.weight -> blk.27.attn_output.weight | F16 | [4096, 4096]
model.layers.27.self_attn.q_proj.weight -> blk.27.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.27.attn_rot_embd
model.layers.27.self_attn.v_proj.weight -> blk.27.attn_v.weight | F16 | [4096, 4096]
model.layers.28.input_layernorm.weight -> blk.28.attn_norm.weight | F16 | [4096]
model.layers.28.mlp.down_proj.weight -> blk.28.ffn_down.weight | F16 | [4096, 11008]
model.layers.28.mlp.gate_proj.weight -> blk.28.ffn_gate.weight | F16 | [11008, 4096]
model.layers.28.mlp.up_proj.weight -> blk.28.ffn_up.weight | F16 | [11008, 4096]
model.layers.28.post_attention_layernorm.weight -> blk.28.ffn_norm.weight | F16 | [4096]
model.layers.28.self_attn.k_proj.weight -> blk.28.attn_k.weight | F16 | [4096, 4096]
model.layers.28.self_attn.o_proj.weight -> blk.28.attn_output.weight | F16 | [4096, 4096]
model.layers.28.self_attn.q_proj.weight -> blk.28.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.28.attn_rot_embd
model.layers.28.self_attn.v_proj.weight -> blk.28.attn_v.weight | F16 | [4096, 4096]
model.layers.29.input_layernorm.weight -> blk.29.attn_norm.weight | F16 | [4096]
model.layers.29.mlp.down_proj.weight -> blk.29.ffn_down.weight | F16 | [4096, 11008]
model.layers.29.mlp.gate_proj.weight -> blk.29.ffn_gate.weight | F16 | [11008, 4096]
model.layers.29.mlp.up_proj.weight -> blk.29.ffn_up.weight | F16 | [11008, 4096]
model.layers.29.post_attention_layernorm.weight -> blk.29.ffn_norm.weight | F16 | [4096]
model.layers.29.self_attn.k_proj.weight -> blk.29.attn_k.weight | F16 | [4096, 4096]
model.layers.29.self_attn.o_proj.weight -> blk.29.attn_output.weight | F16 | [4096, 4096]
model.layers.29.self_attn.q_proj.weight -> blk.29.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.29.attn_rot_embd
model.layers.29.self_attn.v_proj.weight -> blk.29.attn_v.weight | F16 | [4096, 4096]
model.layers.30.input_layernorm.weight -> blk.30.attn_norm.weight | F16 | [4096]
model.layers.30.mlp.down_proj.weight -> blk.30.ffn_down.weight | F16 | [4096, 11008]
model.layers.30.mlp.gate_proj.weight -> blk.30.ffn_gate.weight | F16 | [11008, 4096]
model.layers.30.mlp.up_proj.weight -> blk.30.ffn_up.weight | F16 | [11008, 4096]
model.layers.30.post_attention_layernorm.weight -> blk.30.ffn_norm.weight | F16 | [4096]
model.layers.30.self_attn.k_proj.weight -> blk.30.attn_k.weight | F16 | [4096, 4096]
model.layers.30.self_attn.o_proj.weight -> blk.30.attn_output.weight | F16 | [4096, 4096]
model.layers.30.self_attn.q_proj.weight -> blk.30.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.30.attn_rot_embd
model.layers.30.self_attn.v_proj.weight -> blk.30.attn_v.weight | F16 | [4096, 4096]
model.layers.31.input_layernorm.weight -> blk.31.attn_norm.weight | F16 | [4096]
model.layers.31.mlp.down_proj.weight -> blk.31.ffn_down.weight | F16 | [4096, 11008]
model.layers.31.mlp.gate_proj.weight -> blk.31.ffn_gate.weight | F16 | [11008, 4096]
model.layers.31.mlp.up_proj.weight -> blk.31.ffn_up.weight | F16 | [11008, 4096]
model.layers.31.post_attention_layernorm.weight -> blk.31.ffn_norm.weight | F16 | [4096]
model.layers.31.self_attn.k_proj.weight -> blk.31.attn_k.weight | F16 | [4096, 4096]
model.layers.31.self_attn.o_proj.weight -> blk.31.attn_output.weight | F16 | [4096, 4096]
model.layers.31.self_attn.q_proj.weight -> blk.31.attn_q.weight | F16 | [4096, 4096]
skipping tensor blk.31.attn_rot_embd
model.layers.31.self_attn.v_proj.weight -> blk.31.attn_v.weight | F16 | [4096, 4096]
model.norm.weight -> output_norm.weight | F16 | [4096]
Traceback (most recent call last):
File "/content/convert.py", line 1193, in
main()
File "/content/convert.py", line 1181, in main
ftype = pick_output_type(model, args.outtype)
File "/content/convert.py", line 941, in pick_output_type
wq_type = model[gguf.TENSOR_NAMES[gguf.MODEL_TENSOR.ATTN_Q].format(bid=0)+".weight"].data_type
AttributeError: module 'gguf' has no attribute 'TENSOR_NAMES'

[KerfuffleV2]

I'd guess this is probably a mismatch between the script and the install GGUF version. The scripts also have some special logic to check if there's a gguf module in gguf-py/gguf and will use that if it's there. That's to make it so if you check out the repo and run the scripts from the repo directory, you'll automatically import the gguf from the repo. If you're running the script with your CWD set to something else, then it'll just use whatever gguf version is installed.

[cebtenzzre]

See #3528. For now, you can either run convert.py from within the llama.cpp repo, or install the latest gguf package with pip install git+https://github.com/ggerganov/llama.cpp.git#subdirectory=gguf-py.

edit: gguf 0.4.0 has been released on pypi, you can now simply update it.