Add constant bitrate #161
Replies: 3 comments
-
Another person also suggested a minimum bitrate setting too and I could see the benefits. |
Beta Was this translation helpful? Give feedback.
-
My main worry with this is that it's going to create a flood of models with all different kinds of settings, defeating the purpose of having a single parameter that sets a target average bitrate and then letting the quantizer figure out where the budget is best spent. Dataset bias is inevitable, because like GPTQ, EXL2 inherently relies on calibration. Setting a fixed bitrate not only negates one of the main strengths of the format but wouldn't actually remove the bias since the quantization would still be error-corrected, and the error is measured by doing inference on the calibration dataset. If you want a completely unbiased format, that's just RTN quantization. |
Beta Was this translation helpful? Give feedback.
-
"since the quantization would still be error-corrected, and the error is measured by doing inference on the calibration dataset." |
Beta Was this translation helpful? Give feedback.
-
There should be a constant bitrate variable option that would bypass the need for measuring, and that would allow for setting all layers to the same one desired size. I think it'd make sense for testing purposes and bypassing any dataset biases that might be there.
Beta Was this translation helpful? Give feedback.
All reactions