Embedding quantization

[michaelfeil]

Feature request

Quantization against:

• https://huggingface.co/blog/embedding-quantization

Motivation

Your contribution

Looking for contributors to fill the blank of embedding quanitzation.

@cache 
def quantize_embedding_stats(model: str, dataset: some_dataset.parquet) -> statistics[min,max,median,other]:
      ...
      
def quanize_embeddings()
     quantize_embedding_stats()

[mahiro72]

Hello @michaelfeil , I'm interested in working on this issue. Can I give it a try?

[michaelfeil]

Actually started to implement a rough sketch! But then i continued working on it, and it might have solced it entirely. Want to give it a try? @mahiro72

[michaelfeil]

#247, Completed as a "v1"

[michaelfeil]

@mahiro72 moved it to here: #250

[mahiro72]

@michaelfeil
Thank you! I would like to work on this issue to verify the performance.
#250