Colipali - ready for review. #423
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
This pull request introduces significant changes to the Infinity Embedding Inference Server, focusing on expanding image processing capabilities and improving model handling.
- Added support for new image collection models (ColPali, ColQwen2, ColIdefics2) in
/libs/infinity_emb/infinity_emb/transformer/vision/__init__.py - Replaced
ClipLikeModelwithTorchImageModelin/libs/infinity_emb/infinity_emb/transformer/vision/torch_vision.py, adding support for Colipali models - Updated OpenAPI specification in
/docs/assets/openapi.jsonto reflect changes in embedding object handling - Introduced 'colpali-engine' as a new optional dependency in
/libs/infinity_emb/pyproject.toml - Modified test files to accommodate new image classification models and improve test coverage
16 file(s) reviewed, 27 comment(s)
Edit PR Review Bot Settings | Greptile
libs/infinity_emb/tests/unit_test/transformer/vision/test_torch_vision.py
Show resolved
Hide resolved
libs/infinity_emb/tests/unit_test/transformer/vision/test_torch_vision.py
Outdated
Show resolved
Hide resolved
| if __name__ == "__main__": | ||
| test_colpali("int8") | ||
| test_colpali("auto") | ||
| test_clip_like_model() |
There was a problem hiding this comment.
style: Consider removing main block or using pytest.main()
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #423 +/- ##
==========================================
+ Coverage 79.01% 79.15% +0.14%
==========================================
Files 40 41 +1
Lines 3173 3228 +55
==========================================
+ Hits 2507 2555 +48
- Misses 666 673 +7 ☔ View full report in Codecov by Sentry. |
|
|
||
|
|
||
| class ClipLikeModel(BaseClipVisionModel): | ||
| class TorchImageModel(BaseClipVisionModel): |
There was a problem hiding this comment.
For consistency, wouldn't make sense to also rename the base class here?
|
Are there any tutorials on how to expose Colpali as a microservice? |
|
No, but you can deploy it via docker. i would suggest you to get started with https://huggingface.co/michaelfeil/colpali-v12-random-testing (small one for testing the Api -enpoint) The following command after pip install and via docker works. |
No description provided.