[WIP] Support BLIP and GIT in image-to-text and VQA pipelines #21227
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
| @@ -778,7 +778,7 @@ | |||
| _import_structure["image_utils"] = ["ImageFeatureExtractionMixin"] | |||
| _import_structure["models.beit"].extend(["BeitFeatureExtractor", "BeitImageProcessor"]) | |||
| _import_structure["models.bit"].extend(["BitImageProcessor"]) | |||
| _import_structure["models.blip"].extend(["BlipImageProcessor"]) | |||
| _import_structure["models.blip"].extend(["BlipFeatureExtractor", "BlipImageProcessor"]) | |||
There was a problem hiding this comment.
No need to add the feature extractor, you can use the image processor for the pipeline.
There was a problem hiding this comment.
Yes, but the pipeline still builds with a feature_extractor(which is actually a blipimageprocessor)?
There was a problem hiding this comment.
Now not anymore ;) please rebase on main
|
Hi @NielsRogge, should I remove the return of the topk scores in the VQA pipeline that used ViltForQuestionAnswering only? |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
What does this PR do?
Support BLIP and GIT models in image-to-text and VQA pipelines.
Fixes #21110
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@NielsRogge