[OWLv2] Add method #27003
[OWLv2] Add method #27003
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
NielsRogge
force-pushed
the
fix_owlv2_image_guided
branch
from
26dfef1
to
950c82f
Compare
There was a problem hiding this comment.
Thanks for adding this!
A few notes:
I'm slightly unclear on the addition of this method. I can see a difference in the two approaches however this would benefit from more detail in the PR and code i.e. how does this relate to the original model? In particular, details should be added to docstrings.
It would be better to have a single image_guided_detection method which can be configured with flags. If necessary, we can break this down into private methods e.g. _image_guided_detection_objectness that are called. xxx_v2 isn't a descriptive method name.
We shouldn't commit notebooks to this repo.
There was a problem hiding this comment.
We shouldn't be uploading notebooks to this repo
| output_hidden_states: Optional[bool] = None, | ||
| return_dict: Optional[bool] = None, | ||
| ) -> Owlv2ImageGuidedObjectDetectionOutput: | ||
| r""" |
There was a problem hiding this comment.
The docstring should detail what the method does and how it differs from image_guided_detection
| query_pred_boxes = self.box_predictor(query_image_feats, feature_map=query_feature_map) | ||
| query_class_embeddings = self.class_predictor(query_image_feats)[1] | ||
|
|
||
| # v2 differs from v1 in that we use the objectness head to predict the objectness of the patches of the query image |
There was a problem hiding this comment.
Is this the naming that the model uses i.e. does the paper use "image_guided_detection_v1", "image_guided_detection_v2"?
As so much of this logic is shared - couldn't we control this with a flag in the original image_guided_detection method?
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
What does this PR do?
This PR adds a new
image_guided_detection_v2method, which unlike the original method, leverages the objectness head to get the top predicted object in the query image.It also fixes the documentation example which has bad threshold values.
Fixes #26920