-
Notifications
You must be signed in to change notification settings - Fork 26.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dpt model tests regress after PR #33403 #33649
Comments
Both are failing due to SDPA not supporting the output of the attention weights. Afair, in text based models there's a fallback to eager attention (with a warning) if the output of attentions is requested (or other unsupported operations). Can be fixed in two ways:
Depends on preferences ig, although in the future the base test should be adjusted instead imo. |
@dvrogozh why is dpt affected by my changes to dinov2? |
@avishaiElmakies It's a special case where dpt uses dinov2 as a backbone, see for example: transformers/tests/models/dpt/test_modeling_dpt_auto_backbone.py Lines 101 to 113 in 78b2929
Hence, the hidden failures as the dependency wasn't obvious. |
The failures can be seen in the main ci too, see https://app.circleci.com/pipelines/github/huggingface/transformers/104875/workflows/52541e47-4524-4c08-8d54-b6e800e3d7a7/jobs/1394847 |
@vasqu should I fix that? |
If you have time, gladly :) @avishaiElmakies |
I feel like it is kinda my fault so I will try to handle it @vasqu |
Folllow up from #33485 (comment). On 78b2929, there is a regression after merging this PR:
On these 2 tests:
tests/models/dpt/test_modeling_dpt_auto_backbone.py::DPTModelTest::test_attention_outputs
tests/models/dpt/test_modeling_dpt_auto_backbone.py::DPTModelTest::test_retain_grad_hidden_states_attentions
CC: @avishaiElmakies, @amyeroberts
Example output:
Bisect:
The text was updated successfully, but these errors were encountered: