TF_Deberta supporting mixed precision

[pinesnow72]

What does this PR do?

This PR fixes some codes of TF DeBERTa (version 1 and 2) that do not support Mixed Precision.

With TF and Keras, mixed precision is configured with the following settings:

policy = keras.mixed_precision.Policy('mixed_float16')
keras.mixed_precision.set_global_policy(policy)

However, the existing TFDebertaModel and TFDebertaV2Model throws TypeError with this mixed precision setting.
With mixed precision, TF and Keras requires to use Layer.dtype for model or layer's weights and Layer.compute_dtype for internal tensor computation. But the current TFDebertaModel and TFDebertaV2Model codes do not seem to reflect this requirement and definitely assume the dtype would be tf.float32.

I searched and corrected some error-prone code snippets in modeling_tf_deberta.py and modeling_tf_deberta_v2.py.
This correction worked correctly with mixed precision in my local machine.

This PR fixes #31989

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [V] Did you read the contributor guideline,
  Pull Request section?
• [V] Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@Rocketknight1, @ArthurZucker

[Rocketknight1]

Hi @pinesnow72, this looks good! Feel free to ping me whenever it's ready for review. The issues with code quality can be fixed by doing pip install transformers[quality] followed by make style in the repo directory.

[pinesnow72]

Hi @pinesnow72, this looks good! Feel free to ping me whenever it's ready for review. The issues with code quality can be fixed by doing pip install transformers[quality] followed by make style in the repo directory.

Hi @Rocketknight1, As your comment, the style was corrected and uploaded. Thanks.

[Rocketknight1]

LGTM now! cc @amyeroberts for core maintainer review, and thanks for the PR!

[amyeroberts]

Thanks for opening a PR to handle this!

Just a small question

[amyeroberts]

What's the difference between self.dtype and self.compute_dtype ?

[Rocketknight1]

self.dtype is the dtype the layer's weights are stored in, self.compute_dtype is the dtype used for computation. By default these are the same, but in mixed-precision it's common for self.dtype to be float32 while self.compute_dtype is (b)float16

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[amyeroberts]

Thanks for making mixed precision possible for this model!

[pinesnow72]

Thanks for accepting this PR