Not able to use FP16 in pytorch-pretrained-BERT. Getting error **Runtime error: Expected scalar type object Half but got scalar type Float for argument #2 target**

[Ashish-Gupta03]

I'm not able to work with FP16 for pytorch BERT code. Particularly for BertForSequenceClassification, which I tried and got the issue
Runtime error: Expected scalar type object Half but got scalar type Float for argument #2 target
when I enabled fp16.
Also when using
logits = logits.half() labels = labels.half()
then the epoch time also increased.

The training time without fp16 was 2.5 hrs per epoch after doing logits.half() and labels.half() the runtime per epoch shot up to 8hrs.

[thomwolf]

Which kind of GPU are you using? fp16 only works on recent GPU (better with Tesla and Volta series).

[tholor]

I experienced a similar issue with CUDA 9.1. Using 9.2 solved this for me.

[thomwolf]

Yes, CUDA 10 is recommended for using fp16 with good performances.