What are the hyperparameters for finetuning on cifar10/100, and other small datasets

[Phuoc-Hoan-Le]

Cannot find it in the paper? Is it the same as training on imagenet?

[TouvronHugo]

Hi @CharlesLeeeee,
Thanks for your message.
No the hparams are not the same (see here for more details).
Best,
Hugo

[kevinkasa]

Hi @TouvronHugo, I noticed that those hyperparameters are from 2021, which I believe is pre DeiT III. I was wondering if you use the same finetuning hyperparameters for DeiT III, or if something else is used? Thank you!