how to train the classifier model

[lidc1004]

Hello @Gengzigang
I tried using lightweight backbone to train the tokenizer and classifier ,but i get a bad result.Is there something i missed?

The result of tokenizer model:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.965
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.990

The result of classifier model:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.365
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.733

I changed the backbone and optimizer for config of classifier model :
optimizer = dict(type='AdamW', lr=8e-4, betas=(0.9, 0.999), weight_decay=0.05)

When i start train the classifier model ,the ap is 0.005 ->0.007. SO how to train the classifier model?

Thank you for taking the time to answer.