cannot reproduce results on PACS

[Johnzhangt]

I run your code on PACS to reproduce your results in Table 6. I run three times, the results are:

Best val 0.965909, corresponding test 0.68847 - best test: 0.787732, best epoch: 18

Best val 0.967532, corresponding test 0.771698 - best test: 0.782642, best epoch: 25

Best val 0.962662, corresponding test 0.750573 - best test: 0.780606, best epoch: 17

The average result is 73.69, which is significantly lower than your results (80.85) in Table 6 for Res18.

I guess I may be wrong somewhere. How to reproduce results?

My environment is:

pytorch=1.1.0, torchvision=0.3.0.

[Johnzhangt]

If I do not use ImageNet pretrain model, I run the code for three times, the results are:

Best val 0.683442, corresponding test 0.443879 - best test: 0.471367, best epoch: 26

Best val 0.683442, corresponding test 0.370832 - best test: 0.404174, best epoch: 28

Best val 0.678571, corresponding test 0.382795 - best test: 0.484856, best epoch: 28

I am looking forward to your reply.

[Justinhzy]

Hi, could you run the code using my latest environment? please see #2
If you want to train your network without pretrained model, you may need other techniques which are not in the scope of this paper.