Reproduction of the result.

[IzhiSu]

Dear authors,

I am not able to reproduce the results reported in the paper using the given config at config/mbptrack_kitti_ped_cfg.yaml.

I wonder if the same config used for generating the results in the paper is provided. Or is there anything else we can take note of the reproduce the results?

Thanks!

[slothfulxtx]

Hi,
I've uploaded all my code and checkpoints on github, thus it should be easy to reproduce our results. I've just tested my code and checkpoint on our server

python main.py configs/mbptrack_kitti_ped_cfg.yaml --phase test --resume_from pretrained/mbptrack_kitti_ped.ckpt

The results are listed below

Notably, this server is also occupied by my colleagues for other tasks, thus the runtime metric is not accurate.

[IzhiSu]

Thanks for your reply! When I reproduced the category such as ’Pedestrian‘, the experimental results were 66.25/91.10. Could you please provide me with specific configurations to help me reproduce this result? Looking forward to your reply!

[slothfulxtx]

Actually, MBPTrack reuses previous prediction results when the tracked target is missing. If the prediction of MBPTrack is not accurate enough, it may lose targets for the following frames. (You can find some tracklets of extremely low quality in KITTI.) Thus the performance of MBPTrack is not very robust and has a minor fluctuation. I suggest you can train our model from scratch to reproduce the performance.

[ujaejoon]

@IzhiSu It was the same when I trained the model from scratch. I tested the weight of first best_epoch=~~~~.ckpt in the checkpoints folder and the test results on KITTI pedestrian were 66.192/91.008(Success/Precision).

@slothfulxtx Can you tell us how many GPUs you use for training and which checkpoint you used for testing?