Could you please provide the additional metrics in Table 1?

[sunnyHelen]

Thanks for your great work. I am also exploring this topic and want to cite your work. I need the additional metrics in Table 1, including mATE, mASE, mAOE, mAVE and mAAE of your method and the reproduced ones. Could you please provide them?

[zehuichen123]

Thanks for your attention to our work. I am sorry that we only keep the logs for our own models. Here are our detailed metrics in Table 1:
BEVFormer
pts_bbox_NuScenes/mATE: 0.6931, pts_bbox_NuScenes/mASE: 0.2640, pts_bbox_NuScenes/mAOE: 0.3995, pts_bbox_NuScenes/mAVE: 0.8022, pts_bbox_NuScenes/mAAE: 0.1992, pts_bbox_NuScenes/NDS: 0.4574, pts_bbox_NuScenes/mAP: 0.3863
BEVFormer-T
pts_bbox_NuScenes/mATE: 0.6633, pts_bbox_NuScenes/mASE: 0.2683, pts_bbox_NuScenes/mAOE: 0.3930, pts_bbox_NuScenes/mAVE: 0.3742, pts_bbox_NuScenes/mAAE: 0.1839, pts_bbox_NuScenes/NDS: 0.5153, pts_bbox_NuScenes/mAP: 0.4072

[sunnyHelen]

Thanks a lot~

[sunnyHelen]

One more question~
Are the experiments in Table 1 implemented under the same setting as yours, in terms of backbone, image size, and with or without CBGS?

[zehuichen123]

Sorry for my late reply. Yes, they are conducted under the same experimental setting.

[sunnyHelen]

Hi, do you have the result about the model efficiency, like FPS and Flops?

[zehuichen123]

We haven't measured that. We plan to measure it in our journal version paper :)

[sujinjang]

Hi, in-line with the above question, could you please provide additional metrics in Table 8, including mATE, mASE, mAOE, mAVE and mAAE for your method (ResNet-101DCN w/ BEVFormer + BEVDistill and BEVFormer-T + BEVDistill) ? Thanks much in advance!

[zehuichen123]

Hi. The logs are missing due to the recycling of the machines from my company:( I am not sure if I can re-access the file system once the machines are ready. But I will rerun the experiments once the code and machines are ready.

[zehuichen123]

I just found an experiment record for R101 single frame from my chat history. Here is the results, pts_bbox_NuScenes/NDS: 0.4674, pts_bbox_NuScenes/mAP: 0.3888, pts_bbox_NuScenes/mATE: 0.6858, pts_bbox_NuScenes/mASE: 0.2626, pts_bbox_NuScenes/mAOE: 0.3873, pts_bbox_NuScenes/mAVE: 0.7435, pts_bbox_NuScenes/mAAE: 0.1912.

[sanmin0312]

Hello,

I have a question regarding Table 1. It appears that the performance of the student model differs from the values reported in the original paper.

According to the information provided in the BEVFormer repository (https://github.com/fundamentalvision/BEVFormer), the reported performance (ResNet-50 backbone) is 35.9 for mAP and 25.7 for NDS. However, in Table 1, the values are listed as 42.3 for mAP and 35.2 for NDS.

Could you kindly explain the source of this discrepancy? I appreciate your assistance in clarifying this matter.

Thank you.