Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Training fails to achieve reported performance #5

Open
caelumchz opened this issue Nov 24, 2023 · 2 comments
Open

New Training fails to achieve reported performance #5

caelumchz opened this issue Nov 24, 2023 · 2 comments

Comments

@caelumchz
Copy link

caelumchz commented Nov 24, 2023

Hello, this work is full of inspiration, and we want to follow up this work and give stars. However, there's something wrong with my own train process. That is to say when I start a new training on Surreal with default paramaters and without using pretrained model, I cannot achieve the performance in the paper(5% lower). Is that because my hyperparamaters are incorrect or something else? And I wonder if the rotated_gt_2 is incorrect since it supervises the orient_2 which is the angle between original target pc and augmented tgt in student model, but seems to be designed to calculate the angle between two rotated point clouds?
微信截图_20231124120657

@2019EPWL
Copy link

Hello, this work is full of inspiration, and we want to follow up this work and give stars. However, there's something wrong with my own train process. That is to say when I start a new training on Surreal with default paramaters and without using pretrained model, I cannot achieve the performance in the paper(5% lower). Is that because my hyperparamaters are incorrect or something else? And I wonder if the rotated_gt_2 is incorrect since it supervises the orient_2 which is the angle between original target pc and augmented tgt in student model, but seems to be designed to calculated the angle between two rotated point clouds? 微信截图_20231124120657

After I retrained 300 epochs, there was also a decline in performance. I noticed that the maximum epoch in this repo was not 300 but 600, which was different from the baseline.

@caelumchz
Copy link
Author

Hello, this work is full of inspiration, and we want to follow up this work and give stars. However, there's something wrong with my own train process. That is to say when I start a new training on Surreal with default paramaters and without using pretrained model, I cannot achieve the performance in the paper(5% lower). Is that because my hyperparamaters are incorrect or something else? And I wonder if the rotated_gt_2 is incorrect since it supervises the orient_2 which is the angle between original target pc and augmented tgt in student model, but seems to be designed to calculated the angle between two rotated point clouds?

After I retrained 300 epochs, there was also a decline in performance. I noticed that the maximum epoch in this repo was not 300 but 600, which was different from the baseline.

Can you successfully reproduce the performance through retraining? I still meet a failure after training 600 epochs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants