Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Details about the data structures more_info and cost_info #46

Open
WanFG99 opened this issue Dec 7, 2022 · 2 comments
Open

Details about the data structures more_info and cost_info #46

WanFG99 opened this issue Dec 7, 2022 · 2 comments

Comments

@WanFG99
Copy link

WanFG99 commented Dec 7, 2022

Hello, first of all, thank you very much for your work in the  field of NAS. Here are my questions about NATS-Bench,and hope to get your confirmation:

image

  1. What are the specific units for results such as 'train-all-time'? ( for example, minutes?)
  2. What does training cost mean specifically? In particular, what is the difference between 'train-all-time' and 'T-train@total'?
@D-X-Y
Copy link
Owner

D-X-Y commented Dec 16, 2022

Thanks for your interest. The unit is in seconds.

Can you share with me your codes to get these cost_info and more_info?

@WanFG99
Copy link
Author

WanFG99 commented Dec 26, 2022

Thank you for your response. The second problem above should be caused by my negligence, and it has been solved now. Also, I need to consult with you on the following questions.
image
image

  1. I am a little confused about the exact same train-all-time on cifar10 and cifar100.
  2. I am confused about the test-all-time given. Specifically, can I consider the total time to evaluate an architecture to be the sum of train-all-time and test-per-time? Or is it the sum of train-all-time and test-all-time?
  3. Can you tell me some details about the hardware environment that NATS-Bench is based on? Because it needs to be introduced in my work.

Thank you again for your work and patient response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants