best accuracy find

[outofstyle]

Describe the bug
A clear and concise description of what the bug is.
We find the best accuracy is different from paper given In tss search space when. And in cifar10 dataset, I can not find valid data(valid accuracy). Is it leavet out?
To Reproduce
Please provide a small script to reproduce the behavior:

from nats_bench import create
api = create('D:\Download\coreg-master\\NATS-tss-v1_0-3ffb9-simple', 'tss', fast_mode=True, verbose=True)
valid_acclist = []
    test_acclist = []
    netlist = []
    for i in range(0, 15625):
        info = api.get_more_info(i, 'cifar10', hp='200')
        config = api.get_net_config(i, 'cifar10')  #ImageNet16-120
        valid_acclist.append(info['valid-accuracy']) #I can not find valid-accuracy from cifar10, but it exist in cifar100 and imagenet
        test_acclist.append(info['test-accuracy']) #I find best test acc in cifar10 is 94.56 but not 94.37 in cifar10. It also happened in cifar100 and Imagenet

        netlist.append(config)

    max(valid_acclist)
codes to reproduce the bug

Please let me know your OS, Python version, PyTorch version.
windows 10, python 3.7 torch 1.9.0

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

[D-X-Y]

You can refer to the answer for this issue #38 :)

[D-X-Y]

In short, to get the valid-accuracy on cifar10, you need to replace the dataset arg from "cifar10" to "cifar10-valid".

For the best performance reported in the paper, we use the is_random=False, where in your codesnapshot, the default value of "is_random" is "True".

[outofstyle]

Thank you for answering this question. I haven't seen the answer to issue 38 before.