Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tune] when using the tune.qrandint, I can not get the upper bound in tune.run and I found that the value range is out of bounds. #28045

Closed
LinWencong opened this issue Aug 22, 2022 · 3 comments · Fixed by #28187
Labels
bug Something that is supposed to be working; but isn't P1 Issue that should be fixed within a few weeks

Comments

@LinWencong
Copy link

What happened + What you expected to happen

when i run :

from ray import tune

# This will automatically use the `BasicVariantGenerator`
tune.run(
    lambda config: print(config["a"],config["b"]),
    config={
        "a": tune.qrandint(1, 10, 3),
	    "b": tune.qrandint(1, 5, q=1),
    },
    num_samples=50)

i got following:
+--------------------+------------+-----------------+-----+-----+
| Trial name | status | loc | a | b |
|--------------------+------------+-----------------+-----+-----|
| lambda_87b36_00000 | TERMINATED | 127.0.0.1:96387 | 6 | 2 |
| lambda_87b36_00001 | TERMINATED | 127.0.0.1:96390 | 3 | 2 |
| lambda_87b36_00002 | TERMINATED | 127.0.0.1:96391 | 0 | 2 |
| lambda_87b36_00003 | TERMINATED | 127.0.0.1:96392 | 3 | 4 |
| lambda_87b36_00004 | TERMINATED | 127.0.0.1:96393 | 3 | 1 |
| lambda_87b36_00005 | TERMINATED | 127.0.0.1:96394 | 6 | 1 |
| lambda_87b36_00006 | TERMINATED | 127.0.0.1:96395 | 6 | 4 |
| lambda_87b36_00007 | TERMINATED | 127.0.0.1:96396 | 6 | 1 |
| lambda_87b36_00008 | TERMINATED | 127.0.0.1:96387 | 3 | 1 |
| lambda_87b36_00009 | TERMINATED | 127.0.0.1:96387 | 6 | 4 |
| lambda_87b36_00010 | TERMINATED | 127.0.0.1:96387 | 3 | 2 |
| lambda_87b36_00011 | TERMINATED | 127.0.0.1:96387 | 9 | 1 |
| lambda_87b36_00012 | TERMINATED | 127.0.0.1:96387 | 9 | 4 |
| lambda_87b36_00013 | TERMINATED | 127.0.0.1:96387 | 9 | 2 |
| lambda_87b36_00014 | TERMINATED | 127.0.0.1:96387 | 0 | 3 |
| lambda_87b36_00015 | TERMINATED | 127.0.0.1:96387 | 3 | 2 |
| lambda_87b36_00016 | TERMINATED | 127.0.0.1:96387 | 6 | 4 |
| lambda_87b36_00017 | TERMINATED | 127.0.0.1:96387 | 9 | 2 |
| lambda_87b36_00018 | TERMINATED | 127.0.0.1:96387 | 6 | 2 |
| lambda_87b36_00019 | TERMINATED | 127.0.0.1:96387 | 6 | 2 |
| lambda_87b36_00020 | TERMINATED | 127.0.0.1:96387 | 6 | 3 |
| lambda_87b36_00021 | TERMINATED | 127.0.0.1:96387 | 3 | 4 |
| lambda_87b36_00022 | TERMINATED | 127.0.0.1:96387 | 9 | 3 |
| lambda_87b36_00023 | TERMINATED | 127.0.0.1:96387 | 6 | 4 |
| lambda_87b36_00024 | TERMINATED | 127.0.0.1:96387 | 6 | 1 |
| lambda_87b36_00025 | TERMINATED | 127.0.0.1:96387 | 3 | 4 |
| lambda_87b36_00026 | TERMINATED | 127.0.0.1:96387 | 0 | 2 |
| lambda_87b36_00027 | TERMINATED | 127.0.0.1:96387 | 6 | 1 |
| lambda_87b36_00028 | TERMINATED | 127.0.0.1:96387 | 3 | 4 |
| lambda_87b36_00029 | TERMINATED | 127.0.0.1:96387 | 3 | 2 |
| lambda_87b36_00030 | TERMINATED | 127.0.0.1:96387 | 3 | 1 |
| lambda_87b36_00031 | TERMINATED | 127.0.0.1:96387 | 9 | 1 |
| lambda_87b36_00032 | TERMINATED | 127.0.0.1:96387 | 9 | 3 |
| lambda_87b36_00033 | TERMINATED | 127.0.0.1:96387 | 3 | 4 |
| lambda_87b36_00034 | TERMINATED | 127.0.0.1:96387 | 6 | 4 |
| lambda_87b36_00035 | TERMINATED | 127.0.0.1:96387 | 3 | 1 |
| lambda_87b36_00036 | TERMINATED | 127.0.0.1:96387 | 9 | 1 |
| lambda_87b36_00037 | TERMINATED | 127.0.0.1:96387 | 0 | 2 |
| lambda_87b36_00038 | TERMINATED | 127.0.0.1:96387 | 6 | 2 |
| lambda_87b36_00039 | TERMINATED | 127.0.0.1:96387 | 0 | 3 |
| lambda_87b36_00040 | TERMINATED | 127.0.0.1:96387 | 3 | 3 |
| lambda_87b36_00041 | TERMINATED | 127.0.0.1:96387 | 6 | 1 |
| lambda_87b36_00042 | TERMINATED | 127.0.0.1:96387 | 9 | 2 |
| lambda_87b36_00043 | TERMINATED | 127.0.0.1:96387 | 6 | 3 |
| lambda_87b36_00044 | TERMINATED | 127.0.0.1:96387 | 3 | 4 |
| lambda_87b36_00045 | TERMINATED | 127.0.0.1:96387 | 6 | 3 |
| lambda_87b36_00046 | TERMINATED | 127.0.0.1:96387 | 0 | 2 |
| lambda_87b36_00047 | TERMINATED | 127.0.0.1:96387 | 6 | 4 |
| lambda_87b36_00048 | TERMINATED | 127.0.0.1:96387 | 3 | 1 |
| lambda_87b36_00049 | TERMINATED | 127.0.0.1:96387 | 6 | 4 |
+--------------------+------------+-----------------+-----+-----+

I have doubts about :

  1. “a” can be 0 ("a": tune.qrandint(1, 10, 3))
  2. “b” can not reach 5 (tune.qrandint(1, 5, q=1))

Versions / Dependencies

python = 3.9.7
ray==1.13.0

Reproduction script

from ray import tune

This will automatically use the BasicVariantGenerator

tune.run(
lambda config: print(config["a"],config["b"]),
config={
"a": tune.qrandint(1, 10, 3),
"b": tune.qrandint(1, 5, q=1),
},
num_samples=50)

Issue Severity

No response

@LinWencong LinWencong added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Aug 22, 2022
@LinWencong
Copy link
Author

sample.py line 521

        ...
        values = self.sampler.sample(domain, spec, size, random_state=random_state)
        quantized = np.round(np.divide(values, self.q)) * self.q
        ...

The specific implementation is as above.
Would you mind telling me if this needs to be explained in the documentation; or if there is a bug in the documentation? 😁

@xwjiang2010 xwjiang2010 changed the title when using the tune.qrandint, I can not get the upper bound in tune.run and I found that the value range is out of bounds. [tune] when using the tune.qrandint, I can not get the upper bound in tune.run and I found that the value range is out of bounds. Aug 22, 2022
@krfricke
Copy link
Contributor

Yes, this looks like a bug. Specifically for a) np.round(1/3) = 0 and b) np.round(4 / 1) * 1 = 4 so in these cases the limits are not correct.

Let's test for these cases in #28187

@xwjiang2010
Copy link
Contributor

xwjiang2010 commented Aug 31, 2022

cc @ilee300a
sorry I couldn't find a way to assign this to you somehow ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't P1 Issue that should be fixed within a few weeks
Projects
None yet
3 participants