[AIR] Don't add Trainer resources when running on Colab #28822
Conversation
Signed-off-by: Amog Kamsetty <[email protected]>
Signed-off-by: Amog Kamsetty <[email protected]>
|
Hmm, I wonder if it would make sense to extend this to any single-node cluster case. Not sure if we have a foolproof way of determining that though, if we consider autoscaling. This seems fine as a stopgap if there is no straightforward way if generalizing it. |
|
yeah I don't think ray provides good abstractions on figuring out cluster size and autoscaling behavior. Let's prioritize Colab for now since it has the most resource scarcity (only 2 CPUs), compared to other instances which have like 4-16. We can cover all single node cases in a follow up. |
|
Most people try out the AIR examples on either Colab or their own laptop. Laptops have 16 CPUs, so this is less of a problem. However, we can't have users churn if basic workflows don't work on Colab. |
|
This all makes sense, let's follow up later |
…no-trainer-resources-colab
…28822) Signed-off-by: Amog Kamsetty [email protected] Google Colab only has 2 CPUs. Because of this resource scarcity, we have to be careful on where these resources are allocated and users need to be hyper aware on all the tasks and actors that are reserving resources. In AIR, the Trainer reserves 1 CPU by default which is unintuitive for users. As a stopgap solution, we special case when running on Google Colab so that the trainer does not reserve any resources, and num_workers=2 works for data parallel training. As Google Colab is not distributed, the scalability concerns on doing this are no longer applicable. This has been a headache for me when running AIR on Google Colab and also for users as well: https://discuss.ray.io/t/ray-trainer-looking-for-more-cpus-than-that-of-its-initialized-on/7696 Signed-off-by: Weichen Xu <[email protected]>
Signed-off-by: Amog Kamsetty [email protected]
Google Colab only has 2 CPUs. Because of this resource scarcity, we have to be careful on where these resources are allocated and users need to be hyper aware on all the tasks and actors that are reserving resources.
In AIR, the Trainer reserves 1 CPU by default which is unintuitive for users. As a stopgap solution, we special case when running on Google Colab so that the trainer does not reserve any resources, and
num_workers=2works for data parallel training. As Google Colab is not distributed, the scalability concerns on doing this are no longer applicable.This has been a headache for me when running AIR on Google Colab and also for users as well: https://discuss.ray.io/t/ray-trainer-looking-for-more-cpus-than-that-of-its-initialized-on/7696
Why are these changes needed?
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.