[Train] Colocate Trainer and rank 0 worker

[woshiyyya]

Why are these changes needed?

This PR automatically merge the trainer bundle with the rank 0 worker bundle, so that the trainer and rank 0 worker can always colocate on the same node.

Benefits:

• Enables users to specify additional resources for rank 0 worker.
• Always colocate trainers and rank 0 workers together to make the scheduling behavior deterministic.

Major changes:

1. Merge trainer bundle and the first worker bundle.

Specifically, we build a placement groups with bundles [{}, {trainer+worker}, {worker}, ..., {worker}], and schedule the TrainTrainable with the first non-empty bundle. When assigning worker ranks, we designate the worker with the smallest GPU ID on the same node as the trainer to be rank 0.

2. Set num_workers=1 by default in ScalingConfig.

Previously, setting num_workers to None resulted launching a single TrainTrainable with zero workers. It no longer applies to the current Ray Train, as all Trainers now require at least one worker to execute the train_func.

Additionally, this approach led to undefined behaviors during the merging and separation of the first bundle. To ensure the consistent behavior, we have now set the default value of num_workers to 1.

3. Forbid using ScalingConfig with tune.with_resources.

ScalingConfig should be a Ray Train only utility and it's should not be used for Tune Trainables. For example, it doesn't make sense to provide ScalingConfig for a function trainable, since there's no trainer and worker concepts for it.

Passed Release Test：https://buildkite.com/ray-project/release/builds/9650#018dee6e-e3ce-4376-9f3d-5ad7e250e513

Related PRs:

The below two PRs enabled that the actors with empty resources can be launched on the node of a specific bundle in placement group.

• [core] Fix placement groups scheduling when no resources are specified #39946 #43269
• [Core] respect bundle index when task is scheduled with pg with no resources. #43448

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[woshiyyya]

We are able to launch training w/o scaling_config since num_workers defaults to 1 now.

[justinvyu]

We used to force users to think about num_workers, but now we'll default to 1 silently -- think this is a UX problem?

We do show scaling config in our docs a lot so I think it should be fine.

[woshiyyya]

Yeah there are some of our internal tests are using empty ScalingConfig.
For the users I think it's fine. Our docstring and examples explicitly set num_workers.

[justinvyu]

Thanks, this looks a lot better!

Note about the naming:

• We were considering changing trainer_resources to rank_0_resources, but that's a bit confusing since the rank 0 worker doesn't actually have access to the trainer_resources.
• trainer_resources is confusing because users can't access the trainer at all.
• trainer_resources should never be used to ask for more cpus/gpus -- it only really makes sense to ask for memory (or custom resources), in order to guarantee that all workers on the rank 0 node have access to that amount of memory.
• Let's add a comment about that -- I gave a draft below.

[justinvyu]

Why do we need to add a default ScalingConfig() now?

[woshiyyya]

Because when users didn't provide scaling_config in the Trainer init argument list, the self._merged_config will have no "scaling_config" key and returns None. However, we actually set self.scaling_config = ScalingConfig() in the __init__ function, returning a default ScalingConfig aims to align with it and skip the reconcile logic.

[justinvyu]

We used to force users to think about num_workers, but now we'll default to 1 silently -- think this is a UX problem?

We do show scaling config in our docs a lot so I think it should be fine.

[justinvyu]

LGTM!