[Doc] Revamp ray core design patterns doc [6/n]: ray wait limits in-flight tasks

[jjyao]

Signed-off-by: Jiajun Yao [email protected]

Why are these changes needed?

• Edit pass: the content was wrong: we don't OOM due to pulling too many args, but rather we OOM because the task queue grow indefinitely when we summit tasks faster than they can be processed.
• Move code to doc_code

Related issue number

#27048

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[stephanie-wang]

Why async? Did the example not work previously?

[jjyao]

Yea, I chose to use async so tasks are finished in a different order than the submission order so using ray.wait() makes more sense otherwise we could just ray.get() one by one.

[stephanie-wang]

Would it be better to make the number of tasks infinite so that it better matches the issue description?

[jjyao]

Since the test code will be run by our CI. I couldn't figure out a good way to show infinite on the example but a small number for CI.