[Core][Streaming Generator] Potential perf regression from serve streaming handle fix (https://github.com/ray-project/ray/pull/37972)

[rkooo567]

What happened + What you expected to happen

It is expected #37972 causes some level of perf issue. But it seems like this actually impacts Aviary. @Yard1 will double verify this Monday if this is true.

To fix the issue, we should

1. post the cpp level code to a thread pool. Previously, we called cpp code that doesn't hold GIL inside a main thread, so event loop thread could focus on running python code. Now, we are running the cpp code inside the event loop, so nogil doesn't have much impact anymore (because there's no concurrency).
2. Start running microbenchmark that could catch regressions. (@edoakes is it also possible to start tracking serve microbenchmark separately? )

Versions / Dependencies

master

Reproduction script

n/a

Issue Severity

None

[Yard1]

I can confirm Ray 178ae0f is the issue, running the benchmark with a commit right before it gives the expected performance.