vllm_server.sh脚本问题

[QingChengLineOne]

vllm_server.sh脚本配置：
model_path="/public/zzy/model/Mistral-7B-Instruct-v0.2"
model_name="Mistral-7B-Instruct-v0.2"
tensor_parallel_size=2

cd $model_path
cd ..
python -m vllm.entrypoints.openai.api_server --model $model_name --dtype=half --tensor-parallel-size $tensor_parallel_size

报错：
Traceback (most recent call last):
File "/root/anaconda/envs/moss/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/root/anaconda/envs/moss/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/anaconda/envs/moss/lib/python3.8/site-packages/vllm/entrypoints/openai/api_server.py", line 37, in
_running_tasks: Set[asyncio.Task[Any]] = set()
TypeError: 'type' object is not subscriptable

[wlyh514]

Seems like a known vllm bug introduced in v0.4.2.
Issue
The PR that introduced this bug
Diff v0.4.1...v0.4.2
The fix is not published yet, you could try rolling back to v0.4.1.

[QingChengLineOne]

thank you for your answer. I try to use python==3.10 to slove this problem.