Restrict the number of threads for OpenVINO

[cjh1995-ros]

Hi all. Currently I'm developing with OpenVINO for accelerating dl model with intel GPU.

When I run benchmark_app for my model with following commands, it recorded too many cpu usage.
benchmark_app -m model.xml -hint latency -t 20 -d GPU

As you can see

• OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
• NUM_STREAMS: 1

So I think there's no reason to record 100% CPU usage for it.

Here's how I found the number of threads for benchmark_app.

$ htop
# almost 100%

$ ps -o nlwp <PID>
# 37

Here's my CPU/GPU spec.

# CPU
$ lscpu
11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz

# GPU
$ clinfo -l
Platform #0: Intel(R) OpenCL HD Graphics
 -- Device #0: Intel(R) UHD Graphics [0x9a60]

Here's my question.
How can I restrict the number of threads for inference while using GPU? I think it should be small cause I use GPU but it's high. Why so? And Is it possible to restrict the number of threads with 1?

Thank you

[geunhwan]

CPU is mainly consumed by device driver. Could you try GPU_QUEUE_THROTTLE with LOW?

[cjh1995-ros]

OH MY GOD.........YOU SAVED MY LIFE! NOW I CAN TUNE PARAMETERS FROM HERE!!!! THANK YOU SO MUCH!