SileroVAD single-threaded performance is slow. Can it be multithreaded?

[michaelcrusso]

I noticed that this seems to consume only a single CPU core, and on very long audio files it takes quite a long time to process them. I know that I could probably just use another VAD, but is there any way to improve the performance of the internal model without reinventing the wheel?

I tried modifying these settings under vad.py (SileroVADModel class) as a quick test:

    opts = onnxruntime.SessionOptions()
    opts.inter_op_num_threads = 1
    opts.intra_op_num_threads = 1

And messed with increasing the thread counts to 4/16/32, and noticed better core utilization, but performance improvement was flat (there was no performance gain).

[MahmoudAshraf97]

the bottleneck is in the model design itself as it can be only used sequentially and it's not compute bound so increasing thread count will not improve performance, the solution will be to split the model to encoder and decoder and batch the input to the encoder, please refer to #936 for a relevant discussion