Batch whisper inference

[thewh1teagle]

Whisper model has limitation of 30s.
Can you integrate batch inference into sherpa?
I would like to use it along with the diarization.

I'm still not sure how exactly it possible to batch it but I have some idea:
use silero-vad and aggregate segments into 30s (if there's smaller)
add silence between.
using word timestamps, estimate where's the silence added and reconstruct back the segments text.

thewh1teagle/loud.cpp#11

[thewh1teagle]

https://github.com/m-bain/whisperX?tab=readme-ov-file#whisperx

[csukuangfj]

If you are using CPU， it won't make much difference in speed.

[thewh1teagle]

If you are using CPU， it won't make much difference in speed.

If we process speaker sentences of 5 seconds each time it will process it as 30 seconds, no?
Also GPU is very important with whisper because it's heavy model and that makes much difference

[csukuangfj]

If we process speaker sentences of 5 seconds each time it will process it as 30 seconds, no?

I suggest that you have a look at the Moonshine models. It does not require padding.

[thewh1teagle]

I suggest that you have a look at the Moonshine models. It does not require padding.

unfortunately it supports only English