Running time with prefix is not faster than running without prefix

[GuyEyalGong]

Hi
We are trying to utilize the prefix prompt (tokens after the SOT) to reduce running time in cases we already transcribed the first part of the audio segment (running in sliding windows over long audio)

However, when comparing running times, we do not see an improvement.

Our assumption: For a given audio with the first X seconds transcribed, putting them in the prompt will result in fewer decoding steps in ctranslate. Are we wrong?