Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interrupt_min_words should only apply to self._transcribed_text #1117

Open
seanmuirhead opened this issue Nov 20, 2024 · 0 comments
Open

interrupt_min_words should only apply to self._transcribed_text #1117

seanmuirhead opened this issue Nov 20, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@seanmuirhead
Copy link
Contributor

seanmuirhead commented Nov 20, 2024

Despite the fix to flush Deepgram transcripts, we are still (though rarely) seeing DG send only INTERIM transcripts without a FINAL.

If this is the case, then interrupt_min_words should be compared to self._transcribed_text, not self._transcribed_interim_text. Otherwise, we may interrupt the agents response without it ever being prompted to respond to what it was interrupted by. This results in the agent appearing to freeze

Proposed Solution 1

This line in VoicePipelineAgent:

text = self._transcribed_interim_text or self._transcribed_text

Should instead be:

text = self._transcribed_text

I would be happy to implement this if Livekit agrees
The issue with this solution is that interruptions will appear to lag a bit from what the user actually says, so not sure if this is worth the tradeoff

Proposed Solution 2

Another solution could be having an internal timer of some sort that will use the _transcribed_interim_text if we never get back a FINAL event after a certain amount of time. I would not be as comfortable implementing this but can give it a try. I don't see a downside to this approach in terms of user experience

@seanmuirhead seanmuirhead added the bug Something isn't working label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant