Using the WebSpeech API to remove the Whisper model #103

ndrean · 2024-04-05T18:44:39Z

ndrean
Apr 5, 2024
Collaborator

Instead of using the "Whisper" model to transcript an audio into a text, we can use the native Web Speech API.

Note the compatibility status: not with Firefox .

This can save 1Gb.

You can try it with this script (source MDN)

// recognition.js
window.SpeechRecognition =
  Window.SpeechRecognition || window.webkitSpeechRecognition;

const recognition = new SpeechRecognition();
recognition.continuous = true;

recognition.onresult = (e) => {
  if (e.results[0].isFinal) {
    const p = document.createElement("p");
    p.textContent = Array.from(e.results)
      .map((result) => result[0])
      .map((res) => res.transcript)
      .join(", ");

    document.querySelector("#phrase").append(p);
  }
};

document.querySelector("#start").addEventListener("click", () => recognition.start());
document.querySelector("#stop").addEventListener("click", () => recognition.stop());

and serve this "index.html" file:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Speech</title>
  </head>
  <body>
    <button id="start" type="button">Start</button>
    <button id="stop" type="button">Stop</button>
    <div id="phrase"></div>
    <script type="module" src="/recognition.js"></script>
  </body>
</html>

This could be used (an improved version) via a hook and push the result to the server to compute an embedding.

LuchoTurtle · 2024-04-06T02:22:37Z

LuchoTurtle
Apr 6, 2024
Maintainer

A fine alternative 🤔 it'd be great to see if there was a way to compare the results between the two. whisper probably has this comparison

0 replies

ndrean · 2024-04-06T12:22:34Z

ndrean
Apr 6, 2024
Collaborator Author

All I can say is that the web speech seems pretty fast and accurate, much faster that the ML process. It seems to work only for the english language (even if change "lang=es") whilst OpenAI Whisper can deal automatically with several languages.
I did not find any comparison.

Some explanations on how this works: this does not work offline.

I found another answer here:
https://stackoverflow.com/questions/74945259/does-the-web-speech-api-in-chrome-and-edge-use-an-offsite-server-for-stt

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using the WebSpeech API to remove the Whisper model #103

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Using the WebSpeech API to remove the Whisper model #103

ndrean Apr 5, 2024 Collaborator

Replies: 2 comments

LuchoTurtle Apr 6, 2024 Maintainer

ndrean Apr 6, 2024 Collaborator Author

ndrean
Apr 5, 2024
Collaborator

LuchoTurtle
Apr 6, 2024
Maintainer

ndrean
Apr 6, 2024
Collaborator Author