Windows 11 - talk-llama.exe - microphone is ok but error in audio generation.

[mirek190]

My command to run

talk-llama.exe -mw ggml-large-v3.bin -ml marcoroni-7b-v3.Q8_0.gguf -s speak.bat

Added speak.bat and speak.ps1 to the root folder

speak.bat

@powershell -ExecutionPolicy Bypass -F speak.ps1 %1 %2

speak.ps1

# Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope CurrentUser
param(
  # voice options are David or Zira
  [Parameter(Mandatory=$true)][string]$voice,
  [Parameter(Mandatory=$true)][string]$text
)

Add-Type -AssemblyName System.Speech;
$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;
$speak.SelectVoice("Microsoft $voice Desktop");
$speak.Rate="0";
$speak.Speak($text);

But getting error during generating audio
error

llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: VRAM kv self = 256.00 MB
llama_new_context_with_model: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
llama_build_graph: non-view tensors processed: 676/676
llama_new_context_with_model: compute buffer total size = 159.19 MiB
llama_new_context_with_model: VRAM scratch buffer: 156.00 MiB
llama_new_context_with_model: total VRAM used: 7617.83 MiB (model: 7205.83 MiB, context: 412.00 MiB)

SDL_main: processing, 4 threads, lang = en, task = transcribe, timestamps = 0 ...

init: found 1 capture devices:
init:    - Capture device #0: 'Headset (WH-1000XM4)'
init: attempt to open default capture device ...
init: obtained spec for input device (SDL Id = 2):
init:     - sample rate:       16000
init:     - format:            33056 (required: 33056)
init:     - channels:          1 (required: 1)
init:     - samples per frame: 1024

SDL_main : initializing - please wait ...
SDL_main : done! start speaking in the microphone

Georgi: Hello.
LLaMA: Hello Georgi! How may I help you today?

(This transcript has been edited for clarity)

---

Georgi:Exception calling "SelectVoice" with "1" argument(s): "Cannot set voice. No matching voice is installed or the voice
was disabled."
At C:\AI_music\cpu\speak.ps1:10 char:1
+ $speak.SelectVoice("Microsoft $voice Desktop");
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException

 Hi.
LLaMA: Hello Georgi! How can I assist you today?
Georgi:Exception calling "SelectVoice" with "1" argument(s): "Cannot set voice. No matching voice is installed or the voice
was disabled."
At C:\AI_music\cpu\speak.ps1:10 char:1
+ $speak.SelectVoice("Microsoft $voice Desktop");
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException

Also cheeked manually under PowerShell and voices are present

 Add-Type -AssemblyName System.speech
 $synth = New-Object System.Speech.Synthesis.SpeechSynthesizer
 $synth.GetInstalledVoices() | foreach { $_.VoiceInfo.Name }

Microsoft Zira Desktop
IVONA 2 Jan
Microsoft Hazel Desktop
Microsoft David Desktop
IVONA 2 Jacek
Microsoft Paulina Desktop

[mirek190]

Is semi working now but:

I dirty hacked that a bit forcing the first variable

speak.bat

@powershell -ExecutionPolicy Bypass -F speak.ps1 %1 %2

to

@powershell -ExecutionPolicy Bypass -F speak.ps1 David %2

Seems rorking now but stimll not herar a voice.

If I add manually text here
speak.ps1

# Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope CurrentUser
param(
  # voice options are David or Zira
  [Parameter(Mandatory=$true)][string]$voice,
  [Parameter(Mandatory=$true)][string]$text
)

Add-Type -AssemblyName System.Speech;
$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;
$speak.SelectVoice("Microsoft $voice Desktop");
$speak.Rate="0";
$speak.Speak($text);

changing last line to

$speak.Speak("this is a test");

I hear voice finally but only from my test not generated by LLM.

Seems like talk-llama is no passing text to variable %2 so I can not hear any voice.

speak.bat

@powershell -ExecutionPolicy Bypass -F speak.ps1 %1 %2

[mirek190]

I made more tests

I installed espeak-ng

and added to speak.ps1 and comment the rest of code except of variables.

# Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope CurrentUser
param(
  # voice options are David or Zira
  [Parameter(Mandatory=$true)][string]$voice,
  [Parameter(Mandatory=$true)][string]$text
)

#$Add-Type -AssemblyName System.Speech;
#$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;
#$speak.SelectVoice("Microsoft $voice Desktop");
#$speak.Rate="0";
#$speak.Speak($text);

espeak-ng -v pl+f1 $text

Still no voice.

If I manually add text like

espeak-ng -v pl+f1 "Hello"

Of course I can hear it but only "Hello" on each answer from talk-llama.

So I changed speak.bat variables

from
@powershell -ExecutionPolicy Bypass -F speak.ps1 %1 %2
to
@powershell -ExecutionPolicy Bypass -F speak.ps1 %1 Hello
I can hear voice on each answer BUT ONLY "Hello" of course.

My conclusion is talk-llama is not passing second variable %2 to the speak.bat under windows

[mirek190]

Also tested under linux ubuntu 22.04 with

espeak -v en-us+m$1 -s 225 -p 50 -a 200 -g 5 -k 5 "$2"

before that I installed espeak

sudo apt install espeak-ng-espeak

..and getting error during audio generation

Georgi:sh: 1: speak: not found
main: failed to speak

I even passed straight variable $2 to espeak and got only word "two" on each answer generation ...

Anyone could test talk-llama and confirm that recent builds generating audio or not?

[shadowfinderstudios]

It is a Windows command line issue, they can't pass encapsulated strings without some extra effort.

The speak.bat:

@echo off
FOR %%q IN (%*) DO set text=%%q
@powershell -ExecutionPolicy Bypass -F speak.ps1 -voice %1 -text %text%

And the speak.ps1:

# Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope CurrentUser
param(
  [Parameter(Mandatory=$true)][int]$voicenum,
  [Parameter(Mandatory=$true)][string]$textfile
)
Add-Type -AssemblyName System.Speech;
$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;
$voiceoptions = $speak.GetInstalledVoices("en-US");
$voice = $voiceoptions[$voicenum % $voiceoptions.count];
$speak.SelectVoice($voice.VoiceInfo.Name);
$speak.Rate="0";
$text = Get-Content -Path $textfile;
$speak.Speak($text);

Usage: talk-llama.exe -s .\speak.bat -sf .\to_speak.txt -mw .\models\ggml-base.en.bin -ml .\models\dolphin-2.9-llama3-8b.IQ4_XS.gguf -p "Username" -t 8