TTS server and xtts not works with fine tuned models (#3476) #3544

bivashy · 2024-01-28T13:58:31Z

bivashy
Jan 28, 2024

ther3zz · 2024-01-28T14:05:50Z

ther3zz
Jan 28, 2024

Currently attempting to run a fine tuned xtts v2 model like (with these changes applied: bivashy:fix/xtts-server-error) :
tts-server --model_path /models/test_v2/model.pth --config_path /models/test_v2/config.json --use_cuda true

and

tts-server --model_path /models/test_v2/ --config_path /models/test_v2/config.json --use_cuda true

Neither works and returns this error:
Traceback (most recent call last): File "/usr/local/bin/tts-server", line 33, in <module> sys.exit(load_entry_point('TTS', 'console_scripts', 'tts-server')()) File "/usr/local/bin/tts-server", line 25, in importlib_load_entry_point return next(matches).load() File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load module = import_module(match.group('module')) File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/root/TTS/server/server.py", line 104, in <module> synthesizer = Synthesizer( File "/root/TTS/utils/synthesizer.py", line 93, in __init__ self._load_tts(tts_checkpoint, tts_config_path, use_cuda) File "/root/TTS/utils/synthesizer.py", line 183, in _load_tts self.tts_config = load_config(tts_config_path) File "/root/TTS/config/__init__.py", line 97, in load_config config_class = register_config(model_name.lower()) File "/root/TTS/config/__init__.py", line 40, in register_config from TTS.tts.configs.xtts_config import XttsConfig File "/root/TTS/tts/configs/xtts_config.py", line 5, in <module> from TTS.tts.models.xtts import XttsArgs, XttsAudioConfig File "/root/TTS/tts/models/xtts.py", line 67, in <module> manager = ModelManager(path) File "/root/TTS/utils/manage.py", line 56, in __init__ self.read_models_file(models_file) File "/root/TTS/utils/manage.py", line 68, in read_models_file self.models_dict = read_json_with_comments(file_path) File "/root/TTS/config/__init__.py", line 17, in read_json_with_comments with fsspec.open(json_path, "r", encoding="utf-8") as f: File "/usr/local/lib/python3.10/dist-packages/fsspec/core.py", line 103, in __enter__ f = self.fs.open(self.path, mode=mode) File "/usr/local/lib/python3.10/dist-packages/fsspec/spec.py", line 1295, in open f = self._open( File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 180, in _open return LocalFileOpener(path, mode, fs=self, **kwargs) File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 302, in __init__ self._open() File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 307, in _open self.f = open(self.path, mode=self.mode) FileNotFoundError: [Errno 2] No such file or directory: '/root/TTS/tts/models/../.models.json'

0 replies

bivashy · 2024-01-28T14:12:24Z

bivashy
Jan 28, 2024
Author

Here are readable format of the error:

Traceback (most recent call last):
  File "/usr/local/bin/tts-server", line 33, in <module>
    sys.exit(load_entry_point('TTS', 'console_scripts', 'tts-server')())
  File "/usr/local/bin/tts-server", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/TTS/server/server.py", line 104, in <module>
    synthesizer = Synthesizer(
  File "/root/TTS/utils/synthesizer.py", line 93, in __init__
    self._load_tts(tts_checkpoint, tts_config_path, use_cuda)
  File "/root/TTS/utils/synthesizer.py", line 183, in _load_tts
    self.tts_config = load_config(tts_config_path)
  File "/root/TTS/config/__init__.py", line 97, in load_config
    config_class = register_config(model_name.lower())
  File "/root/TTS/config/__init__.py", line 40, in register_config
    from TTS.tts.configs.xtts_config import XttsConfig
  File "/root/TTS/tts/configs/xtts_config.py", line 5, in <module>
    from TTS.tts.models.xtts import XttsArgs, XttsAudioConfig
  File "/root/TTS/tts/models/xtts.py", line 67, in <module>
    manager = ModelManager(path)
  File "/root/TTS/utils/manage.py", line 56, in __init__
    self.read_models_file(models_file)
  File "/root/TTS/utils/manage.py", line 68, in read_models_file
    self.models_dict = read_json_with_comments(file_path)
  File "/root/TTS/config/__init__.py", line 17, in read_json_with_comments
    with fsspec.open(json_path, "r", encoding="utf-8") as f:
  File "/usr/local/lib/python3.10/dist-packages/fsspec/core.py", line 103, in __enter__
    f = self.fs.open(self.path, mode=mode)
  File "/usr/local/lib/python3.10/dist-packages/fsspec/spec.py", line 1295, in open
    f = self._open(
  File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 180, in _open
    return LocalFileOpener(path, mode, fs=self, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 302, in __init__
    self._open()
  File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 307, in _open
    self.f = open(self.path, mode=self.mode)
FileNotFoundError: [Errno 2] No such file or directory: '/root/TTS/tts/models/../.models.json'

For some reason in /root/TTS/tts/models/xtts.py line 67 is called, but in my branch it has empty line:
https://github.com/bivashy/TTS/blob/fix/xtts-server-error/TTS/tts/models/xtts.py#L67

Maybe you've downloaded/using modified tts?

4 replies

ther3zz Jan 28, 2024

I'm using the docker container:
ghcr.io/coqui-ai/tts

and modified the files you've pushed

ther3zz Jan 31, 2024

Wondering if anyone else is running into this issue?

bivashy Feb 1, 2024
Author

I am sorry, currently i have no idea

ther3zz Feb 1, 2024

OK so, I replaced the files included in the docker with the latest release (I also used your modifications @bivashy ) and I get a different error now.

I've changed how I'm running it to the following:
tts-server --model_path /models/test_v1/model.pth --config_path /models/test_v1/config.json --use_cuda true --speakers_file_path /models/test_v1/sample.wav

the result is the following:

Using model: xtts
Serving Flask app 'TTS.server.server'
Debug mode: off
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Running on all addresses (::)
Running on http://[::1]:5002
Running on http://[::1]:5002
INFO:werkzeug:Press CTRL+C to quit
INFO:werkzeug:::ffff:192.168.1.136 - - [01/Feb/2024 17:03:23] "GET / HTTP/1.1" 200 -
INFO:werkzeug:::ffff:192.168.1.136 - - [01/Feb/2024 17:03:23] "GET /static/coqui-log-green-TTS.png HTTP/1.1" 304 -
Model input: This is a test
Speaker Idx:
Language Idx: en
Text splitted to sentences.
['This is a test']
ERROR:TTS.server.server:Exception on /api/tts [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/flask/app.py", line 1455, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.10/dist-packages/flask/app.py", line 869, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.10/dist-packages/flask/app.py", line 867, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.10/dist-packages/flask/app.py", line 852, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/root/TTS/server/server.py", line 203, in tts
wavs = synthesizer.tts(text, speaker_name=speaker_idx, language_name=language_idx, style_wav=style_wav)
File "/root/TTS/utils/synthesizer.py", line 319, in tts
elif len(self.tts_model.speaker_manager.name_to_id) == 1:
AttributeError: 'NoneType' object has no attribute 'name_to_id'
INFO:werkzeug:::ffff:192.168.1.136 - - [01/Feb/2024 17:03:29] "GET /api/tts?text=This%20is%20a%20test&speaker_id=&style_wav=&language_id=en HTTP/1.1" 500 -

ther3zz · 2024-02-13T15:44:42Z

ther3zz
Feb 13, 2024

OK, so I finally got this working.

I had to modify /root/TTS/utils/synthesizer.py with the following
Line 307
From:
if self.tts_speakers_file or hasattr(self.tts_model.speaker_manager, "name_to_id"):
To:

if self.tts_model.speaker_manager and hasattr(self.tts_model.speaker_manager, "name_to_id"):

Now I'm able to generate TTS via the web page. The main usage for me was to link this up with home assistant (Mary TTS integration). I'll test that today to make sure everything works correctly.

2 replies

ther3zz Feb 13, 2024

OK, I needed to make some changes in order to have the default web app AND for Home Assistant to work properly.
Needed to modify xtts.py
Also needed to modify server.py

I've confirmed that these changes work with both the included webapp and via MaryTTS in Home Assistant (using the docker container provided by coqui-tts)

PantACRO4life Nov 25, 2024

hello @ther3zz , I hope you doing great. I was hoping you could give me some insight. I don't want to put too much info here just because I was trying for a hole day so I have a lot to say. But, basically I have hass, I made a custom voice that work with piper just fine but I found it way to robotic.

Then, I found out about XTTS and I was mindblown by the result. I want that. I am unable to make it work. I was able to have the TTS server running, even integrated it in my compose file for my hole AI-stack.

My main issue seams to be that I have made a multi-speaker model based on the logs/console, and I need to specify somehow the speaker. I used this to create/fine-tune the voice. xtts-finetune-webui. I realy don't get how to make it work. And I tried. I also gave a shot at your modified files and line you mentioned, cloned your repo. Tried the maintained fork, the old one. But, it always result into "not working" because I change some .py files get some error or because it cry out about my model file speaker thingy. Anyway...

I have more prescise info to share that is for sure like logs etc. But I'll wait for your reply or someone else for now.

Thank you !

Edit: ended up using something else for hosting my xtts model.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TTS server and xtts not works with fine tuned models (#3476) #3544

{{title}}

Replies: 3 comments 6 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

TTS server and xtts not works with fine tuned models (#3476) #3544

Replies: 3 comments · 6 replies

bivashy Jan 28, 2024 Author

bivashy Feb 1, 2024 Author

Replies: 3 comments 6 replies

bivashy
Jan 28, 2024
Author

bivashy Feb 1, 2024
Author