a few typos and possible fixes

[teamblubee]

Hi

I like the project and it's pretty good as it stands.

A few issues that I have noticed and I could submit PR requests to fix a few issues.

1. Typos in the ipython code, textcleaner has typo.
2. when loading from configuration files we can do a bit better at handling file paths, we can normalize the paths to allow the code to run on either windows or linux.
3. reading files, there are some errors that pop up if trying to open utf-8 files, the main issue is when opening the files no encoding is provided. If we default to utf-8 that should fix most issues.

If you'd like those fixes in a single PR or multiple PR one for each of the above issues, you can let me know.

A more generalized discussion
I've ran the pre-training stage max_len 1800 batch size 4
the time reported between each epoch was about 1600-2100

the first pass was able to fit in a 4090mobile 16GB

trying to train on stage 2 totally failed so I haven't been able to proceed beyond that stage as of yet.

I might re-train with a google cloud A100 to see if I can get stage 2 to train properly.

here's the log from trying to train stage 2

istftnet.py", line 72, in forward
    xt = xt + (1 / a2) * (torch.sin(a2 * xt) ** 2)  # Snake1D
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 218.00 MiB. GPU 0 has a total capacty of 15.99 GiB of which 0 bytes is free. Of the allocated memory 44.86 GiB is allocated by PyTorch, and 769.47 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[yl4579]

For the first 3 issues, please free feel to contribute to the repo.

For the general discussion, please see #81 for more detailed discussion.

[teamblubee]

You guys fixed the utf-8 code along with refactoring out the loading config script to better handle the file path issues. The typos were still there. I've created a pull request: #192

[martinambrus]

A more generalized discussion I've ran the pre-training stage max_len 1800 batch size 4 the time reported between each epoch was about 1600-2100

the first pass was able to fit in a 4090mobile 16GB

trying to train on stage 2 totally failed so I haven't been able to proceed beyond that stage as of yet.

I might re-train with a google cloud A100 to see if I can get stage 2 to train properly.

here's the log from trying to train stage 2

istftnet.py", line 72, in forward
    xt = xt + (1 / a2) * (torch.sin(a2 * xt) ** 2)  # Snake1D
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 218.00 MiB. GPU 0 has a total capacty of 15.99 GiB of which 0 bytes is free. Of the allocated memory 44.86 GiB is allocated by PyTorch, and 769.47 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

In my experience, even Google Colab couldn't handle max_len that high. The only way I was able to train this from scratch was to use a rented A100 84GB GPU from vast.ai