Add parameters to dataset files

[RConstruktiv]

Just a general idea to deal with emotions and general dynamic of a voice. I couldn't find anyone suggesting it.

What if we could add emotion parameters to the dataset transcript, so the trained model knows exactly what for example a sad voice sounds like. Then you can add the paramters before a sentence with brackets like right now. This way you could maybe add sounds like laughing.

Just a quick thought.

[Alienpups]

Really like that idea... would add a lot of voice dynamic...

[erogol]

I like the idea. Do you have an idea how we can create such a dataset?

[RConstruktiv]

We could either add it to the transcript with [happy] or create a folder structure so you create a dataset for each mood.

[thorstenMueller]

You mean, like SSML. So tags inside the transcript - like: The sun is shiningThis makes me happy?
If we find a syntax for that maybe i can record a new open voice dataset with mixed emotions and tagged this (or a similar) way.

[erogol]

I am not sure if SSML is the right approach but love creating a dataset 👍

[thorstenMueller]

Okay, i'll keep an eye on this discussion and once there's a structure defined i'll check on recording a new "german" dataset 😊.

[manmay-nakhashi]

Probably we can add it like [laughter] and give it a special token to it we can make a dictionary for non vocal tokens(ids), when ever [] comes we don't tokenize it how we should be.

[erogol]

Makes sense however, it looks hard to annotate a dataset like this. IS there any ASR model that can do the job you know?

[manmay-nakhashi]

What if we do it with vc model ? 🤔 There is a non vocal dataset which contains this sounds , apply vc on it and create a larger dataset and while training we augment it, or make a classifier out of the data, for labeling.