Skip to content

Commit

Permalink
πŸ“ update songstarter post
Browse files Browse the repository at this point in the history
  • Loading branch information
nateraw committed May 2, 2024
1 parent 1bf81ef commit 459ccc3
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions posts/training_musicgen_songstarter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -281,12 +281,12 @@
"\n",
"Something I couldn't shake after hearing the results above was: \"it would be so much cooler if I could just hum the melody\".\n",
"\n",
"I can't tell you how many times I've sat down to produce music and just stared at my DAW, not knowing where to start. I am not good enough at piano to just sit down and let ideas flow - I'm a classically trained trombone player. It would be kinda whack to see me whip out a trombone in the middle of a session πŸ˜‚. Instead, I usually start by humming a melody, then trying to find the notes on the piano roll.\n",
"\n",
"> πŸ’‘ __**Fun fact**__ - did you know that Michael Jackson used to sing his melodies into a tape recorder for his producers? Here's his [demo tape for Beat It](https://www.youtube.com/watch?v=eZeYw1bm53Y). Imagine if Michael could have produced his own tracks using just his voice! 🀯\n",
"I can't tell you how many times I've sat down to produce music and just stared at my DAW, not knowing where to start. I am not good enough at piano to just sit down and let ideas flow - I'm a classically trained trombone player. It would be kinda whack to see me whip out a trombone in the middle of a session πŸ˜‚. Instead, I usually start by humming a melody, then try to find the notes on the piano roll.\n",
"\n",
"I struggled with this for a bit until I realized I was just being stupid - the solution was right in front of me. Under the hood, when MusicGen does melody conditioning, it runs stem separation on the audio prompt to remove vocals, as they can make it harder to find a stable signal for conditioning. By simply removing this step, we can prompt with vocals directly! πŸ”₯\n",
"\n",
"> πŸ’‘ __**Fun fact**__ - did you know that Michael Jackson used to sing his melodies into a tape recorder for his producers? Here's his [demo tape for Beat It](https://www.youtube.com/watch?v=eZeYw1bm53Y). Imagine if Michael could have produced his own tracks using just his voice! 🀯\n",
"\n",
"Now, unless you've got a voice like Michael's, you likely don't sing with perfect pitch. We reintroduce the problem that stem separation tried to solve. If your vocals are off pitch, or have fast vibrato, the model will have a hard time finding a stable signal to condition against. To try and mitigate that, you can run pitch correction on your vocals before feeding them through to the patched model. I used some modified code from this AWESOME [blogpost](https://t.co/Kpi023sDP6) by [@wilczek_jan](https://twitter.com/wilczek_jan) to do this, and packaged it up into a Gradio app to play with interactively.\n",
"\n",
"Have a listen to the results:\n",
Expand Down

0 comments on commit 459ccc3

Please sign in to comment.