Tone Transfer - From Google/Magenta

Like so many “big tech company” “music AI” things, it sounds like fucking dog piss, but it’s kind of interesting as well:

From the demo video it sounds like pitch tracking and sending it into a (really good) acoustic model, which is, sadly, drowning in reverb.

I imagine there’s some concatenation-type things going on as well, as you can hear from one of the examples at the very end (low tongue drum thing to violin).

I actually really like it for some of the dumb errors it makes. You can really hear the voice in this one:

and also super impossible playing in this one:

1 Like

Also make sure to play with the loudness slider - it gets interesting when its turned down to -10

1 Like

It’s interesting to see how it responds to other types of sounds.

It is super pitch heavy in terms of where it leans/analyzes. It doesn’t really know what to do with data noise.

It seems to take a bit to render too, which, unless it’s doing it in the browser, there’s some heavy lifting going on in the analysis.

examples.zip (5.4 MB)

It is actually quite impressive - the artefacts are no problem for the kinds of sounds I’d feed them :slight_smile: But I think @groma will tell us that training is costly on this :slight_smile:

10 hours to get something half decent, and that would be on one of their TPUs I imagine.

ok, not for now for me (not really any use-case of making my synths sound like flutes :slight_smile: