Training for real-time NMF (fluid.nmfmatch~)

[jamesbradbury]

@rodrigo.constanzo

Also noticed that you’re calculating the resynthbuf and envbuf , which aren’t needed unless you want them. (unless I’m missing something, you only need filterbuf for this kind of matching)

Correct. I just did this to make it clear and possibly useable in another context where you do need that info.

The whole debounce issue is another kettle of fish which can be addressed over there but an interesting find nonetheless.

@rodrigo.constanzo

Ok, something weird is happening. I managed to get the patch working one time (while testing with different settings). It wasn’t super accurate, but it was producing somewhat useful results. Since that point, I can’t seem to recreate it working. With your default patch I can’t get it working either (from a fresh download). fluid.nmfmatch~ returns lists of 0. .

I ran into similar problems while building the patch and I’m not sure what caused it. At first it seemed to be related to @maxrank on nmfmatch~ requiring that it be set to the at least the amount of buffers inside the master buffer containing the multichannel storage of $1_nmf_filter buffers.

@rodrigo.constanzo

I’ll have a further play with this later today, and see if I can get a well working output (post fluid.nmfmatch~ ) stage going.

(will also test with my “manual” batch processing patch posted above, which requires you to pre-crop your samples first)

I’m also wondering if there’s a way to apply some kind of median filter or percentile selection for multiple dicts as a good use case for training a system like this would be to tell it you’re playing a certain sound, and then doing so (perhaps several times). If you accidentally record a wrong sound, or if one particular sound is way out of place for some reason, to be able to groom that out, statistically. (e.g I’m training a classifier by hitting a specific drum, on one of the hits I miss and hit the wrong section)

Multiple data sets for each hit could be interesting. One strategy (that definitely is not suited to this patch) is to have a training phase where you hit multiple times and it averages out the nmf_filter buffers. They should all be the same length and so the math would be quite easy. Perhaps this is another application for fluid.bufcompose~ @tremblap ?