Creating seed bases for fluid.nmffilter~

So I’m building a patch that will let you pre-analyze bases that will be loaded into fluid.nmffilter~. One of the options I’d like to add is the ability to seed the bases with actual audio input. As in, almost like a classifier, give it musical examples of things I’d like to find in each component.

In looking through the reference file for all the NMF objects, and the corresponding learn articles, I can’t seem to find the bases/range correspond to.

In my case I’m using @fftsettings 512 64 512, so that gives me 257 bins in my bases. If I wanted to create a summary/average of a bunch of individual entries (classifier-style), would I just do a fluid.bufnmf~ @fftsettings 512 64 512 @components 1 and take the output of that, or can I do some kind of melband anlaysis to get to a similar thing?

And if I wanted to make the seeds more @fftsettings agnostic, can I analyze/create them at the highest fft I expect to use, then “downsample” them relative to the @fftsettings at the time of analysis?

they are explained as spectral templates. it is a spectral contour. a spectrogram of one frame. magnitudes only.

it could work to a certain extent, yes. I have had success with that as you saw in the example folder.

again that should work. it is in effect a spectral contour. Now how you will downsample is most probably with ears or in js.

report back if that works.

Ah yeah, I forgot about that pre-classifier “classifier”!

Been having quite good results combining onset detection with real-time decomposing/panning:

The top one (sp.shatter~) is doing 3 component NMF decomposition and the bottom one (sp.scramble~) takes the mc. signal in and, based on onset-detection, randomly enables/disables components in the stereo field.

p.s. in 2023 we should be able to embed media directly! (2.6 MB)

I was thinking of just going dumber with it and doing zl stream + mean sort of thing, to downsample that way. I think what I’ll do instead is just save the bases as they were created and leave it that way. Less faff, and closer to how that sort of thing works elsewhere (loading a file created with one set of settings and then changing the settings generally won’t work right).