A spectrogram in Jitter to click and select seeding bases for nmf

Hello

This example of nmf use was exciting, and @groma dared me to implement it. As I had other boring things to do yesterday, I did that instead :wink:

It is a big patch, not super well documented but enough to get anyone going. It is fun I think. The selection and the visualisation are not remotly as good as the example, but the principle under the hood is similar I thinkā€¦ or maybe not but it is nevertheless fun. It is sort of the opposite of the bird finding nmfmatch example, where we divide randomly first, then classify. Here we spectrogram, then select conditionally graphically, then run an nmf.

I hope you like it. I can comment a bit more if that helps

pa

===


----------begin_max5_patcher----------
5505.3oc28rsiiiqbOOyWghQBP2yocOh2zkCPBlDj7vBjaODj7vNCZHaK2s1
QVRPRd5Y1CNy2dHYQoVxMkMsMocum8haKIKwpJV2XUEK8Wd+6lsn76oMy79y
d+p26d2e48u6cxSINw6TG+tYaR99x7jF4Oa1xxMaRKZmcGbs1zu2JOexsdKy
yV9UuxBu1mR8ZKq7xSW25sXaaq3bkdKKq9g7ZKRZRaDmIwqIMcUVwi7e050o
0etXQ2igeU4OEt6Eo4kOKNW8V3wWrYsWUc4xzllOWr7Vu7rl1T4n77SYKeJ8
ao07waSUYAGX8dNq8I4sUkm7C9UD.Wc1iO0iF4YEoKK2VHwEh5jYqjXV4hea
NBw59oEa2jUjm1JIGnWNY411ty5qNaS6OxSkOitatJoc4Sb78g5zks.YGGFe
O6Nu3X78924QhHh+PCu226Kh64u992K93NCmdJRelCuuZ1IuLY0FNsxaNJxe
lN7irW7qpNsgSHSZyJKF.6HT.S.twH4e7Ue38EsjEzwPVP9j.4iFKezQwhOw
3AOb3I29ipT3NlM6jnXSwP29bZxWAV4m3X+SdIEq7xKK+pWRq7zMUbfst7w5
jMbFrhUkOaN2D1UbSHVHVvNgXQBBFh4eVrS6QZuIskK+J+odb.dAWr5FDeJx
qr1ibqTRF9I.4qLeE+abR2yka4esfK2KjVWj5sQ7q.RpTzWnNnpLiK1JzPzz
j8XAbmKxJ.0B7uKGWojOntH44je7pYnkk4kaqazNsvzMs36toEwT.e9vOTJe
iIx+DRs77Rutyk4oI0dI445D18omjvNNRhCQAwFHrebjGBWXmIdzf5DopPru
kINCXD4LVKay9VRapV5yooLzGf6n3HqSe7i8gGMvEghbBABDl+k+Uua78VyE
iK3VOuUKAxc5u7ExDBNgfA5urNlxcRnM8EMSZvw3nShGfRBuOjyIyBsNO.kH
k6XTWRXVjmHTfzKovU4t5eQG4IzYb.XbLPBcJhpzTNzS05Tt6gKSG3wpvsIt
Er0Y4oBJA3PpvQ0689EgWk44dq.+TAG.31zF5U.+V9sr11zZSs.EE6JpJKDI
M3.5O.Rq0s9PucWey0h3XMHN9Tj1HwRjJhaWhYWgMhRUNXwCEJOJX3CecYQa
S1uKeRXNgzhjQxsJmgDN+rNaoDs2KEUiOlQnSR+EKVYi295u5ezx06nnoQgW
FZJFno4kONOYSk22xZ1lvWynD+MkIML9zHoLooLBx9TTvW+nfq.SJBHnIEI4
+nIqQiIhPhyTlQhD3MPQUKAU3PsMwawpkSdL8U3sxHgF7MPeHBv6m8.Km7vQ
1d8yJylRE8pu6fEOuNujCiZHFz8uTppDt8wTtkwGRKRV.Xie+jW8lDIVDnkL
fOkvHn7b1HJwcb2fRJdblMC.SSUMeEsq8dLscYJ2sg+gL9+ogtQv6kI5bBnB
ABRSrxG.3HWvTLAIXaQkv2xFuL9+13sVC1iOFUFriy6YvAH9RWXc1cLgQHSn
5q+Oh+myxK3NsHgg6i38bN95ruOx6QjmbndgiX3JvBNESQg9RYgPiVfJ9TXt
P99.YNz.tKNd+.f2RZqsIooeup16SYEUaaa7HdeRd7mmcyMXeuOH7D3lrheE
8kau06eh6o9uh+xs7Sy+h+W97Lcjb8q3gXCcTHUvgB7AOEvWUh2i5CKB9TLu
gnHYjch8CsdDh6ezRibDpqzmk96qRV9SsRgjiPa8Q4hSHfNwHoDKk084Ig.S
4KSSaRcqV7xYVgB.V7vHe2NmAhFbcoZC.J6jcNwTw7kk4a2TLRCQc4y6LLud
VOBbnGT.D.Q4KXeTGQduDJ.j+0lJA.B3O4JNQZofQtRaXPHDpSFDgbCbXSjg
fjbmPED4K7mdbUa+nn8IOhVJQvQ3yB8DjUTbCw3igRn6aV2oVHVX8zGsTmiI
uoGEeBCCQzJ.nNnCSc.O2NKBADZPsHJ8HPziRkIF6KCXhOfvvR5vz8goVeld
c91rU2ymuK1r9mdepoba8xTO0e9Tm7Q2e+TcRwWEtbAI5G9DNXS4pTOsdzxv
thSAC9If7gbvQMw2eIEz1NV05s.RLmN7mF4J7my4LPRAzqZF1C.qUWGbZtm1
BPfFbDtbbJnerODmnfq3jelVL2YpHQHYFihg.kQ7MGyEq20p1QqSqRKVIpH.
sj.rqTdxezCV9QH8BGyCYDOP5cghhbVndTHcfTTmEbUBziNTlD6NVcvyYH3V
TCpVnAQ1wtg9DQPuMh8oJPe9XxUMzmUooe8mfS.5nV3iItJGkX.r1AjuOrjy
XS8PzxdFhYNS4lxvNTLQWEGCGD6x93U58os7455kqxDAw7Sv2jwZvy2ig5Ok
vffO+e+D7CYH9clrJopUuGBXjicPJBoxVRvUO1kJR5ljluNftpnQsOUucBRj
uq8gDpCIL55Rh5V3oZkEMIapZ3zFh9UniblUGB0efm0HU.ab8hP2CeS1fhMY
DI3XJWGxoH6Pj3N0PiLNw6ZshD9tykChOUVnRpbKZRvLOa+qm1LiejyLyzin
Pp1uJFZDSvSr.JejyXtU0ATLetkYF2c2LrqVIUSUdlxNpVZguyxREkAaTAFD
bZp6cw9jVLYr6b2hffJSrKKmW7kSt7INWUp2bcS8wTmIEzMyCUohIy76j57K
P1IiIta0zJz204Yb+0MgnpIzo9KF6v4ceXQ0RTNH7nj3Oqod49IQGx567L2w
AjrM7E1OfZwGotS4e3z3GEbcWu8vxIIMmCdhpGceESRTn6V0.PRTQhBwttUE
RUxW8Dq0LSa.4hBbUfHB.UlLfXDDegUfreFhCtpxHmEe5N5BUsmndCvczzNH
VE5T2FQbEWBBGDpbuHVE4pnKd3p6HAB+qzauIh5LB.0mnvanhl8QWOIE8gdY
aSJPeDA1ZZIFeWqPEgvf4k3qafXp4CVpGGcZz4SVXjyxoWmqIznW1VKFD1A2
TigpJKTV6fdevCpqPu+t+Qw9fUWoDFF5L5BCSDzELTwXnn2Jkg4jjH7DjH2k
NX0dM8ONjHzDjHlyHQHUBzd6PhNs.hGRcWv+T0OO5MQ.wOcRT.wcd2IIMApc
3xUlBIheXi2BMT.lyBeXbDjW.H0izinvab1BCmHy7T8BJzymKPQCvPxXAq1u
EDSDaOrUI4UOkr20FScl3QGgQoiMfckEOPd5ROOIvYpGnREBLH3RDhwAW5Rx
QbXKLNl7D3OjK4pp8T2hACcWN4UZMCCt10t1Ir8rB8crVCUnAt1aypoXKBbX
RWgp285yV7RK0RaG0JvgoiUU.yQPh2uvwH4PsRrfPmg3JUBJD2cgQcprODD3
7rOrm4aHuqWfzJLE5yBuJnuZV+hg9G8NJkSKt4F9yXzFK0+K2dGr+R8l6o9x
GkKpG+k9yb2DKsmQcUV76re.YwjdoB49T6.wnWx8zPz2Y4xEC0BKJVh2ANqx
nmBeoLRLAeuNbF6ZbF5HfAAWibJHS5FCoyWIbrqxm.C1jopzKEguvHd6DIPA
6rM5S7Plab7gWp84lmdYDEF26KGhngmT+xgDB91FKaxKLjxg2C1nBlVaGARV
hpaNIehWv7DradEmiztRRryR+LsqepArGnfqZVEmiln10vtKwhpDq1QAvrKc
yjk19jn8kpCqcVqTDCoOU0kjNy1n2j0c5QUx4GmcKXMtQ9tulSmrAIj86oYE
P4ki7wz6DsBvbulrBO92Sppx+g2iI7izVKlNa+colYYpJQicExRdy1EpXhH2
8F5MsibVXPTsHEFrAGQlD8PGGV415jhlpRs4AO10jAPuV3alzzoMNYhxlUSU
Tbj6UnwQh8Psru.WS46hdM3mA6ZU5RdS5DFZByp9CZUdE70sKug+45rj7YVv
iCH30DUKI73ZmQZ6heRx8fwKsVQkTjI9uNKWz0+aDMzwWf22Miqkdvoe2faQ
Pa+sR4CJ7t9SkU.mhzep5zuk0c+r9ylTuTzDgW1tsFnGeOniRKdLkqRqK1lI
AE3j7Y4228.eYKJBkyipc3n9iP8dGUfyQ7Xd4xultZvrFm.VkVjULzE5QWdU
55js4sOLhD5eu1quVzNCm5l6YM9eSqWkTjzggydrNaUYg.LFQsEmta.4q.SJ
iLDYjWuHoRysx4V4zkItHWkP61lEI0hoBUvkvcWrsrLe7k5uOQi8Uc4prhhc
nhskUSeQ4KHhou7hR9E2rums7JMOrs.t5Cbo31GZR91XpcaRdtRrd7i+6IEY
bQhz1LXJ.62eQH.aO0rrtLOeD9BW4aZtxJNW7xzmyV09jbfFxLv+4YUcLQy5
miWk8XZS63y0l7Xy3y7JMB7SscgRJ8g1zMU4hR1ezOXzqcjghjCU.N576SQ3
qCPMDz+Wt3PEhCOuNyB6Ws3zpFmfXr29yIrywBdUcNtWGb2ALGphr6JRyJJk
NmHEdrOwiIwh.eJ8D9yy7umMm++eXYYyM+O+e+W+2+xG73JP2Hh+4m4141yD
QeT9lzG4K4LABxp.90sNfI8n9xLOrGN8AdbIr+OxUKLcBxdeOF2dB.3CR10Y
mY+SGp9++q2rQF5b0kYxAfiInzXiHzZWg+ovAiU6acV2miHBcQlX2GGmuI86
C7RYDY4E61BGsmPgsDfDWWO8BZRTpooNFPuw.G2BTaVQuaL+ZO8ySEshCNcc
r.gP4iQ.QGz5Dn.YHPzAsZmhd+PmKmI7uX0CfuTOjzxkHVrsEl5F5s7QYyeu
NBNkKlbuVWjjqbCn+t2mWCu3Zw6eAIs2ZSKq1QAo2m3m5C5Vu99ibioKYZ5E
TpQc3dVjIIV8tP4ThxwDqkxlo4fCDUaa0lqCp6JDjtBcGh.Dxfc...mtbGIC
iv5r5Fc6LYpyJIjNFEHqtHzkdey7mjcYHcnbnqhAJEryQf5ulFYV4CZSj9CS
D0yipS1cJSyXUXBhu7371eOap7WQOlsi.43SqoJN2r2.8fAYceven272uF8Q
NSvsd+IsY.f3LAdUldU8zdTzE+ki3Nu+CUubZDu59xZ3jllpr5zUdqqK238T
aaUye9ieb4OVxcM5wP587m5GWWVucSyGG7PlKdw9MeQ5SbOXZlmm80z4Oy+p
nkn8HmFMeURaxbwSbNz9elyGl1zjUyKWOWzjaM9UiCwYuKOIAvVKB1gQDvgb
a+t7brwFQPkzkVNm0EL5T71Yqw+BaqQJ7wE779.+Sr2GgF7ltHeG6J8vJoOU
k6iuzEVRVw5xepZ9t5v6iQqS7If2cusBHGQ2tpqOk2eBS9hqplaUG0PaJKIt
Zy.oLdSglDVH5X8k+RtGDqJe9laDeHKjQNbJrvoNFIO9VwqwAl9pWDgc0FSr
iFBU3Dwmd0S34jLRQtlF.0zDK5pSB5ZFeSpPh4XQJUZFiieiIRsdcqGVWHYG
41riyPMkprSiutU5a2KWy+yrkk4y+2KKq92l+ebO2AOc9M6rBmgM7MnZ3E1t
8AETb1loR8NuJvz27UVnIUVkTjp6cuBa+QDZQY8JH293QmQ7p7tFfNtdOtKH
Ax23mj3.+vHw2BY3XJhOsZo2shvaQcD7tk.xRWvvx6Pjf+gOUNgRkU5v9ffN
awiifabrOJNPB29wQ9T3a7SgF+RQhuZppx59j5ywRKNADP+CwDPbrr9kifnb
DFSg23ig+MwT.5OFSAbmT4if5cFm76x9MyeCLA3+GhI.nD9CQC6vtjv2DZgj
+pwYEDlP1MEXpokWm5KznLvMQhuDuFQ8lJ5IlNThHS5gNvHEZiQR7Jf8f3Dx
BCjrm2dPbRr4przPgOvPQvVXnvlP87sG8y+RvSzwBuerJDnwmGREZvHEYCbJ
JzTb5rYIDZInlLRnyk5EXfDElXAbhXDedfEFoXSzwJAmWP7daenyTByD9dQ2
TbftkK6XS0O19mGI2jIWrE3WkCD1fAx21j2XloT2yFGoFYQv+RIprC7XSVlC
o4QBeWDdlcnD1BIMwfajMlHMR12F1lhMXfrgV7n.SvHlMFI5EyCBxEajLYYF
VXbBMYfhsgaxgl38knit4QdYghYqpJyJZa55SnpsLGjMpt2QXxCOWnKvTnCO
IzQf8OTOzAQiyJPGyTnCMIzI5jBrAPmpEcYCnyD4uvv8BcDLEpAAlLBlApzh
HNz15zMCbCNFvUsc+m.bQW.vkYF3JYAFc3qAW74AtWrknFZRvDX1vgtPSLNy
rg53f3KE0SNRFQ8N2E3GXhIl.a3xQfIlXX1vIp.SLWDDagQhYBNEZinKwLQQ
CyJijIRt63uvtpz78UURMBz.281jDYeCFlAtAFAtjXPCrxUAwg11fgYfKyLv
ELXz46fKLXvLIPs6XD7UfKRZGVsWh5su4GbuUXUOnmW1H9Zl3hWrMj9nlXiY
+rGfvFAZU1DrTDTdjM.sCw5R2qjFF5bL3PnXVAXSdjsUKXDvh2qejJfkB6JD
jOzvejGZawLyfVpKBpJMxbFtyafLwVI1JijQKWLxEwwzrgduFO6jXUU5M7Zh
j3OLI41RFgYfxcpM7ShZh2KXa36GkX5hwO6Qxj3Hakb6XjDJQuxAe2OzXmHI
QLQRxFbLDSRRWnyPwCMxANIz4jKTQEfOh.ZqWcHKR5TNKH9d7K9L.GZ6IDif
VhQfan7sRQ.i0ejsmBkhcGLEOLWv8XzPGs+fzgTa0CndvfFyo7HqOoZTz5sg
eGFQVvwGAYAi5Ox1tbhCMQsmMxdFN3r0A.x9SnC37.N14tdqNOzf2Am8q2J9
rWuEldtzMTXn6HbFUCR1f+AYzRhwtv+GiFZhS7+AYhZKjMbNWNRAlLRme8PY
TYJwrQ3lQLiHfADqMVHiFqylFxBLhFZEIOlIdn1i8VXrvFMVmcUMZTzzQLrU
ngWtoKihkCB6asw5frF1QAE0H1Pq3CW+S4.iExZi0AwKrUJjWJ9BhWHi3CsR
ojShMh2vJhxFspYqMRHSFoIWIAgFJ70iFC8KW+f9iNWXCaTkZSrBmD1LoQqn
8zH+siQ1vVG1HADqTs6HybTGamoKipsVaLRFswDPz.qMVGTZjXEScnK1t8wD
8lVoZIgg5hT2LHei7.x2JSUlHWYkAxDtuWqAD1Nc6zpxEiwNsn7cZO4ut0jO
caIe2VRtriQB6Cwc1Be881vMIeOPrAKkXSWaAeLAXzdKD5FU8eLLNphMAu7W
9PVg.GS62Fk2M3ig6axjkKSKZGsyEk6wxHRLFI29hgpsuH5Eyj2olrp425ta
wxYa3PeVWbFFN4NBgkaGz8gxi.IJlvvwxsPYHlQkeCGRYHaCVi5qB5ArimFu
3QQWbtGgF1fP65F.ydrNYU1K8.LE9i5mSB3+S3cZ+1n1Xu71vc2VXTDmxcm1
u85ayjst5n6R29fU2NZs+2ussrGQ6hDUeGHU+z2XoCylEG1ufMf6RNmQ4rU9
X4233LGlOUNK987We++u+imFZ
-----------end_max5_patcher-----------

I like the idea, but Iā€™m not sure how this patch is supposed to work.

I load the Nicol audio file and thenā€¦nothing happens. If I bang the info~, nothing happens either. (I also tried saving it and reopening it, in case itā€™s a loadbang issue, but still nothing).

To be fair, it could also just be me, because I never understood the ā€œbird finderā€ patch either.

edit: Ok, I see you have to manually tweak the spectrogram thresholds (even though there is a default audio file to correspond with it). Still couldnā€™t get any color action though.

colour: you select a class, and a threshold, and you click in the spectrogram a place. If the threshold is met, it will colour that frequency.

To colour about everywhere, just select a class (1) and keep the threshold to -180 and click on a white bit.

I know, not the best UI but this is proof of concept and fun code, no?

I see, itā€™s more like a click+drag kind of thing. Clicking on individual things didnā€™t really seem to do anything because (I guess) I was clicking on specific tiny bins that were empty, or missing the things altogether.

It seems cool, but, like the bird patch, I think Iā€™m not really getting what is supposed to be happening. (I wouldnā€™t have understood your JIT nmf patch either, had you not showed me it in person)

hmmmmā€¦ maybe Iā€™m now too much in-the-knowā€¦ anyway @weefuzzy will pick up the ownership of help/doc/examples/tutorials so it is important you keep telling us when you get or not get something!

thanks for the feedback

Like this idea and all seems to be working fine for me (bases buffer populated, resynth too) but no audio (nan coming out of play for all components). Ideas?

Hello!

Iā€™ve just tried it, and you might get NaNs if you do not put at least a little something in all 3 ranksā€¦ @groma and I are thinking of a way to deal with empty bases but at the moment it is a division by 0 so the machine is not too happyā€¦

1 Like

gotcha. thanks will try again shortly!

1 Like