New Object Early Community Build #4: LSTM (and updates on DataSeries and DTW*) MAX+SC

spluta · April 10, 2025, 8:06pm

I posted this in the other thread. You can use this:

It will gladly do LSTM inference, but the training has to happen in python. I haven’t done much with non-audio effect LSTMs. The very rudimentary example below does a nice job with prediction after it is trained, but I haven’t tried to do anything else. The good news is the plugin is now fairly solid and shouldn’t give you too much trouble.

from my other post:

I have found the best repository for LSTM training to be AutomatedGuitarAmpModeling by Alec Wright, which is written using PyTorch. It is used in Proteus and Aida-x and ChowDSP things. I have found it doesn’t just train guitar amps. It works well with time series things as well. I made an example attached below, which will does time series prediction. I think it could be tweaked to do more complex pitch or word prediction. If not, such scripts certainly exist on GitHub. The python scripts used are included in my repository.


----------begin_max5_patcher----------
2226.3oc0bk0aiiiD94jeEDB6BLAvws3gtBvhE8SKlE6LOLy71lEFz1z1pit
VIpbzCl429vCotc5lxlRVVwIAsUrMkK9UGerphz8ue8UNKyelU4.tC7eAWc0
ue8UWodI4KbUyyuxIk97pDZkZXNaRxypScloeq30pWLe4mtE5hae0M4koTt7
c7adEwsjWySXb0GBp4UKn7U6hy1tnjshqmD9Aj4ty.99P4EOW4iHzbWv+q4d
zeL7WJX5avwYFvYIMaqyWFRAsjlx3rxErL5xD0.c+5DINqcd.ku1eb80xGlY
I9Wkmlxx3sPkydVMycBtAvKi2tkUB36XfRVUcBuBbq5YahKq3fMrm.owa2wA
KYfmJyy1BnUp2OMeMKAjwXqEOOGbuySzxTPcw8NsxIINisJuNi+J8mPJh4Bk
Gmms3UifXx9fbLZMb61ZDfBjF.r1nf7UWv38LGmM8o+Mfjb5Zk5gWRiyDyLi
dcvdipHn6bOgyk16BhvJv4NEnhr7FPE8QAjnkaY74OQez.nhh5MlfheTv.6q
.USjyz.JZCnhyJp6DSg826KJ7sBRd2.9k5rW47A9TsHFNI9AF3GAqyEfU81O
FulkC9gcbdQ0ce3CuHfV87krO3Vj+ueAGV+7O9v+bCixqKY+ipczR15a.TN.
dG16Nb38Y2m8TLem5iZcbEeghJPpDxDtGEu.pVUFWvkiKkJDck3yAjuQN9JA
uRBca0cfaK1.947LF31pU.Wvs0bv+4W+seBb6tJg9Rduh+PvMjrVx7PydgK0
zyUBcolyJlChqlAJJyWJnLeogURHK5pU0kTNS9orJO6QVI+0pEAe0u7a+LSL
nDfl3GnPzG+We7mVHmFK34KZGg.Q2mYjSC5YvmIn29L3HkyBow0AA+xkyuSC
VDFvk5Lo9oh8+qYYqXRJdpfJKun0iYY8lMrxp4Rsk3s3woxQWFypDpe153UR
57YfuZtUpaEeA.JBzRKRDep6XBxQsiPSbm3Oiq.rL48HWVgx2+Natuc4Iq0q
4jIl1sBjsF7HMol0NE2aNIs6+fPgEmVmBph+rXLBmJjqm6MFsidGcsIHzfk1
q+LdAQZZAk8NTY1kqSc9synazFG5WLxl36H8FQDO0JQ9tSNcGrAQ64dZBR39
Sg6EpACdxwT0NEgWpbRsWffIb0+THBcUHIvEd1vUF6Iwj66fUoHLZMvcNx.P
B6HuA3ALP3v8y2Nxh7sMCQz3AwB5pG.v4l.XX+AHQYa7BzqKDLX.NDaXmEKE
5MB0J0X6Bz9hWf0J0M7GiRECgJioG4RsTwtbuYrG9ytYiBI81GuUS3pWDz+3
ZBgkQp0M5niGUKMZ7bzemZoUYmYP0DDNT15VCMb5MzorpJ5V12AUYs55ZWTI
OtPm73hkLQIMeI4+OUkmYRQDzeOdcdEMr5HR3jtt0xZN2HR7GxBvJqHQGBiH
sO1MPlP+5tr1tlrht8G69XM10luHYWYdqS9PUlBqzD.6HMQxwMt5T58sHdMV
lIql+R9O4SMCYuQLkRdlt7cH.ZxoFO3rJ0ke6GMoIc0czIp2.AgUgkQjKvfy
NrlbvRPhoNkh6fl8.K7BIXcGeUEYiaZG2d3eSdFW1T.EFji8PpkY6alk2YlP
4nleerLll3btYs9aPvyOaTyz+Uhg9HkJwUyf4NXMSu0HnQTifMnM7BFP3tRI
f7rN8ryNM9eWTMtIvAGpkFEniA7siC+rCvOm.R46LAQ2ADlC2aadHP6xudDw
Xm9m.O.BHeDKt5J9UdEotJ+0U7Nh26fMhbOUCo+w43lP6f87skIc91lmIwu2
.g3pVEVm276iUxJoYqySA9lT.3gp.fDxkSmtp+brYdJRGMA.eb74hrMHtcQ4
1qSDu07sk40EcfaTuIuZwcn5he3EC4Uylgop.GapcsCHsinP8hQZX17joxId
3.s+oT3G3t2QN3BAnOUFyMsWO39SGCca.XSp0jKa.5MfZ6Quc3qyjBItheLA
v9mWHxOPWauJiPuKk7BkMhSSn+cfDM.9llMtKZ+jGP1m1vYqVNYoUwYxM0t5
0F0t22YjAMRzP0HM90sm5l2XGa8d1dfMU.2+hBZ.JVu6Rgt11q4YitOPGf9V
icqB4O3HYWOaaC4jDIWj+.6.lTT+OxDHh9vRDEzw9DcxaOPmUrf5+tZE.aNf
Gv2OkrbH12928QeL5cF6693ENT7dYxs1013AC6M0ZCNwP0EcqEtnnV0TOch3
9er7fjn8YdfiOyyA5Ft3WSnvu2kXC07m5vQBp+UXOQgmlBGg8OEdTfl9g38d
X64f3du8bHcy8fgAWxaOW0JZBS0pyagyMetnf8eskld91zz9vgetn7G21g0Y
EZCMZs4h+adCw5N2nAX77l7l4ptaUK1+luuPp4m70eM1qxqKW0J+18QF7043
ZVEONSUm1dCR8sHZuQkWtVDs2I+nsxVl94Qksr9XSh18jDs7PYdTQKOOfxAc
RRxJE72LcFIPJOaDGWzjyh9MzBQ2pZNMIgrAjgigjrwcMbLvTfMZuVfe9cQB
BOGQ+xCX1z3hHODTGGjRMA5TC1irURmLshMt8vwP6I6cyQkDdLvD1FmQYGV.
vSURSkxCZkxaDDjGzJHMF7fpyrjzDbPYIOwCmrcRKJWaD0oipPqzfAifrHgS
Ep7rg5ChNGo.Xkn8dsqzHszhJpCcDQSFiXAqXg8GKhjiEyA8Fk.AaraPytLn
SKtvF0oxtcpDKDa3kIjwHV2lrFIiwR.DaLbDzT4h7M4MLRrJXqVPcLnNsZ8f
QQRHqjzYoNXqVeENJLk1Da+MSmwBjdSFHitn7YPQmiB2fHaSSezsjVIZXvYA
0XaE8nmNCBNY9uV0QNzYoYfV4UAOKlVOaKG708GP27UZQwirxplQqDhSJ8S4
pIV3L0SiyzOU8krzoj8Xb63UGHZGZ4pcwb1J4+gjn5.8y95sl2Q9s2qLqNtA
kB3IDop80xuUBUETMRTc495+35+RyZIzB
-----------end_max5_patcher-----------

tremblap · April 10, 2025, 8:08pm

0 is a strange way to ask for the same output length than the querying input. if you ask for AL you would get IC (if lucky ) and if you ask for ALI you would get CES. If you write 5 and query ALI you should get CESIN

now it might be broken, or my understanding might be wrong… so I can check that with the same training you are supposed to get the same thing if you do it manually ALI and ask for 1 and get C so you enter LIC and you should get E and you enter ICE and you get S… @lewardo can you please confirm?

that is what we have here.

Definitely interface will change - not just subject to but will certainly do. So if you need stability for years, you need to go python I think…

spluta · April 10, 2025, 8:08pm

in the example I use PyTorch for the training, but if you want to use keras, it will load those models without needing any conversion. That is its lingua franca.

Sam

tremblap · April 10, 2025, 8:10pm

or PyThorch and @spluta 's object for now - all of it is good, the more we find ways to make sense of it and cross patch, the better for my questions above!

danieleghisi · April 10, 2025, 8:26pm

@tremblap @spluta – thanks so much !
I’ll look into Sam’s github page. I’m familiar with some implementations of recurrent neural networks in python, but I cannot go python unfortunately – this is a course for musicians for which I cannot assume any familiarity with textual programming. So, that’s not a choice, not even for training.

Can I suggest to get them in in the meantime as “hidden” undocumented objects?

that is what we have here.

Uhm, not really – in order to get them I had to manually patch the package. This isn’t all that elegant for me to propose in the course – or why not, I’m no longer sure.

In any case PA this wasn’t meant to be a trade war
I am just thrilled that there’s a Max-bound RNN object, and I am just a bit sad for the complication to include an example of it in the course, but good things take time. Of course I completely understand your point, of course the interface will change – do whatever is best for Flucoma, I’ll find my way around this!

Thanks for the great work to all of you, and thanks again Sam for the pointers.
Daniele

tremblap · April 11, 2025, 6:06am

I didn’t take that for anything else nor worse than good vibes, don’t worry.

For now, this is the interface. It will change, I hope as soon as possible, but this is a massive amount of work. @lewardo replied to me privately, I’m trying to understand it all, in good old FluCoMa fashion (“you only learn for the first time once”).

p

rodrigo.constanzo · April 11, 2025, 9:12am

Not much to add and a bit late to the party, but I did manage to get it to work, but with very poor fitting (though it did improve/change, unlike in your video):

tremblap · April 11, 2025, 9:31am

when I cannot get good fitting at the moment I just re-seed (clear) to get somewhere…

try a network structure of just [15]

danieleghisi · April 11, 2025, 10:14am

If anyone is interested, I’ve managed to produce a Flucoma patch that takes the Presto from Chopin’s sonata and produces a similar stream of notes via LSTM and temperature.
It’s commented in Italian, but just clicking on 3.4.5.6. shouldn’t need any translation
(1 and 2 are requirements: you need the dataseries objects from flucoma and you need ears for fromsamps/tosamps. You also need bach.)

https://www.dropbox.com/scl/fi/6a1vnq8v3kjbtgn1lkplj/LSTM_Chopin.maxpat?rlkey=zv0srnz61kla5jrheivsbtq6x&dl=0

I think I’ll provide this patch in the course as experimental, without commenting too much its technicalities. That may be a good compromise

(If any you spot anything weird flucoma-wise of course let me know!)
Daniele

tremblap · April 11, 2025, 10:33am

now now I’m toooooooo excited but I have to finish what I’m working on… will try very soon. @lewardo should be excited too. and I can practice my Italian!