# New Object Early Community Build #4: LSTM (and updates on DataSeries and DTW\*) MAX+SC
**URL:** https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224
**Category:** Development and Maintenance
**Created:** [March 15, 2024, 4:57pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224 "2024-03-15T16:57:18Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 15, 2024, 4:57pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/1 "2024-03-15T16:57:18Z")
ok people. here it is.
1. I’ve updated the interface so it loads right from JSON
2. I’ve removed the T in the printing to avoid confusion
but the fun stuff: a recurrent neural network and 3 objects! The usual Classifier and Regressor with their classic example, and a forecaster - predicting the future!
Now we chose LSTM as a RNN. Feel free to watch any of these.
[https://www.youtube.com/results?search\_query=explaining+LSTM](https://www.youtube.com/results?search_query=explaining+LSTM)
I will need to find more exciting things to do, I have an idea now, but I won’t wait more, but I’ll post here when I’ve tried.
Comments and feedback (pun intended) welcome
p.a.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 15, 2024, 5:05pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/2 "2024-03-15T17:05:02Z")
Max, MacOS, fat:
[max-v3.zip](https://discourse.flucoma.org/uploads/short-url/3OT4N8tEZlWY6eIqiU20PvlxzN9.zip) (3.8 MB)
SC, MacOS, fat:
[sc-v3.zip](https://discourse.flucoma.org/uploads/short-url/8PxfJXyrI3Vs25xcG6G1Sy3iZda.zip) (4.5 MB)
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 15, 2024, 5:35pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/3 "2024-03-15T17:35:37Z")
after that musical example, Pd is on its way
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 15, 2024, 5:36pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/4 "2024-03-15T17:36:32Z")
oh and if anyone has a good musician friendly LSTM explanation, the same way we explained MLP, I’m all ears. I’m watching all those above but…
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [March 15, 2024, 7:35pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/5 "2024-03-15T19:35:49Z")
I’ll look at it in more detail, but a screencapture/walkthrough would be helpful for these as it seems like things/presses are expected between numbered steps (i.e. just following the steps on their own does nothing in the lstm examples).
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 16, 2024, 10:26am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/6 "2024-03-16T10:26:57Z")
> [@rodrigo.constanzo](#):
>
> (i.e. just following the steps on their own does nothing in the lstm examples).
here it does so I’ll check again later today.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 16, 2024, 11:14am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/7 "2024-03-16T11:14:56Z")
LSTMclassifier steps 1 to 8 as described works here. fit until happy low number, obviously. Might have saved in query mode but that would give some read “no data fitted” which you didn’t report.
LSTMregressor steps 1 to 10 works here too, and so is LSTMforecaster steps 1 to 5 (6 and 7 are optional)
a bit more details would help
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [March 16, 2024, 11:35am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/8 "2024-03-16T11:35:30Z")
Yeah there was an error that was preventing from the first one of working. Then it wasn’t clear if I was meant to press/log/save something after each step.
With a clearer head this morning I (think I) got things to “work” but I don’t think correctly.
I can’t get the forecaster patch to make something other than a straight line (when training it on a “Z”):
Tried with a circle too, and with different amount of points into the future:
Ok, tried with something really simple (straight line), trying to do the same amount of points/orientation/pacing and made sure the fit got really low (0.03) and this is about the best results I can get:
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 16, 2024, 1:58pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/9 "2024-03-16T13:58:28Z")
ah welcome to the wonderful world of RNNs 🙂 This is why people who really know about this didn’t want to include them for us - they are INCREDIBLY finicky to tune. @weefuzzy would say it is irritating, @james would quote him saying it is handwavy, @balintlaczko might have wisdom, @lewardo too… @Chriskiefer is wise too. @groma too.
but listen to the videos - huge datasets, huge training time, huge machines.
so play with it and if we find it is useless, we just kill that branch and not include them. I almost didn’t make it public for that reason…
now I need others to play with them. i have a few ideas to poke at them too but it is a slow process.
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [March 16, 2024, 2:33pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/10 "2024-03-16T14:33:08Z")
In this case it’s more about not being certain that I’m doing the correct thing at all. The “line” version at least is in the ballpark of a line whereas the other two shapes aren’t doing anything at all.
And obviously the answer to everything is "It Depends"™, but does huge datasets here mean drawing circles 50 (50k, 50M?) times, and does huge training time mean 5min (vs clicking “fit” 5 times) or 5h etc…
The loss in that line example is super low, but it appears that the output is sinusoidal at almost all times (for this diagonal shape at least).
edit:
I guess to add, having “heavy” algorithms in the mix is nice, if they enable different/useful functionality. IMO not everything needs to be limited to things that can fit/build in like 30s. I am not bothered about having things train for an hour, or overnight, if it can do something cool. So if that kind of thing is the scale of stuff, knowing that, and knowing how to implement/tweak that is handy.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [March 17, 2024, 9:46am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/11 "2024-03-17T09:46:15Z")
more - as soon as I get how to tame the beast.
---
### Author:  [@philippe.salembier](https://discourse.flucoma.org/u/philippe.salembier)
#### Post date: [July 2, 2024, 11:02am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/12 "2024-07-02T11:02:03Z")
Hi,
I am interested in experimenting with the dtw~ algorithm. Although, I am somewhat worried by the brute force search strategy. I have a large dataset which may involve a serie of 20.000 descriptors of 12 dimension, and I would like to find in this dataset the best match with a smaller serie of about 20 descriptors. Do you think that the dtw~ is a reasonable solution for this kind of problem?
Moreover, I am currently running the flucoma 1.0.7 version in Max and I was wondering whether the build that is posted here (max-v3.zip) is compatible with the flucoma version I am currently running.
Thanks a lot in advance.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 2, 2024, 7:27pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/13 "2024-07-02T19:27:24Z")
dtw is currently brute force but @lewardo has promised fast-dtw implementation for the near future…
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 5, 2024, 1:36pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/14 "2024-07-05T13:36:24Z")
> [@philippe.salembier](#):
>
> Moreover, I am currently running the flucoma 1.0.7 version in Max and I was wondering whether the build that is posted here (max-v3.zip) is compatible with the flucoma version I am currently running.
I can build another version - let’s see what is @lewardo’s horizon for the fast-dtw and if not within the next week or two, I’ll compile the current LSTM+DTW+FluidDataSeries tip with tip of 1.0.7
---
### Author:  [@philippe.salembier](https://discourse.flucoma.org/u/philippe.salembier)
#### Post date: [July 5, 2024, 1:48pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/15 "2024-07-05T13:48:33Z")
Thanks for the followup on this topic!
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 9, 2024, 2:14pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/16 "2024-07-09T14:14:13Z")
ok just for you, I’ve compiled 1.0.7 with all the dataseries stuff inside. It is unsigned code and untested, for MacOS. Please let me know if anything goes wrong.
p
> **[FluidCorpusManipulation-24-7-9.zip | Powered by Box](https://huddersfield.app.box.com/s/er3ig7x32kiw2syi11msp9eiecl7vvwi)**
---
### Author:  [@philippe.salembier](https://discourse.flucoma.org/u/philippe.salembier)
#### Post date: [July 9, 2024, 2:38pm UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/17 "2024-07-09T14:38:17Z")
Thanks a lot. This is very helpful!!
I will test it and contact you if necessary.
---
### Author:  [@andolph](https://discourse.flucoma.org/u/andolph)
#### Post date: [September 5, 2024, 7:06am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/18 "2024-09-05T07:06:24Z")
Any more updates on this?
Thanks in advance
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [September 5, 2024, 7:22am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/19 "2024-09-05T07:22:16Z")
Hello @andolph
Thanks for your interest. There are many working objects in this thread for people to try and comment on. Have you given them a spin?
Feedback welcome
p
---
### Author:  [@pikseliahky](https://discourse.flucoma.org/u/pikseliahky)
#### Post date: [September 7, 2024, 9:59am UTC](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224/20 "2024-09-07T09:59:00Z")
Thank you @tremblap for this! I’ve tried the examples in SC and all seems to be running well. I am now trying to wrap my head around how difficult it would be to implement a prediction system for outputing continuous (control-rate) control data similar to e.g. Charles Martin’s IMPS written in Python (works fine with sending OSC to other applications, but a “native” SC version would be tempting):
[https://arxiv.org/pdf/1904.05009](https://arxiv.org/pdf/1904.05009)
> **[Interactive Music Prediction System (IMPS)](https://charlesmartin.au/projects/imps/)**
>
> IMPS is a system for predicting musical control data in live performance. It uses a mixture density recurrent neural network (MDRNN) to observe control inputs over multiple time steps, predicting the next value of each step, and the time that expects...
> **[paper10.pdf](https://ceur-ws.org/Vol-3124/paper10.pdf)**
>
> 1865.19 KB
[Next page](https://discourse.flucoma.org/t/new-object-early-community-build-4-lstm-and-updates-on-dataseries-and-dtw-max-sc/2224.md?page=2)