# Using an MLP regressor to control synth params (Re-usable GUI Class)
**URL:** https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148
**Category:** Code Sharing
**Created:** [December 7, 2021, 12:13am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148 "2021-12-07T00:13:06Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [December 7, 2021, 12:13am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/1 "2021-12-07T00:13:06Z")
Here’s what I did to create a re-usable interface for an MLP regressor to use with any synth.
Basically you can take this base class and extend it to use with your synth.
A couple of notes:
- The MLPGUI Class expects a `makeSynth` method to exist in your subclass. Maybe some OOP nerds can help me make this look nicer and error-proof.
- You will need to override the n parameter of the base class in order to declare the number of parameters to be controlled by the MLP Regressor (see next post for an example) otherwise you will have an error.
- Your synth should accept an output buffer from the MLP regressor (here called y\_buf) as an input argument. Note that there is currently an open PR to allow `FluidKrToBuf.kr` to be initialized before the buffer is created (for example, if you are compiling a SynthDef on the server). In my example, I am setting the buffer size of `FluidKrToBuf.kr` to the n parameter of the class which is equal to the total number of parameters.
[Buf to kr enhancements by tedmoore · Pull Request #42 · flucoma/flucoma-sc · GitHub](https://github.com/flucoma/flucoma-sc/pull/42)
So far it works pretty nicely for me. Here is the main class:
```auto
MLPGui {
var s,n,name,
<>w,<>synth,dsin,dsout,nn,prediction_buf,x_buf,y_buf;
*new { |s,n|
^super.newCopyArgs(s ? Server.default,n).init
}
init {
fork {
dsin = FluidDataSet(s); s.sync;
dsout = FluidDataSet(s); s.sync;
nn = FluidMLPRegressor(
server: s,
hidden: [7],
activation: FluidMLPRegressor.sigmoid,
outputActivation: FluidMLPRegressor.sigmoid,
learnRate: 0.1,
batchSize: 1,
validation: 0); s.sync;
prediction_buf = Buffer.alloc(s,n); s.sync;
x_buf = Buffer.alloc(s, 2); s.sync;
y_buf = Buffer.alloc(s, n); s.sync;
};
Task { this.makeGUI(n) }.play(AppClock);
}
makeGUI {
var counter,predicting,multisliderview;
counter = 0;
predicting = false;
w = Window(name, Rect(0,0,1000,400));
w.userCanClose = false;
multisliderview = MultiSliderView(w,Rect(0,0,400,400))
.size_(n)
.elasticMode_(true)
.action_({ |msv|
msv.value.postln;
y_buf.setn(0, msv.value);
});
Slider2D(w,Rect(400,0,400,400))
.action_({ |xy|
[xy.x, xy.y].postln;
x_buf.setn(0, [xy.x, xy.y]);
if(predicting, {
nn.predictPoint(x_buf, y_buf, action: {
y_buf.getn(0,n, action: { |prediction_values|
{multisliderview.value_(prediction_values)}.defer;
});
});
});
});
Button(w,Rect(800,0,200,20))
.states_([["Make Synth"],["Stop Synth"]])
.action_({ |b|
b.value.postln;
if (b.value == 0, { if (synth.isPlaying == false, { synth.run(false); "stop my synth".postln }) });
if (b.value == 1, { if (synth.isPlaying == false, { this.makeSynth; "make my synth".postln }) });
});
Button(w,Rect(800,20,200,20))
.states_([["Add Point"]])
.action_({ |b|
var id = "example-%".format(counter);
counter = counter+1;
dsin.addPoint(id,x_buf);
dsout.addPoint(id,y_buf);
});
Button(w,Rect(800,40,200,20))
.states_([["Train"]])
.action_({ |b|
nn.fit(dsin,dsout, action:{|error|
"loss: %".format(error).postln;
});
});
Button(w,Rect(800,60,200,20))
.states_([["Not Predicting",Color.yellow,Color.black], ["Predicting",Color.green,Color.black]])
.action_({ |b|
predicting = b.value.asBoolean;
});
Button(w,Rect(800,80,200,20))
.states_([["Print Input Dataset"]])
.action_({ dsin.print });
Button(w,Rect(800,100,200,20))
.states_([["Print Output Dataset"]])
.action_({ dsout.print });
Button(w,Rect(800,120,200,20))
.states_([["Randomize Parameters"]])
.action_({ multisliderview.valueAction_(Array.fill(n, { 1.0.rand }));
});
} // end makeGUI
}
```
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [December 7, 2021, 12:14am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/2 "2021-12-07T00:14:10Z")
and **here** is an example using @tremblap 's famous _Fart Synth_ by extending the base class:
```auto
*new { |s,n,name|
^super.newCopyArgs(s ? Server.default,10,"chaosSynth").init
}
makeSynth {
fork {
synth = { |y_buf|
var val=FluidBufToKr.kr(y_buf,n), osc1, osc2, feed1, feed2, base1=69, base2=69, base3=130;
#feed2,feed1 = LocalIn.ar(2);
osc1 = MoogFF.ar(SinOsc.ar((((feed1 * val[0]) + val[1]) * base1).midicps,mul: (val[2] * 50).dbamp).atan,(base3 - (val[3] * (FluidLoudness.kr(feed2, 1, 0, hopSize: 64)[0].clip(-120,0) + 120))).lag(128/44100).midicps, val[4] * 3.5);
osc2 = MoogFF.ar(SinOsc.ar((((feed2 * val[5]) + val[6]) * base2).midicps,mul: (val[7] * 50).dbamp).atan,(base3 - (val[8] * (FluidLoudness.kr(feed1, 1, 0, hopSize: 64)[0].clip(-120,0) + 120))).lag(128/44100).midicps, val[9] * 3.5);
Out.ar(0,LeakDC.ar([osc1,osc2],mul: 0.1));
LocalOut.ar([osc1,osc2]);
}.play; s.sync;
synth.set(\y_buf, y_buf);
}
}
}
```
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [December 7, 2021, 12:19am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/3 "2021-12-07T00:19:19Z")
Thank you to @tedmoore for his help with this. If anybody has any comments, suggestions or improvements please let me know and I would be really eager to hear them.
In the coming days I will try to do something similar with a KDTree classifier to provide re-usable abstractions which bundle synchronous code, common helper methods and some kind of basic GUI.
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [December 7, 2021, 12:22am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/4 "2021-12-07T00:22:52Z")
One useful improvement that I can think of off the bat would be a way to save the training sets. I will eventually get around to this but if someone feels like helping with that it would be very much appreciated.
---
### Author:  [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)
#### Post date: [December 7, 2021, 12:51am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/5 "2021-12-07T00:51:28Z")
At the workshop did @tedmoore or @tremblap show you the ability to save the internals of any of the data processing objects with the `write` method? You can dump them to JSON and then load them back in which is handy.
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [December 7, 2021, 1:05am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/6 "2021-12-07T01:05:56Z")
Now that you mention it I think they did say something about it.
In this case would it make sense to save both FluidDataSets? I see that `FluidMLPRegressor` also has a read method.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [December 7, 2021, 8:03am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/7 "2021-12-07T08:03:26Z")
> [@heretogo](#):
>
> @tremblap 's famous _Fart Synth_
I will henceforth call it that way, with due credits given, obviously 🙂
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [December 7, 2021, 8:06am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/8 "2021-12-07T08:06:50Z")
> [@heretogo](#):
>
> Now that you mention it I think they did say something about it.
We did indeed 🙂 The advantage of saving the training data is that you can retrain, but if you are curating various subset of the _fart synth™_ control space you don’t really need it… and this is where saving the MLP training is great: you can save them as various preset and recall subspaces, for instance have a training for long sounds, a training for noisy iterative ones, etc
If you haven’t seen it, check the video explanation of @spluta as this is what he does with a much crazier fm synth!
---
### Author:  [@tedmoore](https://discourse.flucoma.org/u/tedmoore)
#### Post date: [December 7, 2021, 11:55am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/9 "2021-12-07T11:55:02Z")
> [@tremblap](#):
>
> If you haven’t seen it, check the video explanation of @spluta as this is what he does with a much crazier fm synth!
You can find it [here](https://youtu.be/F5mwfVr0nU4)!
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [January 3, 2022, 3:19pm UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/10 "2022-01-03T15:19:03Z")
Hey PA: I notice that there’s a weird click whenever I start the _fart synth™_. I think this must have to do with the `LocalIn.ar` feedback loop. Do you get this click as well?
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [January 4, 2022, 10:35am UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/11 "2022-01-04T10:35:02Z")
> [@heretogo](#):
>
> Do you get this click as well?
I didn’t notice - how do to initialise the 10 faders seem to change the starting sound more than anything. Did you try with something that is somewhat static ?
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [January 4, 2022, 2:24pm UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/12 "2022-01-04T14:24:53Z")
So I tried two things just now. Before I had been initializing the synth with an empty y\_buf. I tried writing an array of zeroes into the buffer. This didn’t solve the click.
Then I replace Out.ar with OffsetOut.ar and this seemed to solve my issue.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [January 4, 2022, 2:59pm UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/13 "2022-01-04T14:59:55Z")
that is good to know. I’ll have to ask @tedmoore about that UGen and its implications in a feedback network as I don’t see how it would change things…
---
### Author:  [@heretogo](https://discourse.flucoma.org/u/heretogo)
#### Post date: [January 4, 2022, 5:57pm UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/14 "2022-01-04T17:57:35Z")
Hmm, actually I take that back. I’m getting the click again sometimes. I’ll take a look at it with Ted when we get the chance.
---
### Author:  [@tedmoore](https://discourse.flucoma.org/u/tedmoore)
#### Post date: [January 4, 2022, 6:03pm UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/15 "2022-01-04T18:03:11Z")
> [@heretogo](#):
>
> OffsetOut
Hmm. That UGen is just for sample accurate starting of a synth. It does say that if it is used with `In.ar` the input will read normally from the bus, but I’m not sure how it would apply to the `LocalIn.ar`.
If it’s useful, the thing I would do in this scenario is to just add a `Line.kr` before the output that ramps from 0 to 1 over 30 milliseconds so that each time the synth is started it has a short envelope.
---
### Author:  [@spluta](https://discourse.flucoma.org/u/spluta)
#### Post date: [January 4, 2022, 6:25pm UTC](https://discourse.flucoma.org/t/using-an-mlp-regressor-to-control-synth-params-re-usable-gui-class/1148/16 "2022-01-04T18:25:19Z")
Three things to try:
1. set the buffer to rands between 0 and 1 instead of all 0s
2. use .zap from from the SafetyNet quark on the output of the MLP. This will get rid of any NaNs that you may be getting right at the start of the synth. Add ‘val=val.zap;’ above osc1
3. clip your MLP outs between 0 and 1.
Sam