Another experimental object with the usual caveats: limited docs, examples, might change, might die, etc
Introducing FluidPolynomialRegressor: a dataset process enabling 1d to 1d regression via polynomials. if more than 1d is provided, it will be done ‘in parallel’. Implemented by @lewardo and quality docs by @MattS6464 - this one has the example in the helpfile
Again Max, MacOS and Windows, although only help is missing for Pd and SC for this one so any eager user of these CCEs is welcome to jump in, I can provide the binaries.
(if a Max person wants to do a preset interpolator for another tab that would be ace, like in is preset number 1 2 3 4 (duplicated to match the number of output dimensions) and the output is a parameter per dimension)
As the existing bit only works if you’ve named your entries the same way (I most certainly don’t!). The bit on the right is label name agnostic and will dump the whole length. (If less are wanted, a filter can be put after the zl nth 2)
In order to predict, does a dummy dataset need to be made ahead of time? Wouldn’t the interface be cleaner if you could just specify a size or dimensionality you want to predict to as an argument instead? Either that or having an easier way to make fake/dummy datasets outside of needing to whip up a little uzi/sprintf helper patch every time you want to predict.
edit:
I could be misunderstanding here. Is the 25point version also meant to mirror the shape you are wanting to regress to? As in, do you need to feed it data, then manually create what you think the regression will be before predict-ing? Or is the data in there arbitrary?
The first tab should hide all that mess at the top (similar to the p Generate_Data subpatch on the second tab. Most of the first tab (with regards to visual info/noise) is random data generation.
So the explanation is useful in a way, but I’m not entirely sure what it does, like in a musical sense. The reference file is also quite math/jargon heavy (is this a typo? It’s in there a couple of times: x to :math"y"). Granted, I’m not the mathiest dude out there but I don’t know what this means: " Linear regression is a special case when the degree of the polynomial is 1, meaning the highest power of x fitted is 1 (it find the best :mathy = mx + c to fit the data)".
I know @tremblap loves his didactic non-sounding producing help patches where “push this button to a see a number, now push this button to see a number that’s slightly different” is the thing being presented, but having some sound-producing stuff would be helpful towards understanding this.
(Obviously this is a work-in-progress, so these comments are meant to be helpful)
ok here it is with very very very rough example and not tested in kr mode. but it should get you started on a fat version (and maybe @tedmoore or @spluta will get excited too)
strange it is in the file but doesn’t parse. for now:
code::
s.boot;
// 10 random points and their square
~somepoints = 10.collect{var x = 1.0.rand; [x, x**2]}.flop
// load the ins and the outs in 1D datasets
~in = FluidDataSet(s).load(Dictionary.newFrom(["cols", 1, "data", Dictionary.newFrom(~somepoints[0].collect{|j,i|[i.asSymbol, j]}.flat)]))
~out = FluidDataSet(s).load(Dictionary.newFrom(["cols", 1, "data", Dictionary.newFrom(~somepoints[1].collect{|j,i|[i.asSymbol, j]}.flat)]))
~in.print;~out.print
~polyreg = FluidPolynomialRegressor(s);
~polyreg.fit(~in, ~out, {\done.postln});
// 100 points to draw the function
~question = FluidDataSet(s).load(Dictionary.newFrom(["cols", 1, "data", Dictionary.newFrom(100.collect{|i|[i,i/100]}.flat)]))
~answer = FluidDataSet(s)
~polyreg.predict(~question, ~answer, {\done.postln});
~arrayedanswer = Array.fill(100,0)
~answer.dump{|x|x["data"].keysValuesDo{|k,v|~arrayedanswer[k.asInteger]=v}}
~arrayedanswer.flat.plot
//compare with the real function
100.collect{|x|(x/100)**2}.plot
::
seems to work over here! The first time I ran it the first index of ~arrayedanswer was very slightly below zero (-7e-11 or something) so the plot origin was offset a bit…which maybe spoiled the dramatic reveal? It’s easy enough to avoid with ...plot(minval:0) of course…
Thanks for sharing! I’ll try to play around a bit with this tomorrow and have a report on your desk by the weekend!
@weefuzzy predicted that someone would see that - it is a known problem of having crazy overfitting (you are doing a 10th order polynomial aka ax11+bx10+cx9+dx8+ex7+fx6+gx5+hx4+ix3+jx2+kx1+lx0 to match a little number of items so it freaks out.
now try that tikhonov number around 1 and see how it behaves. we’ll explain better but it helps agains overfitting IIUC
Ah I see, thanks! Indeed tikhonov did it. I made a little demo patch where you can draw a squiggly line of noisy points in the left plotter and ask for the line that fits them on the right. Then you can turn the dial with the polynomial degree and see the effect. polynomial_fun.maxpat (18.9 KB)
Is my understanding correct that the Fluid objects’ action function is called after the OSC message is sent to the server, and not after a response from the server? I had to add a limiter to the degree and tikhonov knobs so as to avoid ERROR: Primitive '_FileLength' failed. and ERROR: FluidPolynomialRegressor - No data fitted …though they’re still thrown occasionally. Or perhaps my approach is totally wrong to begin with?
If I get a chance in the next days I can try to copy Bálint’s approach with FluidDatasetQuery; perhaps that will be more consistent!
the function is supposed to be called after the response. but I might have messed up the implementation.
but the tikhonov factor and the degree would refinitely need to refit, so bundling the param change with the fit message is probably the most efficient way to send that to the server (although I always need to reaccomodate myself to that paradigm)
