# Regression + Classification = Regressification?

**URL:** <https://discourse.flucoma.org/t/regression-classification-regressification/547>\
**Category:** Pre-Release Toolbox2 New Ideas\
**Created:** [July 4, 2020, 5:14pm UTC](https://discourse.flucoma.org/t/regression-classification-regressification/547 "2020-07-04T17:14:49Z")\
**Posts on this page:** 1\
**Showing post:** 13

<div class="post-metadata">

**Author:** ![rodrigo.constanzo](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/rodrigo.constanzo/32/12_2.png) [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)\
**Post date:** [July 13, 2020, 6:08pm UTC](https://discourse.flucoma.org/t/regression-classification-regressification/547/13 "2020-07-13T18:08:32Z")

</div>

Ok, I’ve built a thing and got it “working”!!

[![](https://img.youtube.com/vi/2KE7eUQy-ms/maxresdefault.jpg "KDTree prediction") ](https://www.youtube.com/watch?v=2KE7eUQy-ms)

([actually had a gnarly crash, which I made a bug report for](https://discourse.flucoma.org/t/instacrash-with-fluid-bufcompose/565))

I’ve trained it on just 10 hits, and am feeding the training data back in to see how it fares. Surprisingly, it only gets the correct results 50% of the time, which is unexpected as I’m feeding it the same exact as it was trained on. The real-time version is running through an onset detection algorithms, so it’s not _literally_ the same data as it may +/- some samples, but I guess that has knock-on effects because of windowing etc…

_(I should note that the training data was, itself, segmented off real-time playing and the same exact onset detection algorithm)_

If I test it on the same literal `buffer~` data it works perfectly, so the system/plumbing is working.

Here’s the patch as it stands:

 ![Screenshot 2020-07-13 at 6.36.37 pm](https://discourse.flucoma.org/uploads/default/original/1X/408c78b8c8f7624ec3250bee2a26c7fdb06425dc.png)

At the moment it’s oriented around validation primarily, so I can play specific samples, see which ones are matched, as well as `fluid.bufcompose~`-ing a franken-sample so you can “hear” what the algorithm thinks is correct. Even though it was 50% wrong, all the composite versions still sound believable, so that’s good.

It will definitely be critical to weigh these segments differently, since the second half (at the moment) has pretty shitty accuracy.

Speaking of…

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

At the moment I’m using the following descriptors:

> loudness\_mean  
> loudness\_derivative  
> loudness\_deviation  
> centroid\_mean  
> centroid\_derivative  
> centroid\_deviation  
> flatness\_mean  
> flatness\_derivative  
> flatness\_deviation  
> rolloff\_max  
> rolloff\_derivative  
> rolloff\_deviation

These have been my go-to descriptors for a bit now, but they definitely aren’t robust enough for this application as an onset detection’s worth of slop in the windowing brings the legibility of them down to 50%.

_(it should be said that I’ve not standardized or normalized anything here, but the input is the same as the dataset, so it shouldn’t matter that much…(?))_

Off the bat, I can take all the spectral moments, along with all the stats, and perhaps even two derivs. Maybe throwing in a [manually-calculated linear regression](https://discourse.flucoma.org/t/additional-statistics-linear-regression/562/8) for good measure. Basically a _more-is-more_ approach, but primarily based on “oldschool” descriptors.

OR I can try throwing MFFCs in the mix, as I did have the best results with MFCCs and stats [when I was trying to optimize the `JIT-MFFC-Classifier.maxpat` a while back](https://discourse.flucoma.org/t/onset-based-regression-jit-mfcc-example/464/50).

That patch, notably, does _not_ do any dimensionality reduction, nor does it take any loudness descriptors (not even the 0th coefficient, looking back on it now). So it just brute-force takes a 96 dimensional space and matches based on that.

So I’m thinking I might do something like what @jamesbradbury is doing for his [macro-segmentation stuff in Reaper](https://discourse.flucoma.org/t/segmentation-by-clustering/510/11) and take MFCCs with all stats and two derivatives, and do some dimensionality reduction on that to see how things go.

This quickly ends up in territory as discussed in the [(ir)rational dimensionality reduction thread](https://discourse.flucoma.org/t/dimensionality-reduction-mixing-the-rational-and-irrational/534/23), where loudness is important, as a perceptual entity, as is timbre, so if loudness only represents 1/13th of the total dimensional space, that’s a bit of a problem.

---

_[View the full topic](https://discourse.flucoma.org/t/regression-classification-regressification/547)._