I’ve been doing some more experimenting with this recently and getting ok results by taking the 104d MFCC (13 MFCCs + min/mean/std/max + 1deriv) and running it into a PCA based on some @weefuzzy code from this old thread. The idea being that you can specify an amount of variance to retain and it would then give you the amount of PCA dimensions to keep.
Perhaps counterintuitively I got better results the smaller amount of dimensions I kept, with around 3/4 PCs seeming to work the best.
After that I’m doing knearest and knearestdist (as above in this thread) to work out the distance to a fluid.kdtree~ with the means of the classes (also run through the PCA).
I suspect I’m getting better results from using a lower dimensional PCA into the KDTree because (perhaps) knearestdist is not an idea metric when wanting to interpolate between classes. The summed multidimensional distance jumps around quite a bit in a way that doing any maths on it afterwards gives me pretty erratic output.
That is to say, I do have a thing where I do get a lower number when playing one class, and a higher number when playing the other class, I just get pretty jumping values around that.
I’m having a hard time figuring out what to test/optimize here (more/less dimensions, scaling/transforming numbers, etc…). I’ve tested normalizing and that didn’t help, also tried some UMAP stuff with poor results too. I have a feeling that I need something other than knearestdist to compute the nearness to a class though.