Yeah totally. The idea is to more-or-less isolate the visualizer part, and the rest can be handled elsewhere. So this can be a loudness/pitch map display as well, etc…
The whole thing took the better part of the day (somewhat on and off).
I was able to get something that displayed 2D points hacked together from the fluid.pca~ helpfile, but it was suuuuper slow (for >100 points). After figuring out fluid.buf2list was the speed bottleneck, I used the dump-based approach like the fluid.umap~ helpfile uses.
So that much wasn’t too bad, an hour or two of solid work (spread out in my case). It still had all hardcoded names and such, so generalizing it was a faff.
The color thing wasn’t massively difficult once I realized/remembered I can just use swatch to do the HSL function for me.
What took the longest was working out an overlay for the playback display (mainly cuz I suck at jitter).
Tidying obviously takes a while too, particularly if I’m going to Rod it up (no color coding here though).
I answer some of this in the previous step, but the logic/headspace of managing a unique-name-per-step is just a general PITA as everything breaks with a typo, and each fluid.dataset~ needs extra code around it to display the contents to figure out where things broke. So in general the “buffer problem” added loads of friction.
Other than that it wasn’t too bad really. Once I added the playback puck I realized I needed to scale the space back a bit as some points were being plotted weirdly (which I didn’t notice while just using normal hover), so I guess that’s a scaled data → jit.pwindow~ thing or something.
This is using my “best” recipe from the time travel thread, which gave me the most differentiation with the lowest number of natural dimensions. In this case 20MFCCs (minus MFCC0), loudness, and pitch, with just the mean of everything.
I am excited about the prospect of being able to hone down to a more meaningful set of descriptors/statistics via the PCA method @tremblap mentioned in the last FluCoMa chat. At this point I’m a bit tired of manually testing different permutations manually (as in the other thread) mainly because the patch has to change pretty drastically for each version of that.
I did think of including point size as well, but I think that 3D is probably already pushing it (for the purposes of this kind of browsing/inspecting).
This is very cool. And I’m in agreement with @tedmoore that it would be super handy to see this as a lookup table.
It’s interesting how different the perceptually differentiated ones are from what one would think. I guess it makes sense that we wouldn’t necessarily perceive things in a uniform(ish) way.
Having a quick look through that page I’m not entirely sure how one goes about doing that however. (Python is not my strong suit)