Hello, I’m having trouble understanding how i can acheive the following task in max:
I have a dataset containing mfcc analyses of a corpus,
and two additional datasets with each one mfcc analysis of a single sound.
now i’d like to be able to use these two single sounds as a sort of start point and end point to query my corpus, like a kind of “journey” from one sound to another.
What i mean by that is that i want to get at the end a list of indexes of sounds in my corpus that starts with the one closest to the start point, ends with the one closest to the end points and in between a given number of points that lie on or near the path between those two points.
Any pointers would be greatly appreciated :).
Depending on what you want to do exactly this can be somewhat easy, or very difficult. The easy-ish way would be to plot the corpus as a 2d (or 3d plot with color) using fluid.umap~ or fluid.pca~. Then do a nearest neighbor search to find the points that correspond with your “start” and “end”, then just traverse the space querying for the nearest match using fluid.kdtree~ similar to what happens in the Coalecence M4L device:
At this point this is basically a metaphor because the UMAP or PCA reduction of the space no longer represents the original “space” with its high dimensionality, but it likely sounds good, or rather, sounds like it seems like it should sound.
The more complicated way would be trying to work out an actual trajectory through the higher-dimensional space and figure out what points actually work well. I suppose that the first dimension of fluid.pca~ would give you some idea of what the line intersecting that higher-dimensional space will be doing, though I don’t know how to actually turn that into a path that you can then travel through.
The PCA page has a good video with an explanation of what I mean here:
https://learn.flucoma.org/reference/pca/
But as I said, not sure how to turn that first principal component into a traversable path.
2 Likes
Hi Rodrigo, thank you for your answer.
i‘ve thought of the first answer as well but i‘m looking for the more complicated answer, i.e. a trajectory without dimension reduction.
So if possible to just select in 13 dim the point closest to the start point, and the one closest to the end point and then kind of interpolating between those two points and selecting points along the way that are in a certain radius.
the pca way seems interesting but i still don‘t quite get how i would construct such a trajectory between points in that many dimensions.
Yeah it’s tricky to think in higher dimensionality. It’s worth building the easy version first as I imagine that will be (likely) the most perceptually correct version of it as UMAP/PCA tend to organize and flatten multidimensional spaces pretty well, just to get a sense of it.
I guess that’s something you can try. Smoothly interpolate between the two presets, then do a nearest neighbor match (via fluid.kdtree~) for whatever you get along the way. No whether or not that actually represents a “smooth move through the multidimensional space” is another matter, as the ranges could be radically different such that any given dimension can overly weigh things towards it etc… which is what made me think of the PCA idea, but it gets a bit more complicated as you have fixed starting/ending points you want.
I suppose with a high enough PCA transformation, there should be a “line” drawn through the space that closely intersects your two points, though I have no idea how you would figure that out, and then do something about it…
1 Like
Thanks, I’ll try it and report :).
2 Likes
