Ways to test the validity/usefulness/salience of your data

[tedmoore]

@rodrigo.constanzo

Maybe some kind of crude clustering after the fact where nodes are created where top right is “bright and pitchy” and bottom left is “dark and noisy” and then cluster the UMAP to that or something?

As @tremblap and you said, this does sound like you’d be rotating and/or “flipping” the space to get certain descriptors roughly on certain axis. One approach might be to analytically find the “brightest” and “darkest” (or whatever) points in your dataset, then rotate the space so the relationship between those two is along the Y axis (for example). Then find the “noisiest” and “smoothest” points in your dataset (whatever that means) and further rotate or flip the dataset to make that relationship generally an X axis. There will likely be some tradeoffs no matter what (these dimensions may not, will likely not, be orthogonal!)

Instead of finding the “brightest” or whatever, you could try to find the “centroid” of brightness with something like a weighted average.

@rodrigo.constanzo

Now, I remember a lot of lively discussion about normalizing, standardization, robust scaling, etc… I don’t know where that needs to happen in this process. Some things like loudness-weighted spectral moments, and confidence-weighted pitch would definitely go before PCA, but my gut tells me all of it should happen before then. That would potentially require a massive processing chain to treat each statistic of each descriptor differently.

For what it’s worth, approaching something like this in the past, I’ve had success with doing a grab bag of analysis, like you describe, doing BufStats on all of it, Standardizing, then PCA (90% variance or so like you say), then UMAP.

@rodrigo.constanzo

I guess the output of UMAP may need to be normalized as well before being quantized. Is that right?

I’m not sure. I think I have been normalizing the output of UMAP before using Grid, but it will probably give you results nonetheless. They may be different though because normalizing the output of UMAP will stretch it somewhat–changing the distance relationship between points.

@rodrigo.constanzo

Is there a way of getting the ‘scree’ plot natively now?

A little bird whispered to me that the data for making this may become more readily available.

@rodrigo.constanzo

Once you have the salience info, is there a newer or streamlined way to pluck out the desired statistics/descriptors post PCA and pre UMAP?

I haven’t continued pursuing this idea (PCA/SVM thread). Also, @weefuzzy offered a bit of a lit review if it’s interesting to anyone. Since you’re generally interested in determining how these slices are different from / similar to each other using the principle components will get you farther than plucking the same number of original descriptors. Yeah, you’ll loose perceptually relevant axes but the rotation above may get you back to a good place.

Also for what it’s worth, since you’re interested in having the axes be perceptually relevant, you could try to 2D plotting on what you’re interested in (pitch and loudness or centroid and noisiness or whatever) then putting that in fluid.grid~ and seeing how performative and musical that feels?

@rodrigo.constanzo

Or perhaps using color as a corollary dimension (red always = “bright” etc…).

With whatever plot you have, you could totally just make the ERAE display color based on whatever analysis you want. (I know you know that).

@rodrigo.constanzo

Maybe some kind of crude clustering after the fact where nodes are created where top right is “bright and pitchy” and bottom left is “dark and noisy” and then cluster the UMAP to that or something?

Or you could do some clustering / sorting before the fact. Break up your corpus in subcorpora using fluid.datasetquery~ for a manual approach or fluid.kmeans~ for an unsupervised approach. Then run your analysis -------> grid pipeline on each subcorpora and then put the “subgrids” in a bigger grid, such as quadrants or whatever. You could even rotate the subgrids to try to get them lined up in some important way (I’m just riffing here now…)