Making a well-sampled MFCC space for audio querying

[rodrigo.constanzo]

@staysh

Can anyone point me to some resources on corpus building? More specifically, wanting to take a bunch of audio and reduce redundancy via the 7-15 dimensional representation of it (MFCC). I’m working with flucoma in Max/MSP, but I’m comfortable in any language/environment if there are other workflows for this.

I’ve not watched all of the new videos, but @tedmoore did an absolute massive dump of workshop videos a few days ago which may be of interest. These in particular:

FluCoMa: Sort Sound Slices by an Audio Analysis

FluCoMa: Concatenative Synthesis on Real-Time Audio

(p.s. @tedmoore the thumbnail of this gives me Max anxiety…)

There’s also the classic vid by @jamesbradbury:

Building a 2D Corpus Explorer (Part 2)

@staysh

A bit more about my use case: I am slicing audio by novelty in mfcc analysis, then doing kdtree lookups via mfcc analysis of live audio. I would like to make the corpus as small and potent as possible. For instance, loading new audio as I collect it and incorporating slices from it where there are gaps in the corpus, and thinning out denser clusters. Real-time look up performance is fine on 13 dimensions, with well over 10,000 rows in the dataset.

It’s been my experience that dimensionality has no (essentially) impact on speed. A KDTree is fast/optimized such that whether you’re looking through 13d or 250d data, it makes little difference.

@staysh

I’m looking for legibility, meaning for me, slices of input mapping closely to slices in the corpus. I figured having a highly curated, generalized corpus (at least in terms of it’s mfcc analysis) is one strategy.

This is, ultimately, where the secret sauce lies. Finding the descriptor space that makes the most for what you want to display/browse/represent. Be it a “simple” 2d thing ala CaTaRT (loudness/centroid) or a more complex/processed MFCC space, or a blend of things (LTP, LTEp).