@output attributes for descriptors and stats objects

So with @weefuzzy’s great abstraction for managing fluid.dataset~-level chunks of data, I had commented that it’s still a bit complicated for things at that descriptors and stats level.

In response to a question from @alicee at this last week’s geekout session, @tremblap responded that after a few times you start to memorize what indices/channels are what, and I have to say that even after a couple of years of involvement, each time I do it, I have to stop and count and think.

I think a big part of this is, that by default, you get loads of stuff you don’t ask for from every object along the way, and then there’s a (confusing and error prone) pruning step before you get onto to flattening.

So what I’m proposing is having something like an @output attribute for the descriptors and stats objects where you specify what what you want them to return.

If I’m after the mean of centroid only, I could then have fluid.spectralshape~ @output centroid -> fluid.bufstats~ @output mean, and that would save a whole load of pruning and “indices math” after the fact.

As it is, it’s a bit confusing, but manageable to to remember what order the 7 stats come in, and the 7 spectral moments, but once you start wanting only a specific stat of a specific moment, it gets a bit messy, particularly if you get into derivatives where you have to work out some maths to iterate over the stats to leapfrog from derivative to derivative.

Also, given the small amount of stats (and spectral moments), one could use words to refer to the outputs you want (e.g. @output flatness or @output 6 if you prefer (I think that’s flatness?), and/or @output mean std min max or the @tremblap method @output 0 1 4 6)).

With something like this in place, you could skip a big part of the pruning process (for 90% of the use cases), and then use @weefuzzy’s names abstraction for things once you get up to a fluid.dataset~ level.

And in an ideal world, this would also skip computing the descriptors/stats you don’t ask for (even if they are generally interrelated).

This is another thing where I’d suggest a / some helper abstractions to soak up the UX pain (perhaps temporarily) because it’s not something we’re going to address immediately. Not that I disagree that these interfaces could be nicer.

1 Like

Giving this a bit of a spiritual bump with no new concrete information/suggestions (I quite like and stand by my initial suggestions here).

As I mentioned in this thread I’ve been revisiting some patches and particularly when it gets into higher dimensional descriptor spaces, so much of what I’m doing is removing/pruning things I don’t want, and never wanted in the first place (every statistic of every descriptor I encounter along the way). So situations where I want the mean of spectral flatness, or the max of 20MFCCs, I now have significantly larger datasets which I have to extract data from, using only indices as there is no symbolic naming along the way either.

Also something I hadn’t really considered at the time of this original post, and was re-surprised to re-read in @tremblap’s explanation in this thread that having non-adjacent columns in a fluid.dataset~ means that it can be more expensive (including adding more interim steps/buffers/datasets) to extract the specific stats/descriptors you want because of how buffers are flattened. So not only is it significantly faffier (my main point for this thread), error prone, and expensive on the front end (presumably it’d be cheaper to only calculate what you need), there’s also another performance (and faff) hit on the tail end when you then have to sift out the stuff you only wanted in the first place.

Was about to make another thread saying that it would be great to have a @peak flag for fluid.bufloudness~ to optionally not compute the peak of the signal for the second channel output, but this could easily fall under the blanket of an@output attribute as one may potentially be interested in the peak output only, and don’t want the loudness.

This is both in terms of saving a bit of CPU for things you don’t want, but also to have less numbers to prune along the process.

I’m just starting to code the LTE(p) idea from the other thread and I’m over an hour into just trying to figure out what indices/stats I’m interested in moving forward (as well as double-checking the numbers/frames along the way). So far I’ve not even finished the L step…