# LTE - An alternative to LPT for people who don't care about "pitch"

**URL:** https://discourse.flucoma.org/t/lte-an-alternative-to-lpt-for-people-who-dont-care-about-pitch/605
**Category:** Pre-Release Toolbox2 New Ideas
**Created:** [July 22, 2020, 3:05pm UTC](https://discourse.flucoma.org/t/lte-an-alternative-to-lpt-for-people-who-dont-care-about-pitch/605 "2020-07-22T15:05:24Z")
**Posts on this page:** 1
**Showing post:** 9

<div class="post-metadata">

### Author: ![rodrigo.constanzo](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/rodrigo.constanzo/32/12_2.png) [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [March 7, 2021, 12:05am UTC](https://discourse.flucoma.org/t/lte-an-alternative-to-lpt-for-people-who-dont-care-about-pitch/605/9 "2021-03-07T00:05:28Z")

</div>

Now that I’ve tested some [PCA speed comparison stuff](https://discourse.flucoma.org/t/speed-comparison-between-different-dataset-processing-chains/807), I want to revisit this to see how this fares. I’m still [testing to see what descriptors/stats are most salient](https://discourse.flucoma.org/t/intelligent-feature-selection-with-svm-or-pca/788/61) in the first place, as well as being aware that [NaNs shit the bed down stream](https://discourse.flucoma.org/t/null-values-from-fluid-pca/803/5), so I have to be aware of what descriptors end up in the general soup, and I’ve come up with something that may be a useful conceptual anchor.

My thinking about this before was trying to come up with meaningful _overall_ descriptors for a sound given that 1) I don’t care about pitch so much and 2) I have very a very small window (256samples) to work with.

I think breaking things up in to an _ **LTE** _ (loudness/timbre/envelope) is still an overarching idea, but given some of the discussion on confidence above, I want to sprinkle in a bit of _ **P** _(itch), primarily to differentiate whether or not something is “pitchy” or not, but this is not nearly as significant as other dimensions.

So at the moment I’m spitballing this:

- **Loudness** (4D) - mean, std, min, max
- **Timbre** (4D) - loudness-weighted 20(19)mfccs, mean, std, min, max → robust scale → 4D PCA _(or MLP)_
- **Pitch** (2D) - confidence-weighted median, raw confidence
- **Envelope** (4D) - deriv of loudness mean, deriv of loudness std, deriv of loudness-weighted centroid mean, deriv of loudness-weighted rolloff mean

So that would give me a 14D space that encompasses the aspects of sound I’m interested in. We’ll see how well that works, but I have a hunch (or hope) that have an _ **E** _(nvelope) vector could be interesting. Plus it incorporates some additional perceptually-meaningful descriptors (centroid/rolloff).

I did think about including individual analysis frames as part of the _ **E** _(nvelope), as I only have 7 frames of analysis (at most) for my 256 sample window, but that wouldn’t scale up, and I think more generic contour descriptors like derivatives (or [linear regression](https://discourse.flucoma.org/t/additional-statistics-linear-regression/562) (or [something even fancier](https://discourse.flucoma.org/t/categories-descriptors-for-contours-envelopes-and-morphology/693))) may transfer from one “shape” to another, regardless of actual/fixed duration.

I still want to do [that prediction thing](https://discourse.flucoma.org/t/regression-classification-regressification/547) where I query for the next 4410 samples which are analyzed in a similar way (or perhaps something slightly different), to then have _two moments_ similar to @tremblap’s original approach. I would also do the same for the entire sample, though I wouldn’t be able to use that as an apples-to-apples realtime mapping.

The nitty-gritty of this will be a bit tedious unfortunately as it will involve a whole load of pruning steps along the way, particularly since all of these stats are non-adjacent, and as @tremblap unfortunately warns in the [Example 11 thread](https://discourse.flucoma.org/t/example-11-verbose/658/23):

> [@Example 11 - verbose](https://discourse.flucoma.org/t/example-11-verbose/658/24):
>
> …it is more efficient to _[take the stats you want]_ in one pass.

[If only there was another way…](https://discourse.flucoma.org/t/output-attributes-for-descriptors-and-stats-objects/669)

That being said, I’ll start poking at this and post code/results when I get to the bottom of it.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////

A couple of questions to end this long necro-bump.

1. Now that we have MLP, is that better (“generally speaking”) than PCA for large reductions (e.g. 76D MFCC space down to 4D _ **T** _(imbre) space)
2. In Example 11, there are normalization steps after most steps. If everything is going into `fluid.robustscale~`, is that strictly necessary?
3. Now that we have `fluid.robustscale~`, is that better (“generally speaking”) for prepping data for PCA(/MLP) → `fluid.kdtree~`?
4. Does the fact that `fluid.robustscale~` is _median_-centered (as opposed to _mean_-centered) become problematic for PCA (_mean_-centered) or MLP (-1 to 1 activations)?

---

_[View the full topic](https://discourse.flucoma.org/t/lte-an-alternative-to-lpt-for-people-who-dont-care-about-pitch/605)._