Dimensionality reduction (mixing the rational and irrational)

[rodrigo.constanzo]

@weefuzzy

Absolutely not, no. In the case of PCA, the output range will be in terms of standard deviations (so mostly within ±3 for data that’s normal-ish distributed). For MDS I’m not so sure, but you seem to be getting some values at least comensurate with your largest dimension in the screenshot above.

After winding down for the day I thought to try PCA since it’s the “dumber” of the choices, that it might go with what I’m trying to do better.

@weefuzzy

In the un-normalized case, you have dimensions of sugnifciantly different ranges, which is going to give unpredictable results. I would definitely normalize / standardize here.

I wanted to keep a sense of the difference in scales between the (currently random) entries, but I guess that would still be the case if I either standardized/normalized them anyways(?).

@weefuzzy

Yes, that’s a sensible thing to do, although I don’t completey grasp what matching in a contextual way is getting at?

Me neither!

I guess one a simple/pragmatic sense, I could “mix” a bunch of related descriptors together (duration/timecentroid) and then use this single “timeness” metric to bias the query.

So rather than having to chain things together (duration > 1000 and timecentroid > 500), I can just query for timeness > 0.6.

This is likely just me entrymatcher-soaked brain trying to make sense of a knn-y world though.

@weefuzzy

(If you think there’s a standardize bug, please do report it)

I’m nowhere near confident that it’s not a meatspace issue with that, but if I find it, I will do.