I’ll try some more aggressive reduction. I was just going with a roughly 25% (well, 10% here) suggestion that @jamesbradbury made. It’s kind of shooting in the dark though.
I do remember you getting kind of funky results where the first few dimensions kind of did the heavy lifting and the petered out from there. Some way to validate the compression (or however that would work) to see how many (few) dimensions is “good enough”.
For my purposes a 2d plane isn’t necessary at all, so as long as it’s less than “all”, it will all be gaining speed back in querying.
Hehe, I couldn’t be bothered to tidy up the guts of those subpatches since most of that is workarounds for a current bug.