Bringing that up to 6d pulls the accuracy up to 44%.
I initially tried 20d, since that’s more in line with the 25%, but after some terrible matching I noticed that most of my dimensions were filled with nans. Don’t know if it’s the standardize/normalize-part of the process, but after around 7d (down from 96) I started getting numbers with e13 on the end of them, and the next dimension started nan-ing.
So presumably efficacy has dropped off by that point, but it’s crazy that it got so nan-ny so quickly.
So short of trying some other data sanitization routine, or if there’s something weird going on with the nan stuff, the dimensionality reduction in this context comes at too big an accuracy loss.