@weefuzzy @groma and @a.harker would explain more elegantly, but my understanding is that you give the OS less time to do what it has to do. in 512 and 64 (the default) the app can squeeze a lot of stuff in between the various smaller blocks and keep the large block happy. When both are small, there are a lot of things that need to be serviced in that small amount of time, including a lot of memory copying to the various processing buffers…
What I can say is that 1) if FFT does it we’ll do too, but 2) KDTree has been improved tremendously for large-ish datasets so I’m happy to have a 1-to-1 with you and we can try with the current tip to see if you feel the improvement?