I have been working with an increasingly large dataset that I want to “knearest-scrub” around with fluid.plotter. Initially I thought that north of 10k samples in a dataset the fluid.kdtree~ performance degrades so much that, especially beyond 35-40k samples, it drags to a near halt.
I was reading this thread (though still haven’t finished reading all of it) and was thinking how I could speed up kdtree, and came up with an idea (that I’ll share in more detail in another coming post): scanning the dataset space with a kernel size and a stride and fitting a kdtree to each such kernel. As fun as it was to implement this in Max , at the end I realized that the culprit was not actually fluid.kdtree~ but rather fluid.plotter. I guess this is because jsui always uses the cpu-bound mgraphics backend, and therefore does not scale well.
So I made a similar thing in jit.gl.land and named it fluid.jit.plotter (but still open for suggestions). It does not do nearly as much as fluid.plotter as of now, it makes the scatterplot (with fixed point size and color atm), and highlights in red. This is all I needed for now, but I can see implementing a total fluid.plotter replacement if the community is interested.
I made an example patch to show that the responsiveness of fluid.jit.plotter does not really depend on the number of points in the dataset. In the test patch there are 100k points, but I have tested with 1M too, but it’s harder to capture that since the fluid.plotter starts to break the whole patch down.
I wanted to avoid using other dependencies from my home-cooked things, so now you have to make it into a bpatcher and manually resize it to 300x300 (which is for now the hardcoded size of the thing). But if this would become part of a package then one can always use the clippings folder with a saved bpatcher instance (BEAP-style), or I can also recommend bl.autobp that can do the same.
Let me know what you think, or if you have suggestions for it, and happy patching!
fluid.jit.plotter.maxpat (19.9 KB)
fluid_plotter_vs_jit_plotter.maxpat (12.1 KB)