Fluid.jit.plotter, for a better performance than fluid.plotter

Code Sharing
1

I have been working with an increasingly large dataset that I want to “knearest-scrub” around with fluid.plotter. Initially I thought that north of 10k samples in a dataset the fluid.kdtree~ performance degrades so much that, especially beyond 35-40k samples, it drags to a near halt.

I was reading this thread (though still haven’t finished reading all of it) and was thinking how I could speed up kdtree, and came up with an idea (that I’ll share in more detail in another coming post): scanning the dataset space with a kernel size and a stride and fitting a kdtree to each such kernel. As fun as it was to implement this in Max :upside_down_face:, at the end I realized that the culprit was not actually fluid.kdtree~ but rather fluid.plotter. I guess this is because jsui always uses the cpu-bound mgraphics backend, and therefore does not scale well.

So I made a similar thing in jit.gl.land and named it fluid.jit.plotter (but still open for suggestions). It does not do nearly as much as fluid.plotter as of now, it makes the scatterplot (with fixed point size and color atm), and highlights in red. This is all I needed for now, but I can see implementing a total fluid.plotter replacement if the community is interested.

I made an example patch to show that the responsiveness of fluid.jit.plotter does not really depend on the number of points in the dataset. In the test patch there are 100k points, but I have tested with 1M too, but it’s harder to capture that since the fluid.plotter starts to break the whole patch down. :stuck_out_tongue_winking_eye:

I wanted to avoid using other dependencies from my home-cooked things, so now you have to make it into a bpatcher and manually resize it to 300x300 (which is for now the hardcoded size of the thing). But if this would become part of a package then one can always use the clippings folder with a saved bpatcher instance (BEAP-style), or I can also recommend bl.autobp that can do the same.

Let me know what you think, or if you have suggestions for it, and happy patching! :slight_smile:

fluid.jit.plotter.maxpat (19.9 KB)
fluid_plotter_vs_jit_plotter.maxpat (12.1 KB)

fluidjitplotter_vs_fluidplotter

2 Likes
2

That’s definitely loooads faster!

Having some parity would be nice as I think I use the color/size things in places in my patches (sp.plotter mainly). I do miss that crazy 5d one we were building a couple years ago, but thankfully there’s a native way to do all that now, even if a bit slow for large datasets.

2 Likes
3

I don’t see why we shouldn’t include this in the next release at some point, even as a release on its own. I could help facilitate that @balintlaczko if you were open to it? I think as @rodrigo.constanzo pointed out some additional finessing to make it more in line with the existing non-GPU one would be all it needs.

2 Likes
4

when it is road ready, make a PR - ideally it would probably replace fluid.plotter if it has parity of feature and is faster, no?

5

I think that js and jitter differ enough in behaviour and learning materials, perhaps even just slightly cosmetically that having both is worth it.

6

As two separate abstractions?

I imagine that would make maintenance more cumbersome.

I don’t really use either (js / jitter) in the core of my practice so I have no horse in this race, but I’d think jitter is more “native” overall.

7

Assuming the drawing is exactly the same and has all of the same features (highlighting, zooming features) there is still the ergonomics of the resize and whether or not people want to have jitter in their patch. I think that the js version is longstanding enough at this point that a replacement poses risk in modifiying the behaviour of what is there, without serious testing, which as we know sucks in Max.

2 Likes
8

we could aspire to a deprecated status - slow replacement… let’s see how @balintlaczko and his team go forward with it, I’m excited!

9

I have a resizable version now:

2 Likes
10

FWIW @balintlaczko I am working on integrating this here if you wanted to contribute and collaborate:

2 Likes