Fluid.kdtree fit hangs above 500k points?

As I posted in this thread, I’m getting some funky behavior above a certain fluid.dataset~ size when trying to fit a fluid.kdtree~ with it.

I get the feeling that fit-ing scales up exponentially as the fluid.dataset~ gets bigger, which could explain this. But I left the patch below going for a really long time (I think 10min?) and it was stuck pinwheeling.

After force-quitting Max I get some error reports that have a bunch of FluCoMa stuff in them (I guess because that’s what it was doing when I quit it):
Archive.zip (306.4 KB)

This could very well be expected behavior if kdtrees don’t scale up nicely, but wanted to post my findings and example code just in case.


----------begin_max5_patcher----------
2786.3oc0bk1aaibF9y1+JHDPAZAbTm6i9k1jtso.MYaPSJBJVTXPIM1gqoH
EHo7wtX8u8NGTTjRT7PdjsiEfxHJN58548Xdmg4WO+rIyRuWkOI3OE7SAmc1
ud9YmYuj4BmU94ylrL794wg41aax7zkKUIEStv8cEp6KrWGMMHuHLSEDVDT7
MUvpnj69lREOU+WvUoYpaUY+4MyJNJQMOcchcpnxKtJSkq+kCKhRStr06HY8
xnjXUgkSfalVXw7uEkb8kYp4ENIgB.SoWDfIroLBs5Ei.uH.RASA5uCOED7+
196lttXyOLn7pQKrBV5re9MrIlK8amet4sKdhZJ3zfkg2nBxSWpUSoQIE4A+
9B8UxCBClEU7G5TKMTc.DRr5.n48iRnwGiPuTkmGdsZOg9pnhf7nkqhUzEgE
gSZSZPcYQISARNkBoBBhf4bhVbjvoPfbyKH.yuHPvLRJBcPIciNycohGVobj
XxjpITSIPEGiVHQcmdx6qDhWGsX5MKJxTpGmLZPcKp.BVaX4npWDr9hbROp.
7gTAwQ4Z7pgQSCsCZWm3UjwprnsNICEOHsv5VA.XjWA.DlGA.GVV6vvi3biD
QAhofZ+gPCQXawulP7nDkookNJ1wXCKkKBSNEfYUuDXcTZFtG4BcPi3gPsDr
GE62D.Gq3BktjRP7TnX6KDQ6whjlu5XvrF8daRKF4QosHHJX1XgskxKTxlhj
0bRgFyKpG4E0k7psvyBSttc4F3S2Uk5lGKSaYST2pN.2uMmhlB2qPDvQmoxE
dtMwG5Sy95eIJPLZXNS5p3fOEipmYBoc0AGahIq89hs+6gf8PnGk++n1Foqj
bzw0vt7u5zvn5d5Dc7ZB0+1beVgRVXxhzkV49XEaLcJhVCryk5JRjGqXeHCM
+XD5YqKJRSFafLgvV6ryakr48wJHGNhkbREsyBWpJTYWpRBmEamI3kNJNwJ2
PnXZsbzLrs3ySVTbjvmUXNOVElMV3rKVU6UXR8ZElbeFyZ15qtRk0HoUveIO
b4p7CDJuCaOAev5yfb3wVfV0ZKNn0G5yb341ZSuJHbwBm132sn2L586OzJt.
5Jn0eq8zy4xo10MDL5H6t.fBvTR80e.X5357ScBch2W9so8C4phG6scD8iBZ
c02k8Z4onT5d42MBWXuAamh1oKdVt1b8l5o7z0Yy2PO2ullWqHyBUdQThsEb
0tG8RmtHPKMsZJFLgFBkzku+zojN6g1LzCkrriWnDnGJQ8.g1nW5V4g7Ek5S
4w8AgPCPjf9Pj1.e6FiS7AkXCQl7.gnOWzYP.beDdfO.BI7AcfOSpNwPg2vm
Jna.wUQ9P0YVoR+9Q6QHWJpvUqtUkkWdyVZnSZ+yoYlOJtv9wnD2GsYdmjot
MZy8yrWILSmHtPmEdclkulbOiLwM0zEprj0Q1YetQ5zjLu3g3cSKlnWgkcpu
c8hnzOWDVrN+xOpRV6JAPKTWEtNtnolX10WEEGOOM1wdmU8M5hKJydOw8sWT
d4p68mB.SQRhd8SZ0xTLDyQL6H8.JsVj1x4.2LI8Jmk.j4VILckET6Hgd00X
yHvNSCskV.nzQAfT.HtQ5KYJZu9zzkZ3VgIhapWo7pqxRWklswjpmpr59WWj
dcV3hHka+nrKL0Yhcpa2BX0eqUsa0Ik5zkZEZzl4TG3UYL9v54gOUCPEyc.a
v.0KMsBhJcufCnBgajffacZUVAFUSDy8xL+wcib+RmDiPoGYtJV6bnmWCQmi
.LG2.YbjC9H..HoAyn0wMlkDPflV2nkWIkvsxfffjbq1fxpl2UoIEaLj5X.+
qUpjfOGljG7Y0xnYowK1VDa374ZFuAYvHMN2ZLHazYFOCy0pybZAa4tSkKzp
VmsA.EB2H8uBB0btMlDgSgX6shjXsjYGQgZMTyIYVzfchWFkXhaop.RXIBxb
SaiGJpNhvKNDeLrnnKGBiR+pvMQmgMLF4Q+xlqC7FC8o2VbocZftXKamVpou
sMIPikz3GpyeP6Fww6B799x4dvSyyd264f5Dv1FUW8ti+4ayhBi6zkbXAuZZ
AjTSRNWjABGXsEFSAdntUVbxl21k8qgnQdCQ+0u72eUlxYXQDewy62NnDxvL
hEx.4DlTVNBH6DS9OdXQV50pjuXUrc.Mo.NyJZ.FECczAhfnAjsfvfPfqJJI
hQrpJpDpcbaN2phgaZKMUtYScpIGk4rOFmea7eY6NEPS0dtzkFOA6sx2T22g
g3BuAwscDKZ9Inz1swPwRtKBHVVFAvbIJ1On0F5SQUp28GcTX7NgxCSx1EJ+
dUh51vNAwjx3hsMpej3ACTdTz3oFOV5ar56psEcGnFi8JuPTgl.kif9ls9AS
RyWaL06CiRdbfbkFtWECZ+Q9m0xuKRid6h41yy.VB0191IgmPupXp+YdbzBU
VW7T6qpand36DU6f90slybfz3YTe8A0htzUsIErADwD1hhwmrsd8bYQ2OuHa
jQQNcLj4bF7X2UFLtvGOEbjOkreLcgJerXji.oO35QN0oL+w9CgzSH.uxMqW
Na7wy1Oo4If09Tz7h90Uu3.3OYdNHJd0UswmmmtRMlHFGkO01lcKwl+NUXgu
DNarffg5t+xgb9htZ9Y8V.cKKIALzTjmvL6eI8Zyp1FoQAyYDNzkLhfg1URR
ZrCIufUA+eV1yFK8Lpd+Z3spqRyV9neK0DLk9RpnWDlcyaRht9aEuwtb4SSG
8f8f17dmN7d27Z1oiVh.LzF31dSIHUAD4U6IR0tiHr+L62xgA2OcufT9YKmq
x.fxmbCe7SgFylgzYqcAibCQ5pmbCiLeWfT2cyKlEqzY2b6u36RSuYRO.ys6
kwdCNb84TuA4hRtYbHjssLlALt.0GAe5PjJ0h1.ADUI8wVRz8tRb5h78rrI3
8Kw60+2gsMestCZ8SrVaTQ+S6Pn7KZ7dqc3cDTYv2uWbStwslruZNZOuzEKE
aJkn+Rn2s2H.Io7nDHkXAubDWLf8khUkmjVEidiyl+jqkg2yLmY5gJS6nfqW
kWK3cQyEEwK82OxMZvaBr8nbO31hPPXJR1LYoIBf2sCMdLUNYkpJp1798G0Q
TatvbFytn0QOiUALvX8dBU0z6XXVQ2g2ebMcq1Y5y6s.v8LO7t30p2.G4p+w
Pd4l1uMXjz6gfJYvrz6RFMG97Djzwg+0GBGOCBPRp6LEv4XLx4sgf5x47OC9
9Lk5H3Pq1qpmZPlo9F+ybeTGYKoHbzrmf.XLW29X.MyYUkLITR8OO9uUKFM+
wqp5EVsnJPYFCeye+WUbb5ciWEhntiDC273FaYPpFUR667TQk.f3TH.nuGE.
61W79z3i.hT03DptRHp6zIUOInWXvUqyVEqFUwoDfqtFDEvjVLLRPYHw.qPi
Wc7BIPpNrgM1lI5gWcMyTyUQ218N20FloZsYaYNP8SqneXN2RTLvhQn46EPz
5xsH53fxx7G7xycs1IP50LI4pjE4u1xfrUI6eeuVOzp.NDvc60DgAjkGwcNz
qXmhvU8cRtq0gqODVjF7AypPm3M5mWntZcbbQuavxtGlSAkBcMykg4X2gdmo
8t5ZCuLOqKb6sxpVTfoCBUmyRuJTiVlPBtvUmAtBIiQRrT3gkp+BpOT4utdd
AtyzMmGF+dMALZK6pNwkJIsyc4dSzwwt7GhxJdH3uccZeMDFJk.W0kZbLobK
yY..F17f7dvrgC7zw2TwZ9OOjiQw5dz4r6fmQXyWUZBsO+6m+am++A3x3QoE
-----------end_max5_patcher-----------

Hah. I put the patch to run and made the post, and in that time it managed to fit. I guess it’s a nice chunky boi once you start getting that big.

Has some pretty big implications in terms of real-time usage though (as per this thread).

Aaaand, I don’t think this is a bug. So kind of ignore the bug-tag of this thread. I won’t delete it as it can still be an “issue” perhaps.

I managed to fit a 1min point fluid.dataset~, it just took over 20min of pinwheeling to get there.

Well, it’s not hanging, insofar as I can see it working in the profiler, but I’m wondering if there’s some low hanging fruit for improvement in terms of where it appears to be spending most of its tim e. Poor Gerard might be getting one of my special pull requests once we’re back off C++ sabbatical.

1 Like

<cough>
Yeah. Changing one character gets fitting a 500k dataset using your patch from the other thread down to about 1.5s.
<cough>

2 Likes

With my extensive knowledge of C++, I can only imagine that this extra character was adding another “+” to the end of C++, which makes it extra-super-fast, as opposed to just super-fast.

Exactly correct

1 Like