Fluid.buf2list performance issues (+ crashes)

Bug Reports and Issues Toolbox2 Related Bugs
1

So I’m tweaking some of the patch for the spectral compensation stuff, and I remembered that we have snazzy new abstractions for dealing buffer flattening!

So I’ve added fluid.buf2list to the patch and on a hunch I decide to see how fast it is, since I’m always dubious of js stuff in Max-land.

Turns out it’s slow. Really really slow. AND super jittery too.

Screen Shot 2020-06-14 at 11.24.06 pm
Screen Shot 2020-06-14 at 11.24.06 pm1254×338 49.4 KB

Here’s a quick test patch showing what I mean:


----------begin_max5_patcher----------
2342.3oc6bszbihqEdsyuBJu3NaR6B8FlUyr6t39OXpa4BaK6POXfBjSmtmZ
le6id.NfCfUrQ1IUSV3fkD5bNe5bjNOv7WOLa9prW3ky89Uu+va1r+5gYyzM
oZXV02mMeezKqShJ0Ca95r864oh4OZ5SveQnaGtZg21nRgWT5FOQ7tmNNjj3
T95rCo5wAqZbaVpHMZOWeu+WdxybQ75n5aIORr9o3zcKK3qEFtiP7W3+nGBg
T+C.zeCSW368+qtozC6yNHR3BMi52fPaiVqIDnps3MZxls5qegfm2Xfkw+PO
PnZ1ONqwo0SJP01e+vCpOd7ZArEdkYdkIYey6+380XgfW78wEx.ALm.Yn6Ej
AjPlHpP3sJJcWb5uL+RQFDDpvBBPCLFMK30CLv6Ev3WCLRQLqXiTXuXnI.fU
nAU+IfAGInA3ZnY0AgHKc.AiEnWloDsDgq+rG4pVFLMI9dN2LKyUpdyOdSMD
Qb3qDuPh3R64k7znUI7l3zUKmaSxjywP519FqdbnVb0eQot2ifB6SPm+3PBK
xFgcaVw9HMaQGKwOk+MI8eiAvddTpEXhOU+uvKESjfuTdj.SrZq4tvkiGm3J
I8GIRS8BdzdOx4EXHkd8JAcKnzNETnKznoragFM7SkFcMl3dMZ+OFZzUBr6z
nQgilFcOBpzuEuUCHhvP8IT.TfVDQWpHp0wGRWGAleyNNhf8annB7QNw5ERu
SVum0sCpu1ISHB3H+NfA2D+NNqfVsP6PA813fUOFu4dqyOHh2yK5z05euHNJ
wBzwr8kwUzArB.CuecmF09c4hM.1sK1vlrpTlL3QEfHmh3D9y7hxX4R9qryr
4Q44MZdViaQgheMSOQAOdro3TSSfiMUveNt99wGaMpPBXBIZcnvfmuPq8jRM
MYa3EoGhOtjpWOqXI8JmZcnLuJ9C8Bbc2MB.fXvdnI9F8QlrfFZnRMicIYq+
S9lFJTx0fbdZbZdAuTF5Ujnh2O18F91nCIhksPcvBRHADPCq+qqQWGuTmSUm
ZVyluqHdSVphkZstnZtl3x.+Il3ZaJZ5QjFk2wMK0djfTOcJiqTbnbUTgZYq
xVCV2oHKKocWGuuD9VQU24woomfohr796rPk8n96dUlry8CM25dJWdH0z6Ro
FhXYYzysQaQTRRkwd6o+knzX44Dbk0tIL0icZ1u4ox0EYIIsjWSOO2QOajZ7
q4eKdi3IMgVzX8VN737ZUp4GWk2DuiWJZ2lHZWY6VJEe2.5MZ5vpJK5kB997
DoTzd.Rak3RQ4SYeqrZf0JZMAfWSIXSK7l6b1p8S1A0ro0Qy2dhPoJACF+AH
0edTgs2TMb59df4suiF692ni5LmntKoqt7hmiR7hS81W5EW5UvyyJD7MdR7f
2b9hS2veow1NUmlTsCzEhOsNgo8oLq7ZIN8uQPeo7yD2GS6AIz3jLXPTsIHM
7Ywu473S4zyt6W2mO6dTUdx8Z0F6iAzB70JtlyRtFrs8I4u4z7OMf6Wj6ncU
.a0NAX8NAH3BhKwU70iqvOGaEfCtY6DP94Zm.LSoidi1Hf9wBa0ypc6ThYVe
z9kq5w58ze+dN8WMcdkRGPK8jN3I4bcEKMd.HcFnx2fNcB.diARiI7MAGCtT
bLRgZU.4ffWWZguFdhphm83zoFXT82MpVlcnXcsLVIJdsAIoe1h3zigt8GG2
0RMNqVSeu7.yRd.5PdfZIOHORzC3HdfXIOPcHNfsjG.NjGPui0BWwCPK4ATu
7PUi04dYtJj5MKMoOXYjPTDu5fvXF2LYRuqvbsKMJ6RxVEkbRDrcEq7CuJBu
y7.Jx1sKge9z0Stx7c1WgWfj6Y5NkzqHyi36ed.HjZU4JtfD9htc0XDvnJm5
bbYJ9vVkBHR6SqCSdO7iQUJpVmcmfhIehqRQM53tpTfYSUoXpJESUoXpJESU
oXpJESUoXpJESUoXpJESUoXpJESUoXpJESUoXpJESUoXpJESUo3pyCXNm+mv
+mLJ5AR2Glfa7Hayt3z80Yl9.c+6o.7yWJ8HgMBIiZdPvwpjq5fT5AW3OkDu
oj38yRR7FHHpej3sqH6P94BCHvDKeHnQwbgC5+JbH+Wa86KavvjtGgq+k1gq
2Uk.MaSU8tC.FpAG3k6OeyxaOXD42C3PcH4+3I3kmMXwvvFmTZ9wweMnx6H8
OsvEzsAWN7iXOL9bXR0d+HSFGQPK.EzYC86wW++PpNnahpiU0H.XB.FYblB4
W+46uHAv2cQBbdU.rKSJAuVxRqj+KOC.f2cF.tug4S+.DhM4CPXc3O.gXiLo
P3tyCv6bZe.eJBw9FD7rLzsxnc72D87W7WfjmmA8jWfQ9.l9JHIz2zV.gx7w
R2qA.JCfzMwBnXp5JDjERPxNQgHPnZTD.BgHp9H3PH.ZtBDDDH6D6SwLrpIF
hhBBUW4Kuh.jcBwJZpGOBiB0CyOPdMvvFPBPSSFAfIxwy.T43zsPQLC+PADJ
jp3G+.+P87S.gr.nZ9CPHnhKX02GAKCAgZDbXHQw9XR.loZA3KEapYBvX+Pk
nEvvPEmRXXrAbfR9iDnGku7JMYnDndTg.JIPyK.HkYlJDDP0bEDGFDZXTBAn
4JXH0.b.luDf0huJMKc9dxBOPhNBMukLzmPyXl2iHiUhNBFsmRwSzG6UPvlW
+ELFaTyXCy0u.LjN8tQIiGCBYf2SS..xAIkB45WlIoYwk7+Yn2uVlW.UUhGF
eohWY7tznjteVKcsPZdKiYBbx62RxxxsZEsJOXPeGHyc+FMAMdx7pCa2xKpj
YajVyiEM65ey7z+SSrqd077bzqG3tM4P7lERoGlzHYxGgk1c68aFXpYH0C7j
iWENM.CFUibH6r3h91Z6muAlN0WpJv5s9yF1zIpy6.0aVOrlPLKHTqnLlKsM
MIVG3dJi6gz9WGoo1PZ7X.uDanDaLnD1FJMJpL91PI3HPIjMVA0ry0QIPyf2
5iRDWXGXEoggtvN.Ar.egLmI0miz.2H01n+htejF5jsaQ1f2ifkDzpStFCaV
XfsTBbsTxlipXsLiGIEFqnLD5BEFnMGcAGiCTf1nahFEkSnsGGe0pL2HyMfM
laAtP0zJJehMZmplFmkO4IvPQ0SdxKN4ot3sOwE8+zVb5SZgNyh88DV7ve+v
+BTfstVH
-----------end_max5_patcher-----------

I guess this is easier to code with js, but is there a way to make the abstraction just use peek~ for better performance in context?

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Also, this example patch insta-crashes for me, like, a lot. If I delete fluid.buf2list it doesn’t crash.

I was initially going to save that for a proper github issue, but this is TB2 object, so I’ll just bundle it into this same post.

On my laptop (Max 8.1.4 / 10.14.6) if I load that patch up, it crashes almost immediately. Sometimes when nothing is happening, or double-clicking the buffer~. I’ve got 5 crash reports here, all of them seem very different, and several mention HIToolbox.

Here they are:
buf2listCrashes.zip (159.2 KB)

2

I tried to make a max external for what is essentially buf2list. In fact bufspill everywhere in my patches is that external but it’s still slow relatively. The fastest method I’ve seen is owen’s jitter method but that has issues with maximum sizes. The best way would be an external that leverages some kind of memcpy or something.

3

Hmm, not sure if I’ve seen @weefuzzy’s jitter spill thing.

Does this patch crash for you by any chance?

4

It didn’t throw an error till I closed it so probably something worth investigating there.

5

Did it crash when you closed it?

6

Yeah - a bit after

7

Shame about the crashes. Because this is only an abstraction with built in objects in there, we can be pretty certain that these would be C74’s. That said, I’ve not had any on 8.1.3, so I wonder if it’s there’s a new issue in 8.1.4.

As for performance, because it’s using js, things are bumped to the main thread. As we’ve discussed before, that switch incurs overhead which makes things like this unsuited to fast use on the scheduler. Medium term, the plan is to replace these last-minute, duct-tape abstractions with some proper objects. Meanwhile though, there won’t be other versions of the abstractions (by me) just to support this use case: as you say, an alternative with peek~ is readily available.

8

Yeah the crashes are weird. Particularly when several of the reports mention HIToolbox.

What Max version are you on @jamesbradbury? Wondering if it’s something with 8.1.4?

Indeed, peek~ works fine, it’s just not as tidy an abstraction.

I’ve not tested yet, but does the getattr-ing happen in the same thread/quickly?

And @jamesbradbury, what does buffspill do with multichannel files? For the most part I would just be wanting to dump the contents of something like a slicer out into list-land, but if I make an abstraction, might as well try to aim for parity across implementations.

9

It outputs the values in the channel prepended by the channel number so you can route them.

1 value value value
2 value value value
for a two channel buffer of three samples for example