TLDR for all the posts above, but some quick thoughts.
1 - the bottom line here is that the issue is deferring a time critical process - that Max deals badly with threads in terms of user ciliary is pretty obvious from the amount of confusion here, but the reality is most max users thing if a metro is trigging something (or snapshot~ etc.) then it happens straightaway, with no delay. Suffering the defer for something expected to take a long time is fine, but for something fast it is not.
2 - the deferring is all about resizing buffers/nothing else, but it is brought about here by the use of storing buffers as an intermediary - the real time objects don’t/won’t have this issue.
3 - from an engineering perspective the infrastructure in the client layer doesn’t care what the buffer it is writing to is (it’s an abstraction of a buffer that could write to any kind of structure you like), and it might in fact represent anything, so the design implications of being able to output to something other than a buffer would remain at the wrapper/environment level. That is to say it is technically possible to get all the buffer analysis objects to output directly to some other format in max without having to touch anything within the core code. Whether that is a design that might be considered is another matter.
4 - the idea that turning overdrive off will provide an answer is for me quite problematic - I don’t think it is generally viable on a retina screen - see next point…
5 - yes graphics might get put at the back of the queue, but even if they do then once they start processing you have to wait for them to finish to get the next thing to happen. That is why the times are erratic without the defer low (which is giving you the operational time). You’ll get noticeably better performance by opening Max in low resolution mode, but the timing for events on the main thread when hopping between threads is basically an unknown - bear in mind also that anything you trigger off that is still low priority, or you have to up the priority again - not nice. It’s sort of OK for response to triggering general events with a low rhythmic tightness, but it won’t cut it for musical timings.