In the discussion about buffers vs channels in the thread about @deststartchanmessages for fluid.bufhpss~, I tangentially suggested a way to visualize and interact with all the permutations of fluid.bufhpss~, fluid.bufsines~, and fluid.buftransients~ to get the ideal “three way” split.
My, not fully fleshed out, idea is that there would be some kind of abstraction/bpatcher that would let you load an audio file, and then behind the scenes it would work out all the hpss->transients / transients->sines / etc… permutations so you would end up with a (visual) matrix of options.
You could then click on each component (jsui?) to hear it back as well as selecting it (or maybe a separate tickbox or something to “select” it). When you have selected a specific layer, it then greys out the remaining options which do not null-sum with your chosen thing.
Obviously more details to hash out, but that would be an infinitely better way to explore/audition the choices than manually making signal paths, buffers, and playback objects to (slowly) test every thing.
I think I can see the shape of this. Dealing with all the parameters would take a degree of the automagicalness out of it (I mean, this could take a while to run on longer files, or were you thinking of a real-time thing?). Offhand, I can’t think how easy the working-out-what-would-null-sum bit would be (especially in real-time).
I was thinking off-line stuff, so you could load a file and then play/test with it.
Loading longer files would definitely take some time, but I guess the idea here would be to save you a ton of time by finding the signal patch (and settings) you like, then applying it over to the real process.
Interface-wise, I’m not sure how to set the parameters for each object in a Max-like way, but it can be something as simple as sending parameter string messages.
For the what-null-sums, I pictured something really stupid/passive, where if you select a single layer, what would be valid (i.e. not greyed out) would be the ‘companion’ parts of that individual process. (e.g. if you do sines->transients, and you select a single sine layer, it would highlight the corresponding residual and transient layers (in the pre/post combinations)).
In a real-time manner, that would be more of a nightmare (though presumably still possible, by just highlighting the ‘companion’ processes).
So you could click around and audition what provides the best balance between all the sounds (or hell, if you don’t care about null-summing, what the best sounding individual components are).