Ok, so I managed to put some of this together. Here’s a demo video.
Hopefully it’s clear from the video, but there are three main playback bits at the bottom, the right-most one is samples 0-64, the middle one is 64-256, and the left-most one is 256 until the end of the file.
There’s a bit of extra played back by each section, along with some fade in/out.
My initial thoughts are that it’s a bit underwhelming. There’s loads to improve here, in terms of analysis/matching, as well as having tighter playback and fades lined up etc…, but with these sounds, being triggered by a snare, the transient of the acoustic snare is over represented in the sound, so everything sounds super “clacky”. I guess that is correct, so that’s good.
What is more interesting is the microsound-sounding sample playback, where I’m just querying and playing back these tiny fragments. And I also quite like the sound of the first bit and then just the sustain, so you get a kind of “hollow” sounding sample.
I think moving forward with this, it may be something where the real-time input is analyzed at a couple of stages, and then that is used to just query longer samples, rather than matching like-for-like.
And/or doing something where the sections are put together from more decompositions, where the “sustain” would be made up of an arbitrary amount of HPSS/NMF’d samples.
The windows are just too short to be meaningful, and something like the 50ms window of @tremblap’s LPT thing is (way) too slow for real-time use.
Also, I think the initial results from the spectral compensation approach are looking more promising, if a bit “slow” at the moment (fingers-crossed for fl.land~ filter magic).