Adding @lookback to ampslice-based JIT onset detection?

So working my way through updates for the next version of SP-Tools and one thing I thought would be useful to implement would be something kind-of-like fluid.ampgate~s @lookback.

It’s a shame that fluid.ampgate~ and fluid.ampslice~ don’t share more features with each other as I like what ampgate does, but need hysteresis.

Either way, onto the question at hand.

So the core of most of what I do has an ampslice-based onset detector (actually ampfeature+gen~ to get gates rather than triggers from “ampslice”) that marks the start of an analysis window in a rolling buffer to do just-in-time descriptor analysis.

What I’ve been doing up to this point is taking the frame to analyze, and subtracting a fixed 16 samples from it (@44.1k) to ensure that I always include the start of the transient, given my analysis window is so small (256 samples).

In looking at fluid.ampgate~ again I thought that it may be better to incorporate something a bit more meaningful and consistent by doing something similar where (in a tiny window before the detected onset) it looks for the minima (what I believe fluid.ampgate~ is doing, based on the text in the reference).

At the moment my onset detection is giving me something like this:

And if I zoom right in I get this:

Which looks pretty damn good, but it looks like something 6-10 samples earlier may be better, or perhaps something even 20-30 samples for more of the pre-transient.

Now, in terms of implementing this would it be as simple as doing something like:
[ampslice onset detection] → [sah of position in rolling buffer] → [bufstats @select low @startframe/@numframes based on position] → [descriptor analysis]?

I guess at minimum this would ensure a zero crossing, or something closest to a zero crossing. But is there something more I should consider here if trying to implement this?

Actually thinking about this further, and I don’t think this is correct.

What I want is the minimum of the absolute in that window, and on top of that I want the right-most sample that satisfies that criteria (so closest to the actual onset detection), in case multiple values are a zero.

I guess fluid.bufscale~ can clip values to avoid negative ones, but that’s not exactly the same as abs.

And actually remembering the discussion in this thread about managing different edge cases for window analysis (mirroring, zero-padding, etc…) I’ve been thinking about this specifically:

Where at the moment I’m analyzing 7 frames, that have a lot of zero padding on each side. So I’m thinking of revisiting this idea of including the audio outside the 256 analysis window (dark green in that image).

The reason I bring this up is because if I’m offsetting the start of the window analysis to be more precise (using the fluid.ampgate~ / fluid.bufstats~ possibilities above), that it may be beneficial to center the onset such that the transient falls in the center of a hop.

So lets say my transient starts precisely on sample 100, instead of starting to analyze there, I would instead analyze {hop / 2} samples before that (so sample 68 in this case) so the transient falls exactly in the middle of a hop with the thinking that this would (potentially) ensure that the transient is best represented across all of the hops.

I guess that would mean that frame #2 (in the above screenshot) would no longer be “centered” but that geometric centering puts all the energy in the second half of the center (50-75% through that hop).

Thinking about this further further, trying to think of what’s the best way to get the local (absolute) minima within a fixed window.

Using bufstats @select low @startframe/@numframes based on position is most elegant as it fits into the current workflow/paradigm, but as pointed out, @low isn’t useful for returning a zero crossing as it will just down to whatever negative values are.

I can peek~ out a list of the window of samples I want to test and then do zl stuff on it to get the zero crossing, but I’m hesitant to add an uzi + peek~ thing on every onset.

I guess I can go into gen (sans ~), but don’t know that’s any different from uzi + peek~ in terms of high-priority banging a load of individual samples.

Is there some other way to get a zero crossing (or local minima (of an absolute value)) within a small-ish (~16 samples) analysis window?

does it need to be buffer based?

The solution, no. The stuff being operated on is audio in a (rolling) buffer, so it will start off that way no matter what.

Basically doing that rolling buffer JIT thing where rather than analyzing exactly at sample 35313 (where the onset was detected with fluid.ampslice~), instead start at 35309 because that’s where the closest zero crossing was before the detected onset.

So what I will pass forward will be an int (sample position in buffer).

so I use a zero crossing sample and hold approach for my dirty octave down, which would do (and adding a LPF would help you remove the useless topy noise crossing)

I think I gave it to you before, but here goes in case:


Ah yeah, I have this patch somewhere.

Hmm, I guess I could keep track of a small amount of zero crossings then poke out the right-most one when an onset is detected but it’s probably expensive/overkill since I’m doing JIT based on onsets only.

I may just try mocking up a peek~-based version and then see how fast it is.

Do you happen to know how @lookback works in terms of the point it picks offhand? Like if there are multiple zero crossings will it take the later one (i.e. closest to the onset) or the first one it finds? And does it have wiggle room like if there’s a pure zero crossing present in the @lookback range, but then a 0.0001 much later, will it go with that instead?

the latency is incurred either way, so it doesn’t matter too much, but would be useful to match the behavior there.

I’d keep the last one - if you need it in control land, with a snapshot~ that you band when you need the last bit of the past that was a valid edit point pre-attack. with sah~ if you need it in signal land.

I do, I devised it with @groma :slight_smile: It looks at the lowest value within that window - so if you have a decaying cymbal followed by a hit, it will go to the lowest point of the decay. does that make sense?

1 Like

This is what I ended up coming up with in the end.


I’d rather not uzi on every onset, but this works alright.

It would be amazing if fluid.bufscale~ had an @abs attribute or something as it would make things like this easier (zero crossing or peak-picking in general). (added this as a feature request on github)