Adding @lookback to ampslice-based JIT onset detection?

[rodrigo.constanzo]

So working my way through updates for the next version of SP-Tools and one thing I thought would be useful to implement would be something kind-of-like fluid.ampgate~s @lookback.

It’s a shame that fluid.ampgate~ and fluid.ampslice~ don’t share more features with each other as I like what ampgate does, but need hysteresis.

Either way, onto the question at hand.

So the core of most of what I do has an ampslice-based onset detector (actually ampfeature+gen~ to get gates rather than triggers from “ampslice”) that marks the start of an analysis window in a rolling buffer to do just-in-time descriptor analysis.

What I’ve been doing up to this point is taking the frame to analyze, and subtracting a fixed 16 samples from it (@44.1k) to ensure that I always include the start of the transient, given my analysis window is so small (256 samples).

In looking at fluid.ampgate~ again I thought that it may be better to incorporate something a bit more meaningful and consistent by doing something similar where (in a tiny window before the detected onset) it looks for the minima (what I believe fluid.ampgate~ is doing, based on the text in the reference).

At the moment my onset detection is giving me something like this:

And if I zoom right in I get this:

Which looks pretty damn good, but it looks like something 6-10 samples earlier may be better, or perhaps something even 20-30 samples for more of the pre-transient.

Now, in terms of implementing this would it be as simple as doing something like:
[ampslice onset detection] → [sah of position in rolling buffer] → [bufstats @select low @startframe/@numframes based on position] → [descriptor analysis]?

I guess at minimum this would ensure a zero crossing, or something closest to a zero crossing. But is there something more I should consider here if trying to implement this?

[rodrigo.constanzo]

@rodrigo.constanzo

Now, in terms of implementing this would it be as simple as doing something like:
[ampslice onset detection] → [sah of position in rolling buffer] → [bufstats @select low @startframe/@numframes based on position] → [descriptor analysis]?

Actually thinking about this further, and I don’t think this is correct.

What I want is the minimum of the absolute in that window, and on top of that I want the right-most sample that satisfies that criteria (so closest to the actual onset detection), in case multiple values are a zero.

I guess fluid.bufscale~ can clip values to avoid negative ones, but that’s not exactly the same as abs.

[rodrigo.constanzo]

And actually remembering the discussion in this thread about managing different edge cases for window analysis (mirroring, zero-padding, etc…) I’ve been thinking about this specifically:

Where at the moment I’m analyzing 7 frames, that have a lot of zero padding on each side. So I’m thinking of revisiting this idea of including the audio outside the 256 analysis window (dark green in that image).

The reason I bring this up is because if I’m offsetting the start of the window analysis to be more precise (using the fluid.ampgate~ / fluid.bufstats~ possibilities above), that it may be beneficial to center the onset such that the transient falls in the center of a hop.

So lets say my transient starts precisely on sample 100, instead of starting to analyze there, I would instead analyze {hop / 2} samples before that (so sample 68 in this case) so the transient falls exactly in the middle of a hop with the thinking that this would (potentially) ensure that the transient is best represented across all of the hops.

I guess that would mean that frame #2 (in the above screenshot) would no longer be “centered” but that geometric centering puts all the energy in the second half of the center (50-75% through that hop).

[rodrigo.constanzo]

@rodrigo.constanzo

@rodrigo.constanzo

Now, in terms of implementing this would it be as simple as doing something like:
[ampslice onset detection] → [sah of position in rolling buffer] → [bufstats @select low @startframe/@numframes based on position] → [descriptor analysis]?

Actually thinking about this further, and I don’t think this is correct.

What I want is the minimum of the absolute in that window, and on top of that I want the right-most sample that satisfies that criteria (so closest to the actual onset detection), in case multiple values are a zero.

I guess fluid.bufscale~ can clip values to avoid negative ones, but that’s not exactly the same as abs.

Thinking about this further further, trying to think of what’s the best way to get the local (absolute) minima within a fixed window.

Using bufstats @select low @startframe/@numframes based on position is most elegant as it fits into the current workflow/paradigm, but as pointed out, @low isn’t useful for returning a zero crossing as it will just down to whatever negative values are.

I can peek~ out a list of the window of samples I want to test and then do zl stuff on it to get the zero crossing, but I’m hesitant to add an uzi + peek~ thing on every onset.

I guess I can go into gen (sans ~), but don’t know that’s any different from uzi + peek~ in terms of high-priority banging a load of individual samples.

Is there some other way to get a zero crossing (or local minima (of an absolute value)) within a small-ish (~16 samples) analysis window?

[tremblap]

@rodrigo.constanzo

Is there some other way to get a zero crossing (or local minima (of an absolute value)) within a small-ish (~16 samples) analysis window?

does it need to be buffer based?

[rodrigo.constanzo]

@tremblap

does it need to be buffer based?

The solution, no. The stuff being operated on is audio in a (rolling) buffer, so it will start off that way no matter what.

Basically doing that rolling buffer JIT thing where rather than analyzing exactly at sample 35313 (where the onset was detected with fluid.ampslice~), instead start at 35309 because that’s where the closest zero crossing was before the detected onset.

So what I will pass forward will be an int (sample position in buffer).

[tremblap]

so I use a zero crossing sample and hold approach for my dirty octave down, which would do (and adding a LPF would help you remove the useless topy noise crossing)

I think I gave it to you before, but here goes in case:

----------begin_max5_patcher----------
671.3ocyWEriaBCD8L7Uf331jHrA7BUpG589GTUUQHdS8JhM0XRo6pMe60LF
1js0YCJgDZN.wOF3444wuAd10weongV468Quu5437rqiC.0B3zM1weSVSdQV
EDlOm9KwxG8mYtjh1n.3614EsnGksBvzwMOoGSTqJnJ0uKoF17qXq4YE9deq
KfGDbUE6I3xoKBN.kmsAP8+rjouitqTlox+Aiu96RZtx7LSRz2mGNIr8TXb6
Qj9Q8JE75MLtdV.YBdOnYtAnnVvWbcaOL6xjjJJe0NOQtJaKUNWywWlWJoVz
HD1eD0.Bj7XLj8nj2WDPVEgfyPDfmokjCiOZEv90da4w8sybyRIJt+n8rHv9
RYGZtXyFJ2rl75Djuh1bto5QVuWx9YcldEOXQf9Glz8m3v8HyQKRSHIQZD84
znXaBV3McKCHyXR3A6bNd4B4ZumYrzvnaoFlBZHJf7+gFd2NaJR7DTUEBJRL
ZhMhyKXk67PdXa5B4VpKHbmeLHIQouuvDd06PwVq0EapxMssM9dveONdHssQ
W+11E5YlPtyd8R5DTufQFOETvDWvrjoxZeoFr0plvfoXuDA1EQBlXSlO7Ia1
tgnoPRh5ZpOwRxSTonwppfOkpL6Z1n1HLQnywnAef7.2meAi+2eAEPcK9a0r
JQsLuOQ6e2Xu8zuhVoX7LESvOLnn2DjPthJO9atNtTGamZzkQc3.nNwDykQT
z.HJDOFLEOD0LbLXhLDlRGClRFJSnKkozgVKdoL01g5zLQFA0q03+z0dAiAS
Xiv79LgFAlFRAA5e2NYrIyJK2RkUcQCbnah7n.LTRlACYbyvXXnjtk0GeDfj
I0N5JscdszX32PLenm+Fg1ZhWy5bmzYmlRnAUaygpxLSh.8wbew8Ov.AeoN
-----------end_max5_patcher-----------

[rodrigo.constanzo]

Ah yeah, I have this patch somewhere.

Hmm, I guess I could keep track of a small amount of zero crossings then poke out the right-most one when an onset is detected but it’s probably expensive/overkill since I’m doing JIT based on onsets only.

I may just try mocking up a peek~-based version and then see how fast it is.

Do you happen to know how @lookback works in terms of the point it picks offhand? Like if there are multiple zero crossings will it take the later one (i.e. closest to the onset) or the first one it finds? And does it have wiggle room like if there’s a pure zero crossing present in the @lookback range, but then a 0.0001 much later, will it go with that instead?

the latency is incurred either way, so it doesn’t matter too much, but would be useful to match the behavior there.

[tremblap]

@rodrigo.constanzo

Hmm, I guess I could keep track of a small amount of zero crossings then poke out the right-most one when an onset is detected but it’s probably expensive/overkill since I’m doing JIT based on onsets only.

I’d keep the last one - if you need it in control land, with a snapshot~ that you band when you need the last bit of the past that was a valid edit point pre-attack. with sah~ if you need it in signal land.

[tremblap]

@rodrigo.constanzo

Do you happen to know how @lookback works in terms of the point it picks offhand?

I do, I devised it with @groma It looks at the lowest value within that window - so if you have a decaying cymbal followed by a hit, it will go to the lowest point of the decay. does that make sense?

[rodrigo.constanzo]

This is what I ended up coming up with in the end.

----------begin_max5_patcher----------
1511.3oc0Ys0aaaCE9YmeEb9oV.WAd+xdZ+HFvdXcXP1lNQMxRFRTooon829
NjzJowQIVxltX0HvNlTRmy224N8WuZ17k02aami9czeilM6qWMaVXI+By1+8
Yy2le+px71vkMeU81s1J27Ew8b16cg0W1sYisAUku01uWU215NWo0EtS79U2
k6VcSQ00+aickKJYJGmgWfnR+6JZ3+gUP+ySOnhp9mCY+hEqCxsd4m9fbteo
uc0U92Vbl.oMe6tRKZWcagqntB8t0VGno10n5pVq6882PYQkcUcWU3tnSExR
YFUr.QLY92wA7yXS.y7KJlev1TiV0T21BpchfLQDrrBkHSfUFl.GeoMKPFwj
I.ZJIftVKxciE0V23szkEsNjqF0ZKA0OtSjhJpVau2uUdUd4Wdvh1zTucP9g
MY9wP7NCBBICC7CWgMXFmSkfCBU64GtZB7ifjRBZC.7.OTV+YKPN2kW1YQua
UYcqMxUivkY5TBSEBSXRQlVIEBBmikLMw7TTyzHEbJIkUM1bmEkG8Wp2z6jD
Hm1DE0PiYJnF.mRrVvjDgRKvbHrQalddC5OCF.7VJVkJJfpHgRBTRFkKHBgx
fMLJlnfDGrIy.LcJYf+5FaEjLHVb.UzhdrbQYc8snM0M.67rXCzmKb.BQD4d
1pE8tFaYtq3NqOPhyyH29dzRKbu9rR4NHwbQkCjx5mk1wGO1345VmMec1Gq9
X0edCnAvesEaKJya7ONeP6e30kk4qtEk6bMEK6b9aCsorqXcFnCWCFwumMn0
RNUqESFLIznMSF7P0SIFkcRkyqreFt4WXdbnknkC2PRueXbI2W1Yi.X9x7pq
muX+mOp1C.UU.oDANyPMbvsTJwJsQ6yXE6hgNEbyRHtyW1hvYCCbxqA7Mk04
t2DwDhHS4yFI4YRACigpTLl.xJwfLwjIiYAOw15hSzVWTcLfGSCyEPzogR4F
pTvMDBgu.I0SF3RZBA9CknF60SD4dTONHanYPsVrQSTJCQI8lZ9aiX5.HljV
Du0cykBwfi8yZAivkG2HODjwo1HeWpgLkHi4oGHdFZ9ZpHVnSKhutota2kBy
PmUpC6mPPlNlMoEyskPqSHxEKb1+oF5iFy.CtjKLZ.1hoCaQhgML5UpwLTfN
L9fRmw9gwGvZJDOSmLj4p+2GO2iXNNiK0JLUP0TnpsTdRwyb4u.wy8XFSyDG
NgzIDOySoi8Gds.4WsIri2IRrErAa6jFNOooA2T114u8gKEZ05LMSAIrzdSq
PfIfwEOczR9EogSnEDuSsRjQHPtKLgSnTgQ.0kIL4j62jmxVQ5dn.ldcXfyF
0TUim..zmggJ1PmXPfsfxkF+Q+vmrkmkxv5cV6seGsta61ub4lwZn9SzGowa
1.GT5I0SV3Qlbrs2nFOb.Ft+8gwBdflNn8pT+4wLkirOb3sAD7ShPdatfhi1
4SlLXCQFC7Cw7Htom.tifZx+lNONKkRElkhwMwWJwwA6PmTfbTV9gOE5AM7A
QGNrqC9AvBnwu9yYk15tlU8V18yvu.8DBVaacEU4d83GuHR7hFj4mjjHGSR3
3EcdRhLFLgSAlviPRdq9YKIX.sQHIcJjjdrdDOYL2VrNb7t6cBgh79RabHQu
9vQV7s7oFdyyUwEiQwoofh3iPR7mqNGPQDfhh+jH7LBVBMASfFiTDkA5ITSi
IUe4dmqhyFghS0ukhy5K+Ao7YJJjZTaDbn6crumldT8xMOWMeL4o3oHlFFP9
34o580NOIIGClToPRiIzfmh7T7w3gwSQ8D9XxxePZyWKHD5VEBxnLBiwUDsR
XVzmDaf8d7wU2r11D5g3xijChJeTzjySz3Q3ryIInnLeLkJYoHrJ7PnGCSrT
IoiwdrjHI1XRJ8lN6lf6LQCYlO7LH16pOvdoPsGagxyRRzwz0Rec6Wz.c9tc
2YaZ2e0Ag.CP7o5PDldQ3qEUwuJBeswdWQ+0GWIuAFevOxVWSPwleujOOdq0
PrZUWw9vU.dfHCCm3GvocWdDIgYXt5aW8eTsM7eL
-----------end_max5_patcher-----------

I’d rather not uzi on every onset, but this works alright.

It would be amazing if fluid.bufscale~ had an @abs attribute or something as it would make things like this easier (zero crossing or peak-picking in general). (added this as a feature request on github)