'real-time' onset detection example

the interface I work on with @groma allows to fine tune these settings, so we could make a few preset in the helpfile/example - tight percussive material, low end heavy, etc.

I can see you still not use my ‘first past the pole’ version where the median filter differential and the dual-average differential are competing. I will test and edit this post to see if I get better results :wink:

edit: the results are different. It definitely confirms that we need these parameters to be available, and more importantly, with examples for each settings, since your super nervous one is failing on the bass stuff (Tremblay-AaS-SynthTwoVoices-M) and on the full program (Tremblay-BeatRemember) it catches stuff I’m not sure I like. Anyway, all good research here for fun!

Also, for info, the decay time of the peak follower (the fast asymmetrical) will indeed influence the reset time of the Schmidt trigger (slower or faster to decay, or how nervous it follows the rectified peak). Does it make sense?

1 Like

Indeed. That’s what occurred to me to experiment with since I wasn’t getting a response time that was anywhere near the lockout time in terms of resetting. So I started looking elsewhere in the code to catch it.

One thing I started experimenting with, but got nowhere useful, was having dynamic threshing or dynamically adjusting slow envelope that “rides” the overall amplitude better, so it can still be fast, but not uber nervous during quieter things.

I’m sure there is some literature on the auto-release parameter of compressor that can help your explorations there. The paper mentioned earlier is quite good I think (Giannoulis, Dimitrios, Michael Massberg, and Joshua D. Reiss. “Digital Dynamic Range Compressor Design—A Tutorial and Analysis.” Journal of the Audio Engineering Society 60, no. 6 (2012): 399–408.)

1 Like

Getting back to piano writing with some more realtime onset detection questions for you.
I looked at PA’s and Rodrigo’s solutions. But for piano with changing pedaling and no need for snare-drum speed repetition, I’m not getting good results with your solution.
Here is a different attempt, but it does not work well either.

I’m using threshold and have it ‘float’ over the past 500ms of average energy.
That technique has worked quite well in the 1990th, when no other attack detection
was reliable - instead of looking at absolute values, I’m making them relative to the past
energies.
In order to prevent re-triggering, I’m now also looking at the pitch and try to control
a time gate with it (slower for low notes, faster for high notes).
This is VERY experimental and does not lead to good results yet.
Any thoughts on how to get good attack detection on the piano

  • with, without held notes
  • with, without pedal
  • across all registers
  • with limited re-triggering

This is the link to the audio file:


----------begin_max5_patcher----------
3546.3oc4cktaiiiD92IOEZMVfAX1zd4ktVfYvrOGMVDHKS6vIxRdkn6jLMl
7ru7PRVRVxl5zMlM.ssapCV0GqpXwpJQ88GeX0lj2oYqr9WVe05gG99iO7fp
IYCOj++eX0gf2CiBxTm1pX5aIa98UOoODm9NW0bTRv1CzrLKXwghOcH4DOhx
UWGLu0L9GQT0UTbd5Sh+wQplLVsx5+jeniA7vWXw6eNkFx0GE4RVa+jEBaK+
xwcMP7efqAkWinaYwM6U1VUWJH7u3rR1ze93ixOdZbL8QqW3q2jD+5n44MAw
6uNeicU7sOTxw1NWmuQWx2kiKpaNMMmayY2GVsiEQ+FMMikDWg1eXUvwiUZ9
gJWhDi98D0Mx8oxlXw5lvkMkR+Fq35sKaMHUvhbA+cJUiMu6PVc91jrklFeh
oHEcihQqGKtgknhsDGbAxOgPHR9sOfbFVDCr6CShzjzWsTfWsONehzsLtDyq
cAf0dNBQNAxK9kKv1yS+KOBF1352rOJI7UpBwAEMlbjFyhOlRynw7.dNJTd3
szcAmh3OuKIlmw9CEP.kjVKGeWPHsyKNN3fFE+2orfnBbb09T11jXIQTaLU1
bQ2I.EEHJ+7LynNi3fisbwBgIA92wAyDL4orMAoxg7MZodTwA4IIQ0OTIuDQ
2wyO7QVbbCTjmbr6Clx1+xUt1MIhCd3Z2a0Qxd9Tr9nOKz54OmE7s5THOHJJ
2LP8a+6AwrCAbJmoGBPfxCRiCDL5KYgoIQQ0vI8Q9VKGYqPaIj9FaK+EUGUU
XPb5riEBQqJGk2x1Sy30aiGrOqdKmsB8RPbF.fNejSaxMJ7Lmd3XjfYpbd3y
MVgoOOoQUKBUsiVq8F1SCSNbPnPTpuW0nJ+Eg1xKIQas9ochYUj5j+jUvlju
QsDiIoA6oVzXZ59Opd0QrXZXxoXdMSOWXQFT4HWXUtKSu154Zf1dJ8Dfu7KB
th5RWy6boM3hlUy+jaUaFvP0zISKJfcgRTv0weMlXSr8exBqMb3O.jvYoPhL
gPw1IFJbfqcEPAgnDD.CFDPKEHbjI3Aqvj3cBSKwgzIFO7rUBB1HEf34OX.A
RrmRHQX9PzqcyrnaxrW5a5SM7US66TRpXF.4o3bcjxU6phcNfAZ5EWOPJb8A
kTgC.bgAb8DKpgxIDIq4+aGN9KbKpanFN.ntFnbEvTK145LBvbRMLK5xMB+q
mewt1LMYOUBXX+kT.yLGBlX6VDe+0k1wymWeX1svtqVbvXKa2NZ5LXN2FoVb
DgfFOtLo1yMwJzWrAKqUnBMtbqPtiPiCuTBQh4E2SSs1K7o2RL2kUTxaVwIb
Z1SVAYV7WneXkH9L8MVF0hSi2ZwSrRobwRs1W2x1b52sqMVAoHW85SAC2uaL
ZgjC+a+SgCgh+VOoBgp0AcaIQhRS0Ep9RCdWWRD0Abg7WH3JTrDy8Ww9EdXS
Uxjx85uLDzb7TKtiLB0Wj2BAZ+7mSpvUFaebPzMAp7IKczAfDo9OCT5ZRW7m
zpePcYnRjRZ1vxV7OH.XAjetryMXCU1rbzVv78GgF4hMQpvf9mSpuq4xWl5A
aAloi1BlLBLCuPXV1wHF2BiTl9mTra.Fxz3lqAV+wcgankR+TGGOsKsV+c38
wwMOcVCzV9QM.rZAlGsFzaYvZxCsDo9+RGtiBORze4MhUnCWzUidMY1hnvmj
lYoiq1RK3VD2i7.sAbFtIRn6BYhrBpkxmV+WhXYlacDiTAwdDt4A+KdHJINU
W8+XhfDjrj5rCCIajJLkLaJKKLHm.W6OmPs1kZnNAr3wHTh9AIZvxzBm8RB+
Sgco9pk2xXw.WIrvQeEV53XZLY5v.oGXobgjs+y4.uLbwcXrBoHNJqjXzHBk
9RsHX0f78SDq.wvJIM7HVZ2RFhzqmW3giYFsxtbsx77naObycHvRs.EDB.lE
6X2FrrySttNpmnAKegfKEXErIKI5DmdevqbgKMdMhfeNsQm5J5i4Eyx8zrew
7j5nrn0KGlFo68OApygqbsVGLZCWtXouaivoMTskzJNe1Ag+qMN+6nqbGCBe
UfZyjiblNAaNbmaNbPZzX3BF6zagXngiXOYNpQ.n1KSXiiB3hkwrizfWCNb7
9tfgBgM87rtdCWVirT4vkEKrJ7sy05qoIcsEPqsklB7TI3IuF2bGbI.rXU.v
+vBdGWN.wK22Vk.D1avYGi39+KKGnXhT3XWN.wao7v099LSXNPo0EGQlWIKU
Z8+hXVPzcTYDmmLesQ8guV.6kxygLZjEbNbZPqgatiCNHU.xwC2aK6kpLHjO
uEoyAloEx5AnYqiyHd3IhwdozL+0eYHUAgInlI4nt.u7s0Qabv3kyREWV518
8ubH5ihoYSUliaZ0RxvCOqyR4UuNm9Bm5kqFZ88Ox11.sO8ngWWDNSZ4jn5t
6Qcbk+vq3py4Dn3ytgCPWQ5uV7KJVpRSxjEuk9dkmhyePAOyK+swBa0qO4aB
anID1zb8T+bgoUvbcvq8JdtvL.d5J+Q1vQKUc9oXUFPlNdNEUbh73siaYImR
CKDNxyD3SV04mszLNKt7g58qkKfSchFMt0WpPXCxLpPXneFoBjgTgvsFYEXN
iTAzTpX1vBSkKD9EOeTQAKda4BvbREdFREJs7FmYR5V8lR.7NSaH6toMv7Pa
PSG8vyn9DwTYHzbpOQLcbhLqTgaeFQPKovhwj1xqigIFRZX3LN1gQFaEZNoB
S0owy4L0HS0oUt2.aWXAeeIMkTEbIkiMmzPcq8ityjFraTadLLgb6ir1rIw6
X5rXyoe5H3ODTgo9l1BnM2pXFSZvk1aL0JzMaBD3bZ6FZaJYPlSpfzK6z0Ow
CrsGSXw770f6BWijIW.4s1W7GVFp.hZu04bSKoPn47FrEMj1YNGO3ZRNyYCx
e1RKZZQEiIXiEicmUAHnwxw3YkN5iK8KqEGROmsXAUQvlqhLmSqK2bDLjNvy
4BCgF6.l5QVaFoCyimw7RGNFiG1yJcX7rknY0JCB2qoTfyFcX7pfQdyJd.9w
P9.1qHjNaCKP2ePfCyUWlUebMdpOa3rNrf5S1cZmLNm1IUty5HuSkO92Wji6
h8y01oYYl0U6KsOyhk6Guzh8mVvSU9nYdsMaquUdlAggzXdsc9VWhPb0Qdmw
1NdNXUePbAxT707xaro453PHND4E3f8cf5ahmGzEe4kV8Qr+qpc+1F24crnn
x69CW.MBrIOOtq1mFrkUufYqgAf0He.zWSN.eO.Q+KQSvKHr7qD12MB3yWJp
DNDqDPc9Nx+b0+Re2t3JCh2mus45BpteCHjpSSNljVtGBuF6W65NwSJ4+ptp
dVF87Vnr9IKPblskl2UGDRgrp2Cov8iUuAqjaOtaeVuU.+b.mmx1bhqk4qL5
zusr18QIaBhx2IZK0SLX+q87dc6im42oYG9NLhE95mid68tQsYbkM3artBUv
d8diMOe6Nom7c8RLuVEIJHMq2dIfK27ujOj8zXwuZGJ.cBEsUw..GcMFppRZ
n9oMsZwkeSVE5MDV8PfPL88PdZQMyK0ROcHtZQ+zd8cYxPrZaH3oFaGAWi88
r0AGP+jJWg6SSdqFieyAdvYvtimVnoQYPCfeZg6ZS9G0aMBS.L4NPnbaZTqY
3BttlAtE.BNDwkZan.CmQunH+tb+A3ZumCbHsuMLbacD2ERpX4LQB0ahGXGc
LC6uIRHbBe4OTslsuo3Q8orLnNsuF+qKCzKqQ6a9Nfv1aB4+p0e8n4+Kq45q
..HW0LG39+RvHuNqmF.3W+Eqyldukb+s.fpkO80F6AZuD76MqmWxzSCqWsrn
mLY+a9NPoX3Wue7eYYPe628KngfAz+XaP3mSfiO5GS.hsd5Lcsba2igvhBtc
UTRXPTYPemlgzcQmXaWmDmI5egEc5mV+148ZXwzO1V+1tcbwQkuPFxrrASnK
.lL8OJ+YUm32eOiGxn9wnfOjtlzw.u8P3y19k7eaYgxkHEj9wUQBs7iii9sQ
wEEY6MwghsL9UaC3AMWplXZ7icDyB4qFnx0ig.RW0Ehs3mgdqIv0D7yGiS9R
lXEuqCX6prNR408JKVS.Am1xRjsT4DhRRNVuviEtjGKVnA+Y4KRlNBey+8TP
Di+Qo0iLVXKa.SxfYjJwtx.3TOtBxw2VaWtmtk8BaGuLtIMt4QQIuoWqp7Ei
RRm2jq2+Ior8LgLf.Z2qeUujGMGwDqgulUmijuNjtI+1N8JmlNimREDTajQV
zozh2bMJBndvWjcF+TLSZYVUF7j14hJHAD07tjcjpeo6bI0UGENjUgLZMDaW
t.e0sqdkdqUuaFDvbk7aTMy2nFhKzwqk0sKrrL8cM7xtFLttFXRWWI0.CumL
niplA5A2QMFitcAPO7dxyDdBNA8D1fNpCQSzr2wn4PmvjNF1dGONMBjQrLTW
weiqmfFNpBGaGALkkFqbpIFxbPyxvVOXwoURsdt7thn5XGFgPS4wQ2SlzQ3K
jWzyA23MEorOZ7FhrwaGxKeyP18aExluQHUYhnQl1z7Z6YXqQ1099iCKqZlk
QsQjMsAlIsqjEs1xfVgS+cl4rgk0rAlwrgksr1yTV2YIqiLjU3L4SO1cVwpm
QrpJVkBah0N5HWP4jJr0hXjFDw9Hsnfa93QmRPMDRAqsKzbm.Ftxyga6rbMR
mfv1HekriKxln9kbXsPZXxHqZagKS0Xwv0hpnOTHOe4uth9fqmm.4dp0ec4k
0ac1kPMpt1gYihUeZkMP5RMlQDhU.j5WBdF6OXIKw07mO9+.Kpner
-----------end_max5_patcher-----------

I’ve done something like this with @jamesbradbury for his distorted pedal steel: make a very short HPSS, and run onset on the percussive channel. That is quite sturdy. Maybe @jamesbradbury will share the final bit of patch?

This is already in the gen~ examples here. There is a proper object coming in Alpha06 doing that with some very good controls I’m currently experimenting on with @groma.

@tutschku I am doing some piano segmentation myself . I’ve found really good results using a variety of preprocessing techniques (as described by @tremblap above) and some of the less straightforward functions in fluid.onsetslice~.

The setup I used for the lap steel is applicable to the piano to it seems, and so the last bit of the patch looks something like this.
hpssonset.zip (861.1 KB)

I’ve provided a short piano sample too, although I suspect you will want to test on your own material. fluid.hpss~ @masking mode 1 is also useful, although not so good for real time as it functions best with a big fft window and thus lags for real-time use.

EDIT: Apologies if the patch is not super clear, it was cut out fairly hastily. Thresholds aside, the key is to have a really small @harmonicfiltersize and very tiny fft’s for everything. I used @function 9 with fluid.onsetslice~ as this seemed to be the most responsive. @function 1/2 work also, but you have to be very particular with the threshold.

Here is the “really fast roll” test/comparison patch I showed during the 2nd plenary.

I also have some audio files I test it with, but perhaps too big for the forum. Several options for testing here, along with the ability to add in some noise to test the robustness of the settings.

Archive.zip (11.2 KB)

2 Likes

Ok now that I’m wrestling with pd, some old demons are coming back.

In looking at (presumably) @tremblap’s JIT-NMF example in the pd examples, there’s a del 2 in there, that makes me a bit uneasy:

I presume this is there to time-align the analysis window of 128 samples further down the patch, but using 2ms instead of 2.90249 (128 samps @ 44.1k) to allow for some snapshot~ “slop” in pd land.

Since snapshot~ doesn’t have an offset inlet in pd is there any way to ensure that the sample taken is always the correct one?

I’ve basically coded this up in pd and don’t notice any discrepancy between them, but since I’m not using change~ anymore, I don’t know if @a.harker’s test example is still valid:

If this isn’t valid, would it be a matter of adding an additional offset to the delay time of vectorsize - 1 to try and “align” it?

edit:

not seen this before on the forum!:
Screenshot 2022-12-21 at 6.37.41 pm

There’s also also the *~ -1+~ 1 before the samphold~ too. It works fine without it, but I presume this is because samphold~ triggers on a decrease in value in pd?

yes.

I think that was it. It was long ago…

Had to do some more debugging today, but it turns out. my “256 sample delay” bit wasn’t working properly. Fixed it now and it seems like it’s working correctly.

Doing the debugging trick where I fluid.bufcompose(~) the bit, and comparing the results from Max/pd and in all the hits I’ve done (not super comprehensive), the results are the same all the time.

It would be good to force this in code in case the accuracy above is accidental.