Tracking percussive sound for video movement

Hi there,

im doing a patch for a video piece that needs to track a percussive sound flaming in quite a fast rate (less than 20Hz).

I’ve got some good results with different descriptors, from loudness to onset_features through spectral flatness.

But im quite far from the effect im looking for as the difference from the louder to the softer hits is huge in loudness and the softer hits almost get no movement (And you can still hear them).

I’ve tried by compressing the buffer (with ears~) as if it was audio.
and still, the softer hits get very little movement.

Does anyone have a hint on how I could scale this data in order to narrow the difference between louder and softer hits?

here’s the patch im using for the analysis,


----------begin_max5_patcher----------
3711.3oc2cstbqZqE92IOEb7zezcmDO5JBNyblImmi1yjAaicnaL3Av6jc6z
8y9QW.GvHrEf.618zlDivHs9z59RR7mO9vhUoeDluv4e67qNO7ve93COHuj3
BOT94GVrO3i0wA4xaaw9v77fcgKdR0VQ3GExqmmdLacnyOAexYUPxtp1SNtO
8XQbXg7aCKu5gfh0uEkr60rv0EpdGhAdKoO4fbYKAO43SE+DgVBb9ekeI0yo
36GBUeiEKN0TzF4XHc0u+L0qVWGkT0yHw09qGeT7imLjTSBem+DaQoayR2m+
88qRi6OQh.tRhzGIHOFbnDISKQBG.QtMNk+LzSInKPIPJSPIXOeAMPAFPIO4
rPxZb5V1llsOP9.c0PitKN08YA6CKBydMLIXUr74At4zOfNwzO5tl9AT7zR+
DuYg96PH+Xxgf0e0ArT8e5wGrA3CTpNyicc7gOSvAimZ8G5T.PfVSAPWJzOh
7g..fhvUMEGkDtN8XRQCdirv7vjhfhnzjW0dG8wD.VASrkD.FfX9THCxvLVE
GFl1ackDvTaPHnHcyp9aK.h8UjDdo+4TqpoAXY.6uXhkLFH0B7IR4A6Rsr6U
psT52tTq6jKzOVu37.RGafb8cL.jxYwYTOpmqm2HboCimbW5hOFsY4piaQwQ
4ECd5tz2UF55DprezQr9vYiXy4Zsy+gyK4gwbRwIN8cm8QabdKZ2aC1e.Dzq
staDvP2CzhHnI2idNVrML6G8ln8oRVZJ1U7KhaOrw2zQn5TKdpUowmocV4rN
NLHq2t0PJoXBPiEZVO7Aj+a0HPKFPlZLHyYST9g3fu+51vfhibeX5sPOk3uj
.ccYDLj5i7IPrvKEYHcvJGY5u2JTsjNvdjNmCbyfzsSgx.z0O26cc5saVd5s
NZVlKQyboRr1GNjPa3zQzd9HQL2sDXR7NHih2kSsUOW3RvT6NP51s4gEb2A5
MyhdDUYYzDlE8LJLqYIXfLJtLfFxh.rEiB.ZTnw2grJ+QZ59gvnnGQGIiBwd
LJcQuuG7sPwjjygvr0Gyyi9V3f0npnVHZnoCjPrF4964GiFJc3BkJL8bU1.w
.yHjsQwbl28g0bNsBbO4RVchcdxK1HSKRuiXhP9LOZCHKGP1T6jbA2ow3dqv
zCISSJBqbNzW9gq6lPWwB.gSd.+GNDlrwIKbSuEC74zJWSVSUYXDqcJ.bGZJ
.ffol9OjEJAfA6WrdT.nIQH9CNQHjIOQHUjuyQLO5UHjh8pySLVYdhFHhG.c
K+pgjgpP.CmonlcBRRBCb1It8WyR27JBfvBM38WWgu2R2ygEhJcYPeqDcMax
YbdOKpn+dB3ymmkYOQZ5DSFZcAoy2jdPbzeD1+TlfTIGiP6cYQ5bRkhm97hN
rnf7gxjhAU49lLX+67m5o0AmT.Oprf1Tn7miIK.zoujVwoG2jvaanTIiZJUp
Wk7jmHyvfr7ka9N2a5n0hL6V7F2D1aNOycqz4kLQM5bv7+5qIgg7nMYNurO3
qgGOviOj57RPQgrjm76LLNLHOzQbw8GNlDU3.sjkOEPBIp.evJqad0fxsoIE
4BMKhGD9TbqFhvmz+IdLmBp3+lEEDOmZFk+364Q4CxNnVOn78TgIfsgFyIOG
yGNg.8OyQLMTOSZdzE0GIu0owoYpqBDUix2Ev73OTLCweVbNOJw+yqRzdIXs
tptBY8IosNQElUhZkvlJN2uElkGklTCBdPDwQsK+Psuh.q+cEM38zoKEkntD
8zkxB+VT02+yqFjwg1BNtx8iUNx+vsZbKdLoaByR3A6WMwKm0KGRx4Wg3S9g
f0purfMnp45xypLWn7QCq9MS3qVErwYf1Emt9qgapAQ74MdPFQI0W7.MZdS3
1fiwEu1Pa.Zo112VND01nVc.OrXWVzlzDwfnwTg3xUc2u5.UDGsNwHuijfCZ
9xbFCNtzQihZscLeUPlXlpLWEnpFKRSia1zouWb31hxlODwcztIJVjdn6Fyh
181E9tqR4Mt+ROaYK4udLQ05qblhhWyC9VSztHHNtTKPyG+GAIQ6CJBKhTSA
73npZTkul2xWmkFG2fdUs7MMsrgyjuN78nMEu0Hq5hV32dzgJlnEmlk2DsKL
un40JB1k27J4EeWA50tzwUkBwuVDt+PLmJZdCbwin7h72ReOu7FqXzpC.etx
JqKTWWkZiqeQuXzDkgHKY0aUqJ1NqjGSZPwSU0t1ZVuh10VVT.mMRNyF6CUl
VJUzLAXRCezG.f.jImxSWX3s8bumvChLKvSCqwWpF2c6PR2viZgr4AzWq693
JRKzoalG37fNmVSDJOlO45bZ7lOW8e8GzvhUCw4E2vEA0jnm1oDzfkIQqzC5
cqAxL9.NzQwrMXXyCKs7x.nkD5YqhHFvB3zoZsdyDGqRyZdL2nV9fUZUgT7H
FZmdU7XUZQmEtoUG4NhjLVHvU8KDo5mWlnun9Hn+hF8YmkY5lIlUErYZRtni
GpfluprLLLT4KfMUpCw2MJ0UnjPRSrZ2TKwQUVqeIJYiP.rDHcdY61B9eTvg
obGH.Qb7bdF57Be9OKZsCrocA5X0v4xTqddJwBJ0XyBdeIOv3AzwGwouKW+n
xOHVKgN+DZvB2HZYhhzVajw5UFD.t01AztxSGLNQXtswIlg912XIo14xR81i
T4qChEBxm31dFh7VxCjg44UdUIa2y3kt.LhhcdgSqk2qHQnpOIuGXCe6zkay
AnTkvVxDSF7fpZYTlnVjG0Kn2.D08A+ivOOWtEmNQJq3nGkcqYakojO7iCbi
0+7gz2+YHX4u8aO8Sag+hHcie4K+xOy+6+0+wGr7KCV5+DN5S0r3qwiUMI1+
twRtZopyE9W+lv.dbtCb7beZMtXbFQtX7Xv+N3AsqOQBDXlt0tuMbnFAu2cn
9RaJhZaHh9ykUYYl6hWalLFyFLYzaep2ztsg5OmnXUvoYcXqQ9zmNVe+t4Yj
yJlP0hXZY1rh8T26Byo+xOFLGlXQ82lCCoKmurwZ17etLXtUInyFpuP2E7To
IgGREwUfEgHHT4ef+Uc.iRWFl+OhmKF5Q87PtOI84PTSQl2XYtbuuPMHsNpA
sqE.2xUT9nQMD99LJVYrnH.+eKqEWKpUvp0ij8Y3R6JTS8zTWBB1J0k3lGuZ
UBSOaIfM.nChJCuG3qwsDlMSgJ49IvqJb6yDn949dw4kSqV5paiqKH8fXkT3
3M93yzAzXic.7R7kjad8xx6ZinpAu.FEog1BXKNsZzAW0whad95as0KFPdPc
osUrwHtlrEL5qEX5sFu5XqZXKTCgLYKazWTibqQMw9XJ2tVNYXYoVMMQas1g
Ss.IzMuTNet+WNse2NWvzvrkODG9XLjQ6El9x7MOEwXjoiSKhfcmpby482kZ
cm+VvgvAK3Rkqces9R3gr5hY59pr2kBxc321Eq1MxxYkRG1Sf1vOt4IM6pwz
f8OChKWLXx.8D+ipVOcsctUonCzsyZvKTr+ZMrNc+9vjlq.V4SJYS3GcAOet
XmEZ36XIrJoOQ65wMEuV4TWUj2NMImMg4bNuSq86e8yDaHtQil256nPjqIiF
EUC2IYT3Z3fPjh5IaPHJnuYiBwhjXxFFdFNJHsuwzrMpMYA3lNxfcOvfSx.i
YJO7TJIQMcTHJd7jMJHFOIAmvQA1TAZLZJGETSGEBr.NUiBhoiB5ThEHEId8
QQ6g69nMGRiRJJM7A4wF62J9DpZ8uSUGiyM+j5DeEwHZ9dh1lScEXTeXJt4S
GnIX5vUUz.eR6O4ohnj.8O8o4zBiwvBj0sMFzzLzL06.B9pyXhy.0ymwJQdj
Z5nD4mFJAXp4xozMGjolHPSoZQDsmypSynfbWLi.6ommSS.I9FNJXS4fvT1B
+ozHAjXb.IS4nvT1B5jNi..8DL5V+aoNVptkFtVcyZtyIiLMkJmRGmMcF2ct
Czy3HzoCxFLgnod+kbKZZybOiJmmpNNAVH1x3adUs83eMnnHKZ0wBUhkpe9H
zqsw8t3zUAwms2q0sKue7yAWOOZKZjFeyO+JpVu0kGGnsRQeeNXbAyxQcXQ5
tcw8+LqBUdj9wLkNiRzejmCuouoO1DrtiS9avEe8uHESX9Ukj5rykj5jm6Te
zyHV+.+3pGDonqRNX05rx2fC3n7ncIhySiKv8BoS9oLijteKHaeZRz5AP00I
ZfcHZ6czDNR8OX1n0+bSkKqJ62HXqwpCU0xYXHvJm1btKlIBev70HjacFaeq
bF6MWTs0NMII9Ml7sy4GIZpggSEu8sC0Wrck6V4JVhS7FNuTqbtcUYWBv2UT
W2m63.LjbckIpWrIThguhGN8+ZUQZMLT3K4.NtpqbYQogDp4cWl3AK6AANuM
JtPb9b84ocoYGFc9yh1yAiAJ0BJC+WDCDrUCECfn6YPfaQ.19rlCpNMyuHln
NE.5IV3+2OrvCeUnPdrR1Oj3z1A79DIPHxv3JTmll8DKFmDh7Q1bgZnflyS1
PI.0NICMpsvUJVWEk0H5+VyJF2098nqGUO0LYuSZOglqd5r4jKW26ym3fSeW
S7mBdFr2bgujQHWLN302DtUBblnQw6OyOqRoslGI.C5ZpmEnQpIbLHazSXSz
mQcsQOYhRaJxF8jqIrH1.8jLznqIv4aqdBdsdhYqd5pJQrBGAdt3xMygA7Tn
XDYhnruMTLZhjrM5GuYpenlXwlhsQOYDKuUTX3OWzDwDVdhMT2RLRcqUTMYB
GAhXidhLanmIRSL8tyfFm5Wi5ZfMTzC6w71o2wfMpFrGDq48riW4KwV0qpDO
2kiNTnYSpAYhTCxFRMHih74hnOEoVOFtZ1ltc21XG2DCb4gXCoczrIsK6Izb
zSPiLSOM4HwntlLEdaAMJACt21t15AlBQl6RldAbLvS8JOjTqzIm8I2xEGh5
s9kKbzJagFESiMDvgF4fmMbZEBLsmfismLw9Qaq2pjud16uEQeb16skydmsz
980R2uqVN+8zhbQU006mkG+qG++vc9XUh
-----------end_max5_patcher-----------

I don’t have ears installed so can’t properly test the patch, but some thoughts/questions.

So is it only offline analysis you’re trying to do or will it be realtime as well? For real-time stuff I’ve done a ton of optimization in SP-Tools which may be of use/interest.

The spectral-based onset detectors (fluid.bufonsetfeature~) have an accuracy limited to the fft settings, and, in my experience, are not really ideal for percussive sounds due to the metrics being used (as well as the lack of hysteresis in the thresholding). This may explain why you are getting poor tracking with the softer hits.

fluid.bufampfeature~ is much more consistent and (much) better suited to percussive sounds but sadly misses out on the first onset in a buffer if it’s near the start.

And for the last bit, are you trying to compress the range of values you get from the analysis or trying to get values at all (for softer hits)?

so, the analysis is saved in an audio-file that is later played in sync with the real audio.

im getting accurate results, what i need is just some sort of scaling that makes the lower hits more noticeable.

im getting values for those soft hits yes and im compressing the analysis buffer.

im currently compressing and expanding the rendered .wav of the automation in Adobe audition. :smile:

kinda working!

will try with bufampfeature then!

thanks!

1 Like