Creating [resonators~] tuplets using fluid.bufsinefeature~?

First off, all the sines objects totally slipped under my radar as they came out after I was actively playing/exploring things, but for some upcoming SP-Tools updates, I’ve been messing with CNMAT’s resonators~ and just doing pure sinusoidal resynthesis roughly based on the Resonator device from confetti (which was based around sigmund~, since it offered multiple peak outputs). Obviously can totally get that now from fluid.bufsinefeature~/fluid.sinefeature~ which is fantastic!

So other than wrestling with resonators~ (wanting to do the same functionality natively, for reducing dependencies, but also it seems to be super buggy, leading to loads of instacrashes), I was thinking that it would be great to create some new models for it.

There’s an IRCAM patch for this (MaxModRes), but I can’t make heads or tails of it, and it seems like overkill for what I’m after.

So the tuplet format that resonators~ wants is frequency, gain, and “decayrate” variables for each filter. I can presumably use the frequency/gain outputs from fluid.bufsinefeature~fluid.bufstats~ @select mean to give me the frequencies and gains, but the decayrate is a bit more complicated. I would think that the derivative of loudness would be handy for this, but I wouldn’t know how to translate that into “decayrate”. I had a look at the resonators~.c file (github), but it’s pretty impenetrable to me, so I have no idea what the “decayrate” is actually doing. I’m guessing it’s some kind of dB over time value or something like that, but in what units? How to compute it? etc…

So yeah, any thoughts on how to generate frequency, gain, and decayrate tuplets in the fluid.verse~?

Yes, they’re not quite the same thing, which is why it’s not immediately obvious how to get from one to the other.

If fluid.bufsinefeature was doing partial tracking rather than just peak tracking, it would be a bit easier, because then you’ve got some groupings against which to try and estimate a degree of noisiness (perhaps using the standard deviation of the frequencies in a track), which the CNAMT res-transform object suggests (slightly obliquely). Unfortunately, I don’t know of anything that only does partial tracking on a list of freq / mag peaks, either real-time or on buffers.

1 Like

It is still on my radar to try to extract the voice allocation from sines into an object… I’ll let you know if I get anywhere.

1 Like

For resontators~, decayrate is inversely related to the Q of the filter. Sharper filters ring longer, so have a lower decayrate. It use to be possible to generate resonant models using IRCAM’s ResAn, but that’s long gone. (Although, I think the source is somewhere.)

There are a collection of more-or-less scientific ways to generate decayrates. There’s a patch called sin-to-res.maxpat in the CNMAT mmj depot that does some of them. It’s in here: CNMAT-MMJ-Depot/patchers/synthesis at master · CNMAT/CNMAT-MMJ-Depot · GitHub I thought there was another one by Daniel Cullen, but I can’t find it quickly.

The truth is that some pretty simplistic approaches to generating decay rate work pretty well. Most acoustic sounds will have higher decayrates as the frequency goes up. Lower partials ring longer. YMMV

3 Likes

Ok managed to push this a bit further and got something working ok.

Inharmonic metal stuff is quite problematic for this (as one can imagine) but I found I got much better results when only analyzing the first half of when splitting via time centroid (a FluCoMa-based solution @a.harker helped me with a while back). My initial thinking was that the derivative (of loudness) would be dogshit for a long/sustained sample as at some point it becomes quite flat in decay and that would overly impact the statistical summary. But I found that it also improves the peak tracking too, which makes since since the (perceptual) peaks would be more present in that part of the sound file anyways.

Still not optimized the resonator~-esque playback (using an mc.reson~ hack at the moment), but it’s somewhat ok in terms of the overall duration/decay, based on the derivative of loudness.


----------begin_max5_patcher----------
5689.3oc6cszkaiakds8uBFclEyjtZF79QV44jMyhY4rKYN9vRhpJ1lhTQjx
ksyI92dvCRVTRjhPR.rjS6paKUEIk.te39B.26E+i2+tEOV9kzpEQ+4n+Zz6
d2+38u6clKouv6Z962sXSxWVlmTYdrEKK2rIsndwC16Um9kZy0YwQUujUu74
n5xn5rMoQKUO1txrUQIE5+kj+0ukF8eR9uhRdJIqn8KHOqHcY49By2Bt4ha2
kVo9zI0YkEe7fmf17DqKKpqx9Vp9ZHPLn2kKR1Xt7h+mz7OmVmsLoss1ln5e
YEO8wcoKqsDMCIhkDDgfj1efnGhvXZLFQfBHgRnHDAqtHjSUMSDiECh9+a9B
K1uIqHOs1.Mvd8g0IKS6eM0CVtut8Ia6sYqL8yxG+sekSVnu1+78uW+xCNNX
Tj9h5CexXw1HE9UVjTWpd+uuOs8Arcg5utM0R7KpxdRMtrnidF.ffXHQS2HH
R+FDXdCgFCEPCRwvAnXzACKo6ZnyFBUgiY4oeNcWkhGn2Ww6VjrcauK+tdeD
M57akluHwCcWJqvdIZ2k1k94r1OOp6pI6Tzdshv2uyx+7EFYwqeMkqR2UrOy
zUrWTMN0zkLiHZ9tpsMC7lAt1a+JZp4TsuvHl2HBvqHoZ39o7xkeJcUOtD0v
11zhrh9xDGb6UoqS1mW+w9hDlAqAteKe4f2rSv4+dWlhqn8QdZW1pxBcm3fg
A8kaaNEahQ3P+5qDi4IJR1NvGVw.nvjQtYkhH2W8XxN8nzi4o8GlVTWVle3s
59b4oqqat81rhhiPw5xsieycYO87Y9rOVpt4ly8catS0G2WXu6GULD0erJ4y
Gh10I44MxrG90+kjhrMI0oZEmVUZc2LsHQQnOWsbWYd9Azq8Nedf6rRwfuL8
krU0OaZn9LCpGOaaKSzhtQ4UYOkVUe30pSdp5vqTU+UKn26R6erQ.9i0oa1l
qnhCe.knQVUc0ykuT07fsLZ8AfWsB0WftuBvCt94TDdnxv5nGix6emSzB9XR
wSKdP0i6XcGSWnR4GUfa+AI.XkwBNLl9PjDFK68CmiOTK4n1KFPaIp2c5owj
.WzdcichFcP9Gx9ieOR..mEyNxxwXHFsQIGRCQDxwlNF07w3lPNFTfzYBTVl
ms7Se2ifBgoAErbZPAdwfBd1XUTJWvmETVmWlTOsnkfcpnERZrpfwAfugOW7
MUKSxSUnj1M0HB.X9MP7YwrqDtnFdIHFLMdwtT7hMW3UxiUQS.O2BKEx3+JA
F.oNpzmPjxYxpjmRGlmJs9yI46UrUp+Cp3oT5TtMFJj10sWAKrFqHBifGDAB
f7GhLS7SJ+lUNXtJpEytQAuibAfxouBTfPvUgQyDP8akcyJ2qfi0NmDE.lHx
bgM0Q4QvnGOO7nbgLSuxHMdTNEZwkmpdRIKaTOAtEFI7HfEet7TRuXMp4yeN
vpAnZ+m9OuNtKHPycwbPxiNJfQFwKJDeFE8hvQenVMOzmR2UEA8qfnchJXhF
pf.GbmBew19lKGN2WXAqIEE692UIGxEZrRHtE4vQXqnykRqukGkojCcAqtJL
hXckheKJ1GYptz4Rxayx38EaSTyraBYNGma2fJonp4wn4lHgXpdHw7AUax9x
pxWJlBq1rOuNa4yIEEo42Dtw.Ay0AwLxgYVM9uG8AMjTollFadPODwNcP1zv
G4RgOuNUGS+X5knyw0Uw7Jt60wIZvEOGXY+NY6Ng0ewNMefhUoeo21DLSPzj
vifFX3gCtH3A4Q3w14V3lTBWuJL5eHRkmibtYY3b.PFW2LXD.Ady7KutaJ5s
Acj0H2Pq56OLRUUte2xV1jVGaeH5PJcUZUcVQ2tK8W6lfk4AcZn5h6FXnicC
HMjcCpi8BY.6DHxcPm.itC5DDmYJZ4hCW2.5pHBLT8BWGQvgTPk55PR6XWX5
EHGGQnAuW.bsW.CYu.4RuHjhpTWMiPCJ2ozwdAKjVQXtZEgGxdAmdIbmn.0K
DbmUgKBHX3JegHn1QbVFI38BnyFUQgra3hZqV2gBinpqLFcNnFFYUfqVUCIm
QmH3zJtFGLZtXa.wsPG3Sq9nMHu9XRc8trG2Wam8R+H76hBFomxKeLI+nHHZ
nXU58u14tvvobrXakFGYBruGS1oit0j8qxz84nr5AihUzrGEqJoVbrdMevl.
sQw2ZlpeXBUUlXgGwVRbarAekXI1yXIkfiA89ABIODI4wr9WDgeHRfBIHy8I
HiiixyppSKzruiv0hmcjVHXmhzTCqKmGvXslw7I1hTXaYxpnG2udc2lddoLw
HOCsRtYu9Xl8cfGTFU30.lJyA0kEiFW5GrbuCF19FByt.2u954IuIiHcF3UD
cmBsqS28QaL1ZncOxy.iixJxpiLT1BeO1CeEaZicmvL1SwWyX+KJmCVWtay2
Gc32FeWOb1eQqO6fstbXnPXi+Ti5DL2FlNT7XnAcPJuIV4VXkv6FKV90MO1E
l7GDwNBOlKG+R2dsMbPTbVoDrgUfvfwDAfq9eDfCARARGakFtjaOONvzqgZ2
qjK1eVB63su+bDHPLP95..mOSU5nk5zMMKr9hj7bci9f9ESr3qduMOlVz+q4
zvs4.Lg3aMIivcTG8q2.6AWZMLLD6AyMvaZ1CnGEFZxvLzYjIbheQRhgTFUJ
QLFTHnZelgjySwXWoXDxmtWnMEZlzwM4jLZbLgnFpgPJDhaRcA8thYcYf0E9
0WBSv.FJPPeBIIazjUT45nsoIep57Px.g5JxXM.prMfTpLZ+ONV4SOC6IJF5
Sa.Z+K0gCbDFcc78PlLlc7XrMnU4bOInCYdlhsokys4gHz5FDU3KhD3Qh7So
ecJKfm9lK4pozrJ.Tv4svCFjnICnB+Zn4CCe8dy2GBnwbpjg4WIurxRkQ9UZ
13dnMyC8fiLTwawLY5HGD1qSlgBlIWPxJVW98nCcFdbO5aca2EW769Em38aw
QVSr3iuFQd4o.Iwm501TUWVkrYa02mJmre33zbYvUPApbcCInbXyqDsDOCEK
PLIR.fRJByPvqUMOZ.S4ROBG6TsUZzYUzO4b.TN7azCv.sKYzkqMXH5j60Dx
WOYh+R6DINuTxkOfiz4dJgBAJ+2UW0bQA2SpEgveGlm9DtUOL0lV6Hnc8Ho+
LY8+Yx5+yj0+FSV+CmW2gZJetbajgQchXY0HORj5ovwPDLEwnB0r33J2dsF9
8ajbhv9LdmOG.7WRxWtWOtDU+b5QERm0k6hRSV9bj1Eh7z3n+uSdjrJymaao
xeonrByend5znWTCkoQOmju1bszhzcO8U8i+ndwXSsEom05zEIqV+Eqti5+2
WktdetogeRYpRMHDkDozrUkpm9shKI5uolqVwS+sEssjpeEknzMqDd9ZTkZ5
3qpdH5ypur5pHETmtKaYRdzp86L5FOHOeGZKRGkAvlT.X6JTXWpONwqi4Xvb
MlevP3Db91pPDUM4cpZF8bFlRXboxUuNe77LmOetPgFlGi9lIv.A70PY2969
lpoyEUO756c9QeKgKChht4JO9rt92DNJ2d9oQLKksjESDRJWhELHFB0EyKNM
.4m1rU0Q5k852V1ezfP5rAhwPHBU..PNSfIsq1quSLMxh6q7eoAAZMYDfDfA
hFKeOFQ2VPxEFO.UMIIT.QpQyLlynOLHIV0Yj89SlRqgOJEIMrdloQB3Ttfg
DHl.w45MmNDUEIvLoexr5eNfP9Zo+lR8FmEqToIgHDjfPHpf3XFuOt9M4HIP
5h4N+0rRkTcpo2y0OldROAI80XiIiNp+pysDplkZ8dchxdVVPCm2TbPXhkKQ
xiUSmTBgDpBgkLlNnzXAv.4rUAb9b5W1tK5+XML5Ope0GlFFTSlHHnzbUsD9
S55SGzCJ6aDSwhXrPHARIAvvX0alhDX.z0ymu5IwSJ212F8gukq9L5koIBi3
L9s6+divGVMEVlTnKGuBBgBg.tdcJYAvCd4rUf7f.DI1a7UCtlW2VM3XrR5x
LgPqdrtLwGJkFz3ePpuh3YBZ9CJWP8HqiNLHT7MbtZhwTBhZRXffTWtfymJo
p8aBmBIhxNmfCjPfRsDQ4qktr2HBf5HrX9zG4CNpVE1XdLUwHw4JTh.nDcH7
eZnk5AVJLeFK0aSUwf6pwaO3tGSTdLTnLEPADDFhkbGE9t357VSfX8CrvWCh
Mjm3gQ5CNWJzWmuOaU7i6WizyIwA7xootzX9ank1CBjgn34N2vUd49UEoUUe
O5CpNnsNJAi9vKpIbV9hg0i.jrnO7b4Vyeo84J5Ce5kT8VEq20Jft35sOUGv
n5O31jUqzWV8qQUo4JjLpsI7E66fqEMfdKB7iw8tXtJbLNWaNBYNt6ZJtCCY
ZyhEWRIw.9Fmg63Pll8J6Mtka4m1a2jsxrS0MrmHaAzEyDwRrTMKPlZhyBDE
pWKZSTdc5sB03K+NnT3nLKcGT2ULiutkt7gLa4ckUOjHgqYJeH6CNWdjBZQC
3x5DiKxyo1BsJHVISyERlDCXTJkoWoiguyAeck6VYiHQXXpjItRmfPZqwwNA
BcOTfxvgj6G4LiGKjr+tpLhEZ0xu40REmKddvf1Kbk4TbWnT7TLqSMFHLcMv
k3yLzAE1T0bXfmrR8H3H262iUMF8obX0PI2ODObx82ORtGrxjnf2SxTSgZJk
mjFisaR20kFiZJXYYtMZ70mYcT.TxfOL3u0uM5uBvWUNioq7wU4YqdsNZLQ9
fTkVqHK0WRaR.Y26McXa.5+PcgBM7L4EJfcZRRxAutemM4Td226xjhUeU8hM
Hv6RC7mJJUeo5SKpif5qHaaHmOQxaIvFFKL5s.zsK9zki2DL5tCuEAGtGsFS
8ogTUncA9JzT.EjSgVlDEi5WuczoIDrIrYd6zT30BcDr6XqyOvHX.NTg3zZA
US0V4MDFo9DFWuK8uuOsXYVpWMfg3bGMfwYu4Fvn9LqmqpK2dco6rD0jBZFN
LLzSY0GwmIxtdifT5e06+yX6Fz0jXmfArOnqNaHS5d5qD8Dh8ZhdpLFsII+0
LX5BGtGlnElT4mC8TULf86vL5TzTOrj+7vW9m4y4OymyY5vWNotb0i2Vrc0T
PHsA3bPNrb+AJN2rA+fMk29wNx19UOcHuZ2xPns3eDjHOBNWLHpmKay9MtAL
NenIB6cHPwI+HG7L5HyJ+1COC6wDUa4AzANlKOfLl+H6+WTupb9VeXSmrS6w
zgGQai3dhCZgF22jyBuHbW93DfHdWdmkZgMzLO13xNmCTtPHjHVnNGxHuYmC
YNknRV3.i.F7PfZvCLKTIpD897b1hQtC1qd78voiBTdOD9FNObHCcu.9VGNM
NGZQP3cQvrHCXTzAu8sv9eS1JywKeYpdYd5+axxqbYsvX7oq7KiB85xZAI+N
bcsfjl5cHxtymbaMQA+yE15mKr0OWXqfsvV05bDZhYi1kgPW7rRIf.jWPrYa
Z5YQYmEXFnbFO5VFhfwBJWmF0MFNz4MUS4IGFf4tSmSTxcbx0k6wtZoHa0vH
HrQjeDYiZvEn0WCHCE.NG77k1c6ReJ.ocGhQhwGKqQk1MNifCvRGhlMdoGcU
W80ARDVy4kfnWjLH04e9bjqWAoZXfmT1yUkQPT3Vadp7G+hewvrTfFeeYAo7
0MWZw+kHxsyGAMrNDpzNEmPvGQn+.AHC5TDAxrkPkf.OykwseMBFF3Aaqlcg
QKzrUuK1+srI3eZMj09tS.lZVGm.XHovUeHQWbIJX1J2EJ014oEAPocqGkj9
u4a0zyVYFUASlsGYSxrYfiYEHgbd.PNw8VwGUMgLY+ensRVybgH89XCBGjgf
ai4+frCgP784NDJH2C4VsyYRX.2DF2SxbQfQBm1aNXnqvEN0Knj6g5rAIj7l
NKfPnAj6z48r0fEgrWfbVFA9FGPA54lG1vZvcYjPgEtlxuhvlC1tFcEgsWbO
DsKV0.f2bkmcJEcueLioCt6cNxYxUcXf5b2CEYJqJTmT1RtKpzJAsTS3bUdH
zUeGnq5ZeqiUHz+tGpP274dZyYUKjihwDLA.3XrPWqG83gfJDy78ofZ52Vkr
76KllvPTXL93igSh876gdYYs3PmuzXel1hUo4m6rk9jMG6rCnXyIuLA5o7yj
dUIha0VUGa29Mq1k7x2GLGl6hkoyEJaWT5MKZJu61UzhXCkCy4R5vP.oWibT
HTMMqszy4mbxt5qKz9Zn51DTF4oDTVdUUPjIkNkPQu8jzahiRjOSh37juN4I
p7qGSvSppscDxdxHykd5j.VBuJZd+lGmnRiLssigNN6aOp2Adh73xY5.z92J
yJ51scuDWsVdb3Tm58XWkE43PTr.XDOVr.FDHn.qEIuUq.XXuqzcsl2RoRYh
SA6yVy.v1S.af7zBSBDv7jBYrzqztww5oTxcVxlQsm62Ftbg2HSlOIycoJk4
WaPyKE8srRD9hB8oz7i6WuNc2jVq5xY0I0s2RzViUBecr0CQdknWs5qWutpl
rvUBhEGO+.n.4qxZxB+Zg55XgsjJgxiEXDAxUSviwXPnT6l7.kaMOvdiDdjx
6J.8qyUSMOs362hEJ3ozKlKhgTFUJQJXQHzGIKRe4ACBDBj3zStfIN0BbFLH
55a3Qfw.meAWKeANDngNqRzmJAMGh.aRSJLGQAqR2k84ptfYxa7KpIWqEjDP
BkPQHkoe8BkP7jRCDKDXjqGdCpuzdWSYAPPN9DcXnyvgt0s3fyObz0W2ovMQ
GNLVA5TUCC3BJmPL0e9tyP1aU379TKkUgcSBV3MMQPH9tUUTiIplTMyapafP
zLou4lGrGRmRi+W9vQDPPLCUoD1WmlnSsREbrdcckh1UjWkUYhQcgUsRecNc
5LLWVqIoxTz5TMc2ea0zX1kqW0eeA5VdX54sLfeOCorwNA9KUYCdfkpIH9Do
K1HZv2LOVMDqOkc5i1YEaUcqxWhzLAM+4yYO8r94Vlmscqdn.GH02bbLoeD2
QvZnj4I027qBQ0auy9rqyU5gWzKVyR83nlZcGncY3Lb6KbdMf79NobKfAC2T
owskIXEzHDDgnc4OzmXLWFXXVFF2QCFxUzvzENLjAsfyw67XCDMvNQePnwM1
QNRuMb7jwAmaJhqsD7FaINxgVRuGW81x2ihJh4nsAm11vv21R5vjM5lZZFvE
xlMLUClglFDB.+vJLx4Ndrv9lpwTGZZjOjawtH2hodnkPtv+x7BMAcE8tUcQ
H4AhaifdvPvc5TSaHRju4NgNYUQ5CqJtH8iY9fmwgFh5Cw.ovEEZ9pgfNzP2
pPfzEUHRuPRPWMAB7sIP2ZZTHjzcuo8u+NRWjzYg.vcpoYi3kGL7MMLLNX5j
1A8IUws6GsKZwkBOH1xcQ4Z+Jj106snKsDGGDGUcooIjPnf3HofoyBrqukPW
.OimIRWZZTPFYcngEuUMLOHZ.oTmz.5CGunt3hGwGsj4KgMkOdBOnZk.lKOh
vxaPfDE9lFwCgfgaMcHzBgYuc3M6tlngvP3OD1IWG7whFfDt5u4M2RbWgyat
kbhkA3iVxkwIjOzshbQ2JxKiStX90G9GCcw+MigdjOZoIYx8AI4TKA8hjKDh
bk461aJWZIYPVMZfSqDtWbyG5xRKx7TC41rcOrkr6L1Q0dYcibTMW9n5s7o0
Z4wqyxGWikMIc2X0V42+Oe++BaDEgT.
-----------end_max5_patcher-----------

Here are a couple test sound files too. With works well.wav you can really hear the difference analyzing only the first (via time centroid) makes. For the other two there are weird quirks I can’t make sense of. The problematic.wav one I think is just an inharmonic/perceptual thing (though even that seems weird) and detunes?.wav confuses me. It sounds like it’s a bin off or something when going between full/half analysis, but that doesn’t make sense as the fft settings stay consistent.

But yeah, pretty promising results so far!

audioexamples.zip (466.9 KB)

Actually listening to the examples (attached) again, problematic.wav seems even weirder. When analyzing the whole file it has that ‘off by a bin’ sound where it sounds a halftone flat. But when analyzing the time centroid first half it’s pretty solidly detuned by a tritone. Is there some weird frequency/fft/peak phenomena that I’m overlooking here?

Ok did a bit more testing on this tonight and it seems like @padding 2 has having a massive impact on the accuracy here. To the point that it seems like there may be a bug or something (or a gigantic hole in my understanding of the @padding modes). I’ll try and isolate it and post a code example tomorrow on a clear head.

Basically with @padding 0 I get good results across the board with the samples. The time-centroid version is better, perceptually-speaking, but the analysis on the full sample still sounds “correct”.

It seems like there’s either junk data being included in @padding 1/2 or the inclusion of zero-added stuff in sinusoidal analysis severely weighs down the averaging further down the road.