Fluid.bufmelbands~ not outputting bands at certain fft settings

[rodrigo.constanzo]

So I’m working on tidying up that spectral compensation patch from the other thread and I’m noticing some weird funny business where I’m not getting values returned for certain bands from fluid.bufmelbands~.

Here’s a patch which shows what I mean:

----------begin_max5_patcher----------
6085.3oc68r1biajbed2eEH5bU4wIwZdiAIeYum9RE6KNwNkqT1WoBjDRBdA
AX..kVYW29aO8LC.H.H.3PP.JY6UePhZvioeOc2SOM+o29lqVl7gfrqb9Wc9
Nm27le5su4M5gTC7lh++MWsw+Cqh7yz21UqR1rIHN+pqMWKO3C45wEKbf66t
zf+OG+nrD3edePlS9Cgw2m47TPX55xGIJLNXUxtX8yQJFbaZPF7Z8yCShusy
63tj3767WEnFCWLV3Z8bmr7Gtg4U99i2sILNJHOq9cpd5rveT+zDzBz96MYW
d4MWN5xcKWFEz9wi82nG6p+RPziA4gq7Kmws94qTH5soAqxMDSLxi.yhCVPV
fQTLkKQdXLGIkW6Pn5qwA3v4uodE+829V0ut9L4A7ENqdvO9d.Tbt6tbGEB6
78WcWHvi+9qbByu1Y4tbm+cm0Iw+ivkCBbd5gmgwcxdHYWzZfokmGjNRFE2R
Fk6qKFEVxULCBirfQELgGlQXLIi6csCkQOKFEPNS2EVN6qCy1F4+7lj0Zvid
HsgSGj1zjHTNpZRzuAfkmEjmqz3JeMlaO+4sAFr8pqb9a8SKjddZRAWzjTfH
vftR80HSrLKagyxj77jMNAwqcRtyIaK.Oo613Dkj79Lm61s58Aca6fdTQRpk
hj7wHRZD9xBaxMmMAUfOnX.BPbEKPtXgGGyITvDiKVcA2o1ZBcAXmXG.12rJ
Jb06ASG2cWPpy+T9C.4VanIqvPhhS47TZHX7H1IO4edjbKlkbK1OC3VDs1BS
RZwrHD5rvrHKbL7k6RAvLywOM.VG9I+mybB1rM+4QZS2VEH5qJa5RoVif4cH
0mOKK7NSTeaWQk7Zj5S83MVEAb5Y9X.3ENO.twnVd8JKIZ3oinc5NcXVnUJW
vA62HM8hHlXZBZAXX1e8RvgPaoInWPZhfqWeyH7f0u+wSS.kvL+6CNflzlfT
C4ocG.AYPeuNM+q3R8pBRiSkmm+TwAOAP8AHHDvUldl26Fecbz8D7ujzKNds
c3IlX7XwcOhV7LY9OV9x9M+U0qqAX6b0+l5W+l+K0uAuBMiLJpDDtSdRbGTB
xv1L25qLjCwBcaPregAQzoIInEyFzaaWgVNfiauJEyDPxDK6CZTHgPHOy3vq
SDoytJiDSW3x6hJw0VHn7IlJ8YhtjV3yOhxXFDEIZKN3oVkXpMTr0INIciej
xd9gHrPLh.QOMDVHLtB.tI3RERjT5wAGEbQXv8X7AlLJsNXPzBLEVlILJ3wf
zLPbsFn.AEucasgeSsGQQd9gD8KRdc0PgwlgvUCkF7XX4yyqF0OEvib.I1kZ
VZ6ChxvQTuFHz9zXHhe0axLHvnJ.IMKQsjHXoyrDrlyUd48TFOpw.fwu.ACY
Bk.smZ.776iRzAFu2vDv.1FDGFWWEtwkWGbm+tn7aquNNlrnyqW5kPmWrZc8
eWZneTEFbeZHDOnBHZvJTCWNcfqOZtNlWGYz2Qr+1NdXPXDnK8bwL.I2kszO
UwoJrRSJuXdRRTyKU8bQA2kWb4sgwwsnh4Ia6+hog2+v.OqIUFC8t0WI61cw
lqdKHTjea45g6uO+nnBE2lu9O3GGtwOGblZSgeXUWzrR0CYqRShhZfulq7XG
WYMHjuJ3ov04Onmn5BCvsGtsTH5pJt75v6CxxaNVt+8YMGIK+YCQu1P6VVnD
eaNDUTDfEMuAP8HLKO6gjmJ8EnTPqNAXe5pqqTW2JXiwGxZXSKhqWlm3W+J0
W.n93cYRreyhGwzXu9PYb.1y0DKUSeEeS4x.EVXldhgedx5k8PLvWbhAyqLo
IuDzh+ga.Ei9nE8SLHiiXbGDiR9QoHEAMh8DFwjltHL+zD39B2ray.DF5HHL
jiSXfXABism9XBpzkcokYxchbh5izvFgBzfjlFQjMrEELV9xHw7XvG1l57Y2
gc9svuINuKakejepxkIGbeTJOqDh5wglgU752QlyxfMWavFahsaxouZ7uGhk
newJzrZWF6svS8iqKBbGQ5QDfUZis559rp8AtHWU0W2WiFwqC9PMulmDhkAe
5iZwGgRHxFQ.J0POjjB5AchoG6cbVEfdOtCoAO006lPkkrKcUIetbgLml.G3
BXdXbUbDe2dcRGrkR0mLTPsEJJ.24AJXFT73PActgBq4HyETHrFJlSRgsv.a
NABOKgBxLBDDKgAk4sd.hhAKSLgNYrqu0Dn8sphIHb4tbiYk5YZ4jBH79njk
9Qshhqq3Ee6df6TSf2tn7vrnv06yzcVPNXFGtKCgv3uSsbHEdebBXsVsa18r
CHzQmU3R+OXnyIY5.FTE4L9n4NSvvK7DtHD7AOAi3I4pLhZ7.ggGYBC6Mm4X
uWK4LmwaVDDHt6Ym17dxS5OF4beZxtscQOXmPVgG+toTlnTUU4fbkbF3ZgPf
wbLLHdN1DosAAu+iN2byMaBh.Vx5raUYZKqqMTp6DiSOM1dyPd6jFfXlczWr
fyAYchKi3JjRAuPReF1IMGTWHLaJRLd8HXGBcgnWkbA3Poj4hYvnBCdN8X6t
eLzg0IBiNAgb5fJ1Wu+u1R.vnC0yOW9c+6OVW53Wts3pSiZEQUN035mg6BWw
Wrc4pSF67rQWlxVqCyYfG9cQEHroXOxqRV0QWRiHzhybfxzpt9L18u.zi9nD
hWFJwA6.JlRmEs.PLHMWWxVNeFrXZeEFhX1UKJL20Yg2hwz4PL3tncgqW.BC
ZcgO57NSnFMDKdmQO4PMm2UixQcdGPBJJ7MZWhQnSPL5nqfTINYCAkx7pW88
LOWlR9xcVInkzpOBKnVmrlDq7t24c2E3q183rlj5ZEusRN.7s24c.0RefJHH
3EUd5JvHj5eAxl4IYn5L.3Q6hEfe4XAHzBWv+Erf.R0LJ7qqcjjySldYqRb5
wvfmzazaXTX9yMzCu6NfrZfnaz.zMzZQDZ1D80oP7nUaNZYLK53Dye.b9+9G
pO9A6yZyK73gWX48k07e0HIoqMEyPUDRkAIuNXSBHzDudAfxa8y6ZgZYmryx
Wke58EIE7p2opOBsw2zcYODjsvO7N0+9tnjjspjbo97iIQ0ypfsVzxfvo8iF
TFfhLGzEiBG1Tzidygdm+lsY.+J.Lj8P38OrEdnRUGP+.jXSRctgygO5mkCp
Ja2sUY4p7+Vm7TrCUBijEk7Tw0IDDe+.5awLDPy0SVTP784O.ZjDPYMIFjUB
xdHIZMXyF9ePitZ.9fkDeG6s5YsjaAm4XZnT0QIfHHRrKUB+lnBlRRO77JwP
lpY2apKAoBsyCICR5fR3SwZtTtn2xTlI+0b8GQHBSFjL4ZofXv9T8G8o5O5S
0ezYV+QMKV+l1C+SwYZ+ByevO2QIxAfR9GcxKNIsANJdkSFrVTTf5jxoF5QP
5DVZKOww+wjv0N.eQc5OCyx1Aq1WeZ5ZkmonLmNxdhRL0afIBNrmoVFcubkc
.XMZq4jEVaol13sq2EujlfEYM0HOYgJ0dXlTHwBFR.K+H4W5xU4ldKACIZpK
vI6JeGywjycgj6RQtLJ3CBA7K4knxU7WaLz6.9U8XVezI7TWuS1VpST9dELo
6rH4LXkTPmdiFkJGB2EbWL3YlvkH4XtR43UekTnjDrp5ATpVy1FEW9xOJT3N
mUOfqs6YN8W5aWcuw.AR8IoqC6L0idzoX2lrOPchJs7si9S38q4fg3dZSQTy
A5VfMNvf9TvPeJXnOELzLFLzpcoov0JC2YaRVnB8cVFnhuYeyl3TCwQL8dqP
wlC7N7GIELTnNDatfUTU9zLVOl7.dFhx4uacXhCX5YWedrwldZfKQ4UNUT6.
+htf37eHM.jfc7cxeJwICDBfH999qLsujhFWx2ekJJYypsNIwAk2FDHsghk+
Dn84jk37zCfjkOrj4y+nRViiIERgYWqh4NM.Butr8N367neDD28pG1E+9Zuq
jhWPP+wfytXwfWIfZ1jQyoxFc4Bfp1VNWrIXpzx2WM+iriBXcQ+SFeQ+28dW
2a0vSLdJfVPqYF.Ijpv7cuzgz2pUGza+uXxR4gUzHWcKuPRWvgeTDHIii3Xg
Qf7xRg9Wbv8d5kPzWMm7DKOPXfa5dbIE45gHHrjnJc.voULFi7XPnydH.oHu
D4OYc1VkygAerWhMZZ03YimXqR3x0sNqY1djy37EDOrfgITNsHaEcJoSuzR5
aSdePMyu8vG75+HwPGCa.YManKpIXM0EnlDtmq.HkJYVW9BW3Gf55QHbBW.D
SOxklX9a+nCiAxr8Y6v60ioiVQ92Ks10SK4VmVCNKPHDrTRwTIXQQvdIDb0N
K8QGzvTbo7maTbyxfXvqYA7SCKFf85KsAZ6LOvEuhLOX705RZQXnidIcxO5k
VJGwp4XuEIFuuPCu3m9R5Lb5KItr1VwZ6KtNjbKnSiI4CW9MZvy1CKoZg8Y6
HapL9ZGTbH3tIb81Dv2prxNGnIIzxEL3GLy0kwcERrpDfgq01cfCuuFu7pRr
C+xh2J6ls1OnJPi7xBZTtkrDDWejlEXImhHtTF3M.A7ssMK4v6qa7FMK3My1
yIZGDnUIQlMcn3bCt+W3Ey0tiQYmhB7.vqWSh90ECQ8PRFBFTOBBbgBLFh3D
hqq9fzrXPVO23GhP030Z6iBmbfCiGdeWRVu0jRs13rSJmYaO5zSXuQW7.R5b
ONBXhdpxqP3oPNNB6Q.LV55QwRhgB.AuxbYdbIC6IOpniJfBV6jQHPZADN2y
7izyzI7aYCoi66RJHYOoUaDY9IsytnDx1lcPGTlICgGzmI89W0iSS6iZP4a6
Wq2Dxa+xf3c68Mqb6R6lhr796BihpPj2bvc.AJU3WdA5dcsK0f.P7XXrTgjP
Pytp1fhx8SWBm2hBW7b3xGDHHdHM0goNZqb8mjJCoZqMc7nj8yIPhMyDxSZL
OQUCgOPT5MW4GeewFy5hpGYlpuelrMIsZWpWP8Z7b6xStO0ecXgCxUpd64W6
2acyQAGtyC21QkkB0w1u96nKg3Jl5Wrak+bvHqPlA3km.csI2TVwCktHtTZ9
jjQ68Qq3lBPARe+pzBHbMex71tHLyqqSNattHrlmqVmV0mNc0fljQ7b0HJWz
vrf+pUvKuwySIfLtlHxJwQklgZr1V7BhB1z9wck.4vPSg0SjlOAuI8JKKFvn
Dykio5am.lg7Lehi01mZ+fYvLuRQ1ZN0JiYZ.FKbIFMRIBoNGSseA0SiAX07
+bKDk3Wq9dR3qC1DtLIZcy.5UIfQOU2FFqJPlfJgOpGAKLfZokARcInYUo6q
9c42peMneMo5cRO5kV2yf7c8o1hfcn5YuQkFIu66zoua.Ok4p06LZ+LWjlgo
3WzNzrVEE3mV6owlnq7.CIbCnADcW5Q0nLoFzNkngBhaFzZduoWz7spJRyV8
l1TjF+Zlg2M9ePTqj2NtJd+.KHAbLYPi5TkYM2BE61r6WKLSEwoUBeslSBq4
.dtQ7puDmx7MiiuTfdyC79Ex7cMivklYN7SGwHrqT4060c9otezWM9AOeRwG
pod5RDsyF+IHMqg2ZQnT2D+rJIa1wpeeztf5sb3gg7t77D6BqOoWjoBI7TCg
ufHQZxSwmEVrG14U1TnWXr3O7r+4gDHBD6fYA.WJkXrLPvXL8xgDedZPvYhE
Zt.U8iIp.kqiWND3KAK6w49mEJHYHgvDJi.gE5UmcEdXO9kCO9eChhRd57PC
HRNWyZDXo1UTUNnrywVtGBImaDc2lkAoeND32YgltU9WyAGn3ZLFWessYEIR
MN1pvhQ4rnEveegXHXfTpWgUB2hjN.7WuKh8h8387x85M.KjKFoucOJSf7JR
4hK9hnjl6u0VjtUPZegedhyWnNaFWMEv3urO.XAquupvHaVTj7onUfznw6Yw
I.q61gwrzxAqNg6ch8SRS253MVxhdNFigUq+IQDlKGVLwUsrBFylC7V+UzbO
7b2K648q69i0rz14VGD4+7G0s1pt6QUXjbJX4m.xC+oNOWW6XyRqDMy+gtY2
dWVLt69KGbs1cAnQS.Z9Edc8B9BeNeyVW0eyF22q0tlykT2n+Y1gM6EiYxyA
iK51aiCe8Ply0oKegn8o4EbipS7U+9ZVcZFzu8FJWPDNr3aZTSM80Zzq0k3O
fPa6LwoWpYh5YwLo9R07rmIBxlYBMEyD1Fp2TLSTwEiOguTyTyuTF5alZTtV
spXkwy331L0Mp2mVEOxLO0j4.qshfKDcO0jyZpE1H+RmBgJaDea90OwEk71B
9lHgJZiBOxhuQONuY5nLxIalHGal3la57lIaXbT9TfS1n7OI7I6zClEKbd13
Sg1g8yFKk1MUMpZ7osP6FOCxFe7ZQJmHFjUSMdVLQh7rhgIeIm7lKU0pbbUm
+eUTcPzsPzdRkrijJAwB8gpD0y0lbNHxJVHZJbmt5sbj4hOIykU9IHljohOB
oAKNZCiGhr1rItivMa0XnTSRqFBUqlA0gMBp9aBTsa.T5DH2pVhMH6Qpg3VI
+9mdaeEMQYpm6tdgGSsBOx5Ddj0Hb20EQ+0DQO0CQY5wu9s8mr+lI5utjWG0
963Y.sqVkwUyHirHCGWs8NYLgplO+npi2wWCumS86NpZ28rpa2SnlcOy508r
UH9R+77gTHp2p39tZ9N1ZuuQSF.0tPg+kgdp0O1bqnZUQ+N9B9cfh8cTE56n
Jx2gKv2SsR5NaI5u8a9yCIJW4UTSwS0J3lRAhnZzNFwSPxwDFVEx7hoHX4Yu
3k1qgt0CvBZQ46gcYBOuhOg7NpZ.G4JLGWSAmZpSHLvgHdCpFHOxJYLAFbC2
TIjDAqnvNvfEhAVICq7wyrBqRuQKX3V5sW+ZC+kmWmlbeP72nEOlFQb8VWEt
5zEyOTk65yxS381oodtk0GdggF0Pb5zHd1rhwpVo9vOMJg5AkcsCy5R1srJa
65S8K750qmTmvqql72mGDG7n+Km43BY0eesuQZ6Vh8vUcjUhGnhOM4f0erpM
07JBn9b+v3OZITAxuUViN7SSOnk8THXXYHf6.Mg9NVLSLLQdUAT+G0+ZstOm
psy1bmqfYowfifwCF70Qmiojd8EAqGhV0EMPzGA6Pk1YyrFDOWZ3GVkmdhVQ
lO.R8s28GOAwtiZ93bjBmRL6ulrtr3g5O14tWGchzfrv0iIEgOtIDqRK7DAM
55feHfoKqHGtn4L.ZeU3p7iSqdwEf+J02i.4u5713qWkrM3TrXLJMm8I61q5
T2LGxBei+xSUHvx3It5kSx4a.e1WdTGn66XNYyRjynmPeSx8pvvNQlR44hSs
XDih0gFxZrCIufdA++r4HarzEj79s9OFbWR5lILfgQ6Un9PjOIH2Z+z2eSr5
7mbiNSFmdpNX6On5UoyuJw9RM.OAY0CeD40IO4GSdF8Zl7iNrgXapiqRrf0I
geRjT9ghS7CBUTd5Swqhb5xa6SX+AeX5Ry1.YAF8yLIs1dqy6cOHrEsOXeHV
FE.KYZ1IveeRx6uZxD4Bie+XjPttwumf7uV8pA1BRVsXOUnZU8CueBym8p4c
uDrbq85bO0NJU5PiXV7Lclahi+Xm78efZw9cjPe6z5e53KjOzTNIpIG1Mcd4
bVJR4JwwcgtctQPdrhRVPWWlEexUZQ0RXUyd3rwqF8UDKvo965Oiqi+bw2Ng
Vc0GKb20lN4yj.VM5XOSccTbxclmQ1UdlYe.rzl+DIU0T6vNtX8trisIcafN
qyYKYcXGzw5n+spq4LU.X8tii0P3kwH4gc9F6SfhMc6lIB.azUaNsT7LPmrY
h.tVcrFqAOq5RMSDL9eGr9jgO2JuewUwVfJVwXpgulcKG6Ig1zgbFt63LsH.
4miHvAswG6EQro08b1.31coaiBNImSYHieMDNR3okgIRtfHszCM2pZRT8cKm
zj2ak0iIU0LMXUP3iCuyccIyTEb2dfCUu.HmFfqceQxJJukcCnQzGjNaDJKH
dc1qsUPNrILMc5dcVrq1zzkNarpp4J0+tQOTCUZBl+r7f61EEkezMXoc4XJ4
brIi3BpK0Tz6BP6ZnM7RcVWb02pnJn.UFDbkSKQ0fTmLNQjtRieFzJIYJwi1
5qigwEp9KH8P0lqdMcdAdRkMmmO8jeh87PFu+.4LVwVZKPHJtYAV2DvUM8ki
jFKjhMnCmkVP8AqFEaYw.kE4eLLM+Ym+z8IVTYjGqV7OcBq4nyo2AOELkssf
EpaTMu8u+1+e.a4ZGJC
-----------end_max5_patcher-----------

It appears that there’s something strange with regards to @fftsettings and @maxfreq where there appear to be gaps in the bands.

Stuff like this:

Or this:

Is it the case that depending on your @numframes, @fftsettings and overall @numbands, there will be “holes” in what can be represented.

Is this a bug or a byproduct of the process/algorithm?

If the latter, perhaps this could be explained in the help/reference and/or capped to ranges that will still produce values (like how @numbands gets capped).

Also, is there a way to figure out (without guessing) what would be the most resolution I can get out of @numframes 256? I was aiming to limit it to 200-10k Hz as my low frequency resolution will be pretty dogshit anyways, but I still wanted some resolution in the higher end.

Those settings, however, produce tons of gaps. I’ve played with the min/max freq a bunch and it provides some different results, but no obvious pattern to the gaps.

Bumping the fft settings up to @fftsettings 512 64 works right away, but since my @numframes is still 256, I think I’m just filling the rest with things outside of my desired window(?). Or, in other words, the max fft size can’t be greater than the @numframes can it?

[weefuzzy]

I don’t think this should happen, no: presumably when the resolution is isn’t great in the lf, something is going screwy selecting the bins that contribute to a particular mel filter. That said – not having looked in detail – the algorithm may just not be tractable when the number of bands gets too close to the available resolution. I will enquire of the guru…

@rodrigo.constanzo

Bumping the fft settings up to @fftsettings 512 64 works right away, but since my @numframes is still 256, I think I’m just filling the rest with things outside of my desired window(?). Or, in other words, the max fft size can’t be greater than the @numframes can it?

The fft size can be greater than numframes, yes, because the fft size is only constrained to be at least the window size. So you could use @fftsettings 256 64 512

[rodrigo.constanzo]

@weefuzzy

That said – not having looked in detail – the algorithm may just not be tractable when the number of bands gets too close to the available resolution.

That’s what started occurring to me when I was whipping up the example patch. Either way, it’s something that’d be good to know for reference/knowledge purposes.

@weefuzzy

The fft size can be greater than numframes, yes, because the fft size is only constrained to be at least the window size. So you could use @fftsettings 256 64 512

This starts getting real brainfuck-y for me. But would @fftsettings 256 64 512 pad out the rest of the frames (meaning whatever (potentially) suspect math is going on will still render usable bits) or is it then working with a “bigger” number of frames? (with the rolling buffer setup here, I believe what would be in samples 257-512 would be random since it’s what happened at some unrelated point in the past while the loop was going).

[weefuzzy]

@rodrigo.constanzo

This starts getting real brainfuck-y for me. But would @fftsettings 256 64 512 pad out the rest of the frames

It pads the window (256 samples) with zeros to bring it up to 512 samples for the FFT.

https://jackschaedler.github.io/circles-sines-signals/zeropadding.html

[rodrigo.constanzo]

Awesome!

And to follow up with a naive FFT101 (128?) question. This is beneficial in this specific circumstance because “something else” is going on correct?

As in, with such short windows, should I zero-pad the other spectral descriptors (and/or loundess) as well?

[tremblap]

@rodrigo.constanzo

naive FFT101 (128?)

[weefuzzy]

@rodrigo.constanzo

And to follow up with a naive FFT101 (128?) question. This is beneficial in this specific circumstance because “something else” is going on correct?

Possibly ‘something else’, tbc. The mel bands process basically involves making a filter bank in the spectral domain, and it looks like some coefficients end up being zero. Possibly this is a quantisation error when the number of bins relative to the requested number of bands is too small, possibly it’s intrinsic. But, yes, it’s a workaround here because the zero padding gives us an effective interpolation between FFT bins.

@rodrigo.constanzo

As in, with such short windows, should I zero-pad the other spectral descriptors (and/or loundess) as well?

I think it’s only of possible benefit when frequency estimation is involved, so I certainly wouldn’t bother in the case of loudness (which, IIRC, only uses an FFT for the true peak estimation anyway). It’s important to remember that it’s not ever a substitute for just using a longer window in the first place: i.e. it doesn’t magically give you access to information that wasn’t there before. However, you may find that the interpolation from zero padding yields better results for you with things like spectralshape~. For pitch~ it might not make much difference, because there’s already quite a lot of work being done there to try and come up with interpolated frequency estimates. Also remember this comes at a (slight) extra processing cost.

[rodrigo.constanzo]

Awesome, I’ll test this out, particularly with regards to the processing time.

At the moment all the things I’m analyzing (in real-time) (loudness, spectralshape, melbands) come in around 0.3-0.4ms, which I can totally live with. Particularly since I’m shrunk my overall analysis window from 512samples to 256samples. I basically now have around 6ms less latency “for free”.

My @fftsettings were the same at the bigger analysis window (I just had more frames for the fluid.bufstats~-ing), so will see the impact of speed now.

[dcocharro]

Greetings All,
I have a few naive questions regarding [fluid.bufmelbands~].

1. The documentation says: “Each frame represents a value, which is every hopSize.”
   When I read the data from the buffer, does this means that I get the results from each analysis window reading sample by sample or by reading a signal frame every hop size?
   Basically, I wish to understand how to read the data properly.

2. Second, I’m importing a stereo audio file which later is converted to mono for analysis, with SR 44100.
   I noticed that fluid.bufmelbands buffer has a different SR of 86.13.
   Is this related with the hopsize? 44100/512=86.13? How come?

-monosample buffer

-sample_MEL buffer

Thank you!

[rodrigo.constanzo]

@dcocharro

The documentation says: “Each frame represents a value, which is every hopSize.”
When I read the data from the buffer, does this means that I get the results from each analysis window reading sample by sample or by reading a signal frame every hop size?
Basically, I wish to understand how to read the data properly.

What you’ll get is a value that represents the reading from that specific window size, and offset by each hop. So you’ll have a single value for every valid analysis frame of the overall analysis window.

@dcocharro

Second, I’m importing a stereo audio file which later is converted to mono for analysis, with SR 44100.
I noticed that fluid.bufmelbands buffer has a different SR of 86.13.
Is this related with the hopsize? 44100/512=86.13? How come?

This confused me at first too. Basically once you’ve done something like this, the “sample rate” of the resultant buffer doesn’t matter, and isn’t real anymore. The buffer~ is just acting as a data container, and is no longer audio.

The general workflow would be that you would then feed that buffer~ into fluid.bufstats~ and get some summary statistics from it, or you can keep/use all the individual frames themselves.

[tremblap]

@rodrigo.constanzo

This confused me at first too. Basically once you’ve done something like this, the “sample rate” of the resultant buffer doesn’t matter, and isn’t real anymore.

Actually, @dcocharro was right - the new buffer’s SR is the accurate one. Indeed 44100/512 is 86.13

In the resulting buffer, for a stereo input, if you request 20 melbands and with a maxNumBands argument to 20, you will get a 40 channel buffer. Each frame is one hop forward (hence the SR) and channels 1 to 20 are the 20 melbands of input channel 1, and 21 to 40 are melbands of input channel 2.

[rodrigo.constanzo]

Well at that point it’s not really a sample rate any more, in what that normally means.

[tremblap]

it actually is. Think about it: the sampling rate tells you the frequency at which you should play a frame. 44100 = each frame is 1/44100. 86.13 is the speed at which you should play that buffer if you want to sync to the other one.

[rodrigo.constanzo]

@tremblap

86.13 is the speed at which you should play that buffer if you want to sync to the other one.

If one were to “play” a sequence of numbers which represent the contents of the melband analysis as an audio buffer, sure. But in this case, correct me if I’m wrong, the sample rate doesn’t mean “samples per second during audio playback” like it would for a (conventional) audio buffer. The 86.13 is just what you get when you divide one number by the other, but has no meaning beyond that to what is being analyzed or what was analyzed.

[tremblap]

@rodrigo.constanzo

The 86.13 is just what you get when you divide one number by the other, but has no meaning beyond that to what is being analyzed or what was analyzed.

it actually does mean that. Let’s explain it for everyone’s benefit here.

If you wanted to play a buffer the old way, you would

1. take the buf SR (44100)
2. take the current SR of the audio (44100)
3. divide one by the other
4. move your playhead forward by that per sample

for instance, if you audio is 48k and you play your buf at 44100, you would need to increment your play head of 0.91875 per sample@48k to play at the right speed the buf@44.1. Obviously you could sample and hold, or interpolate for better sounding.

now in this case, your buffer is at 86.13Hz of SR and your audio is at 44100. so you need to increment your playhead of 0.00195306 per sample@44.1 - again you can interpolate or round or whatever. In this case, because the original analysed buffer was at 44100 too, the value you get it 1/512 indeed (0.00195313) but that wouldn’t be the case if your original audio was at another freq, so our object would have given you something else than 86.13 but the accurate ratio of your hop and the original SR.

I hope this helps?

Does it make it clearer?

[dcocharro]

Thank you all for taking your time replying!

Blockquote What you’ll get is a value that represents the reading from that specific window size, and offset by each hop. So you’ll have a single value for every valid analysis frame of the overall analysis window.

So if I understand, to read the the energy of each mel band (stored in each buffer channel) we begin at sample 0 and from there (x*hop_size), correct?
This raises another question, what is the data in between those samples?

I also noticed the slight different duration between the two buffers:
monosample = 7912.086168 ms (348923 samples)
sample_MEL = 7918.004535 ms (349184 samples)
with a difference of 261 samples.

I thought initially that this was extra samples to represent the energy of each mel band, but dividing the sample_MEL buffer duration by the hop size:
349184/512=682 samples with mel bands data (which matches the info reported by info~ in the snapshot above)

And I suppose that answers partially @tremblap comment, 86.13 corresponds to the frequency for every 512 sample.

[rodrigo.constanzo]

@dcocharro

So if I understand, to read the the energy of each mel band (stored in each buffer channel) we begin at sample 0 and from there (x*hop_size), correct?

It actually starts before sample 0. I made this diagram for a different thread, but the analysis starts such that your first actual bit of audio is covered by each bit of the analysis frame/hop:

I think you’re using a default hop of fft/2, so you won’t have as much overlap as what I have here.

The stuff in light green is also zero-padding generally speaking. I made the diagram there as I was trying to do something else that would incorporate audio from before the analysis window, but that’s not what generally happens.

@dcocharro

This raises another question, what is the data in between those samples?

There’s no data in between the things. You’ll get a single value (per band) depending on your fft settings. So the more overlap you have, the more temporal resolution you’ll have (at other expenses obviously). I tend to use an overlap of 4, but 2 is “normal”.

[tremblap]

@dcocharro

86.13 corresponds to the frequency for every 512 sample.

at 44.1kHz, yes.

For understanding overlap and padding, I recommend the bufpitch helpfile where there is an example and a way to query so you can see what is what.

[dcocharro]

@rodrigo.constanzo

It actually starts before sample 0.

I see the issue, specially at the extremes of an audio buffer/file. Thanks for sharing that.

In my case I wish to understand better how to read and use the resulting data stored in the buffer for the purpose of the project I’m working on.

@rodrigo.constanzo

There’s no data in between the things. You’ll get a single value (per band) depending on your fft settings.

I believe this pinpoints what brought me to the forum.
The SR reduction which reports the MEL buffer duration equal to the original sample, plus the buffer visualization (buffer~ and fluid.bufview), lead me to think about some kind of different representation.

But in fact, my initial expectation was correct, each ‘j’ sample (index) of ‘n’ channel (mel band) of the MEL buffer, represents the MEL energy for each signal frame analyzed. I went through the documentation but I wasn’t able to confirm this assumption.
I hope it’s perceivable, sorry in advance for any english mistakes.

Thank you all!

[tremblap]

Thanks for the feedback. It is good to know that it is not understood within the current documentation as we are working slowly towards a rewrite. Just to point out:

@dcocharro

each ‘j’ sample (index) of ‘n’ channel (mel band) of the MEL buffer, represents the MEL energy for each signal frame analyzed

That is the case, and well said! In the help it is shown in the 2nd tab (mutichannel) and in the doc, the 2nd paragraph says:

The process will return a single multichannel buffer of numBands per input channel. Each frame represents a value, which is every hopSize.

A more thorough explanation of FFT and padding is going to happen in the mid-term horizon indeed. The BufPitch has a little graph that explains the overlap and frame counts if that helps in the meantime.