Spectral "Compensation"?

[rodrigo.constanzo]

Ok, did some more testing in context.

Since I’m querying based on on descriptors, for the most part, the source and target are kind of similar, so fairly mild filters are produced.

Things like these:

(the normalization here is for display purposes only)

Since the corpus is finite, however, it means that sometimes I get stuff like this:

Where I have a linear amplitude (per band) of like 9.0, which is pretty chunky. I’ve cracked up the regularization, but I’m not sure what the limit to “a little bit” is, so the highest I’ve gone is 0.50, which is waaaay higher than the 0.08 I was initially using. Don’t know if this starts to fuck the numbers too much at some point or if there is such a thing as overregularlization.

So my options are regularize more and/or clip the compensation filter, but I’m thinking that I may want to try to compensate spectral shape, but decouple it from the amplitude compensation.

The last version of the maths that @a.harker posted estimates the loudness of the playback with and without the applied filter, weighted by the k-weighting, which I guess includes an overall loudness compensation already?

If I want to remove the loudness compensation part of that filter, would I then normalize the resultant filter in a loudness-weighted way (doing something like maxgain + an additional pass of k-weighting… weighting) to get a filter shape that would be the same loudness as if the filter shape wasn’t applied? Then do loudness compensation in a completely separate way.

I tried a couple of permutations of this, but I kept running into a weird/crazy bug that I’ve seen with vexpr a lot over the years, where I get stack overflows without any feedback. defer seems to fix the problem, which leads me to believe that under certain circumstances vexpr doesn’t like getting numbers from a high-priority thread, but I’ve never been able to isolate this problem (even though I can 100% reproduce it in a larger patch). All that is to say, it’s been tricky figuring out the normalization step as half of the things I try result in a stack overflow.

[a.harker]

The last version of the maths that @a.harker posted estimates the loudness of the playback with and without the applied filter, weighted by the k-weighting, which I guess includes an overall loudness compensation already?

The whole point of this calculation is to produce a multiple that removes the loudness changes induced by the spectral filter - or in other words to make filter that compensates the spectrum but leaves the loudness unmodified - for which we must know the loudness before and after, but it is only an estimate, and won’t be as accurate as the method used otherwise (by the way the k-weighting is part of the loudness estimate, not a separate thing). That is the decoupling you are talking about here - you don’t need to do it again. There is some fairly fundamental understanding of what each stage of the process does exactly (in conceptual, rather than technical terms) that I’d suggest you try to get straight in your head. At the moment you seem to have misunderstood the goal of the maths I posted.

[rodrigo.constanzo]

My bad. When we spoke the approach morphed a couple of times, so I wasn’t clear on what the final version is doing here.

In that case, I’ll mess with the regularization and try clamping to see if I can keep it sane in a wider set of cases now that I have a patch that does it with loads of different samples.

[rodrigo.constanzo]

Aaaand got something working well. Regularized with a medium 0.2 and clamped the filter with clip 0. 4. so nothing jumps out like crazy.

It sounds good! Surprisingly the different time scales don’t sound too crazy when applied either. I think doing the like-for-like compensation sounds the best, but since the final filter goes through some smoothing and averaging, those crazy spikes tend to get flattened out some anyways.

Reworked the variable spectral compensation amount too so when you go negative (i.e. inversion) you scale things up some, since things weren’t getting boosted as much as they were being cut, in the inversions.

I’ll try to work out a variable amount of compensation in dB as I’m doing it in linear amplitude at the moment out of maths knowledge/efficiency (the way I was doing it before no longer works with the kind of output I’m getting).

[rodrigo.constanzo]

@rodrigo.constanzo

I’ll try to work out a variable amount of compensation in dB as I’m doing it in linear amplitude at the moment out of maths knowledge/efficiency (the way I was doing it before no longer works with the kind of output I’m getting).

Ok, got a version of this working nicely in dB land.

----------begin_max5_patcher----------
3671.3oc0cs0qaabD9Y6eErB8sdNB68K8olzzlBTm1fZWDTXDb.kDOxLlhTf
j53iSPxu8tWHoHk3kU5rRVQvPldWdYlu8alc1YGJ+Ku9UyVj8bTwrf+bv6Cd
0q9kW+pWYZR2vqp92uZ1lvmWlDVXNsYoQeJawOM6NaWkQOWZZtLXQvi0stMr
b4GhSW+PdzxR6MGy3yo2EfXh4TIjRnTBgvkPttQooKzbPvOVcGR2sINMIpz7
Lg6aLaWYcqnpViWYD.kPcOoV.rmV4m2FYe5yVDltd1cAydLIKTIQ+n9z90W+
Z8W28xT7eNIHOZ8HZNhoTr6BvP9bn.CjPfPHoBLQpZTn6ZXMG4plCGTy0Z84
ovahJJBWGcjFeuXNGhYBZv8j4LDjAQA2CoyoH.ExCtmNmiQbBI.NGHf.nLfL
Gg.blHfOmIoPFQ0BgPAzlArs4QEQokgkwYoOjDmFsLaWp4wAYSBsPBaNBBXD
EFqQVFkcWfhgcNXK7XrkMB35UfkoPGgfavUJDQwhf6QyILIFoPX1bLVx3vf6
kyYb82RMXpwdEZxHRJybH.PAbnC.KcRfUx0eqrS8DRh49FI2kTFWjDuJJeTK
P3bo5CUp.Flhsfz7CkJJobNR8GDVQa3XkNR.ZUkyZopwqSyT2qj3ke7.mQt5
gpP8uK+bRz.nBBUK6Ew+r4jDM5Rd3lnxn7GhRCWXuAfILz2+LUxmBrr8ZTKh
Vq7mmOEjrUciU21o89QTDSLiAEHHW40moH52EPwiyqvtxq7MsJI9on4qhCSp
uuOElmpFILc91sJkKeee8MYGgpmQCJIVc27s3EO8lxE2rQXFvwXF0S7USOBe
JZ0Cgkk4wK1UFs+nhJXoBWzpdxtnrGqatt81hPbZboFrLOMJXud12Ycr.24r
Ttn9nFpKF7DxRWOzXQmSbSE4GBzj+95ONU2+8CcBEeHKuzoGUMV22cYmRsar
9o19Mzw5+xOFiaCVlsYaTZgwS+Wsw3ke5fRXpIq4DlxzDiIXNEQTzVrul4DM
5LmskNk+6C3cOFmD8TTdgRaZSElEtcaql6RI2D9SY4sbfpaJN01zdtTdzSw0
WOso0vbEFUp.nc41A7mYj8i0axTSwjtK1bmpGDqEIyfigztMbo8h0ig0cuG0
oRS3J.iaAU.vl4ZnsbLnF7Wmjs7iQqZSmlkoFXiSaOUdmtWE8XnZhvGdLKsr
dJDHZdu8+XkD1amMj8uJeuyuWMacd7prTsPzYjP2b8i68pvIL5FssxXNizvs
8bwJ5gBVFnyBkRtqXQXtdfpxYAptyxrrjtc0bcIQOVV0813zzCPwxrsC2Yd7
5OLx0tHS04lwt2ldJTl61dePwIJeP6ls64EljTYA281+bXZ7lvxnxX6P.Bzz
o0g4GJVlmkjzQes87TO8rRwwWF8o3Ukev7fZSFTmd71ZRzrlQ4UwqiJJ61VY
35htsz3JqUS6VTYC+PYzlsIJsn6InrNhKJUtS+TQ0IVSzZC.6WFZaa51tC6z
9XtE65Z7w1s1iuPnTNmKa8gpm6V4fjzpMgf02BUGz43vNHOzIIkzV9NxQY6U
r1Zti5oL7LVUF7Xvj3ERsPEv9Op0Bp.EXO3kpmoAL3f.FZ..C6.fc20F49C2
qW8EX9TfGUgSs.OLTnyM.YtJlX8GjZ0d.HRsVjiW0kGHaB7MDYqXYXRjdEDv
45ucB95k6wj8v8.NferSF+jiheWYnyfXMPWvjnmN17VePHtNDPPOfG4R.dxa
DvKbQwzXEhfOdVArjeLXYWe245ja.rhHtQMTuGp+qyB75ioYWhrmIZD9sjUp
E2z+AbdrNhhg0iIJbNmdfwruwQzsANtZQYVnC91TKpRZS0WGnBKu.FnXxsA1
nh6X5YMAUofh0COhAu.A0hA2PtuTmW7lcabGmTqL+HbhSdIgiMPnrXnagxFm
dMWBPRPhSFaDf37VizIGxOdBGQ2cEM3dJ54s4A+wGgA+I02nf+h1KeXtNCQA
cFNGNiJi6BizSLFP8VR3GqzAxRzgHN91v8ly1tXRUFMAWIaWD51z1cwzVuji
WFfMI3pUh9hBlEO.Tgmz7sdC58KTYj4oPCrMhAVcDWPSDCGGEgEf.igLfSN4
OSFoZqtWlsYSTZ2TzYteoqhdtUlouZPWuYN65gctsDoSA.QdD.sBzDHHQdL4
iwwmGBNrc4.N4ocVP9YQu1mrecwELPJbMJtt+9gshrc4KqG1pcsFzUQWEUTF
m1r2Gue+BTBHN5x3jkBkWq.jCRQs3d4jBnCRgNOxWVovkQDcNdtXRgdUENIE
3JP6xHEPWkBvkDKPNxKpE2KmT37HxESJbkchuj1H027osTuj1HDzovNuT1H5
Lv4lGbwkDKDmx7HWJof5pG7ZP6xIEtfEhKI6j5rMxkzeA0U1I5hJEtxNunyr
KNoY1uTwZUS5llcdr3lkuxV9PfurhlAfv8KZvKhnIcUzvC6eqpw5pYppx.sk
myHkE3IUFIqSxVDlbPsezWUlbbYwYTttqsvVkbGhUU0J2QXjonI269cHWu6w
mipGOWeRcbsMvyA5omyT5itx6pmb+rePPGTHjOzHF7JAcX9U5AgbQi3d3A4h
B0d8ocLqNnVJ0OiCpgxCpexiqcxgqaxCqYRiGFa8icfo79jjuaUb1aME+2Ce
WT5NqGh5RTrKRrX8iwIIKyRxNpzQqSBzLauMkoY84pKGdjj.0kdCXNFh45WH
.0QpCL0909jZXtFX8EQnDI.oOUBCwITyQBDlh0GAN3xP6eV.nz9D.RAfXOR0
D7N86Hy9KKLccUwNxaUwdayy1lk2T0mywxlyeWY157vUwUosAzw+4cUDobUu
GV1d12bh5qoMwqYv3M6VFN1.P6s2PQz92aiRCdaXZQvai1DuHKYU6WFgDEAQ
o.cFB3Hf40..LGx3HKDJ..HoCpXRts45dHNUyViZfUrDAMvJpYbAwEDbWX0I
lRCJN.YwwAvtzEQCIQvATgvdzQB3gzEFEXgESAgwsGYuSWD1RELEtbopiN5M
VCrFkkTKSZSDcasA3NWCgSgXiRijXgzdDUMKr7vQkNWlDPfbyyRcMDtQqEDj
jatATVy0onRaNTN4BENYAZ.THrGoDYDp5Y5EChuSEMzTFD0kW86aMCc6MbSu
GydSf99up7AykANA6T6FQterKIR4AeOXZ19G0vgZHfZIgJtq9UixISRy0V+0
umsAc9x7rQ3Abk1uBF8Xe5lmht3njpm5yZTQ3.ChpAT7T1mVntui7mI1O7t+
9IPk+GedUd15nz2YXPcdeRruoisoO5o4Myw.QPJy5USOxZl1P9kmn5lu1u3g
VzOOULNOkB3LinAXTLzvSgpgAzXyi.0gkYc9qImlgIdc.ZCySgLrhVadBbBS
JqNBbvipuYQHLHDXioShXDCpRkPkYuOo3lMHLd4IPy+1nznmBm8B791uIwKx
YcOgVu2mKVxsdLwxJeM5lnX+Pk6nDhlf.O9nyx.XLdtbbZNoxMZeGMb7RNdQ
GvymBi8IW8q2UVp.nQXrGG.gnYzDTcj2EquoohptgDpuMLN82bTpTzsFWaGe
j+EshOEq7CLlvcDoFV4aX+WWDYBcSIT+y1+hBzuL0+JZOCG.8oK8awOn65dm
KaJIwm30ahVcZ1gtpEriNYeJ1p0ykG+7xx7jWfz6UAR+Bi+aimzqSy8wsBC4
eksJpXpPd5ed1yY19a.EdZWHSXb6UoY2lEmt+rimz7BHZee7xxowpu3imeu9
cuu7lKZi2tLaaj6garOq0Rr9Suw9+kCjeW3B2UEGmpsOlyjdQ7pRoVs1hICf
tmUSL3LkWwHgdW1Z8pl7V3riLnf4p0sCsWAACMK+iTssJ9Tk9ualXikthv6O
D9TziY4a9M+FpIXNs+oOuNK2XUX9GuOU+6ow8lDOLlx0elIHMlk7lM1oYKdD
FszCIrCNA0y64pv6IqqatJZxKzIlD69b9LYVh8BS4mpJID.n5EmxG2JzorYH
KRhTdls6a4Wmk8wg2XjQyPKXxjssOU8Gcf+R7laR3uKHyGtf.p2nbwoebT2+
GMtuOSzLf15o8QsMi52QMSAI.Qyb4XyU5R5YY1GYy2dHCstKSWQWjddmINxK
nCC.8s0ZSeUm54eXFbcz6buIFYjmlWLS9ncMY+ftzd9RGrThNThoCg9vbi.j
jppOPJwBd0QbgCKPf0DrAsw8YsEf+zqMgOyZ9chqec5z2pmdnxhtqzgWYNNH
07fARf278pU3VuVgtrJODlhjcmGSa038wgN+j75qwBODZpnYi8O9nQb9xE5p
R6tdO5JNk9KoN05Zc31nX6W4RWS5VqZ5y6w2ZeSs+5jcQ0+.c67p+wPd09Xu
2Yjz6tfpDv7rOkdxR30wIoUB+qeN7zEP.RRs6hOmiwHqsCBpBmy+B32lGEcF
RnA8ZREGjoiIv+B22o7rkVFdxhmf.XLaRBY.kvYfRlDJo9WF+OQqNY4i2DJJ
rY8NfpYL7s78+hRRx9z3h3wEuEUB.hQmaFQs0CCW.El3qUWCkPuXJ.52iJfY
6K91rjyfhzj8BpJ9Gpsvf7dYqscW91jyK3AdS0zQfTlvlKZsafwKyILf.rAF
gn.lzXDfDTFxq0pTdzxn3mFem65iSzrdl85DnccI5GgytDEMsv4Y7YDkCMY0
DA7ppLVwVk3QQ6qCKpHJcUws1LH6AY+a60a4oB3P.2tEUDFPVUG3bnW4Nkga
OgrD9lvxrf2nWE5Lu87KJidbWRR4osAK5WZEtARYMwpqSEPSIJNPQQJnTnMM
5LLGaemIX.y+OX3SP0pTWGcBI3BavI3F5OFIwRwsx56K0uGj2Ruu.eRmMmOe
BD+uINu7yA+s0YS8p7.zCglUYhqF4Tl+UagfSu3.NVI68moTnTBrQmpnzjps
bmo+sUtasc2EXwryCXsu5bC7a9+q+0W++AHf2C7G
-----------end_max5_patcher-----------

[rodrigo.constanzo]

Now that I’m making strides in the “using a KDTree to predict the future” idea from this thread I’m revisiting the idea of compensation, but in the context of MFCCs.

From what @tremblap (and @weefuzzy) have mentioned on the forum (and elsewhere) the 0th coefficient closely correlates to loudness. I even remember @tremblap posting a comparison example, but I couldn’t seem to find it now.

That led me to wonder if that 0th coefficient would work as a vector for loudness compensation (ala the approach mentioned much earlier in this thread).

At the moment, when I do this with vanilla loudness, I have an incoming target, I query for the nearest match, I subtract the difference between the two in loudness and then apply that as an amplitude offset after the fact. This works really well, and is the basis for the spectral compensation discussed in this thread.

So I’m wondering now if the 0th coefficient would play nice with a similar approach. Either by subtracting the incoming samples 0th coefficient from the corpus one, and manually adjusting the amplitude of the matched sample, or in the use case from the predictive querying thread, manually offsetting the 0th coefficient when querying the second part of the process.

Perhaps/obviously using a standalone loudness descriptor might be better, or perhaps leaving “loudness” out altogether, might be better for the predictive part of the analysis since I’m more interested in spectral morphology than I am loudness, as I can take more info for that from the real-time “real” input.

[rodrigo.constanzo]

@weefuzzy

Afraid so. I think the below works: give it a frequency in Hz., it should give you the linear gain of K-weighting. I think it could be made more compact, at the cost of being less explicit about what’s happening

Returning to this today, and double-checking something.

In your p e(njw) subpatch, you have a hardcoded 48000:

In order to update this for other sample rates, that number would need to be different (so in most of the examples I’ve posted (where I’ve not changed that number)) the loudness compensation hasn’t been correct. And it can presumably just get its info from dspstate~?

[weefuzzy]

It’s hard-coded to that because the coefficients in the standard are only given for 48k:

To get equivalent coefficients for other sampling rates would require a bit more reverse engineering. This seems to be the bit of Gerard’s code that initialises the filters for arbitrary sample rates

flucoma/flucoma-core/blob/97f68a56e9d773d67a556e78cc937252b4e77f89/include/algorithms/util/KWeightingFilter.hpp#L22

Which, I guess, is based on what this library does

jiixyj/libebur128

[rodrigo.constanzo]

@weefuzzy

Afraid so. I think the below works: give it a frequency in Hz., it should give you the linear gain of K-weighting. I think it could be made more compact, at the cost of being less explicit about what’s happening

Super old bump here, but to clarify this subpatch here. I initially read this to mean that I can send arbitrary frequencies in and it will give me the correct gains at that band, regardless of the sample rate that I’m at (e.g. 44.1k). And that the expr (-$f1 * 6.283185 * ($f2/48000.)) inside the p e(njw) has a hardcoded 48k because that is what the coefficients are at in the standard.

Now I’m starting to think that that’s not the case, and that this only works if I’m at 48k in the first place? Or is this a generalizable bit of code that is agnostic of the sample rate that you’re operating at?

[weefuzzy]

I refer the right honourable gentleman to my previous reply

[rodrigo.constanzo]

The:

@weefuzzy

…I think the below works…

part is what threw/throws me.

Does that mean it “works” (at 44.1k), or it “works” (at 48k as the coefficients only work for that sr)?

[weefuzzy]

@rodrigo.constanzo

Does that mean it “works” (at 44.1k), or it “works” (at 48k as the coefficients only work for that sr)?

For that patch, yes, but upthread I show how coefficients can be calculated for general SRs.

[rodrigo.constanzo]

lol yeah, that was the literal previous reply… I was scrolling way up to re-read the posts from back when we first spoke about this. It’s quite a long/dense thread!

What’s even funnier is that I have essentially bookended the answer by the same question asked two months apart.

So let’s agree to meet here again 2 months from now!

[weefuzzy]

It’s a date

[rodrigo.constanzo]

@a.harker

OK - I think the maths should be the below - basically I’m estimating the loudness with and without the compensation and dividing one by the other. Took me longer than it should have done to get there.

In the interim, and while I’m necro-bumping this thread, I’d like to run the maths by you (@a.harker) real quick.

At the moment I have the maths you posted above implemented like this where each inlet is an incoming list consisting of the gain of 40 melbands:

If I forgo the k-weighting compensation (for now (and/or as a selectable option)) I think that means I should be doing this instead?:

This seems like it’s putting out the same/similar results (as in, I’ve not fucked anything massive), but I’m not super clear on what’s happening mathematically here to compensate the spectral contours.

Does that look about right?

[a.harker]

It looks about right, yes.

• Top vexpr avoids divide by zero/small numbers
• Next one divides one set of band amplitudes by the other (that sorts the spectral contour)
• Bit on the bottom right aims to normalise the loudness or amplitude (remove the overall amplitude change from the spectral compensation, which would otherwise try and match the amplitude of each band in the sample playback to that of the target, thus also changing the volume.

Is that clear?

[rodrigo.constanzo]

@a.harker

Is that clear?

Yeah. Well, much clearer at least.

When tidying up I realized that I never 100% understood what each step was doing (other than nominally).