Trying to adapt the Autoencoder example to use spectral frame descriptors

Hello,

I am trying to adapt the Autoencoder example to use the spectral frame descriptors instead of the MFCC.

So I have a fluid.bufspectralshape~ object instead of the original fluid.bufmfcc~ object.

With this modification, does it make sense to keep the fluid.normalize?

Also, when moving the mouse pointer over the fluid.plotter, most points seem to be associated with silent moments in a sound buffer actually containing few silences. Is this caused by the results of the analysis itself? Then, how can I select results before feeding the regressor?

Thank you in advance.

Hello!

One way is to only analyse material that is above a certain amplitude. bufampgate is a very powerful tool to do that, check the tab of the helpfile that ‘removes silence’

I hope this helps.

1 Like

Thanks @tremblap ! Seems it will do the trick.

Also, what about the fluid.normalize?

By the way, I still got a lot of silences in the output despite silence removal in the buffer @tremblap !

Please have a look at the patcher.


----------begin_max5_patcher----------
6381.3oc6cs0jiabc94UUo+CclRUkD4coPeCMfeZkkcxCQxUJW9MYWSARBRB
sf.z.fydwU1+O42Q9ik9B.I.HPiC4.PRsZThmYI.Gft+5Se5y8y+7q+pW8v7
zODl+.52i9Yzqd0+TdkWoul5Jup5Bu5gsAeXQbPt9K9PR36Sm+KO75x6UD9g
B80iSCVNOHY8g6jtuHNrn3i6BMufGz2E82qtex9sQIxug9whqtZzR8SS9JdC
lvO7v1ETrXSTx5GyBWTXddXGO2Y7Wibm47ZD2S8SBYlSiWfYLXdC5q9+70ek
52xe8Zvy32G7T3pzrset+o1J4ru3gWa8eDGkq+cMHX99UqByRB1peNODreYT
ZRZdTbXxhvG5Dn3cBTLq.ElyUniK1W8KFyQ+Im9wJ2KFqVjtcaXRwojG+nj7
.sRNyxe3LH.n1lW9X8h9woij.vxjx4hmTpIw78EEoIGFNxghYd0OMgdsV9+z
ey+9w4Qlb4tHL6wvjf4w5uryInkygG6pzjhCDHeeVTPL5OjFub.PLNbcvhOp
dbKRiSy5GfwGdPYo6SVFpumFK6G2878U66n9UHdabuw68mQNynLeGxq63ef6
e4hdXSx5ZOKr5s4LS34Qnb0+hIb3ddsdRQqSRki23nEu64s0+5uxiuNq799W
vBOlR0LaItXEAvW1K88xI6uDtM8IIvfJ4Smi1mq9njYdv1cqCJjWIMIOr.kt
Zk5W6BhxNGtdNN14lWtHPXyHtDOgmS4+gk6IoT.qIbGruqiviQwTAQikPtDd
J3s1KLmof4vCf7CGOybd4lG7Y.oX6Ppml3hv0rzvbCHxlh460UXhyPFBGPxP
v4jIWFh9jvbG5uIIW18QytqbTQZ0Nrb418v7+1CinrmNBHvASXnX7XCJ9Yim
VXVI.Tg.R18RB8mByxkR.V+6+pGB1sq10eU8+HEv8Kl80du930hRLWid7ZYg
OEcxiV9ryjyrB4zZelgp4CtGHCTOozkRJp8QG4jZVJqFZ50LEEW9tfElGfZs
8v8qIyfFfvT89LWhF7Hd90vKE293zEuybRjygqltKLIJYWVXtjMQPQ4b338W
FtJXebwipiKyi9jgy.YV2egUkCytuaySaONMVmEsTJWtbfzbkQc8pWojlfaX
dzXNo+JIA655OWRmIwm9tatbxtOedPlZgqjiG4vcKRSiadui+kwgqJJu+tnj
j1.ZQ5NK2MKZ8Fa+0ySk2cq0Gu9V4OtOwb6GkDIEOlK4605KFDGWtUu0a3CA
IQakmiVDYVOHNGuqQ1oM4KxRiiaNuM25ott0RI4+hv2GsrXi9k0f7P9GDsqh
v5giK6KiVGlWz5hEAqyaco7hOZVBpes8yK2n+XQ31cwxYSqugbmijGd9lz2m
W9MqH+Z.EGUOuwF+5LQadiVLSk6MyCVGdbSccNpx4dgjNKqXkRzTz2fa70NU
t1iD1mvDkT+N0XjREMdlcvKk5nUiCyz7FvXmSXk1q17u53ILUrltXfp4oNMw
o0gxMiEYnbo.d4n2pN7MLA4L.ZUpEfMTC2GpwghZNFTiwgfZzqJpYDH4yntM
qw.x9z7.6y.4XCgbLidKUzaDeHHG4phbEn4nX4+2b6fkwbaGoztTLi3ODlgw
Fg+T+jhcgfXrQGwrxJahYiQbGFhpSV4wuMbwrhQxWtFgxmFHZvcdDG+ZPjO4
9iQ++xajxF.gC0kfODn3igmtQV4qO9nmIWN4gSOSel.FSFyOoNU+z5ju9spr
jPCwrzChjkgeXZvJyX4gyliKyCFXPnL3ngy3fF0zdINJoW4O0iY0WnGbJOce
1hJBlJYZP3lS.k.nQIGzp6mOdFNRNUgubc1CF0lQjCjAC4JLXXPGLJ5l1eys
QK2kFkTTtPQvdZiyhqLOe0Gl1of6HNEvX+awTPJEDh.hjfL8jD9P2rTg7S8f
gBZmK+5LXFsMKZZKg2Q5L0+dRG+UPD.vTL4fY0qXT.SJVbC11x.OEpnb58bO
yGNZfyGTFsZ4iF6z8nxB.Qy2WXNQrgsaOSaHsNNcdPbKq9zoMlNLhtXio2Rg
fih5tHNLH60klzvwhyL6wp4jNsZtuUqlS3FwY70THdtSWLazmODraSBv1i.f
CDvNPfBlwK5cYEhNs.wXhEqh2GsblDQjxJtKMO7yn2Z1cYPGzaqsMBLf0xoT
.vILH+NYXpPYhqON8o3YQEgYHxkMq6biBl.x8RFpCtyMj3fXb0Xeyb8cOqYN
ExLujEgPLcrH50yyqhkr5CST9WL.obsebHxv2.ktx37wu6BcsOlAh+XYvL45
7qLm4OHwTIz94wj+AGD8jw0992.1GMNx4Qo52Q0CmAvm3zQjv3SvbWOWhfwc
78dMxX.xVysgfOWqvGWXfOg4YSnSG90JNqFxE9VCmJ.yaP92uLhPHrgBHD7z
smoLnhN4z4RJolzUn2llTrIKLeSZ7RzaHNxKrZUsqPkWQhM5HlPdT95hMHFC
WdQyg6MtrDj2sL88IpOdXPpjOeQ5di4kHP2KOUTvdfVIqNPwPO6dM4.nbmy7
AHru.QL8rRBSwlyRLQDFkLcab6UEixCM2iobGpvm4BRKikRJ3rvcwkgpAFrn
EdVOIfX72.lYD31kMcRVbU4j4YUdJlmWsY8TxGSod79HPqvCFIrOdHRMboyN
Cl5VEpl3INFvxTOGvQrkZlY7wnj0n7oMcr5y2Ie9YAw4aB10lg+AcuPuUNNW
rIHIGge3B385LyyHDHyWKTnlsK8HWXGpOoyKcd7k8v.VLpb0Ey6KD4JDVIA8
MgAG06VITgJLyVFEDmtFspdrj2I4y4ddjqGf4tPazG+ITi1d0qaQZxhfh+u+
2RKZr7ecu7Go44QxynVFhxkZ1gx2KOuJoRcu8IQ+i8gcK1CELtvsKhM1uVhT
fM55y7lB0y58f5ePv98e21PIkw2sMe23XKPpGjj2xDK6XG502TfRxAUfXKGb
aPx+.z7zzB6aHZlOGCttynPP.yFBL06VYiG8oK5LyJJ47W567bXJnD2Cicqc
FvUYw+ofi1a2L6MLENhA8gRs+dcXcz9AOopYpuVP1GOOXzEDLxKsi9DZN8d4
bTIn93vxvtN4RQ10VtxH5NaBy3y3nmBmsNHplgppS4zwsOYdmGsNQEC40+Wm
lAG1E6.tpOzZNvIKpdfxiscjrwbiD25pDoR9k9PzCINhJjtUhflDFe3.Qm5I
tVGQheo2zN3FMaNT6of38goq5HgGNhSwoIqsrjz36tUt7pUm3Xfe271QZv5M
Bmt+B4aRyJf9xpnB57IIEon3P.iyNlSEifm91GWDkGGsLL6.wPdXgbpUN4+Y
i4UbloD4P8eBJVH7OFkACpEgUYiU4jygIFXlbDq644l3ZVTwhiZrBoXJNkTt
yT9nNKQhed5JTK81UBhDnmyt1TZhqY364HlIZm4eXhysvyXHsTiIHty34bL6
Y8cIHHLJvKtI9CDkWjEFrcTm018iQ0r1rQPvlPq48qR1HP.OtuI9TlTtHEoq
WGaQdnyHmvG1DKVs7WoNUt2LiM7OjyrrTohsWVZY1obN.lwTGStoRuAtN9+J
LbGRkqh5LBOuPJyL5ei8uijhHEEiJ1DhVEUfjCi4gYn7vvs5rXMuHcGxjT4y
PeOZcZ5RjVBHTTNJPme9xsaDGTTh7YHuV3GB1tKNb.+wLrAarKuMGW4I9SNs
gRzAYJk8qFWSe8qjCdWmJ4fK+BpjChRqf637RM7X7W4EWoUd1kTCObLFbhxd
o7sLAK8tWokd5ErzynDESalTO+SXn+Bcv3RGvuRzAjKgNfS0GdKnuvAX7W4Y
WoUdmKXkmPMUtK5Kq7SvJO85rxy8tfUdZo5PL1Kq7i+JO45rxSuDQ8IDsUOb
Ee4uv2qZ4+4v8YxUhjvh2ml8tbjJ0WRTZnuI88JEvku8zj7hr8KJPKCJBfGQ
2b6Yu.slmpb8+Rozr886KRCSVjV2DgMH7+wfhzy.CICigktJlPOIQn6IIna3
EJEnOEvfJNbqwk4RJMcL6YLkSU.Ro29JlvBSW3mVFr3yO.1TbL6Q6SYlLP0V
d03EGF+pRk9CJFJneG5OlErtxJZZqmQ9iHS8J6YstY027hxfusLd4c7lt0sd
m++2Ygp3N.sJKcq1ziIJmLEGkGtDEkraegIAWRBeulmW9w5XPOA5zEAS1CIS
gg7f4eZorDanf3jqJn8WyBzFYUhVMNy34QqXOTHMtkfJLk+BN6FPq7WBUYyz
XOsslAjBiRnlpbRU3Xecm0+4pMD5Y9O8e7C+PyS+ujIM0N+7xDPhQl7Ycedk
QxHrHD8tDoHPg4Eim+KsGrRUgZhCumpizj601LzvS4dhW5CgJsVhKlRBLekb
rRgZI9LLV3pkssJRo0ozB0V7R6zE9QAfeXiPj7aPlSqq9paBC1gx2llJ2vrD
sNKHYerTVuLakd0x3e5bBjXeHjRk6gDzWJ8pVK8plTXyrkiWl48hWp7puT4U
eoxqNoUdUa0Yt7cggKii1hpUr2GuJw2wTr2Rd8ZRCRshId2gkb0u8y1wk1mp
bFnCf5vmo3laNk0TKptupjiNHGDVR7LS9SO8GFprEdQHEdP5HiYkYDyYw36v
R5nRWVXzR.pfp8gTtPAJeiArX2e0b1OEOKoXCBCnnFeA.jOr8bkoASGU4k6A
7QdbUv14wgSCD4ADhLpP45eWBQYgqmFzQ.josnVBVdWANJyvk+wsySiubF03
Kt1pVR6HzMtGA99iMsR8xbcTMpCIxsQwJCyrHMYYs95w3gXTX.lpSGUynbjy
nbz1RYyV.S+Jc1mhmse.cp.ZeJgZQQT.Ji1qBosTJEhho15KHDS2TQ6orF3b
OZnBQKUPZpNr1pP0X0lVq.0bcHsWGVC1g0hcHMYAnM6vZzBQqVPZ1dVZ2ZSC
2g0xc.McGPa2Az30tVu8p4aOZ+1mFvvzBFplvcqM7ILiZeFvoegAOKn44AQI
qR+by7pEPWinpsXAoEZc3ebRTeL7QI8l9IV0AvTFuwX+9x48dNWwuwcqcZbq
Sjm9UkBzplMkhytKdbd3JCBtZpsQk4DU2ZqaU5uaMnF9gcYnuYEF8cxeRPeq
xnPNCCxCgqDK3JENtVld4Xr.J4J9pgrmV98OOxKBbXfXrFTOUh+ApF+VqH+8
UU9mZv6j97.fDVa3V9vI1P5LfXuyCgwSCB2RRrS5ABcf7cUzp6B+Oo3UeZs.
2V8.2TR3ubBjKdXxNoXzaYX1ybpUwB2zGIwNlx.kItj0e5lMAOq0ggmfLsp1
JCZohvkie3lL+nmy7qc2M35MLwmyvjAZXV+KzTU0gpt6mnw9EJ49.U68Nq36
smKikEXZEJLsDtSEbbx+eUvvrIcGRolpphGq9bPRP7GkJK7ZkwYVjl7TXlId
g1klGoKANkew3zz2semIzxNbuxPpxT3RUF9nw.nyrW8L5dYCYJGgoSD3v0+x
ysxB7MMkyTDJt8FPOWKqp8KoGqoSimeg3.slOsJChuMdPyFv76TBYOUc5K9f
lxmoIIol98E6NzqYeqbH56ZGeNQxPfnCGnsWMs.KF8NrS5o61fnuAi9Fxz3B
etCLPpzyqd92gVzON3imWy+DteX60QH.8seoeXwD7swQQSTO1yEpef7+MPO1
yc3drm+LgJ0nLLf+hnI6o1..qugIFtMrQ0ZsvcKOE2yD5oM+zQitwcm3N8jh
mHvlU0z2f1Xfaff3oevfofGLmph0CoYKM9oz4tXH10x7ggHdJou.2Y4plLSI
dAl9x0axGLtmEsNdZAFLXfYxwE3shvouU94Acr3O4CEvcjuq.kK3wxzuBwAu
Dw8m7sQb3sgxo+rTvKR7ou+hROqEoAZunlxsAVmki09vM3vM3SKOfSKhu1fZ
kMmN0GNSwJ9xrga9q07OCaMoqDDSJ6guY4eVdXxxSfvoEl5rnraMs2cMQ1M2
iOiPEXRYERU9eG5xvjqZmcP2kWGol6p0D7zzHGnFO4R42fZu42qcQQnpmMpy
0+8lZvYYK0AsRUHWPULNRq01FMo4adzxFwH2Ek8u16uiTxgTZW4HBuRiieMS
44clhBfBYNVW.5NkuGbx5acqfmYqfqwG.X+gq7L3xl+1webEAlrf2elk9E+A
pb4N0l9k8fta8r2Vt8tKNs.sIZ4xvDTbvGU8DmHIToBX3bao264jXu9b.PFo
jhQ3+Rh8BHwdwl.SxkU13fbdIydeIydeIyduUY1qo7Zrb+1cSSdPQGzEOkIS
A13UiaUdhY08o.fmKAZFNKLElyjqD75dykolVMUY0X5ypCfkhqsewyiRp23l
Q.CtvlpWElRua8U5PtKtO+k5SABAt+Fveo9LnbVJ6hkeI3uTev1EiN8tSfB1
Nn9roev.1oOjo2z45WAH2ZSsOX9skMGyCiUMbjga83micf7sWtNIl9rtTcgS
6ZHhaeaKpgd0CUzyVuIMuPJ3slyjiCr9WToxKB3ZEigaGkxDR1qAFZl1MsEZ
MabRDlZu6wnTspt95IpaeTOqVcfL5kq2g5a7eCYbLboGD6TwKKPV2.Kaen8e
SLouXey4VcLxgm2tPnqLVzWPtAVze+bIG7h5kiPzFopxwJ0kGuBSn.jiMJM5
D+FzO4TVeacXApxDj1r1lMZfN6G4Vo84Fkf5zcE9uTW8fY9MwKle6Eyu8h42
tOL+V2mkBgW5YTJhFzpAkwDgwrABxcXBLbhHVimY3DDXvC1z6V6PhqaO7HON
VGvKSB.45BDfJajkd28FpTWN4eip9xOMlpzkADwLZXhYeYkVGBfDLLmeCXlR
g.H2WeuuXrRo.rs3bmdqTJ.axTwzaXPA3vjWL8wwpK7koo29stLvHC42flL8
Ww1KTvAnSek8B8nfrWn2ACCpBfROcS+Ra5vCgKYmW5pZuvUwoxWgcCC0pgh+
75u5t3Zs2prsAZD1cvffTYKfSr88vFXiLQs5CUGGuLr+Ft620msknc02t7GN
fPcc5rYyNX7gRGeqJevFiikck4L.QylbuRGgBL9VYk42srHKL7yH0OQuU9NS
BiVuYt7vibD8xr4ZmszMGHPSUynzcBcKTeTCpcEsiuhmG4.S.vuXcya3tfbX
7V8YbHHgfeJR36c84RZPBkwfHy2uxhuR0lL4rvAHLHXxU+K3zhKmh3Wx2GAd
s15An+RdP158JckxOTzbN9mpr9+Ag3LfrJFiKpIv0vMUJBj.F1TRevBxgeMn
PIyakcRiJukhfcQIx2ER9Ojubzajqnkm.21pROOVNTHdNtySfJigmogBqWfo
DDpYunQlGLwpKOwXcAMf4ROcGGl6cqXBuMVhKqkhljml8YTRB3MmMjfmcLMm
XBGcpNwqKSuvqmLKvNOLhCDRLyI4Xh2M3jbytMmC61j66TGtWiF6TyS9LyDJ
g8b.yzNGMrvoj6ExJzaCVTD8jV6eDF8V4PPBKMulI+Ozo+QtTvLDQ8i2NWMI
0UCNr7SxgPjjItRAWza0cR4r.oZFRJP48RUmFreq7Sxm1SAREMMOa8mqVfFn
OidOP1KFV0FtIatnlSe3NWawWKxBRxWcf9d7oxoCiADS04By8uUT4V7AwyWP
V7.c2aCF3Q5yoCWIYWKarue5XeN54O0Anae4De3.E4xm2y2WTjlLP7KNZl6w
ZW5koS5cWQUGa0deacBDdRkulibjwwFdQtr+Z1QKhax2falrkkPNzeMKb6b4
ISu4G9S+3a9Oiiji2O9S6yiV7GR+vOEFmN68AO0OpzSEoG.K.qb.bYyz07SS
a8k5PtUr.ZxFb7XCP.PgXl7Xry4N4MeultJqDLZ6tiJHoi5jTyBYRegEd8RM
SmvO72H1CxaDi4i3qT.6UNdyRPuPt+n89HtPdgsFUsKXclx4HurvgZJOIX7r
QZQP77GgFloS0.bDfPyXhZxW8ZeXTFfdXHCPul9W9AHVpYrGB7FoaR6Jzyyb
Yh.ZHfGssVtW42GLLl0ypLYLfXGeX7KcFOVzN.YQiGwSEnvdkjQ7UBh38.XL
JuRNrYYerMvWwAQa5t1GPoDXTwR0TMTvjRaE42v3AiKCulYNWeC7wa8xCz9.
h6n8BAsIva7lftfD+iQGOQw.smycDkvkBiZmNdKhLXrxnimL0zq8xXq5ZaeS
QOqbP3FyPvEll.Lid3SSD+CXiZA3QsTGEtKV4x.ulWZh3byYfTjY7DDfBSNf
w68AZAh40iDsNW2gvTc.sKHA9ni2xLLyNPrqsEwUaAEOp+oNjjS7z1Vx5MEL
SVqx0hUn+z3L6.wNlLhV3.FbNhbiAsKc7lftfTJhCP4bIwPGgTGmn6kAVuI2
yTHZMTc5OMNh7AZu2nAk9vr323o0iOHAv7su3U1I3p1wZbgPyOQXlLhxbZo9
Siy5CnEHuwSZNOPlbve7NDT3Clj.ONuPPPpX73evAMCcwi6KbPEGYimdULPB
NwFOHkAhOh+3wGgASVTFHYocv03Un+zDIZErQMEeFiZuR8V5YTOFxjBZPOdK
sTPb7bGO+kPAIg1HpgNEzgfL138BAIwDYDEoGjZLdincV.wSW3OhF1ADGuNe
ekNqsUo6P+pZWxNZWtN5nTcXoLcbRI5nLwy5qzb70ek7K7+iYwy.r
-----------end_max5_patcher-----------

Hi @amundsen,

Regarding fluid.normalize, it makes a lot of sense to use with spectral shape as the various descriptors that outputs will very likely have quite different ranges (0-1 vs 0-20,000)! So scaling will help ensure that all of the descriptors are similarly weighted during the data analysis.

Also with an MLP it’s usually good practice to try to feed it data that is scaled in some way–often this off loads the MLP from doing a bunch of scaling that it would probably do internally anyway. Also, also, since you’re using sigmoid activations, the output will never be outside the range of 0 to 1, so the MLP would never be able to predict values outside that range (ensuring it would not converge!). So yes, using fluid normalize in this case is a good idea.

Regarding removing the silences, I’m sorry i don’t have time to dig into the patch right now. It might be worth changing some of the bufampgate parameters and seeing it if removes more silence?

1 Like

Thank you @tedmoore. The explanations are clear.

I still don’t understand why I have so many silences when playing despite having raised the level thresholds.

I got it: I have this issue with silences because in the original patcher the FFT settings of the analyzer match the multiplier (“* 4096”) in the “cheap granulator”. Problem solved.

2 Likes