Nans from fluid.standardize~ & fluid.normalize~?

So I’m building a “remake everything in fluid.dataset~ style to see how slow it is” patch by following example11, and I’m getting some nans along the way.

I’ve some head-scratching and isolating, it looks like I’m getting nans something it seems like I shouldn’t.

Have a gander:


----------begin_max5_patcher----------
3207.3oc0bk1aaibF9yN+JHzmkEl6ihhVjsaaVflc6hlTrnHnvfVZjCWSQpR
RYG2Ea9s24fhhThRZn7HaECDkwTb360y60LC8u8lqFca9WTkih9CQeJ5pq9s
2b0U1KYtvU0+9UiVD+koowk1aaziwOnlmWr3qiF691rUKRxRUU1ulVewaWMe
tpHKdgxNIDkcyh4SmVdSYUbU4MySiqpTYpYsdH4qpV+TX0WcYb0zOmjc2MEp
oUNlDC3S.iivhIRAVhg.2OD33HMQLeEiLAD8epeBtGZ0SKUtoOZdZdb0nwGb
PZRo8+G07XRiuUkZYMP8UJVkpJZegrb6klllL891W+gJ8E5L0jYVcR9s+50v
QlK86u4MlOF6oEXZ9hEprp0ptJ0Wr5lQ3IQO94mhhKTQUeVUphxhyJ+yqus4
4YUkI+Oqd.Al.5y5A60Z.Z8DZrn+fJ8AkVzhGseaEBILFDpr1XAjteDDswBY
siHVKikg.yimpZyJsTVXRH0VnIQkUIooQ42+JphjTrUMHn6hmkjAqgvgTCAm
nwQIkQo442W95plDXj00tGudXsFbHpITHUS+wquNpLegpxvyQZ80c4lAOVjq
+7yJs230W+mFMbkTeA+jV0.ACmPoLFDU6Pw0xOPZUCfVpg1hLHjh7cZVMZYw
Jc.7nY4OlEFoivc.dJ07eP.4PBDRFRABLIpPEOKJIyD5Lxk85UDuysANwVSJ
7DhCbJ5lEpxx36T6nallphK50Bi5UngG.+hsYngNKL19IBcnz0i501SOE4KS
8ndx6HdKKR1.Hdd.XFwFJhifVwCsi30NyuLj1n8KDC1FQX.WRa9tIs6Qj7zh
ABnEqPSSUzPEYzQsaLsWGmvn35rKPjN8BC6gHOdehMLfh87zUIylLKtJtTU8
0HW4zKT5Z7plcit5yEC.DiOp8mYpvFiPXwZ6uoBafG.faiytqS4z8qZPgD+O
OoppPqILMlzUy3qdZv9ILfXB0zHhbBjBjz09ILSwsb9I5n.oAGwXD43Tc9qu
FF.Rsf2WQHRObVV2b0ggGP1kYHdLg4Bwi7HDO7BMDOjS2aeYmbHdH5RNDesc
iwkANDOjbYBT0sH4bSgfiiTEWn.UIj45NluaeemLPUdIiSWa1vLdGfJvjV44
ATAWl.UgtmVpoKG7wwo7KTbJWxLxPeKOwICS4vKYb5ZqFEDXXJmdlKYNLEAU
awgP7tUICgzPUkLG7R0.g9egQyTWbQu8O.kfPoYvuTsVEFsRclLLRmIiHv.d
sRwTxEjIBkVAxeU5pZepoAGHEI.SD.NZcEoZMz9ZwhwNw.qTVvQNkUwYyhKl
Etlr5UQzWKWPPv54hRBdFmF3SXx5fDtUVivmfPLBjsFPH08dBdNKTCK3qX53
n3Yyzh+cpHfFB25WgXr8Ba4a4QXmg2oWsVgKcqm7IuhqjyUz1+6JUwSAxooV
ZwPqKADFrjMPQv8Lzngk4gqsgZ6L1J4B5yvUfFROAPnfwtcBA51iKoozqS.E
GzV0iuuwFFkD4yAbXiB.O3FgIs12DDRdpqQC9ahhqqkVcGgHLm.VmCjyCZs0
jfhzKTKSMaHWXJCxVaCwlriHN0X1AzV61PzuNPXNbv42cKtpvY98ny4FKr0j
2mVnShc6MnqFJa6i7jkWLWuqpoLeUwz0zxs1Qii1vNyTkUIYwUI4YstGcM31
apWsuuDR2F7woTPHDzGBABAkLODzwnDM.ThoeFviPHR.nCk4gtiFDJ4AgPgf
PDpGFoPXiH93IgcFxmEgvPOjHXHzcHufCcuo7hYtiQH77SZHseRCddj1GrII
DPFjWXFQHnDxCLCBFBJ4S3WnLDTB3SLDTHnjGDhE.5XM0G0JEDJw8IFRHRRB
8waBFDYxKuoPfwsgweQhP.8A5gChL4i2ztghbEfFub4Cphx5a1RCcU3+ZtMZ
rXr8WSxb+psFZc2EOjr99sGN9QwE5hmqzUNupvcrB+BiLxM0bcb8rUI0YUzR
mljkUOktcQuMmHw2tZVR9GzEzup7leTksxULuVnlGuJspql316lmjlNMO0wd
W07M5tEpqKej6aGWe4l68SQfIHIABEZ0xDLDyQL6H8.JsUIa0yAtdRDJQBPl
akvPbB0NRfvTrYDXqog1PK.T5n.PJ.D2H8kL6DQ6oo6cH0c5N4MGuSMTnHeY
dwZSpdpxl6eUU9cEwyRLGkz0IPclXm51BjJzeqUsa0I05zEZEZx54zF30XLd
+p0mLz9M.ldxrx4MIYFTjpQbwRDzJtnF8EhKH3thqWVvFoaOFQOUrcMihFim
fCnBgazNL31lQFEHs2Ky7C2Mx8jNKVwZ0ToJU6comWGQmi.LG2.YbjC+I..H
oqNt8A9UGL3erTkE8g3rxnOnVjbad5rM8pFOcplA5PDrw.ZUpj0xtwEwbs1D
QyfK1dpbgVE4zw.nP3FoeJHT241YRDNEhs2JRhER2HJDvjaib5LMIf.4V9TO
GBm6nER5nOksNx2yzg3GiqpNjCQ6ims4.TrmiI8m1zIwyjg942VcicZfSxOE
Lt0GaCZZIHsW3C6NJzR4CcO.oVqSc9FZWJN9aaGcumVf8z2xY8sEIwoGz+zu
HRcUkRpI0mywfvAVkpQmhOlOlSa22H6zBBh9W93ea.dXhKALlewHe0KIneH1
O7zrh76TYezphN.RiB3LKSBXTLzhzfPDD4Ql.BCBAtRkjHFwJzTIT6Pc.TJz
TTlK7sAZZqgfut7r8iRgLrFTa4NNgIk0i.awlMEW2E.XpDzlIUKYTlynZ7+s
TWFHHtcItSldNiXKGneQOkIuIDJVxcA.wx53FlKQwgAd2w.HZJbb2QmjSwAw
9uSkodH9fndRcbw9Fs+hX7bRaAc8SQuGn6t.iPhU+tUUU4YGBwtaEAhFD.nd
TvYqu2jd7RiodWbR1W8jqzPzlfa6NJ7rV4iIZvygXtc7Af0.pMebV3IzEES8
2KSSls90tredp+lx700u2zjCOVSe5g9CtbdiO790a6Z+5p9jV19xjsqS6Yis
08yUj7koUECLJx4igTUphuNTU4Ilu4EMzyOkOSU5ek8xv6q3Q0EAUfOdHjWP
GzeZ0haGd7rcSZdFXseNYZ0w0UmDHHnrYgpTUcwUswGlluT4e4FaV0ZI17SO
we2JJwKpR9iw2NTPfmsLL5UTnzMyc6QKftmlO.9lh7LVIzGyuyzo0.MJXttI
bnKgBACsc+Q5rCIuhUA+uVbjMV5ET89Kc+SXUnJ0DLg9ZpnmEWb+0YI284pq
scq9b2Eu9W1N3QPaAe0IB9R10c0I5IBfuKUa+qI.oIfHuYW3Z1ONg8wz7DZV
wIuWB7ffT9UKmqJ..3nf8nPCoNS5d2YiCtxtf.tBa9QluIvvauAF2lpz48ba
332kme+ni.Y2reF6LHbPtjr6OX3+cvAaVKZFv3BzdD74CDZDdsY.HZR5isj3
va8v4Kx2KxdXebIt2M853yp2Ub33San2+1qfqeaa3dw9i67Y6YrS34CwbAwM
4dWOY+h4n87ZWrTpoThiWBcuF8lDdzlPpq8ZNzZi.jj5SefThE75QbQPaPXQ
7WXlWBhftSP83vH51dDu1yeux+VFRa0jASfsuTF92vJBSQxtoDMNYvPaG571
zDJaQ.JtUzrw96N5.w44ByoRabuidAqN34bN055c3mUz8Z4LrEcq0Y5K3KAf
6kX56RWotFNvt+wPd8NYuIFlrcjqPxfE4OlMXN7nwVCHG9WdJd3LH.Iotia.
miwHmuCBpKmK7L36JTpSfCsZulkhCxLkPDdl6G0Q1xphGL6IH.FysHgLfl4r
pRlDJogmG+mpYCl+3MU8BaZcBTmwHz72+Vkll+3gYwcO2bTI.HNXtYD0cZZ3
BnvVJudNTB8rI.nuEE.61W7t7zS.hzrvITc8OT2QCJ3GaskqJVldZEOvaNMc
DHU6+aCRYBCPOREpDfqvHDEvjVm.jfxPhPJWEpopjGN7N20Glno8mMxDn84R
LLLmqEECrv6L9LhNflrNQ.u9jBqQqR7A01uLnnRU1rxKsLHaTxg22q2imJfC
Ab2VTQX.Y8Y4lCCJ1oJd4whC1ZcrdebUdz6McgNJXzurRMeUZZ0Q2fktZGBT
WWnspvlZ0MKEv1mkxsOVjBJE5VAXFlicuaFLsKIKrJUmPUcRaZjuspGB8ARv
EtBavMtNXjDKCZzyJU4k06KvilUy4oSJMkmmz7thDt8eo36eI.AF6ms8TbsY
SG2ndaONv6wx2mTT8Tze8t7ishxPoD3JbUi1I06FOC.vvS9L85d04r6fmgiJ
WVamr+As3M+9a9+Pw8mD.
-----------end_max5_patcher-----------

It needs this buffer to be loaded in:
flat_mfccs.wav.zip (916 Bytes)

I thought it might have been that there’s only one entry, but even when I make more than one entry I still get nans (with this patch).

Is there something I’m overlooking?

There’s only a single row in your dataset, so neither normalizing nor standardizing makes any sense.

Possibly this isn’t the cause of the NaNs you were chasing? But if it were, I’d suggest making sure the result of of any query has size > 1 before you try normalizing / standardizing.

Interesting question as to what should happen in these circumstances, cause I can imagine different things being useful at different times.

  • Return 0s?
  • Return input point?
  • Throw an error?

The same applies if you have multiple rows, but they’re all the same (because the range / variance is 0).

Yeah, that’ll be it.

I also got my rows and columns confused, expecting it to standardize the data there in terms of the single row.

Without knowing the underlying maths or anything, I would lean towards returning the input point in a “x + 0 = x”-kind of way. As in, “the data is already noramlized” since there is nothing to normalize it to.

a warning in verbose mode, plus whatever output (all 0s maybe is the best as it should be dismissed)

maybe an error…

this is an interesting question.

I was getting nans in SC if I tried to standardize a DataSet with 1 or fewer points. I had to add an if statement to make sure I wasn’t doing that.

1 Like

Same thing, I guess. It is a nonsensical thing to do, so maybe an error would be the solution. After all, nothing useful would ever come from such a fitting, even if the NaNs are replaced with something cuddly like 0s.

1 Like