Yeah something like that would be great.
Don’t know if that starts getting to faffy in abstraction-land, but ranges are going to be quite critical for most use cases (ala example11). So some kind of notation or syntax for specifying or denoting a named range, as opposed to having having add column loudness1, add column loudness2, add column loudness3, add column loudness4 etc… when cleaving bits of fluid.dataset~s together.
Well, now you have one. You could even just wrap it around fluid.datasetquery and it would be like a whole new object. It’s pretty unlikely we’ll add this to the compiled version especially quickly (or, even, ever).
What am I missing vis-a-vis the extant addRange message? The abstraction above would allow you to pass a labelled version like addrange loudness 4 to add a range of four columns starting from indexof('loudness'). Is this not what you’re after?
Yeah that would work, though it would still require knowing how many loudness columns you have, which may itself change. But I can’t picture a solution that doesn’t require a whole load of stuff.
What about making a dictionary with the logical taxonomy à la Rod and a translation in columns numbers?
You’d need to know that at some point in any scenario, no? And if it changes, you just send a names message again.
Is what you’re after something where you can specify a range as part of making your names, e.g.
names loudness[4] pitch[2] timbre[12]
as a shorthand for saying your data is laid out as four columns of loudness, two of pitch and twelve of timbre?
And then you’d like to have an addrange that would accept a range label like
addrange pitch-> addrange 4 2
? But also to address columns within a named range using a dot-notation like
addcolumn pitch.0
?
That complicates the abstraction somewhat, and adds some overhead (which would be true whatever language it were written in), but still within the bounds of very-possible-in-Max
----------begin_max5_patcher----------
7081.3oc6cszbiijb9b2+JPHOGlIVMvUl060dWaeYivG1wQ3qc2QGThPpwLT
fx7Q+X1X+u6LKPRQ.T.Djh.TSzhSLpkPABT0WkU9pxLq+waeyU2L+qYKuJ4O
m7tj27l+waeyaBWhuva172u4pGl70amMYY31t514O7PVwpqttrsUYecU35yW
jrXRwj6yRxlkw2wxjuju5SIoIyx+srjGmO6a2r9t6xVj7iSlsbdB7y2LYY1z
jamutXUdw8+Tx1G4r7hrvU4mKt4hOtHaI8PmrJedwGidG4SK6G27q+L3UaeX
EqeHuXV1pPmGd5hyWuZ6UEaeESVc6mndxGWjc6pRHwBXpVo7RM5LBkVXUWm.
dWpA8RmUIcn2hV75DT5SEWmHkohjOvOu+4aeK+iq6Ir9P1xkD30.VuKe1JBy
lMe8zB5VRwj+5eIQb0ogJxnnBFEU1hUkWZ02dLqDRt5pjOzNfo8hl.FBtTqz
Z0VCJAu1nbDJhBFvP7LCXSlNknCIxvGyodWpMQdhfEdY.KvQDRJiUHrfQ.Ni
0RWT3dVfUaKZYvhvwju7o4yxRJwsebdwruUt1cKV9zZyJHDbdVjQivTI.Vqm
9Og0BVOMh01TMZjVsVhfVfF+tEYLky4FFJG7KSVuj5bIK++VOYQVxMKlb6uw
Cft3LsOj37mI9NFHUonIePCBzZrdlx.kozZXgBoETFmFzL0BNHrc1sJBfmV.
UYjZG7kGz7dpkF+7buGjFqj.ACsRfHC.uPpDDT..iLZHUqERmGrBixXL1Afy
RwDpkcbhem5CI2st32lub9puUxr4cf7CmFyFmd3Y1nboNuS3TDqEPPLiagjB
A4HvXNJMkZ3QAqOELnCAEwd0CfDYxGSSdMtmGG2hruPioFfvpjYI2NKaxhxe
FEENF1pxVQgqI1ckuhs+R23hLEUDyWC8SGpMZIwUwnIpCmPhBiAQiPXtNgXV
e9vkIEEyKWiDZ7ukOaVxz7aWUJ9Y9c2sjFmISJllLKq39UeZYxck5Yxrp+WI
cGms9ghjYStIa1x2W7eW7mKa6c+xGRnG7+afbaB8aeJaQ3Od2e+Czcurrk2I
+v6K9eVu5Om79hRJSQxuP+9tu4uj.uuX+udxO9KI+oD3mR96zss64r4pz++2
+oDYRi47GStiGWk85Xy31yjTCsESAozHkdRwUfje.WmnMVR4KTJEFzZkdERD
218z8Z+mW1hMScal6dCq+Y1myVrbyTzl9DMy83i6c42r2Wgmv+04gGj65cWJ
un7RvtKsH6y4a+95cWcxBZTshFRqWDHku5qlsbE3Gy7oYKJVmGdRkWjH81zk
BDYAlzON41xuLSKts4mvIPDTi.zp.JXB76zV3Izfnhue1bR1+z8fbZU1iYE4
E6yZuRySytax5Yq93cyIqex+8PW.X4yQZ+tMcwnMxChP+++ZQ9jY6FA2uHe5
7BtSTYpfu71WGM3zkis8GLg6nXxiQ9xDAHgKsz3RZPtd4MSVvyT2LKaOtwD8
874yp1ztu2rr6Vso4GyKJpghql+X6MtH+9O0w28l4TiOz0yNzxxOttnr0OxK
S+3xIetJZuZxrYaXEU8w+0IE4OLYU1p7xo.TrqwrhIz.8SKucw7YypLdKa4y
QZYJQjea1Wxmt5SgWz9DCzsm+3Nde6lkmlSrIVU8Zqlb+xpWY4puUB56co02
rYQ7GWk8viynQQ0afVdjub0xOM+KK2biaIz1G.dxc.6unde95UtdW72iwi+K
yWOaZR9pjaxRJHrII+tjujwtAft7utdIw6eAw.H42mQbpolk+G6+v1SPZdR9
9sTggpZ+VhwTMBiUbuVZHNMmsZn7e1srpEtvnOkUuPp.u2wZu5XIoDmYqSRV
.6kNmBEr0L9T89RReyVooaXqc1lA1AZqK9044EsgZZyIfZxtPMFxB++gvLvm
JqiYj8ko.ReHU0zzGGPWTAU09X3wLhNjjT0JloaEyvSiR6zwKRAesGLFMoju
QQJuQJ3KsW.7Z174+15GaExTWFHSiojLdxhQznMBxvQRyekPlZsJe.EAB6HH
y5BXKRqVcZxpI0KM7SdQvuXr0hAeFyXyUi4okHaCtT9VgKYqvEzIbcJqNUJS
pTv7wTVKcEmH3IYLMPpoHkykD0l1OtP2LRef6VS5IMqU36TDJzI7wuySiAmv
jJL.PjZJvJotVv7lQhAWM0XtKeAopRoEpI4jMpy9xjusjrfLJNSlLP5VOcmi
6a.yRw4FlOHDydcsodJPJp3OTCnPQRdoUzxKCDyZtmLuHig2fhgjc3ILJN+t
vuxDRWmrbNwu7QVgRZxnMNpY2lOaYBzF5i9KBK0ny.7dk.jNhNk0fdTBLQtd
PlAp5otpfFSV2J0J5NA75YxSECZPqsnwpzDtnzsoxidrU4grkg9ORtcq30ov
DU0Edcyjh6YZrs+6ISuEkoJoiQPWIRWMjtciGUiOh1p0cn3bac2VTrOl2AFR
kZxPNBxTdoUwa9aTSUjithPDit6WLucMGgWNr4PqIEbRoDcBxhXh3yRr4fKi
flI2d65GVy9KYmalWMOYZ1h7Omkrb0jEqRdjzwbUo2mylb6m1rUoQmEBOs14
cBtys9nmJUKJkoFMZsrUQJi06jzbfZr4dluJaQafkvN5pEEc8cL4Ln9BXZ8z
L1oXsBWvKFuQPJMpY6cHUHUNxNGl1xN5NiXqd2j1LsfYF43SgEy7PmK0IY4v
TCdEHPR1gEGa7Jr2a+KhOVYuppCYmjOu5TaF90RLimr3a6zk4To6zRHUZ3Oz
Ubze438x0Ve6tJcX9jOmM8ik6.xGmrZ0h7aVupz82uoAF9lqxd3lp61vl90h
IOjQ7v9X4t.zwM7vjGeb2sr6N1MidNlaCSEs5V71E8HFVZcf3XJIaYbFlaJl
54ONmv5EVf8DhLrgjRQsYnsAry9aiQXTULM6q6sIfUvtm12KNtKZY2LB8Ut8
3f5x4qWb61Q4N+HkTs2MMa4p7hc5Q7tm3njz2UrGc2fcReBzitQXSPFt9Q3o
2q9gV0.3dHeZoBUk2mJ1Nl3koj5gdMoXBJ.m0PBP.CsDttObibmC5fVzqA8f
RDv1SlH6M3CCU+f8Z0QfGvftnnW8isqhGn9g7n5GPWKJjwzRPpRMJMoNJGBS
D+SOaAOnSUnqL3Z46SD+NGrAs5nFzXWC5nKviNpQAVpltCMH3Iatsi4n1z2Y
ZVYogifisOpeXOJFTFRh9JMfM+c3XDf8lgTneLX3w1mdO5G9CIcDrFxTpZlt
a7oJgPZQN57nUI75DmooXzH23fg8tiE66XLSZwap6sWjTf2Pi.RodMsnWQse
cbzo4MNXSz9du.bPk7f8teHGTFApdKIdXYLt8o2i9gYPYPC8V3nYv0X7nXHM
f3QuDTfCOdf8oerUD6.1O5GdLnBrr8l48VJ5nFd+18iH2C4ek+woDIj2Oa9M
SlUK7EiEnju8oN24Kz8mEM7ssGQ3aicEv9GHD8iEc2nMEQsxJ7jh3RVZ40I7
tQEzhlj9a.PVMpRFv.1uLP7Cwq+lLKKqbOSxKll+470SlsIj8Wl79h7cQqup
sn0mCP+2WPP0+11PzOkFyk+Bt8Wja+E006dLWW44veoJ+Mlbck+V1VL6uo2F
cV2blx9MkpRlWnk91BZe2qAs+tf12G7MtwHeMn8eMn8eMn8O2AsekHwns3ZQ
OTwgwA2VAcX6BLk7..18uO8wBi8dLlWz1VtbJ4gf3.6nhLLpUpxD+cz2hvG2
Jk+tEyenTxaKC9wemUM9ljBRM6SPRRp2QJKQJqgb.NAPzcDrl7z56.XKxUaS
1ZsudLYrsHmscYsGVdaaxbqH2sGxdqI+U4KizVWo7WnjJDrUQv3xf6gb39HK
9fxi6oL4NjK2OYyGP97AkQeP4zGPV8gkWePY18PtcejceLxu6PF9Aki2sr7t
km2sL8Nkq2lr83x2aQFeujy2SY8wk2WmQSc17MZ+fABbUd9oI+.Tuw8Xza00
arsfBoaN98fqeqwXnK0F7FrM03jRh+t25b.esXoYW03tnhnxmKV1PzYyjUoK
nzbwgRsMEC.FGCvVqxnbRs2Hjb997RBJajYdM.S6y.LwCBlWej.JuULj1GDU
owKAA5v34m2kCQuc9C2jWjUpgWRZRYX1zA.65DfkiK.KIYzbneAVgVKPktLX
ifWRH75eOuK7z+LHXkGBOqmy.0ia3VqCOl.7Fi2pR7hBbKN.2UmnS7Edd3az
3VrCPk0pTBoDV5rZOQnZrpPs9R8GJtrN3EAW1R.0oScdkhymRiZSzeNx7XqG
BlM7bbmvkXDjvyPE+AMke3ZfD6HA9iVpDVoyyEnf3AkYWAlYKAm4.h1kHPGv
M9LVxK5orm5XImEkFPnwRvzojCKXV0vzFQ8ZSLN19CFAoarOgtFamcGaUnqV
jFbTTAmbOTbD8Pq+BzCs1ioG5tD8PywzCsWpYYnu8P8k.C0GCFZpFTbiSOrQ
zXdnd3XigGEyF3RLI6NhdHdvN3dMtuiZO17T4jblU2wUQrXq3rm6JckN5+.1
xlL38iepnyQY7derJNlCE.Gu1FsDUj08FTaFrJ7Tm.UaE4.uc7AJRabnNPEc
+XF+RgU1WebQnlM9i+PNj7W9KIvO0FvoF+LDLJElF4372Q.FXAxXwfQ4hQOm
JumyS5VWNJtHYfZzM9SvofpQPlx3TdvhH6ASb7KNDsWXHb1y8Nn2KzxiMQKP
K3RQjRpjNPCZKmOEi+xxCTBCblKB4UY3FnD5TqlrpiVBJzfCA+HVyB1Gh3Tl
u0kftKSIoKFQk2yAhHBZMP+sVBbN57RLIvGeV7kjTZsOkAHGo1.WTysbdbL9
o8cm03Pi9BtlSSx27Fk2XTZC.dDtHE0vCVmzLWlxj1FFSZSpSasJq14jfyBl
Qrvn0n3ATVm7+OCoKeq0bKi+ERQDXSnjAhTv5UBEvpJHBEnUP4iFkPSmrZRa
Fes6fZAZrs8OcFtzLVN11jrQTfrqIU8Mg3pcG+.0vFZbFN6Sf5QNzlqGd+ll
WO7xsMutpdHKs45Z959lW2DFehlMXCM.MavUFoOMavGZPFYrIBsDaTGF1Pjw
MTB7QF4PXnC1yh826pSDveTqSD5QuNQTtJDbkUFBlauGTN9bB5Eekgv02TMy
MjYRT3g2mDIx6G5dQevBuX.SC3dOgXNTdnpKM.x4RUNT4XEYEfUH4LuoslFr
TzpuCqgLwJ08jJyMjcBWeSlwlUlf4KlVFduhAom46a9mOjkGD4Yi72xEkp5t
UwKRMdimqoWbr13saWKzrkgp3Kz2UBlgrD3X5acuvLjbb81y1rM3i30QIYCe
aS29wY5t2K2Ml1WuCCC5qNF4cci9JUSmkWS7hwaKKP8MVUF4FGJBt9VoA7C4
hOeeK3HQVeTG3EPDtbRhF24sBxfPNV21TlWis.o4cNTLb585.82WoncsjRtd
hEuHVhEmDxq3M0wU0S2FT4FShkYvjgZKy5337BzGQlAepGicQNt1TNSpkOyH
AjTCUp3SqMm+0bFdaNCCkExdoP7ZNC+ZNC+ZNCeQxYX7hkyvgMMf3DvmzmDq
fxO3XtKm0jSULeuyY3e61rsmX5k0f7je9uxRkRlTJo45DRvbBOwdew16.Zo.
j+XX6tZyy6Nbz2rKucS1YYR4nhkjVU9QTZLg3E0dLqtPUI+xzp0jRTmZ2VKA
XOfN9awbGkjcb7od.Ku9zSB53ktaCzZVimQqDrWak6CbmrLKodGMRugzO5CQ
AxEY2m80GS9wz+zO8d5y69w2I9Y+GJ+iOzl62OA.V2qSgudWWsC.NH0oZqyq
bd+FuxyoS0vrshcteDfbz2OhvpQy1cd3bs+CmJ5zHIGpBOCPUlnjaj1lxQBj
2tgBv4+Cv9wf804vCoa4vd5dBDGzxPZecE7P1Kv9Ng.CITzy4C0AKGqZ4FcY
fRtiJm.QmBCkh4vQ9SjlFJOidrz5c4ErMYorvz13JRSC3rUeJu4tgjts2E7+
.C3uKKIhOt4nJsRY.5p8caxGY.+14qKkLfM8V1tip4gs3Iprb3IhFEuQEf2i
b3sJQUpwJHcIMVhd1BVhV28ccU1SF7TlszsYdYoIytW8W1q9K6U+kMj9Kqsn
j7re9mEbW1AOqmkodNolsBGHjDGSVuef3F.7QvIH.NiJrkbIu.Qp78GeP3Nn
mL1QPKkUjJUFsvJEZqRpT7w8rFB5N9RI8SL5KR5mnLMwKVqZj.KoVJEDwkJ3
qA2v.WmThFZjid9yECnb9TuPCFOothhziQpGLGX2oWGLv42oCwVHwI.fVJ7J
RYMEnsFluyKAexzEQDW7qCae7OfI+PamL8ZynSPY7oFvXr74jgw6zg3WvYSk
NAoCbnp9DJV3d4nedSee1pjVycUv+hXsWLQfpWlmQin3ER5UDEG0hTvaPDDJ
gjXhwYl42OmQimnzZv9hQZMfzjkyaDZkv6IdFxKRNb2YlHCubTFDs5TGZkJO
XjZqHjeViOii+TqaILHN27W60YRsPlBVqSRhfzBTfFFqjtTiBIhJD0f1Dhho
wmzpWLYA3kRNrYSUMH5D.cQmxZQGGohBu96IlrcoqwsyKVMIuXYGJbfitBGw
lCAiJEsbRIJUNIZI0ytNwJFDFGcpsu97qreccQCVMipTMYzrGC0qAqS8LNZf
wQZ6oUhQe2oiISltjzKAOnTjp7Ru4O.aUa+in4gbuAM1iL+BFtjZA5au.t3o
VyPlASJwyOR2OC6BYuoNGxD6r2mzrvP1K58Ir7PNgz6SY1g8Lcru6K7vdj61
6S92gjcAz6iuP+PiE86PTbPOaKOlYjtCFC.SMMrLJblvoAI.dkjLjLDMVxHZ
927FGr.Lp2mhlC5gnYuCTiuSiSinVyVIJLfnQgAFMJLTsEEFGv50nVAHgHIx
DWnsDJmm8NflLe0Gyx09NG0vZ0NsTsSqTGh4jPTYDaJQbolRDXjoDxjCzKo4
CDHNKZG6wu+3Nmru8bU1UKWTPWbFScOf2AO.zBqWnznWWdhkKS0.BjUcZ18f
QM6MlgaOYzl3DfgsOv5zlqKn9676KxWlMc6w81izWJY0mVLe88eZ6W3PA2kw
ddNiTUVHUw6VfVJjRkxxosCPTjb1jnIfy30RtzoAk0QFo7DOraaCPZBG79Us
b8MjXlUDA9zHzRZ77L3QTkZ4AJmRmRqHDkZRUp2ozdEsJUh5P3rANCcQiEQi
xAJo03X2.dhPQUGn8DTTVcu1VUpRB0OojU4ObyhrHffxcDbwNwETxxT5BcnS
Xjdi95DsK0akFGxos.H7BtBWZpkGMGIfTwAYUFipyzDskOwk3THW6AgQoXFu
FgKMTZ.EzrufVD.iDug1OEriAAxAOkmihNQOmjA+yZdtMB+6xmwkcSYx+dhH
UGCDvAmVmqze.poQqinsEBT5hSrWuFRddi22XCdXvo.jHG9T.eVava5nwJUa
TLA.55JufzJQxQLzlyH5Wig2DmODNyds90X380X380X3c3hg24K+1C2Lu0f2
PO9oMq.R8jdwdxlMtFLSpGF17PqvJ3hGOfBcHrT01QOZuZMJK6HWFGrfhiDe
pU.Y8EgRnwxw0r0w1V3cRmEU.6CsKxg4PmAEm6RDSbrCAzVsSHLg.JfnqRYe
DXrdR+KVOL+kLl312YJMLF8kSDw0.EUDJJQAQtIDzk0bjE98SrZL1gd.JaLC
vUwKi04LNiFjjl0VyKivL9jhmK0EIbtTjsXnDQkA4JPkUAkkdIEI5w5TH6uB
vXewEofxKUfBVmHjcfihnBYO3PplAB15tWVmXIvkJFTqiUJYpQ3Ddofs9W.b
Q6ypR8.QlIPkkH7Po4kmTH4Kk5dOHvTIYdHiXDImPFRzaZooDb7Vtn0RsQyW
70fFreAMnb7CZvn5364pnNhfz5z7g2txLXAM3Ku3hqNeB1+fbXwosRhspS9G
gJJuruansz+RHHrFxXyS56YDFLjEaV0wUTu6HtN3SK3FpH4.RTF611fY4FOF
JMAo5FZGz39FvxuQeGuvEtHlqG3BoduJ18CYstuuwf3PFBh8M7pFxEgf53nH
5rp1qp4vGR.QpRo7jd2FfVg4z5nqAibeW3BVDLnQ.aeiFXI9cZjUsfTRIKos
X3YWdq1mcD7zKtMQqFzQKtMV8Prsf+9rjkyxuM19+CxQAAhu4vHKsUqrBOYM
nzPMxU0PWpiycQBujFcY.Sd9wiYYEwPCbTPCRgWxzNKMTMFEo0qR0VDbU+jz
6rEi.QiR..FkZ8TThAZtNE7NgDokDHRJRYNyDCsE2.SlN814yV+PQYrxDCWF
gfjwQq7Qzp8FvP5c5Y+zGKvA7lgL3I1E1PsECE9AGIPx7Xi2hJo24Xe8prsD
uPfbPfhIESS3Stu7.Pvm7zwHIDCNPnI8a7ZZsfDbdmTXcsgChgfIwuRZj8zw
qPj8oo5HWNzgRTrSOA.zoREIOkLWyDRrN77CDqKBPAFKDBOB9kxN4WdR7LiE
G1iBjDzpJDvoa3avqYlS+yNVo6pvJso5UbIs1nPm8fPWaUP4dCkjFXofSII6
MTfgjAw9lVB1Tf9AI7wgn.AuJBVFdWUcuTIzVWa9M.bSs3cULlnEU3s6khFM
lE68qxWwt8CWsYeFuIbrdSN0H8lT8A7fpmzi0NQ6dF3Yud2vYAP0iDfxEWzC
R3WC0qWte0Bdq4qV3PIldjrBulifeMoTsQGNJHI8EZFfpMtyywX5fSTPWiIk
CZNljZM6ZCifKKOBqxXjsLlhbmOywDOWqND+I0YgzC5E5oOauK7PCqyxJJqs
OCKq+r8tfC9tLmi2En603xdFdWLMn9PuIemrJjPpl8nHYMImUFd1.aPk5QKq
Ck1hRxxb9nY2QLEZ3ull24ybDYbikbqQ6EMVx78iktEhQSIlQ5Eg6W51OMoS
jcojgEBNedHaLzAWwqjonP6bBq.zJHXXZbgSMuyyvHBeViHR69TR7uhCnGNs
rCha4DekqeZDWYxp.D8Xainl24YXDAOuQDMGY4f9DUDR6cfLvAiOCR4p2JY8
rlCiwVGRMuyyvPpuJE8rdQ5dPMXJumm86Qzi2yyWDaeHFNGbFL8YJRdNFRXe
eSv4.7NzaReln5NncLmCxaEzS5tmMz0GacUxwxpZUUqppdrQWwQL0xLL9sVK
ivpkMXMyDr1yBr5Y.VXWSKSGlZN+Y2tb9vjuZ1kGXaSzppPxjauMqX0symU1
qdWBYR607OjdjKEkbNg3cBtBkuWFywdiK7c9XdAOhy19cEWu2O1+aby868Nf
Mst4GhzcF9d8louETmpdl9b0CTuOeq6L1e5tx.du3NK9PtxfUgRMmXnhToE0
pvuQZZqgMc+yV2phK+i2wNET8t7Yy1Mf1eSx25pxqtewjo4Ok35aF+vtY6vo
U80Q+MX+boK70vseMqyEpu3w9slesseKjT4xGnpjanp3eitDT6aMo39MYcmc
uTG6wEyeb9hcoeXpzu69WuZ9tAZi5+P7oupqN52r39oiPOntByYJhrRfgei2
jQ+ISYUxjokb87s+y29+CBnn9zB
-----------end_max5_patcher-----------
That looks pretty clean!
I haven’t really sussed out what all the use cases or edges cases are, but mainly thinking of a way to shore up against everything breaking if you add one extra stat, or drop the 0th coefficient etc…
Granted, that may still be the case since, if I understand the abstraction, all you’re doing is manually tagging columns with symbols. So if pitch[4] becomes pitch[5] at some point, everything after it will get broken as well(?).
This would still be better than nothing, as if you have a fixed set of descriptors/stats, this would be an infinitely better way of navigating things than indices/offsets. But in thinking about the “testing code” stage where I just want to try the same thing but with an extra derivative, or without this stat, or an extra dimension of PCA, where that doesn’t become “a whole thing” chasing around numbers spread all over the patch.
If things change in one part of a system but aren’t reflected elsewhere then shit will break irrespective of how cuddly the interface is. Perhaps I misunderstood that you just wanted something similar in essence to entrymatcher where you tell it a bunch of column names to use as aliases for numbers. Are you rather imagining something where this sort of labelling would happen automatically? I really have no vision of how that could work at a framework level without seriously curtailing the scope for experimentation.
If it’s something you actively want, then the same process applies: start wrapping stuff in abstractions that account for stuff that ends up being boiler plate for your purposes.
I mean, with the above you can spell out your whole data layout in a single message. This surely makes it terse enough to keep different options on hand whilst trying stuff out (or building the message dynamically).
Having to specify names/labels, and amount of columns associated with it would have to happen with user input for sure. I guess I was picturing (in my bump) something closer to an entrymatcher interface where it wouldn’t matter if there was a new column inserted in certain descriptor/set (e.g. pitch[4] becoming pitch[5]) with regards to everything else in the dataset. With labels being proxies for indices, things wouldn’t automagically update on their own, as opposed to labels being labels, where it didn’t matter where in the dataset they were.
Or I guess something like a dict / coll where the order of the data inside doesn’t matter, only what you decide to call vs, um… a buffer~ where index 3 = index 3 regardless of whatever you want to call it.
Totally.
Part of this is a deliberate exploration of “all possible descriptor spaces/stats/dimensions” (not really) to find out what performs the best in my particular use case. I’ve yet to find a good workflow for doing any kind of variation on this that doesn’t involve touching just about every part of the patch every time something changes.
In a more practical context (post-experimentation), this abstraction would do 99% of what I’d want and would use.
I think we’re maybe at cross purposes about rows / cols?
entrymatcher seems not to support varying the size of entries (and their labels). The number of columns is set when the object is created, and sending names with the wrong size list gets you an error. I don’t see how what you describe can be done, unless you mean adding a new entry…
Again, the things that don’t matter for dict / coll are the labels for entire entries. Neither has a concept of labels for individual indices within an entry.
fluid.dataset~ is exactly the same in this regard. You can insert / delete whatever you want, the relationship between individual rows ( = points = entries) is maintained. buffers, meanwhile, are abstractions that represent a single row/point/entry; just like a list (which also has no label indexing).
What we’ve been dealing with in the last few posts are labels for the individual indices of a point in fluid.datasetquery. I’m not completely clear on how the scheme suggested above does less than the equivalent labelling in entrymatcher.
Maybe what’s confusing is that entrymatcher encapsulates stuff that is spread across different objects for us: it fulfils aspects of fluid.dataset~ (entries of uniform width, recallable by label), fluid.datasetquery~ (filtering entries by criteria against individual columns) and fluid.kdtree~ (recalling sets of entries based on distance).
Yeah I got my rows and columns confused.
I gotcha.
I’m just picturing a workflow like in example11, which is a bit more real-world in terms of chaining things together.
You analyze “everything”, and can then label things accordingly up front (via the abstraction above). So you’d have pitch[28], loudness[28], and timbre[168].
In the next stage you filter it down to just the mean and std which…um, I guess indices tracking is still unavoidable (you’d have to know what is what inside the smaller subset of timbre[168]). So here you would addrange timbre[0] 24, addrange timbre[84] 24.
The second subpatch is much improved with the abstraction as you can specify filter duration < $2, and duration > $1 and then filter pitch[2], addrange pitch[0] 4.
Ok, that was me sort of thinking things through verbose-y.
So for querying/filtering off individual columns, and/or moving grouped sets of columns (e.g. “timbre”), this approach works really well. At a more microscopic level there is still the land of indices, stats, derivatives, where you have to know what’s what (e.g. which index is the median of the 1st derivative). At this point these are buffer~s, so the dataset wrapper abstraction wouldn’t really work there. I guess a similar abstraction could handle that too, where each column in a buffer~ is named and accounted for (e.g. “pitch confidence mean”, etc…).
I was thinking about this some today. In the latest geek session @tremblap mentioned that he would have a binary fork at the top of his analysis, and if the pitch confidence is below a certain level, the search would get directed to a KD-tree that omits the pitch part of the descriptor space.
This makes sense, and as a vanilla core for stuff like this, covers some useful bases.
What I was then thinking about is what the mechanism would be for this.
Would you do the normal analysis, stats, and flatten stuff, to end up with your normal “point” to compare against, and then literally peek~ out the pitch confidence, use a vanilla [> 0.8] (or whatever) and then fork accordingly? Or rather, is the idea to do this kind of forking outside of the fluid.objects~?
I guess @tremblap is presently working in SC, so perhaps this distinction doesn’t matter so much, but when trying to keep everything in buffer~s, as is the Max paradigm, having to jump out of that feels like a weird data detour.
I’m also secretly super excited/amped about @weefuzzy’s hinting about signal-rate triggers for fluid.buf-based processes, whose elegance would be defeated if you still have to exit into the control domain (and back) for part of the data crunching process.
///////////////////////////////////////////////////////////////////////////////////////////////////////////////
So all of that is to say, it would be great if fluid.datasetquery~ (or any future improvements/addition to this process) would allow for some logic to be computed, and a result passed. So something like sending a filter confidence > 0.8, return click message to fluid.datasetquery~ would parse the information, as it presently does for the first half, but rather than being followed by addcolumn and transform messages, it would instead report if a query is true (like [==~] or whatever).
This would allow for the kind of forking that @tremblap has in mind (and is presently doing(?)), but lets the processing stack all stay within a fluid. chain.
In the CCE. I would do (I am doing) analysis much bigger than I need, then part of the research is to see how I can precook it according to a stream. So fluid.pitch would be used to fork which bit of the patch the query is going to.
I meant more in terms of threading/datatypes. Like, in Max. At the moment, if I understand your approach correctly, you would need to base a lot of decisions on a single sample in a buffer~. So to retrieve that value and assess it (> or whatever), you’d have to peek~ it into the list domain, then once that is done, carry on with fluid.bufcompose~ -> fluid.dataset~ etc…
This is specifically in the context of having pre-baked KD-trees, one with pitch and one without. So above that somewhere you need to make that decision and then carry on.
In my head this would be a matter for fluid.datasetquery~, but then realizing that you have pre-baked trees means that it would happen before or separate from that step.
If there is ever a fluid.datasetfilter~ or something like that, then it’s a moot point because forking and filtering wouldn’t matter so much. It would just be baked into the query.
@rodrigo.constanzo I’m a bit confused.
Is your input in this process a point in a buffer~, or a whole dataset? Talking about peek~ suggests a buffer, but about datasetquery~ suggests a dataset~.
Then again, I can’t quite imagine what the meaning of result would be for a whole dataset (any value > thresh? all?), so maybe you’re asking for a transformpoint for datasetquery~, which would return true (aka 1) if the predicates match for that point?
I’d temper your excitement if I were you: I remain to be convinced about the soundness of the idea in general, based on the experience of implementing for SC (where the need was much more acute), and the bottomless can of worms it opens on exactly this basis (of course no-one wants to leave the signal domain once they’re there, so suddenly we need a whole suite of signal-triggered everything to query and manipulate buffers; whoops, we reinvented framelib…)
A more philosophical issue I have with implementing this for Max is that I think it’s going to confuse expectations about processing speeds and synchrony. In SC the dataset related objects have a control rate hook for their equivalents of transformpoint (i.e querying the model against a buffer), and that happens in the audio thread (buyer beware etc.). By contrast the buf* processes can be triggered at control rate from a synth on the server, but don’t run in the audio thread, so (in fact) you end up with a little added latency from signalling between threads. I don’t think this latter feature is at all useful in Max.
This part is me speculating on @tremblap’s setup, where he will have two pre-fit KD-trees which represent an LPT space and an LT space (sans pitch), where he will do his nearest neighbor matching from.
Above that, presumably, he will have a buffer~ which contains all the analysis points for an incoming (real-time) frame. One of those points being the mean of the confidence of pitch. So based on that individual sample in the buffer~ he will choose to query in the LPT KD-tree, or the LT KD-tree.
So my peek~ question is a speculation on what @tremblap is planning on doing here. I guess the fluid.datasetquery~ wouldn’t make sense for that as, you correctly have pointed out, it’s not yet a dataset, but rather a single datapoint. The concern/wonder still stands though. Is the idea to peek~ out what you want and proceed accordingly?
My own personal solution/suggestion/approach would be to “entrymatcher” it where you don’t need pre-fit forks further down the path, but that it’s built into the same object/process.
It does end up wanting to be an ‘all or nothing’ situation, where switching threads at any point limits the speed/tightness usefulness.
I have no idea what’s going on under the hood, but if all the buffer/dataset/etc… objects could take signal inputs (and gave signal outputs), that would let you cover nearly all the ground doing that. I guess the expectation in Max-land is typically that what you trigger would happen in 1sample, which obviously isn’t going to happen here, but there would be plenty of use cases that would terminate in a signal output, or similarly triggering by the process.
Unless a bunch of things change, I’m likely still going to have to go into Max-land for some other list processing things regardless, but it would be great to be able to have a direct signalrate processing chain for core stuff (when possible).
that is the idea, yes. Forking from pitch confidence for 2 different searches. Not proven yet, work in progress.
indeed, suddenly everything becomes tense… continual accuracy is complicated…
signal rate is no panacea. it just means it is more tightly sync, not faster. actually it might be slower, if you need to make sure that everything is constantly in sync. for instance, jacktrip to play online needs to add the worse case latency in the buffer size to make sure there is always enough samples to play… so at times, asyncrony is your friend towards a most-case faster principle…
What’s the second search? Or are you just doing two stats of the confidence? (like mean and min or something like that)
This goes way above my head, but isn’t audiorate triggering here just about keeping things in a single thread/domain? Things wouldn’t be faster (in terms of processing) or tighter (in terms of synchronicity), but rather not having to change between threads from audiorate clicks (from the onset detection) and subsequent analysis.
Even when I was trying to do stuff with framelib for the onset/analysis part, I lost any speed gained there by having to jump back into the Max domain to carry on with the process.
take the pitch confidence for that frame. if high enough, consider pitch (kind of LPT). if not ignore pitch (kind of LT)
Will keep you posted if I manage something working.
In that case, what’s the first? Or do you mean a search for the corpus (dataset) and a search for the real-time (input)?
I’m mainly talking about the realtime part here.
Yeah, curious as to your results.
Well a point-wise transformpoint might make sense, but there’d probably be diminishing value with more complex queries.
Indeed: you want nearest neighbour and range searches to be embodied in the same query structure.
1 vector, rather than one sample, but yes. Trouble is that for the buffer-based analysis, there’s no way of keeping that promise, as it depends on how long the input buffer is, so the processing for those objects has to happen off the audio thread (in which it is Not Polite to do time consuming things). In this situation, a signal rate output is no more useful or accurate than you using click~, and arguably worse because it gives a spurious impression of accuracy.
The only future route I can see to a pure signal-rate chain would be to take advantage of mc for the already real-time descriptor objects, so you get a bundle of signals rather than a list. Again, far, conjectural horizon (although, this is all TB1, so someone could totally PR that one day)