(more) ways to get data in and out of fluid.dataset~

Now that I’m playing more with the TB2 bits I’m wondering about how to best approach certain things.

As I figured out in this thread, creating a subset of a fluid.dataset~ with fluid.datasetquery~ (specifically as a way to filter a query via a single parameter) is pretty slow. Granted, “creating a subset of a large dataset” is not what I’m interested in doing, it just appears to be the roundabout way to filter something using conditionals, which are only present in the fluid.datasetquery~ interface.

Another thing I tested today was getting some of the data out from a fluid.dataset~, which may be part of the process outlined above (filtering a dataset), or just be part of another querying/filtering process (perhaps pulling up the individual timeness metric of an entry). Or an even more realworld example would after finding a nearest match via fluid.knnclassifier~, wanting to find out the specific loudness value of that entry for loudness compensation.

The process of getting data out from fluid.dataset~ seems to be the inverse of putting stuff in. I have to pass it to a buffer~ and then peek~ out what I want. Having to go through three data types (fluid.dataset~, buffer~, list) comes with a cost of processing time, and using an example dataset with a modest 300 points in it takes around 0.15ms on my computer. This isn’t “slow”, but if there are many of these kinds of steps and processes in an overall chain of events, it can start to add up.

Example code for extracting a point into list-land:


----------begin_max5_patcher----------
2956.3oc2atsjahjDF95teJHTr6M6HSPlbdux6ygCGcfjJIyLHf.P183Il4Y
ey+hCMHgjPxn1ytNBqVhJKpJyuLy5.E+wyOsXU1qpxEF+aiOY7zS+wyO8j9R
3BO076mVrO500IQkZwVrWUVFsSsXYcYUpWqzWmLMXSCaSCGSCWSCOSCeSi.S
iPSCxR9uTNIBPhDjHBIxPhPjHEIhQhbrHGi6iHGKxwhbrHGKxwhbrHGKxYKx
YKxYiFTjyVjyVjyVjyVjyVjyVjyQjyQjyQjyA8LQNGQNGQNGQNGQNGQNWQNW
QNWQNWQNWnBhbthbthbthbthbdhbdhbdhbdhbdhbdPWE47D47D47D47E47E4
7E47E47E47E47gQQjyWjyWjKPjKPjKPjKPjKPjKPjKPjK.VOQt.QtPQtPQtP
QtPQtPQtPQtPQtPQtPXl01YXnsfk1BlZKXqsfw1BVaKXtsf81BFbKTiZzfZn
gilNZ7n4iFPZBoQDXDAHQrllnFfSD.EARQ.UDXEAXQfVDvEAdQ1ZG.TCfLBL
i.zHPMBXi.2H.NBji.5HGsOCpAnGA7QfeD.HABR.gDXHAHRfhjq1MC0.fj.I
IfRBrj.LIPSB3j.OI.TxS6YhZ.lR.pDnJArRfqD.KAxR.sDXK4qclQM.dIvW
B.l.gIfXBLl.jIPYBXlBz9+nFfzDPMAVS.1DnMAbSf2D.NAhSg5PFcLCBZ.y
YvbFLmAyYvbFLmAyYvbFLmIcXFpAXNClyf4LXNClyf4LXNqiK0Al0QlnF5XS
cvoN5TGdpiO0Anf4LXNCly15fYTCvbFLmAyYvbFLmAyYvbFLmAyYGc7OpAXN
Clyf4LXNClyf4LXNClyf4rqNkApAXNClyf4LXNClyf4LXNClyf4rmNKCpAXN
Clyf4LXNClyf4LXNClyf4ruNwDpAXNClyf4LXNClyf4LXNClyf4bfNWFpAXN
Clyf4LXNClyf4LXNClyf4bnN8mN+mkYa92j3T05rCo5jvD2b0rCUIpppeOWU
mZewBiO2TT7Fc55rU+5Gb416R5g8woRMzo342tX88QeUp4p4QUq+Rb5tWJTq
qpu6HEg0RIMIY5G16ettKkbjgnLDAYY7YbK9ymeFerbhi2jp9lzWOY3lJiDi
UKNq5tzXwpnzcip1Ndip1znpMed01U.mnZveP9is3wJ5qDfbe54pCUUYomUi
Nu13tnqKVDsWUoJdQkFsJQWSqoqoW.vgR5AQ2bjHSnvNsedWJ51jLoguSzQ2
sxtMqXejVe7tUP2n9tRhtS7uwHnV+LvN+NfcIe4rg8yDGmarN+PU7dUwYsAh
CC.2m6.YZUpnw5p+eJhiRVLh4w4NRsg6bY72U0oRMstd.gjX2tuGgu+RLivN
Gh92AQCqsNMlGo8hSTeUUTFKN.u0MdZQTdduK+Tup.a5ulouQAK6tTbZ8knt
KUn9Zba8c6tZTgnGUhRbnn158pWqYB2lrMphzCw56T8EE51zkzbDV8x7n00U
F3ts3dIEYs5iI7f+DpGY.i02YMDGkcIYq+M0lddpB1yUowo4EpRUZUTUSmuq
3MpsQGRpdY.iHS2PWYdTc1+wjdaSGdza0nNROsXWQ7lrTzkF.Fb41F+SXILP
Ec6qZZIRixGoxhaoXkNSgkhJenbUTA3VSPL2VXUVVxvh5pWhZaUSw4wooGYS
qxxOegEw69xEp6pLov8W5dqKo7kCo0k9h3hT8RYzWGZsqhRRZh8Gd6eMJMVx
HqPvuVcs5JrNQ1WJWWjkjLPeqK4qiTxFwkes5awap9htgL6waQ737VWpEcTd
S7NUY0vqUEsqb3UJq98ZiduKcXUSH8KUp84IhVLT.IXItrp7KYeqrQvVGs9F
f2V3c+P79IRGb8iRnVm.qK9cX9Op+0GaTfQRCZ0qjQx4Yqy+ik8fHZ21O678
0oIx1uWUOWzEwoxXReMJwHN0XeoQboQgJOqnRswPrGpAc7zMpW6k2oYvklTP
2o8Yv.NCGzYkw.6ykG08xC5brgmuCCOcTCc07amad3V5rsxZQzSKU+oU6v0O
dqpLP9ZjX+hl1gClea1V6e91VrcJXoNdu2F2OXzslu42v5bVCK+vMrMoUbzo
UrY8ZndGsqOvTAt+DcWcB9+5LAd+LMs9vG8wlHPqO2neYOyi+YMOVSx7b97e
N9Sbzez+LJkIfVZHSvStWFQoaZlAfLYfl4FL5j.3+9XHCleCYcj4sZGifUqw
PdQi2Xyf5skmfcK7LS5T2UQ4iaUKyNTrt0l0XZLF1sk4YWEm1szsO0kEzXpA
G2Zeveh8A9A1G7lXePFi0fdP8A2I1G7df1AmI1GnGXev9FXwipOvSrOXe19P
yEa27kEXI0adod6CdIpppHd0gp5v396lzMsL2osMJ6RxVEkbzJXGasxO+lJb
iaK34dbx6TU4YRlNIlQxslen9WKtomugcv777MZ1ne7T6veZ1HTdd2HzOzMQ
zS0Nn4ipf9yiB56q0r1M5M7G5AYbF866IF6JxNja7wumHxCmO7frt3yAXbk1
9FT5Krq990iA6xAl82B2f.utkiL21fCeO9h5rdtHKe6umk62xCty9Bl.R+v4
rCCO0D369Hr.4J0u8WSJh9nc8uu1aMp1aeud8A7HN.VOjfbI.nRYzla6188Y
+Yx2uA7tZsrdu5macc0gsaUE2BruzS9i3Y8g11n43Pv7in5WczKq6ezK+Ycv
KlejCdsM4P7FyMQUQkpp+xnLdedhxE+9p455P+3N72.0suN0ImQdBt0qH68X
7Nxhct8fdmYZ.Obtcv.61jomUu+w1K0mhp+FOfmi07LfGN.T.4AglbeS.42s
j726Gjuc3i+A46p8v8rdbOF+JiXiUFwWbFrSfybv7DuiC1mdWk8LCFvYZo9T
M9Hx4uNQEUbio3oYZ960mSCKxj5mcylWhyC6CHtNItrZ6gzTUxYUXHx3LNbw
r3T2jNyVv4ImAC2GRJ8updMuv3erkM9W34KSF+h7Cx3ikqiRhJv4m3Bqga7Y
03MON.so1qOJZL8PmPW8n60yu4ikQ6yKuXN9IL2Nq4YtcMQ8Nx5zOcB8ANOt
0yzyhbGqmIbdVNSa7f36e5bbrdHADk4EhJu0HZylZ2g+4lIYKF+znNOoEZcB
FKYHNe1OnY4PX3EiYXlNyyDcpS.hUw5ze7OKDIv7+SsD.ZVmRfCNlt88KrBW
pe8AdDSIXyg8423VVNSyHfpmn6nQANznJqt6L7IDUq6GuC3MVfS14aZviB4b
OJlv2126SrxSskvzDM3qzR5tCMGsDckVBqNYdzoqY8bmiFJbBMzbPIaZBMDM
CMj19eM2AG64pkn2kVxdJFOuYnkbemnj+M3NzlyrXS8oll9wrkS.Z1yhi3D7
Caiy+wBslRLrybjrXJZj+bzP9SPir44v1YMkVZNhg0cW5cqktFmrmCNYOINM
GCH5vSwG2YFFj2wYJszbjfvdJCx6LGY08euZHmIMqu4pgt5XuywLjbbmpm2O
r+vTFk2cXZjYZPwI0zGECtOtdc9MKPv20S+dw4YM5CP01juZoN15EiD5qW.l
9WmpnVuCy6vZ7IdXcxBjN5kyBs5QuTVG8BYc5Ki04eQrN9kvReFfpeCTNZQY
cmFGYgndcuJVsuaSCMIQqWqRqVmkT2q9jgkYvR7gcHSd3a9VgAVNBW5Y9wpX
004k3Tnwp15Zsr2G8qwpc8ZCpozlOrLcaiUV1fuBoSc7KWyh8RuOt0x2G2CT
3dGqywU4AJqCa6xgnOX6ytN5uw9NtTS2e15VC1Gfw6X2iUcabRRmB0+He0ta
BK1UDsIV0siaM5O0QaO4e9KG8aT+WeMc031p4GDHVtki9sSqVas3PKJT6UY2
3UguIWhNpVQo6ZdQ27681ZkWjgyHb66+moc2oQK5PUVmh1Fa1cpyFGeCiNlF
E6+hUMAuKMybD2JKV+MQmsCuaOq5jLm40s74+74+KkLqWLA
-----------end_max5_patcher-----------

I did notice in the fluid.dataset~ helpfile that you can load data directly from a dict in a json format. Since there’s no reference file (yet) I wasn’t able to see more about this, but it led me to wonder (and desire) more ways of getting data in and out of fluid.dataset~ that potentially avoided going to and from buffer~s.

This would be fantastic for programmatically creating fluid.dataset~s without needing to worry about buffer~ plumbing around it, especially if you’re only trying to edit/modify a single point. Even better would be to have access to some of the syntax from fluid.datasetquery~ where you can send a getpoint message with things like addcolumn 0, addrange 0 2 and fluid.dataset~ would spit out that information directly (similar to lookup when using entrymatcher), rather than dumping stuff into a buffer~ and then having to deal with that there.

it is used all over the place! There isn’t much a doc would tell you that isn’t in the help with working examples…

This only works in all in all out approaches. You still have to go through buffers if you want to update a point.

Mainly want to format plain text in a way that I can load it into a fluid.dataset~ (using dict format as a backdoor).

I guess that’s what I’m talking about and asking for her. The ability to manipulate the data inside fluid.dataset~ directly, sans buffer~s.

that is already in the helpfile. There is one that assembles a dict in the dicts via json of dataset. The syntax is there, you can doubleclick the dict and have fun and then output it via dict.serialize. The format is pretty self-explanatory and not optional. If you are curious about any other objects, the format is always the same as dump, with load.

you’ve asked for that for about 39482094830298423 times now, and we replied 39482094830298423 times now that it is coming one day. @tutschku also asked for it and will also get it. He might have asked less times though… :smiley:

1 Like

I did poke that and got a json-y thing of {data: stuff{}}, but I couldn’t find a way to feed that message back in (using fromsymbol and tosymbol to avoid the , problems in Max land), but I kept getting parse errors.

Hehe, I’ve has for buffer~ skipping on other things, but not specifically about fluid.dataset~. I guess the spirit of my request is similar though…

there is some @weefuzzy magic in the fluid.labelset dicts via json if that helps. Max’s dicts are a pain as they do not behave well with (for instance) single item arrays (they are parsed down) which is a pain with the rest of the world of json…

1 Like

Ah yes, that is a bit of gnarly stuff in there…

I shall (eagerly (and apparently repeatedly)) await native solutions to the data input/output.