The slow and steady process of converting my analysis/matching to ML-land is carrying on, and as part of this I’m going to start simply by replacing my real-time analysis and matching with the ML equivalents, as a jumping off point.
The first step of this is simply removing the final peek~ -> list part of my analysis chain with fluid.bufcompose~ in order to transfer everything into a fluid.dataset~. But in building this I’ve run into a… kind of small problem.
For my centroid stuff, I’ve been converting everything to MIDI via ftom 0. for better matching/weighting (alal @tremblap’s LPT patch). This presently isn’t a big deal because I end up with numbers in Max-land that I can then process however I want.
If I’m going to fluid.bufcompose~ stuff into a fluid.dataset~ directly, there doesn’t appear to be a way to transform these values…
Same goes for dbtoa (which would be useful for things like spectral weighting).
In @tremblap’s LPT patch that happens via js data-manipulator.js, which is a definite no-go for fast real-time processing.
And the “normal” data sanitization (as far as I understand it) doesn’t deal with these kinds of transformations either (e.g. normalization or standardization).
What is the ‘intended approach’ for transforming data post analysis and pre .dataset~-ing? Or is the idea to shove frequencies in there, and let data sanitization and/or dimensionality reduction “deal with it”?
(I can obviously peek~ out what I want, do maths in Max-land, then peek~ everything back into a buffer~ before fluid.bufcompose~-ing what I want together to put into yet another buffer~ to finally end up in a fluid.dataset~, but that starts to get really ugly really quick…really slowly)
Is it? I thought it always ran in the low-priority thread, so it bogs down the moment anything else is happening (which is also my experience).
(for the offline version, the js stuff is fine, but it’s the real-time counterpart that I’m thinking about, since I want the distance to be comparable, or at least in the same scaling (log vs lin))
also important for you to know that you won’t be able to do log centroid for instance, since it needs to happen inside the algo. This is on the mid-term todo list, or as suggested before, can be undertaken by anyone with C++ chops and fed back in the community via pull-request on GitHub. It is around line 38 in this
What will be less trivial is to add a parameter that allows lin-lin, lin-log, log-lin and log-log, and to think of the interface that makes sense, etc. But that is what we have a few years to make sense… but first we need to get our ML interface up to speed, which we are already burning the midnight oil on…
Interface wise, it’s probably not too hard to have an optional @scaling attribute or something like that (though log-log should be the default for spectral stuff). Just a bit tricker for something like loudness where you may want dB for normal matching/ML shit, and linear amplitude for weighting. That being said, the latter could just happen as a separate @weighting attribute in fluid....spectralshape~ where you give it a loudness buffer (in dB) and it sorts itself out in linear amplitude.
definitely not. For you maybe, but not to be in phase with literature that most people will come across. This is why it is long. We cannot take a single musician’s hunch to implement things in an open manner
Yeah, JS for your causality breaking RT stuff will make you sad.
I don’t know how far he got with it, but @a.harker was making a mode for FrameLib that worked on buffers. That would allow you to do pretty much whatever you liked, really quite fast.
I meant in terms of interface idea/design. Most definitely “not easy” to implement it…
It does default to something either way, so the open-ness comes in the ability to change it, not so much what the defaults are.
If a user(/“musician”) typically never sees the numbers themselves, as they would go from fluid.bufspectralstats~ into a fluid.dataset~ (quite likely with that abstraction/wrapper), wouldn’t log-log work best in just about every possibly way there? (Mainly thinking that all the data sanitization will just push around the linear-ness of the frequency domain representation, which isn’t good for matching)
Or maybe even @danieleghisi’s ears thing, though I never got around to playing with the old version and have no idea what it does in practice.
For now, and forvariousreasons, I’ll be using some kind of hybrid approach at most I’m running a parallel coll or entrymatcher since I’ll need access to meta-data and individual numbers, so I can just peek~ → ftom → peek~ things that need changing.
Given that your RT inputs are so small, I don’t think you’ll be losing a great deal of time to this approach.
If you get bored of making the same uzi->peek~ structure over and over, you could probably make an abstraction that steals the trick I stole from bach for making a processing wrapper (cf fluid.segmentcorpus and fluid.processsegments)
I’ll have a peek(!) and see what they/you did. For the most part I’m just doing a t 8 7 0 since I’m only after some stats for any given descriptors but abstracting it would be most welcome.
From my understanding, Framelib can access regular Max buffer~ objects, as well as his own ibuffer~. The ibuffer~ access did not work in the public release, but @a.harker corrected it last week and it works in his dev branch.
From my understanding, Framelib can access regular Max buffer~ objects, as well as his own ibuffer~. The ibuffer~ access did not work in the public release, but @a.harker corrected it last week and it works in his dev branch.
What @weefuzzy is referring to is the ability to run frame lib in non-realtime which is a real thing, and works as far asI know (in either thread) and has some fairly easy to use mapping/scaling objects. This allows you to read into fl.source~ from buffers and write to buffers from fl.sink~.
That stuff isn’t on master, because I decided to finish my object review first, which is a bit painful, but almost done, although it was almost done for a while, but with a different level of almost. However, it’s hard to say if it’ll be faster or slower the rolling uzi/peek~/conversion/peek~ loops
