So I just come from 2 days of a reality check with the amazing Wet Ink on some of my attempts to do Computer Assisted Orchestration through Concatenation (CAO). Here is a long post on what I wanted to do, what I used, in and out of FluCoMa’s toolset, and what I want to do in the near future. Any input, both from experienced (paging @danieleghisi and @tutschku) and observers welcome.
What I wanted to do
CAO tools available are interesting but not very good with gritty noisy modular synths, and complex textures, like the ones I like to use. So my idea was to:
try what is available out there to see how it reacts to my sounds:
split my sounds through flucoma (nmf was the plan to split and recombine layers) and run the software mentionned in #1 above.
What I tried
Orchidée is on the IRCAM forum, which I’m lucky my institution has a subscription to. The good side: it came with an extensive (yet problematic) database. The bad sides: it did not work on any recent OS (and I’m not one that updates compulsively, I’m still on 10.12, but it does not work beyond 10.10, typical!) The second problem is the database itself. Some clever person decided to normalise the sounds instead of keeping them to their relative loudness. I would have thought that the latter is the only usable method when reconstructing sounds for real life use…
Arachnid was fun but again relying on a strange database (the same but older) so I got strange loudness mismatch for my reconstructions.
AudioGuide was fun. I found a few bugs but the dev/composer (Ben) was super quick to reply and fix. It opened a huge (interesting) question of querying, and potent descriptors, as it allows the users to define the search in a flexible, creative manner. All of this is digested and will inform some design for the 2nd toolbox but I’m open to suggestions. What AudioGuide is good at is segmenting too - a very interesting interface of how to define thresholds for amplitude base segmentation.
What I was missing
- the ability to take a buffer in max/bach from nmf for instance, a small object, and then ask my database for the nearest match, without bouncing. Even with bouncing, it was always long and complicated. I wished I had something a little more like our FluidSound browser, and this is soon going to be a first contribution to the 2nd toolbox (in prototype first) - as part of the project’s original goals of taxonomy/browsing, so it is good it confirms the need of having such a thing. My attempt at a distance visualiser has also raised questions of descriptor uses and usages in matching, confirmed also the non-intutitive use of those beyond the CataRT model I used in the past 10 years (with the Sandbox#3 paradigm although there was some reflection already in there on descriptors usefulness and relative weight)
What I ended up doing
with arachnids I heard something that was quite different from the target, but exciting and inspiring, so i reconstructed by hand the ‘solution’ in Reaper with the sounds from the database, with manual correction of relative loudnesses. I had to score all this new editing manually but that yield the best results. In workshop, the results was similar enough to the maquette for me to be happy.
with audioguide I did a parser to Max/Bach so I can load/see/play the ‘solution’ before committing it to score and/or audio. It was fun, yet again quite far from the target. The Bach bridge was useful to solo tracks and sieve through the solution.
surprisingly, I was not able to segment by noisiness, or pitch confidence, and it lead @weefuzzy @groma and myself to discuss promient feature we use to segregate, which is often multi-modal, so we are thinking in term of research here (under the first tooldbox header) on how to make better informed segementations…
I hope some of these points will trigger some discussions - I’m noticing how my experience of musaiking is quite grain-based and immediate, and how these CAO offer different constraints, but I’m interested in any thoughts!