@tremblap and I have been bouncing some ideas around corpus exploring and applications that really focus on understanding large personal collections. I’ve been searching for existing applications that attempt to do this and came across Sononym. Seems really cool so I’m about to try it now.
I’ll propose a question to everyone here to try and start discussion around the principles implied by this kind of corpus program:
What kinds of long term analytical procedures would be interesting or useful to you? Imagine a program that sifts and sieves your sound collection(s) in the background and produces a number of insights and metadata about What might those insights look like roughly? One angle I’m interested in is being able to start at an incredibly broad entry point, as an exercise in maintaining naivety (at least initially). The metadata and analysis would let you start at this ‘raw’ entry point and burrow into the analysis layer by layer unpicking more fine details about your corpus. Imagine starting with long field recordings which are segmented at different time scales and some sort of network between time points is created. The connections between points would be grounded in some sort of analysis that is specified by the user - maybe you have a big drum corpus and want to look at amplitude based descriptors or you might only have short violin melodies where pitch and tone colour are more important. Relationships could be further discovered such as similarity and antagonism as well as groups of time points that might belong together, ala clustering.
Anyway, these are some of the thoughts I’ve been having so it probably reads like a stream of consciousness right now rather than articulate and clearly expressed ideas but im interested in what others might be thinking about in this ballpark of corpus manipulation.
Seeing stuff like this makes me wish there were versions of this kind of things that weren’t designed for people making drum loops…(exclusively).
I guess there are macro-level descriptor things that start getting murky. I guess the MIR stuff goes more in that territory with things like tempo, genre, etc… Obviously not very useful for the kinds of things most of do, but I’m personally not terribly interested in browsing for samples via the kinds of perceptual descriptors we normally use (centroid, loudness, etc…).
For me, some kind of meta-data-type thing would be more useful where I’d want “metal sounds”, or “glitchy sounds”, etc… Things that might be harder to parse via audio analysis (perhaps).
I do like the idea of being able to explore and curate a corpus in an interactive manner.
Not sure if this is what you meant, but it would be amazing to have something like an NLE where you can have segmentation that’s decoupled from destructive file edits, which can then be rendered to a static corpus if/when needed (perhaps including normal transcoding options like stereo->mono, re-sampling, etc…).
This could also be badass as a GUI and expanded version of your FTIS thing.
I guess there would have to be some kind of boundary thing where it switches over into a DAW or “composition” environment (for something like a developed FTIS), which opens up another gigantic can of worms.
OR it could simply render a category/group/whatever, and you import that into another environment to do musical-y stuff.
I’ve personally gotten as far as producing clusters that are fairly homogenous, but its up to me to peruse and audition these to know what the homogeneity is. I think that to know these kinds of labels you either need to provide examples (i.e train a model) which could be supplied by someone else curating your samples for you or yourself to get more tailored analysis. Most software packages veer toward the former side of things and curate hard for you, with the trade off being that those programs perform very well inside of a narrow set of scenarios.
Same - ftis is still in progress but one idea is to be able to create a number of pipelines and to compare the results through audition. WIP though massively.
Yes. Sorry to keep tooting my own horn but I do this with ftis the corpus is never directly edited and you always keep a trace of any processes that are separate form the source material. In the cache for example:
I really love the idea of creating sub corpora and being able to maintain a hierarchy of ‘parent corpora’ that can be traversed upwards. That could be really powerful if you want to connect corpora too.
My gut feeling is the best approach here would be a number of ways to render out to environments agnostically. The best format for data going elsewhere is probably JSON (max/sc read it easily) but also being able to generate stock patches or scripts ala audioguide which produces csound scores to hear concatenation results.
I think we indeed often talk/think about large amount of data outside of their time. It would also be interesting to think of macroscopic/clustered scales. Analyzing a form, at any possible scale, with correlations, pseudo-periodicities etc. And why not later making some simple markovian magic with that.
I remember for instance some DAW showing similarities with colors.
I spoke to @jamesbradbury about this via chat back when this thread was created, so got some of his thoughts, but wonder if anything else concrete came from his or @tremblap exploration of the software.
Not yet. I’m thinking about this in the context of post-PhD whenever that is as a more full bodied research project. It’s something I’m keen to explore but just don’t have the schedule to fully commit to it with being on the last project + write up of the dissertation.
