conTorchionist

Hi, this is conTorchionist (or its embryonic version, I would say): https://github.com/ecrisufmg/contorchionist

It’s a project that is in its very early stages, but I thought it was worth announcing here, not so much for its current development stage, but because it might be useful to others and also as a thank you to the community and the developers who have created inspiring tools. Besides, obviously, Pure Data, SuperCollider, and Python, I’m referring to libraries like FluCoMa, timbreID, nn~, librosa, and others, which have been fundamental to the development of conTorchionist.

Briefly, the idea is to have a nomadic and flexible library, based on libtorch, that allows for the use of the same DSP (machine listening, but also, in the future, other things related to synthesis/signal processing) and ML processes in both real-time (PD, Max, SC) and non-real-time (Python). Wrappers for other languages/environments should not be so hard to code, as everything is interfacing a shared library (and libtorch shared libraries). Another key idea is to have a widely used tool like PyTorch/libtorch/torchaudio as a foundation, which greatly facilitates development and integration with existing ML+ML experimentation workflows and processes. The structure is heavily inspired by FluCoMa, but it is probably simpler and follows a logic of core Processors and wrappers in different languages (we have been prioritizing Pure Data, but we are gradually creating things for Max, SC, and Python as well). Since we use libtorch, conTorchionist also has some other potentially interesting features, like using the GPU for calculations and DSP - CUDA (Windows/Linux) or MPS (Apple Silicon).

We are taking it slow, so a lot is still missing (documentation, testing, etc.). That’s why we haven’t published a versioned release or binaries yet. We hope to do so soon after verifying the consistency of some things regarding DSP calculations in PyTorch / torchaudio (which, in turn, use librosa as a reference) and ensuring that the initial wrappers compile properly on various operating systems (we have tested mainly on Linux and macOS, so it’s likely that some adjustments will be needed in the CMakeLists for Windows).

I hope it will be useful and that we can gradually make things more organized. At the moment, it’s just me and a great student (Vinícius Oliveira), plus some always quite hallucinatory and unreliable AI bots. :wink:

The paper we presented this week, on SBCM, is here: https://www.researchgate.net/publication/395473005_conTorchionist_A_flexible_nomadic_library_for_exploring_machine_listeninglearning_in_multiple_platforms_languages_and_time-contexts

Or if you prefer figures: https://ecris.cc/2025sbcm_contorchionist-slides/

Any feedback is appreciated, of course!

Best regards!

José Henrique Padovani

1 Like

Wow this looks amazing. I will definitely give it a spin later this month, as I am doing pre-project research to see what to add to our lovely library, so any contribution to interface questions between musicking and creative coding is inspiring!

I can provide feedback here for now, so more people might be tempted to try it?

1 Like

Hi! I’m honored by your compliments: I have learned (and still learn) a lot with FluCoMa, so thank you so much! I’m not very frequent writing around here, but I’m always reading not only the forum, but also the codes in repos.

And yes, for sure! Your feedback will certainly be highly valuable!

Just some notes: Regarding the Max wrapper, it is important to say that we still need to design FFT objects with a more appropriate logic, especially since Max doesn’t rely on [block~]. That also is the case for SC. We also need to implement objects for buffer access and reading, similar to what FluCoMa already provides. Furthermore, we have to better address multichannel handling and the possibility of sharing pointers to the internal tensors, which could be very beneficial for certain workflows.

Since we are still building the foundations of this thing, our main focus right now is on establishing an accessible (as simple as possible) and viable development scheme for the various wrappers, and on ensuring consistent windowing/normalization for the RFFT-based processors (spectrogram, melspectrogram, MFCC).

The various wrapper interface was magic done by @weefuzzy - loads of headache since they are each ‘idiosyncratic’… him and @a.harker wrote a cool paper at ICMC this year talking exactly about this sort of issues - I cannot find it online though, so I hope they will jump in here with a magic link!