For a while I have been a big fan of the CLI tools now and have used them fairly extensively across Lua, Python and some other environments. More than I’d like to count I’ve written the same functions, classes and code plumbing to wrap up the command line calls which is obviously sub-optimal. I had a look at writing my own Python to C++ bindings using pybind11 or something fairly automatic with manual TLC applied afterwards or through configs but even then I think there are significant issues to overcome, particularly because Eigen complicates this problem. As such I’ve written what I think is the last set of loose bindings to the command line I’ll have to write. Yay!
So, here is me sharing it:
The interface is very Pythonic and the underlying code is fairly repetitious at the gain of not using any strange inheritance or multiple inheritance as I have done in some other side-ideas (https://github.com/jamesb93/FluidChain). This makes the code very readable and easy to understand.
From the README:
A basic call looks like this:
from flucoma import fluid
from flucoma.utils import get_slices
from pathlib import Path
source = Path("~/Desktop/ec1.wav").expanduser().resolve()
ns = fluid.noveltyslice(source, threshold=0.1)
idx = get_buffer(ns)
print(idx)
Nice work @jamesbradbury. ISTM compiled bindings should be possible w/ Eigen (isn’t that also what TensorFlow uses for its matrix ops?), but then again I speak from a position of cheerful ignorance about making python bindings for anything…
I think it is very feasible to use Eigen in Python, in fact there is a guide here which is quite nice but there are some limitations that mean adapting the code so that it is Python friendly. Again, I think there are some inefficiencies in binding, in that a copy is made for every matrix operation where you convert the matrix type unless you structure the C++ code in a certain way. I am super noob C++ coder so a lot of this is guess-work:
One major limitation of the above is that every data conversion implicitly involves a copy, which can be both expensive (for large matrices) and disallows binding functions that change their (Matrix) arguments. Pybind11 allows you to work around this by using Eigen’s Eigen::Ref<MatrixType> class much as you would when writing a function taking a generic type in Eigen itself (subject to some limitations discussed below).
I would be super up for writing a compiled module for Python if I understood the plumbing of the core codebase more, just to get around the friction of having to have the CLI present. Alas, I don’t, but I think that could be super cool to have as an option for scripting and wouldn’t feel like such a dirty hack.
If there was a small example to work in the future from I would definitely be inclined to take it further and create bindings for all the objects I had a go this afternoon shortly and found that working around CMAKE was a level of complexity that would take me a while to understand itself but binding some simple functions and classes was quite simple and readable.
Yeah, the CMake is a bit involved (but not as complex as it first seems). It’s mostly to do with managing the dependencies between repos, and avoiding too much repetition for the objects themselves.
If what you’re looking to do is make python versions of the objects you know and love, I’d perhaps hold off a bit until the code base has settled down a bit more: compared to what’s in Github, the internals have already shifted around somewhat for the current development stuff and are likely going to be refactored again before too long.
Happy to share my basic tests and talk about them! They mostly test pythonic things, like was a file created and are the dimensions of that file correct (channels * samples). I was able to catch that bug that I posted earlier with memory resets and some misnaming of parameters on my end.
I’ve spent about a year thinking “if only there were Python bindings to Flucoma” and considered doing it myself multiple times, and have only just discovered this
Is there any reason this is not officially part of the project and listed on Download? I think a lot of researchers aside from myself would benefit from being more aware that this exists.
One of the reasons I am interested in this is to be able to take advantage of plotting and interactivity in Jupyter notebooks, have you played around with that at all?
Have you also tried combining Flucoma algorithms with other python-based numerical computing and machine learning frameworks (PyTorch, JAX, Tensorflow, Taichi, scikit-learn)?
Partly because I forgot it existed, but also because it’s not ‘ours’ (i.e. we’re not going to develop or maintain it). I guess we could profiitably link to @jamesbradbury’s various bits of wizardry as third party extensions or something .
TBH, it hadn’t struck me that there would be much of a need for FluCoMa in Python, given that between them librosa and scikit-learn cover almost everything in the toolkit.
