Mlp controlling parameters of a vst written in rust!

Code Sharing
1

Using baseplug and neuroflow, I cobbled together a (GUI-less) VST that, with the help of a neural network, controls two aspects of the sound (left and right volume) with only one parameter exposed to the host. I tried to independently control resonance and cutoff of a lowpass filter to show that it was truly two parameters changing under the hood rather than one control for “panning” but I turned out to be even worse at DSP than I remembered lol. I think with some slight improvements this could streamline my VST dev; I spend a shocking amount of time right now just trying to get my parameter controls to be intuitive. I’ll note the code is still a mess, though. I couldn’t figure out how to load a file properly so I literally just re-train the network every time you load the plugin lol.

Here’s a github link to the code and some binaries: GitHub - audiodog301/mlplugin

1 Like
2

your friend is the write and read will write/read the state of the MLP object to a file. It is in JSON so you can even read it, and/or use training to/from elsewhere (scikitlearn for instance if you are a python master)

3

Hi @liamfissell!

Welcome to the Discourse! This sounds like a cool project. I’ll be curious to see how it develops. I don’t think I can help you with either the baseplug or neuroflow stuff, but I had a thought. As you know, doing stuff like controlling volume, panning, filter parameters and stuff is quite easy and flexible in Max, so maybe it would be interesting for you to try prototyping some ideas in Max using the fluid.mlpregressor~ for the kind of control you’re looking for. That way once you get something that feels right you can go back to rust-land and implement it?

You could even see if there’s a clever way to convert the JSON output of fluid.mlpregressor~ to a format that neuroflow could load?

Cheers,

T

4

That’s exactly what I was considering actually! Max MSP is perfect for prototyping this sort of thing, especially when FluCoMa tools exist. Next time I’m not super busy 24/7 (this weekend?) I hope to look at neuroflow's .flow files that save network state and see if they are similar to JSON from fluid’s mlpregressor~.

5

(update: it looks like .flow files are going to be not practical to touch at all, but neuroflow has a possible upcoming feature for serializing to JSON)

1 Like