Fluid.sinefeature optimizing spectral features and no click on update

Getting Started and Wayfinding
1

Hi all! Thanks to PA for giving me some time a few weeks ago to help me with fluid.sinefeature, it was good to know @numpeaks can go as high as ca. 100. Here is an amxd I have been working on. I didn’t simplify it so you can see what end-effect I am after. I need my own oscbank for individual FM and possible individual phase modulation for each partial. The device takes live audio, at the click of a button captures its loudest 10 peaks, then there is an option to play counterpoint, RM, and AM around it. And a cool feature of fzero~ (which I use in combination with .sinefeature) is that if the threshold is below zero, it keeps playing, adding another sort of counterpoint while the incoming audio is silent.
So for future development I have two questions:

  1. I would be interesting to update .sinefeature at each onset detection. Problem is that you get the features of the transient, but if you delay the update, you of course get a click or jumps in audio. I don’t want to just use fluid.sines because I still want my own osc bank for reasons mentioned above- unless there is a way of getting under the hood I don’t know about.
  2. I have been messing around with the fft/window settings of .sinefeatures. Probably need a crash course, but any tips on how to get good results would be great. As it is, I notice that the fundamental is often missing, which is why I use this in combination with fzero~.
    The compressed code is too long, so I am attaching the .amxd, hope that’s ok.
    fluid_sines_features.amxd (405.1 KB)
1 Like
2

You can do a circular-buffer-type thing where you use fluid.bufsines~ to analyze a window and go from there. This is my go-to approach in SP-Tools, and is what sp.sines~ does. Depending on the @fftsettings this can get heavily skewed by the transient, as you mention, but I find it quite useful still.

This isn’t as relevant for realtime stuff, but for offline/buffer analysis I found that computing the time centroid and then only analyzing the audio before that point gave me much better results. I think I work through (including code examples) in this thread from a while back.

I wonder if maybe combining something with chroma or something could help nail down the fundamental more? I found I used to get really good results from sigmund~, which I think takes a different approach to that too.

Also, getting a bunch of missing file errors:

go.onepole.hz: go.onepole.hz: operator/abstraction go.onepole.hz not found in gen domain dsp.gen
go.line.ms: go.line.ms: operator/abstraction go.line.ms not found in gen domain dsp.gen
go.ramp2trig: go.ramp2trig: operator/abstraction go.ramp2trig not found in gen domain dsp.gen
go.ramp2trig: go.ramp2trig: operator/abstraction go.ramp2trig not found in gen domain dsp.gen
live.tab: can't find ef-delay-ms-icon.svg
live.tab: can't find ef-delay-beat-icon.svg
1 Like
3

Hey thanks Rodrigo for the quick answer! I am a big fan of your SP-Tools and am using some of your gen~ code for onset detection in this example here. The idea of a circular buffer is one I have toyed with, and I’ll give that a shot.
Just want to clarify that with fzero~ I get great results for the fundamental, which is why I am combining it with fluid.sinefeature.
Sorry about the missing file errors, beginners mistake. Will report back in a few days after getting this sorted and taking some of your advice. Thanks again!

1 Like
4

Hello

Sorry for the delay in replying, and for the forthcoming delay as I go offline for 4 weeks… so thoughts to get your going a little more I hope:

That seems possible. I really depends on what you want. If you want the sound to keep on playing until you get the next valid frame (post onset-noise) or not. One could imagine to delay the right amount the sample-and-hold of the stream that comes out. or one could reset a running average (like fluid.stat) after a few frames. it really depends what you expect to be between the attacks? Do you want average, or following. I am not clear there and both options are fun.

In my case, I usually make a bank after attacks. In my old bass clarinet piece for instance, an attack was triggering 3 ‘snapshots’ that were saved after 10/20/30 ms of an attack (if I remember correctly), in a coll. That ‘pool’ of 3 times 10 peaks was then pooled for resynthesis and I loved how it sounded.

I hope you have seen the great resource compiled by @jamesbradbury here ? If it is too mathy, one could say that the bigger the window, the more precise the spectral/pitch/frequency resolution, at the cost of averaging in time (large windows will not be responsive as they include a lot of ‘time’ in them)

so for sines, I like big windows, like 8192 at least if not more. I usually use a hop size of 1024 so I get a new value every 512 samples, or 4 ms at CD quality, which is usually overkill. The idea of a short buffer being analysed and average, like the circular buffers I value so much and @rodrigo.constanzo hinted at, is also a good idea. It all depends on how accurate you need it, and how long you can wait for it. if your material to analyse is slow-ish, then analysing from 10ms after an attack for a large window could be fruitful… but not knowing the musical context it is hard to imagine what would be good.

I hope this helps?

1 Like
5

Thanks, PA!

For my purposes, an average would be sufficient, although I want to explore the possibility for following at some time also. The musical context is to create a sound that is not a 1-to-1 resynthesis of the incoming live sound, but could be its ugly twin, and can respond in real time to timbral changes (a bit of latency can be acceptable). I would then apply some frequency and/or phase modulations, which is why I can’t just use the sound from fluid.sines.
I have better grip on the fft settings now, thanks! Now I have some time (and a stipendium!) to come to grips with this so need to get to work.

1 Like