HPSS/Transients-sliced "realtime" granularization

Code Sharing
1

So based on the help from @weefuzzy (and @tremblap) in the last few threads I posted, I completely rebuilt my “Cloud” effect, which basically does onset-driven “realtime” granularization.

Passively, it uses a bunch of fluid.stuff~ (fluid.bufloudness~, fluid.bufstats~, fluid.bufcompose~ (as well as some HIRT stuff (irstats~, irtrimnorm~)) under the hood to improve how the grains are captured/stored/sized/etc… but the big improvement is that it now uses fluid.bufhpss~ and fluid.buftransients~ to “three way” decompose the audio, and then lets you control the balance between these components.

It’s also packaged up as an M4L device, so all the dependencies are packed up together nicely.

Checkit:
http://rodrigoconstanzo.com/party/cloudv2.1.zip (edit: changed to latest version)
44%20pm

I still need to go through and clean up the code, as well as bundle in the Windows versions of the externals, but I wanted to share the code/sound, and get some thoughts on interface.

It’s kind of a tricky thing to explain, so I’ve wrapped up all the complexity into a single “timbre” knob. So at 50%, everything is active and the sound is “normal”. At 0%, it’s only the harmonic component, and at 100% it’s only the transient component (with a crossfade happening at around 90%).

In context I think this sounds really good, as you can push the granularization more into the “background” by pulling away the sharper transients. Similarly, you can really crank up the invasiveness of it. My initial build of this only used HPSS, since I thought that pure transients would be too much, but I decided to go “three way” with it today, and just adjust the scaling so that you only get 100% transients at the very extreme of the “timbre” knob range.

I’ve also used an adapted version of my C-C-Combine grain playback module, which has the ability to change grain window type. I’ve included that here, but I don’t know how useful this is across the range of possible settings. (i.e. the more you turn the “timbre” knob up, the ‘shorter’ the grain sound via the decomposition, and hence the window types make a smaller impact). Open to suggestions on this.

UI is also utilitarian and bare bones for now too.

2 Likes
2

This is incredible. Thank you so much for sharing! So many interface decisions that make sense - the single param for timbre is a clever one in this context… I’ll need to play with it to really investigate (it is Max7 compatible, right?)

p

3

Yeah. Should play nice everywhere.

(Easiest way to test is to drag the amxd onto an unlocked Max patcher and run audio to/from it.)

1 Like
4

Seems like the sweet spot is 96% timbre which is consistent with the literature. Very clicky and pointy but still retaining the juiciness of whatever signal is coming in.

I know this is an instrument for you to play with, but have you tested perhaps a small switcher for other kinds of slicing? It might be interesting to use some kind of spectral measure to trigger clouds or to at least have the option.

5

There is a surprising amount of play in that upper range. I can scale it down so the crossover happens at 75% (rather than 90%) for more of “that”.

I’ve not messed with it much but that would be good to play with. At the moment both recording and playback are triggered by the Max-version of the onset detection algorithm. But they can obviously be decoupled and/or look at different things.

For the sake of latency what triggers playback will probably always remain onset/amplitude-based but what does the segmentation of the JIT buffer can be novelty or some other metric.

6

Yep, seems like the knob provokes the most out of the material - I like it! In fact, I am warming up to the simplicity of transient extraction with the tools myself.

7

Indeed. I’m quite pleased with it, and the fact that you can push it completely into “just transients” is nice for certain use cases.

Do you find the grain window options to be useful at all? I’m still on the fence about that, or at very minimum pulling it down to just offering a couple options (sine, rampup, rampdown). I think when you’re doing normal/long grains, it works well, but anything more transient-y just gets lost.

8

I like the current mapping a lot as the mileage of noise to clicks is limited… maybe 80% (which would give you 30% range of pitch to noise then 20 of noise to clicks? Here you hit the complication of a general mapping for particular sounds - your mapping works well because of your material (percussive, decaying) and your playback (granulation is very transient sensitive - it betrays the process) but other types of material probably would benefit from a 2 fader approach - again, I think you should not change yours, it is beautifully idiosyncratic and consistent because of this!

Always!

What follows are ideas of interface that might or not be relevant to you:

  • I miss a volume control - the blend is never satisfying for me since I might want something else than a cross fade - and in this case I often have granular volume build up that needs compensating - it would also be good to have a volume variation control like the pitch one
  • I miss a ‘size of the past’ button (do I want to keep 1s or 10s or 100s or 10ms as my grain pool)
  • the width is great. You dig a whole in the middle, right? 50 is full-stereo range, and higher digs the middle?
  • I think activity could go wilder, like much denser, but again this is your implementation, which has limited overlap, and is very ‘Rod’ sounding. I love it a lot :wink:

Do you plan to distribute the code opened too?

p

1 Like
9

I tried to record the output of this demo, but I don’t have the screencapture with pro audio software on my machine. I still recommend this highly:
54

Also, good result with the ramp-up envelop (subtler)

10

Why not three fader?

Yeah I get that. In this case it was just a practical improvement to an existing effect/idea, but for other things that more fully explore that, I will probably do something more comprehensive (though still wrapped in a mapping metaphor of some kind). (i.e. if I have my sample buffers automatically NMF in the background, having some kind of interface that lets you play with the balance of those)

Perhaps a different fade curve? At the moment it’s all linear with both being 100% when you have it set to 50%, and then fading down from there.

I have adjusted my grain window amplitude compensations now, since the cloud stuff was always too loud as compared to the dry. I think the multipliers I came up with work well when the grains are all really tiny, but for these bigger pieces, it’s too much.

Interesting. Totally possible (though I just stripped out the per-grain amplitude adjustment stuff, to save a few *~s. The way I viewed it is that was sort of “built in” as what you record into the rolling ‘granularization’ buffer is anything (that triggers the onset detection algorithm).

Something like this would well in an ‘advanced’ tab or something. And it could be interesting, but I don’t want to over complicate the vanilla version. Hell, most of the parameters could be rslider-esque controls (or even more hardcore with @a.harker’s randomvals options for each parameter).

Hmm, I can maybe add a small amount of random variance in grain volume as an always-on feature, just to give it a bit of shape. Or tie it to activity and/or pitch.

Hmm. At the moment it’s fixed at 1000ms, which gives between ~6-30 “grains” from the “past”, with the current analysis/processing settings. I’ve found that that works pretty well for morphing with you, but also not having too much “latency” when you want to change.

A very “PA” take on this would be to have your normal “record everything that has ever happened” buffer, which you can then (manually) select a window from to be available for “Cloud”-ing. (either as a static/absolute, or relative/rolling selection (i.e. the last 2s of time, or this specific 2s of time)

It’s super brutalist. Each grain is panned hard left/right randomly (at the time of grain creation) then the width parameter just “pans” that towards the center or full width. It’s “fixed”, so you don’t get an overall granular spread of panning, but it still works well in terms of giving you some width/shimmer along with your dry audio.

(the effect also sums to mono before recording, even though the thru is left as stereo. I didn’t want to manage two of everything, or have a double CPU hit, and given there’s already panning involved, the mono->stereo grain thing works well)

I’ll take a look at the scaling, as I did that back when I first made the effect. More than anything it controls how many grains will happen after each onset is detected, but I think it also effects other things.

Of course!

Obviously not until the library is released.

Part of this effort, beyond making some of the effects I use often more compartmentalized and easier to reuse (literally dragging in a single .amxd, instead of finding all the abstractions/dependencies), is I want my FluCoMa performance patch to be very legible and dissectible. So I want the core items to be abstracted like this. (I’ll probably post my waveshaper on here too, even though it doesn’t (presently!) use any FluCoMa stuff, as I’m curious as to people’s thoughts)

It’s a shame you don’t use Max8, as they introduced an object called sfrecord~ that you may find very useful :stuck_out_tongue:

11

Actually, kind of like @jamesbradbury suggested, this can also impact how grains are recorded. So say there’s a massive rolling buffer (an “everything that has happened” buffer), when the onset detection algorithm fires, it queries the buffer, as it presently stands, for novelty (or whatever), then use either/both loudness (derivatives) or spectral (spread) to determine how long a snippet to take, and build up the rolling buffer this way.

Another thing I tested a while back (before I ran into a @startchan bug) was doing something similar to this idea, but with NMFed buffers.

I never got it fully working in the end, and then moved onto other things, but the core idea I may return to.

Basically, I would run audio into record~ @loop 1, and whenever the loop was done, it would decompose the contents of the buffer (at that moment) into a fluid.bufnmf~ @components 2. That was then sorted by spectral centroid to be a “dark” one and a “bright” one. I then ran that into fluid.bufonsetslice~ and copied all the slices (100ms long) into another buffer, which served as the “final” buffer for granlurization.

I was using a motion sensor on my left wrist, which tracked when I changed direction, to (re)create an NMF-y split of the material I had been doing in the past (i.e. moving forward plays from the “bright” buffer, and movement backwards from the “dark” one, and my absolute position also scrubbed through the position of the buffer).

As I said, I got hung up on the @startchan bug from a while back, and then decided against the motion sensor since I have enough going on already, but a process like that could work well in this context too.

12

I managed with the one there. I just prefer to be able to change gain of both independently… interface decision, you know :wink:

in a new version?

indeed - although I can settle with the last hour only :wink:

extremely indeed. Brutalist is good at times!

yes, but balance instead of pan, keeping the stereo material as stereo, is also valid. I would have liked a ping, pong feedback to it which would benefit from this, but hey, if the code goes public, I can customise…

Actually, the Max window being a good place to say where the external are from, I’m tempted to add instantiation messages (like @a.harker and Bach and many others) and then the project gets the creds anyway…

add that to the video editing to sync my noodling to it and you start to suffer… :wink:

a visual feedback of when an attack is detected could also be good, no?

13

you could also use my seeding trick (a low and a high start) and therefore you would not need to sort them after. Check the bufnmf help tab on updating bases.

14

Yeah, I’ve made a bunch of little tweaks, and will repost it once I feel it’s done(ish).

What do you mean by this?

(you know you can open/edit .amxd files in vanilla Max right?)

I personally find this super annoying, particularly how BACH does it, where it spams the Max window whether or not you’re using any BACH objects. Even doing it when you load an object/external is super annoying, but the startup spam I would disable if I knew where to turn it off.

So please don’t do that…

Hmm, I do that in C-C-Combine via jitter, mainly because I had a lot of UI space to spare in the middle, but here it’s a bit snugger. I could put a bang object or something, but as it stands, it always makes a sound when you get an onset, so it’s sort of self-declaring.

Wouldn’t that either color the NMF-ing or potentially be wrong if what I’m after is more a centroid-y ordering?

15

I hate them too… The problem is that sexy usages don’t get the credit that would send people back to this community we try to build… I like the HISS way, which was pointed at via a flag in the Ableton reverb when people opened it…

The Fluidstuff is BSD3 so you would need to have a read-me to acknowledge the object uses instead… is that more palatable for you?

Most probably… you can try them :wink:

16

HISS handles it well, where you have links, text, info, etc… when you open up the device. A “normal user” doesn’t (and shouldn’t) care what kind of libraries were used to code something (for example, loading up a fluid. external doesn’t spam the console with info about all the libraries you guys are using to code them).

There can be a separate info pane, or link/mention elsewhere in a release.

1 Like
17

On a side-note, perhaps it is worth putting a sticky somewhere on this forum on what BSD3 means in terms of usage. Do I need to do a readme for the scripts even though I don’t include the binary itself?

1 Like
18

I think if you do not redistribute the code then you are not affected by the license (well only in as much as you use it, but not in the distribution of your code). But of course if you do not include the binaries then I would expect some sort of documentation to let the user know that the binary is needed, otherwise the script won’t work…

19

Yes, so the BSD3 just covers redistribution of the compiled stuff (obviously) and only then does the license come into effect.

20

in effect, BSD3 is super permissive. If you use the code, in any way including commercial, you only need to give credits. In your case, because you will tell people they need to download from our website so there is no issue there at all - it spreads the love and gives an anchor. If you were to package the tools with them which would be fine by me, then a sort of read-me would be needed