Getting Started with FluidBufPitch

Getting Started and Wayfinding
1

Hi again -

I’ve been following along with @tedmoore’s SuperCollider video series and it has helped answer a lot of questions - but I’m hoping to understand pitch-tracking a little better within FluCoMa’s toolkit.

I’ve adapted the example from the 2nd video in the “Corpus Plotter” series, “plotter-2” with the intent of taking a sound file, slicing the transients, and then re-organizing the input according to pitch.

The outcome is pretty murky - I think my code definitely has some issues that I’m missing - but I’m also curious about what the approach generally is, in terms of creating a sense of an “ascending” corpus? Is Spectral Centroid usually the preferred method for this, in order to handle polyphonic incoming sounds? Maybe a different SpectralShape feature, in general?

Any help would be appreciated. Thanks!


~src = Buffer.read(s,FluidFilesPath("Tremblay-AaS-AcBassGuit-Melo-M.wav"));
~src.play;

(
~indices = Buffer(s);
FluidBufOnsetSlice.processBlocking(s,~src,metric:9,threshold:0.36,indices:~indices,action:{"done".postln});
)

~indices.postln;
FluidWaveform(~src,~indices);

(
~play_slice = {
	arg index;
	{
		var startsamp = Index.kr(~indices,index);
		var stopsamp = Index.kr(~indices,index+1);
		var phs = Phasor.ar(0,BufRateScale.ir(~src),startsamp,stopsamp);
		var sig = BufRd.ar(1,~src,phs);
		var dursecs = (stopsamp - startsamp) / BufSampleRate.ir(~src);
		var env = EnvGen.kr(Env([0,1,1,0],[0.03,dursecs-0.06,0.03]),doneAction:2);
		sig.dup * env;
	}.play;
};
)

// analysis
(
~indices.loadToFloatArray(action:{
	arg fa;
	var pitchInfo = Buffer(s);
	var pitchBuf = Buffer(s);

	fa.doAdjacentPairs{
		arg start, end, i;
		var num = end - start;
		start.postln;
		end.postln;
		i.postln;
		FluidBufPitch.processBlocking(s, ~src,start,num,features:pitchInfo);
	        FluidBufCompose.processBlocking(s,pitchInfo,
			destination:pitchBuf,destStartFrame:i);

		s.sync;
		pitchBuf.loadToFloatArray(action:{
			arg fa;
			~pitch_features_array = fa.clump(2);
			"done".postln;});


}});
)

(
~selected_indices = List.new;
~pitch_features_array.do({
    arg val, i;
    if((val[0] > 120) && (val[1] > 0.5),{
		~selected_indices.add([val[0], val[1], i]);
    });
});
~selected_indices.sort({arg a, b, c; a.asArray[0]>b.asArray[0]});

~sorted = ~selected_indices.collect{|a|a[2]};

)

(
fork{
	~new.do{
		arg i, ct;
		("playing slice: %".format(i)++"ct:"++ct).postln;
		~play_slice.(i);
		0.09.wait;
	};
}
)
2

The pitch tracker outputs two numbers for each analysis (i.e FFT) frame: a pitch estimate and a confidence estimate. So, when you run it between a pair of slice indices, you get a bunch of numbers because the slice will (almost certainly!) have multiple FFT frames.

Then if you want to try and arrange the slices by ascending pitch, you would typically boil this bunch of numbers down to just one using FluidBufStats to get, e.g., the mean pitch over the slice

		FluidBufPitch.processBlocking(s, ~src,start,num,features:pitchInfo);		
		FluidBufStats.processBlocking(s,pitchInfo,numChans:1, stats:meanPitch, select:[\mean]);

The above squishes the list of pitches in channel 0 of pitchInfo down into a single frame buffer (`meanPitch).

Then, if you wanted to arrange each slice in order, you could just build up a language side list of each mean pitch and then call .order.

Polyphonic material is indeed harder: my hunch is that spectral centroid could work ok when each slice is from the same source, but if you have a diverse corpus it might not work as well (for seriously polyphonic stuff it may just be that it doesn’t make perceptual sense to try and arrange the slices this way.

2 Likes
3

Thanks, @weefuzzy - I think I was getting a little confused by the distinctions between frames, indices, and slices - and I’m a little more clear now.

The issue that I think I’m running into is that I’m doing a secondary analysis.

I have a set of slices, giving me the starting frame and ending frame. If I send them through the code you provided:

	FluidBufPitch.processBlocking(s, ~src,start,num,features:pitchInfo);		
		FluidBufStats.processBlocking(s,pitchInfo,numChans:1, stats:meanPitch, select:[\mean]);

…that potentially gives me a single mean comprised of all the slices, right?

If I wanted to get a mean for each slice, I would think I potentially need another FluidBufCompose to accumulate each iteration, something like:

FluidBufCompose.processBlocking(s,~meanPitch,
					destination:~comp, destStartFrame:i);

…where “i” is the current slice…

At the moment, though, this is resulting in an empty buffer. Maybe I am still misunderstanding something here, though. Here’s the overall code body, in case I am missing something elsewhere… Thanks for the help!

	~indices.loadToFloatArray(action:{
	arg fa;
	var array;
			
	~pitchInfo = Buffer(s);
	~meanPitch = Buffer(s);
	~comp = Buffer(s);
	array = fa.clump(2)
			
		fa.doAdjacentPairs{|arr, i|
			var start, numFrames, dest = Buffer(s);
			start = arr[0];
			numFrames = arr[1]-start;
			i.postln;
			   FluidBufPitch.processBlocking
				(s,bufPath,start, numFrames, features:~pitchInfo);	
			   FluidBufStats.processBlocking(s, ~pitchInfo,
					numChans:1, stats:~meanPitch, select:[\mean]);
		       FluidBufCompose.processBlocking(s,~meanPitch,
					destination:~comp, destStartFrame:i);

			};
1 Like
4

No – because you’re running FluidBufPitch etc inside that doAdjacentPairs loop with start and num being passed, you’re getting a pitch estimate for each slice. There’s no necessity to use FluidBufCompose from the point of view of getting an estimate per slice – it’s just one way of saving the estimate before going on to the next iteration of the loop (you could use a FluidDataSet instead, or just sync and pull the estimate for that slice down to a language side array one at a time (probably quite slow)).

Your code looks as it if ought to work. Certainly, if I alter the plotter example with something similar, it works. What slicer are you using here? Do you really need that clump?

// analysis
(
~indices.loadToFloatArray(action:{
	arg fa;
	var pitch = Buffer(s);
	var stats = Buffer(s);
	var comp = Buffer(s);

	fa.doAdjacentPairs{
		arg start, end, i;
		var num = end - start;

		FluidBufPitch.processBlocking(s,~src,start,num,features:pitch);
		FluidBufStats.processBlocking(s,pitch, numChans: 1, stats:stats,select:[\mean]);
		FluidBufCompose.processBlocking(s,stats, destination:comp, destStartFrame:i);

		"analyzing slice % / %".format(i+1,fa.size-1).postln;

		if((i%100) == 99){s.sync;}; 
	};

	s.sync;
	"has % samples".format(flat.numFrames).postln;
	defer {flat.plot;} 
});
)

Note that this can be made a bit more efficient by preallocating the comp buffer to fa.size - 1 up front (otherwise FluidBufCompose is having to reallocate on each iteration).

2 Likes
6

I see now. I was using FluidBufAmpGate, which required some modification to the array processing, since it already produces pairs.

3 Likes