Batch Slicing Example

[areacode]

Hi there-

I’ve been looking at the Batch-Slicing tutorial - and this SuperCollider example in particular.

I had two questions about it, for those who might have more expertise.

1. In the example above, the process produces a dataset, 24 rows by 1 column.
   When calling “print” on the DataSet, though, it only prints an abbreviated version of the DataSet:

DataSet 2448: 
rows: 24 cols: 1
0     3626.8
1     8345.6
2     8443.8
       ...
21     3379.5
22     2943.1
23     3580.3

I’d like to get a full report of the dataset, if possible.

2. I’d like to use the values reported - in this case, spectral centroid means - in order to name files exported from this process.

I’m adding the following bit of code on line 61 - but I know it is not right, since ‘sliceMean’ refers to a buffer until it is somehow converted to a number in the dataset. I’d like to get that number directly at that stage, if possible.

FluidBufCompose.processBlocking(s, drum,
destination:dest,
startFrame: start,
numFrames: numSamps,
action:{
dest.write("/Users/a/Desktop/sctest/" ++ "ex" ++ sliceIndex.asInteger ++ "__" ++ sliceMean ++ ".aif");
				dest.free;
										});

Thanks!

[weefuzzy]

To print out a whole dataset, call dump to get it as a language side Dictionary.

To get the sliceMean value in the loop you’d need to call Buffer.get to retreive the value from the server.

[areacode]

Thanks - this is helping a lot. Some of the details of how FluCoMa work are allowing me to understanding SuperCollider a little differently, so I apologize for being slow at first.

Right now, I’m hoping to use ‘FluidPitchBuf’, in place of the ‘FluidSpectralShape’ in the example.

In this case, my understanding is that FluidPitchBuf writes to a single-channel buffer, but it is writing a stream of “paired” arrays consisting of the pitch and the confidence calculations.

As FluidBufStats is seemingly designed to work on a single stream, I’m not sure what the ideal settings are for analyzing the output of FluidBufPitch. Do I need to break confidence into a separate buffer?

Attached is the original example code, with the annotations removed for the purpose of space.

(
s.waitForBoot({
    fork({
        var drum = Buffer.read(s,FluidFilesPath("Nicol-LoopE-M.wav"));
        var slices = Buffer(s); 
        var sliceAnalysis = Buffer(s); 
        var sliceCentroid = Buffer(s); 
        var sliceStats = Buffer(s, 7); 
        var sliceMean = Buffer(s, 1); 
        var cond = Condition(); 
        var data = FluidDataSet(s); 
        FluidBufOnsetSlice.processBlocking(s, drum, indices:slices, metric:9, threshold:0.5);
        s.sync;

        slices.loadToFloatArray(action:{ |slicesArr| 
            if ((slicesArr.at(slicesArr.size) != drum.numFrames), {
                slicesArr = slicesArr.add(drum.numFrames);
            });
            slicesArr.postln;
            slicesArr.doAdjacentPairs({ |start, end, sliceIndex|
                var numSamps = end-start;
                sliceIndex.postln; 
                FluidBufPitch.processBlocking(s, source:drum, features:sliceAnalysis, startFrame:start, numFrames:numSamps, action:{
                  
                    FluidBufStats.processBlocking(s, source:sliceAnalysis, stats:sliceStats, numChans:1, action:{
                       
                        FluidBufSelect.processBlocking(s, source:sliceStats, destination:sliceMean, indices:0, action:{
         
							data.addPoint(sliceIndex, sliceMean,{
								cond.unhang
							});
                        });
                    });
                });
                cond.hang;
            });
        });
        sliceAnalysis.free; sliceCentroid.free; sliceStats.free; sliceMean.free;
        data.print;
    });
});
)

[weefuzzy]

No need to apologise for anything. We had to do some contortions to get this into supercollider (as few as possible), so there are some unfamiliar ways of working. It massively helps us if we understand where people get stuck.

Meanwhile:

• BufPitch produces a two-channel buffer (this is the general rule for all the multi-feature Buf objects, i.e. a feature-per-channel)
• BufStats will do an independent analysis per channel

So, if you remove the numChans:1 from your BufStats call, it should give you stats for both the pitch and confidence estimates.

[areacode]

Ok - so, the default ‘numChannels’ for ‘BufStats’ is -1, which means it will read all channels from the source. If I remove the current argument for 1 channel, the test code produces 24 rows and 1 column, seemingly only providing the detected ‘frequency’ without the ‘confidence’ information.

I was thinking since the source is mono, but the features should be two-channel, I then tried changing sliceAnalysis = Buffer(s, numChannels: 2); This didn’t impact the overall output.

I was also thinking perhaps sliceStats (a seven-channel buffer for each averaging function) would need an additional channel - but I presumably want two sets of sliceStats (a total 14 channels?)… and I wasn’t sure if that was the expected approach there.

I’ve shared the code again, with the new modifications, in case there are other mistakes on my end. There was a slight mistake in the original documentation regarding buffer channel allocation - "Buffer(s, 7); " actually only allocates 7 frames, rather than 7 channels, although I don’t think it was a problem.

(
s.waitForBoot({
    fork({
        var drum = Buffer.read(s,FluidFilesPath("Nicol-LoopE-M.wav"));
        var slices = Buffer(s); 
        var sliceAnalysis = Buffer(s, numChannels:2); 
        var sliceCentroid = Buffer(s); 
        var sliceStats = Buffer(s, numChannels:7); 
        var sliceMean = Buffer(s, numChannels: 1); 
        var cond = Condition(); 
        var data = FluidDataSet(s); 
        FluidBufOnsetSlice.processBlocking(s, drum, indices:slices, metric:9, threshold:0.5);
        s.sync;

        slices.loadToFloatArray(action:{ |slicesArr| 
            if ((slicesArr.at(slicesArr.size) != drum.numFrames), {
                slicesArr = slicesArr.add(drum.numFrames);
            });
            slicesArr.postln;
            slicesArr.doAdjacentPairs({ |start, end, sliceIndex|
                var numSamps = end-start;
                sliceIndex.postln; 
                FluidBufPitch.processBlocking(s, source:drum, features:sliceAnalysis, startFrame:start, numFrames:numSamps, action:{
                  
                    FluidBufStats.processBlocking(s, source:sliceAnalysis, stats:sliceStats, action:{
                       
                        FluidBufSelect.processBlocking(s, source:sliceStats, destination:sliceMean, indices:0, action:{
         
							data.addPoint(sliceIndex, sliceMean,{
								cond.unhang
							});
                        });
                    });
                });
                cond.hang;
            });
        });
        sliceAnalysis.free; sliceCentroid.free; sliceStats.free; sliceMean.free;
        data.print;
		~d = data;
    });
});
)

[weefuzzy]

Ah, my bad – I didn’t think ahead to you needing to add to a dataset. FluidDataSet is quite fussy: it expects 1 channel x n dims to represent a point and will simply ignore extra channels.

You will be getting 2 channels and 1 sample from FluidBufSelect, but that needs to be turned into a single channel thing. You can use FluidBufFlatten to do that.

(FWIW FluidBufSelect is redundant in this particular case, since we added an argument to FluidBufStats to tell it what you want directly, so if the nesting seems like a bit much once you’ve added FluidBufFlatten, it can be reduced again).

[areacode]

Thanks - this got me on the right track.

One last thing, just for the sake of thoroughness:

If I remove the FluidBufStats also, my understanding is that I’m still getting spectral analysis from FluidBufSpectralShape (or whatever the analysis tool may be) - but it’s not averaged, instead giving a continual value for each hop.

If I wanted to accumulate all of the FluidBufSpectralShape analyses into a new, single buffer - I was thinking something like the following should work - but I’m not entirely sure where I’m missing the mark. It seems that the data file is much shorter length than the audio file, so maybe this isn’t the ideal way to compile a buffer.

Thanks for the help so far!

(
var previous = 0, inc=0;
~src = Buffer.read(s, FluidFilesPath("Tremblay-ASWINE-ScratchySynth-M.wav"));
~slices = Buffer(s);
~freq = Buffer(s);
~mags = Buffer(s);
~data = Buffer(s);
~audio= Buffer(s);

FluidBufOnsetSlice.processBlocking(s, ~src, threshold:0.6, metric:9, indices: ~slices);

~slices.loadToFloatArray(action:{ |slicesArr|
	if ((slicesArr.at(slicesArr.size) != ~src.numFrames), {
		slicesArr = slicesArr.add(~src.numFrames);
	});
	slicesArr.postln;
slicesArr.doAdjacentPairs({ |start, end, sliceIndex|
		var numSamps = end-start;
		var fluidFeat = Buffer(s);

		FluidBufSpectralShape.processBlocking(s,
			~src,
			startFrame:  start,
			numFrames: numSamps,
			features: fluidFeat);


		FluidBufCompose.processBlocking(s,
					fluidFeat,
					destStartFrame: previous,
					destination: ~data);
		
		FluidBufCompose.processBlocking(s,
					~src,
					startFrame:start,
					numFrames: numSamps,
					destStartFrame: previous,
					destination: ~audio);
		
			previous = numSamps + previous;

				});
	s.sync;
~data.write("/Users/a/Desktop/bunko/data.wav");
~audio.write("/Users/a/Desktop/bunko/audio.wav");
		});
)

[weefuzzy]

@areacode

It seems that the data file is much shorter length than the audio file, so maybe this isn’t the ideal way to compile a buffer.

Because the data is for each hop, so the analysis buffer is effectively downsampled from the audio rate by a factor of hopsize. So, give or take the padding frames, the duration of the analysis buffer should be a predictable amount smaller than the audio.

[tremblap]

@weefuzzy

the duration of the analysis buffer should be a predictable amount smaller than the audio.

and we have a tutorial-in-progress entitled ‘where is my data’ - so far done in Max but @areacode if you can read it, it is quite useful…

[areacode]

@tremblap - I’ll definitely check out the tutorial.

@weefuzzy - I suppose what I’m asking is how to have a matching stream of analysis data that takes place in the same “time” as the original audio.
Is there a way to re-up-sample the buffer?

[weefuzzy]

@areacode

Is there a way to re-up-sample the buffer?

Sort of depends what you want to do. If you want to audition back the trajectory of a feature in a synth, I’ve put some code at the bottom of this to get to started.

I think upsampling a buffer is a trickier proposition (unless I’ve not thought of something). If it’s not too big, you could bring it language side as a collection and call resamp1 on it, then send it back to the server. Otherwise, it would take a plugin and I don’t think anyone has made one for that.

Anyway, a hurried example of ‘resynthesizing’ a sweep from its pitch analysis using a simple synth. I’ve just hardcoded the magic numbers here, but obviously you’d probably do something more elegant in real code.

//make a test sweep signal
(
t = (0..88199) ;
f = t.linlin(0,88199,200, 2000) ;
t = t/44100;
x = sin(2pi * t * f) ;
b = Buffer.sendCollection(s,x);
)

//if we do a pitch analysis, we expect ceil(audio frames / hop size) values in result 
//(this actually depends on the padding mode, but this is the default)

//default hop is 512
(88200 / 512).ceil //173

//Do a pitch analysis (just get the pitch, to keep things simple) 
(
~pitch = Buffer(s);
FluidBufPitch.processBlocking(s, b, features:~pitch,select:[\pitch], action:{
	~pitch.numFrames.postln;	//173!
}); 
)
// check it's not *too* awful
~pitch.plot;


//play original 
{PlayBuf.ar(1,b);}.play;

//very simplest thing we can do is just step pitch buffer frame by frame every 512 samples: 
(
{ 
	var ph = Phasor.ar(0, 1, 0, 512); 
	var frame = Stepper.ar(ph < 1,min:0, max:173); 
	var freq = BufRd.ar(1,~pitch, frame, interpolation:1); 
	SinOsc.ar(freq)
}.play;
)

//but because BufRd will interpolate, you could get rid of the stepper and put fractional frame numbers in to smooth it out
// YMMV, I'm not hearing an earth shattering difference on laptop speakers 
(
{ 
	var ph = Phasor.ar(0, 1, 0, 512 * 173.0); 
	var frame = (ph/512.0); 
	var freq = BufRd.ar(1,~pitch, frame, interpolation:4);
	SinOsc.ar(freq)
}.play;
)