FluidProcessSlices crashing

Bug Reports and Issues Toolbox2 Related Bugs
1

Greetings,

I seem to remember something about this recently, maybe @spluta knows, but I can’t find a thread on here that addresses. Basically my FluidProcessSlices crashes before completing all the slices. It will get through a different number of slices each time, so I can’t figure out what might be causing it.

I’m on SC 3.11.0 now.

Here’s an example.

(
s.waitForBoot({
	Task({
		var target_path = "/Users/ted/Desktop/SCD/flucoma/quick brown fox.wav";
		var corpus_path = "/Users/ted/Desktop/SCD/flucoma/200613 eurorack 01 last 10 min excerpt_MONO.wav";

		var dir = "/Users/ted/Desktop/SCD/flucoma/";
		var timestamp = Date.localtime.stamp;

		var target_loader = FluidLoadFolder();
		var corpus_loader = FluidLoadFolder();
		var target_slicer, corpus_slicer;
		var print_index_info = {
			arg index, buffer, name;
			"index:    %".format(name).postln;
			"n slices: %".format(index.size).postln;
			"avg dur:  %".format(buffer.duration/index.size).postln;
			"".postln;
		};

		// load target and corpus ======================================
		target_loader.setFilesArray([SoundFile.openRead(target_path)]);
		corpus_loader.setFilesArray([SoundFile.openRead(corpus_path)]);

		target_loader.play(s,{"target loaded".postln;});
		corpus_loader.play(s,{"corpus loaded".postln;});

		s.sync;

		// slice target and corpus ====================================
		target_slicer = FluidSliceCorpus({ |src,start,num,dest|
			FluidBufNoveltySlice.kr(src,start,num,indices:dest, threshold:0.57);
		});
		corpus_slicer = FluidSliceCorpus({
			arg src, start, num, dest;
			FluidBufNoveltySlice.kr(src,start,num,indices:dest,threshold:0.6);
		});

		target_slicer.play(s,target_loader.buffer,target_loader.index,{
			arg target_sliced_index;
			corpus_slicer.play(s,corpus_loader.buffer,corpus_loader.index,{
				arg corpus_sliced_index;
				var process_slices;
				//target_sliced_index.postln;
				//corpus_sliced_index.postln;

				//print_index_info.(target_sliced_index,target_loader.buffer,"target");
				//print_index_info.(corpus_sliced_index,corpus_loader.buffer,"corpus");

				process_slices = {
					arg buffer, index, dataset, action;
					Task({
						var extractor;

						var mfccBuf = 4.collect{Buffer.new};
						var pitchBuf = 4.collect{Buffer.new};
						var specBuf = 4.collect{Buffer.new};
						var loudBuf = 4.collect{Buffer.new};

						var mfcc_stats = 4.collect{Buffer.new};
						var pitch_stats = 4.collect{Buffer.new};
						var spec_stats = 4.collect{Buffer.new};
						var loud_stats = 4.collect{Buffer.new};

						var comp_buf = 4.collect{Buffer.new};

						var master_buf = 4.collect{Buffer.new};

						s.sync;

						extractor = FluidProcessSlices({|src,start,num,data|
							var label = data.key;
							var voice = data.value[\voice];
							var t1 = FluidBufMFCC.kr(src,start,num,features:mfccBuf[voice],trig:1); // 13 channels
							var t2 = FluidBufPitch.kr(src,start,num,features:pitchBuf[voice],trig:Done.kr(t1)); // 2 channels
							var t3 = FluidBufSpectralShape.kr(src,start,num,features:specBuf[voice],trig:Done.kr(t2)); // 7 channels
							var t4 = FluidBufLoudness.kr(src,start,num,features:loudBuf[voice],trig:Done.kr(t3)); // 2 channels

							var t5 = FluidBufStats.kr(mfccBuf[voice],startChan:1,stats:mfcc_stats[voice],trig:Done.kr(t4)); // 12 channels. 7 frames
							var t6 = FluidBufStats.kr(pitchBuf[voice],stats:pitch_stats[voice],trig:Done.kr(t5)); // 2 channels, 7 frames
							var t7 = FluidBufStats.kr(specBuf[voice],stats:spec_stats[voice],trig:Done.kr(t6)); // 7 channels 7 frames
							var t8 = FluidBufStats.kr(loudBuf[voice],numChans:1,stats:loud_stats[voice],trig:Done.kr(t7)); // 1 channel, 7 frames

							var t9 = FluidBufCompose.kr(mfcc_stats[voice]  ,0,1,0,-1,1,comp_buf[voice],0,0 ,0,Done.kr(t8));
							var t10 = FluidBufCompose.kr(pitch_stats[voice],0,1,0,-1,1,comp_buf[voice],0,12,0,Done.kr(t9));
							var t11 = FluidBufCompose.kr(spec_stats[voice] ,0,1,0,-1,1,comp_buf[voice],0,14,0,Done.kr(t10));
							var t12 = FluidBufCompose.kr(loud_stats[voice] ,0,1,0,-1,1,comp_buf[voice],0,21,0,Done.kr(t11)); // total 22 channels

							var t13 = FluidBufFlatten.kr(comp_buf[voice],master_buf[voice],trig:Done.kr(t12));

							FluidDataSetWr.kr(dataset,label, -1, master_buf[voice], Done.kr(t13));
						});

						s.sync;

						extractor.play(s,buffer,index,action:{
							action.value(buffer,index,dataset);
						});

					}).play;
				};

				Task({
					var target_ds = FluidDataSet(s,\target);
					var corpus_ds = FluidDataSet(s,\corpus);

					s.sync;

					process_slices.(target_loader.buffer,/*target_sliced_index*/target_loader.index,target_ds,{
						process_slices.(corpus_loader.buffer,/*corpus_sliced_index*/target_loader.index,corpus_ds,{
							target_ds.write("%%_target_ds.json".format(dir,timestamp));
							corpus_ds.write("%%_corpus_ds.json".format(dir,timestamp));
							JSONFileWriter(target_sliced_index,"%%_target_sliced_index.json".format(dir,timestamp));
							JSONFileWriter(corpus_sliced_index,"%%_corpus_sliced_index.json".format(dir,timestamp));
						});
					});
				}).play;
			});
		});
	}).play;
});
)
2

In case it’s useful, my server is averaging around 30% but peaking around 130% every so often.

3

try changing the tasks from 4 to 2 or 1 in the extractor’s .play

sam

4

Thanks @spluta. However, I just tried it at 2 and it crashed, then at 1 and it still crashed a little less than halfway through. To be more specific, my server bar turned yellow, progress froze, and activity monitor said my scsynth was at 99.5%. After a few minutes of hoping, I killed the server.

Any suggestions appreciated!

T

5

The other thing I did was add a little pause inside the class. Changing:

if(jobs.size > 0){perf.value(jobID)};

to

if(jobs.size > 0){SystemClock.sched(0.1, perf.value(jobID))};

which adds a tenth of a second between each analysis.

1 Like
6

Putting the 0.1 seconds between analysis worked for me! Thanks @spluta.

7

Hello

I’m trying to reproduce (no chance yet on this, @tedmoore was there a number of items at which is started to misbehave? ) but I notice that you are passing the target_loader index and commented out the sliced_index. It should still work though. I should still hang and I can. Trying harder here. Will report if I manage to crash it.

I think maybe a JSON writer for the utilities could be good though, @weefuzzy, and Dictionaries do not allow such niceties natively in SC. I know we have some code done by @groma and you in our classes to do that (in Dataset) but I’m not good enough to mod the classes and use them.

8

ok I got it to crash at around 900 items. I’ll put a ticket in and we will try to find out what is happening.

1 Like
9

If it’s useful at all, my “target” was a 4 second file that sliced to 18 slices. My “corpus” as a 10 minute file that sliced to around 4000 slices.

It would crash at different spots, sometimes after processing around 50 slices, sometimes around 600, sometimes around 2000. I couldn’t perceive any pattern there.

Yeah, I was just trying different things to see if something different worked. When I passed the loader’s index, it was fine, but of course that means it was only processing 2 slices.

10

ok I’ll wait to hear from the chiefs and will probably run with special compile modes @weefuzzy cooked for me where I get some verbose reports…

1 Like
11

I’m hoping we’ve fixed this in Alpha 5, but it’s a slippery one.

2 Likes
12

I don’t think it is fixed, but I did find a bit of a workaround that is great for this 12 core Mac Pro I am using, but also works on any multicore machine. Basically, it is the multiple server approach. I am right now running 40. That is probably stupid. Efficiency stops getting better once you have 1 server per core. 20 was probably a better number, but I wanted to see if it would work, and it does without crashing.

(
~bigExtract = {|num|
	var server, mfccbuf, statsbuf, mean, flatbuf, path, paths, ds, extractor, extractor2, mergebuf;

	server = Server(("nrt"++NRT_Server_ID.next).asSymbol,
		NetAddr("127.0.0.1", 5000+num),
		options: Server.local.options
		.numOutputBusChannels_(1)
		.numInputBusChannels_(2)
	);

	server.waitForBoot{

		mfccbuf = Buffer.new(server);
		statsbuf = Buffer.new(server);
		mean = Buffer.new(server);
		flatbuf = Buffer.new(server);
		mergebuf = Buffer.new(server);

		ds = FluidDataSet(server, ("mfcc"++num).asSymbol);

		path = "/Volumes/StretchFiles/SynthAnalysis/3_soundsAnalysis/sounds/";
		paths = PathName(path).files.select{|item| item.extension=="wav"};

		extractor2 = FluidProcessSlices({|src,start,num,data|
			var mfcc, stats, writer, flatten, merge ,mfccBuf, statsBuf, flatBuf, label, voice;

			label = data.key;
			voice = data.value[\voice];

			mfcc = FluidBufMFCC.kr(src,start,num,numChans:1,features:mfccbuf, numCoeffs:20, numBands:40,trig:1);
			stats = FluidBufStats.kr(mfccbuf, 0, -1, 1, 19, stats:statsbuf,trig:Done.kr(mfcc));
			flatten = FluidBufFlatten.kr(statsbuf,flatbuf,trig:Done.kr(stats));
			merge = FluidBufCompose.kr(flatbuf, 0, 19, 0, -1, 1, mergebuf, 0, 0, 0, Done.kr(flatten));
			writer = FluidDataSetWr.kr(ds,label, -1, mergebuf, Done.kr(merge))
		});

		extractor = {|server, counter, action|
			var extractor,path;
			"extract".postln;

			path = paths[num];

			Buffer.read(server, path.fullPath, action:{|buf|
				var slicer,index;

				index = IdentityDictionary[(path.fileName.asSymbol -> IdentityDictionary[ ('bounds' -> [ 0, buf.numFrames ]), ('numchans' -> buf.numChannels), ('sr' -> buf.sampleRate) ])];

				slicer = FluidEqualSlicer();
				slicer.slice(buf, index);

				extractor2.play(server,buf,slicer.index.postln, {"Features done".postln; action.value(buf)}, 1);
			})
		};
		2.wait;

		server.sync;

		extractor.value(server, 0, {
			ds.write("/Volumes/StretchFiles/SynthAnalysis/3_soundsAnalysis/analysis"++paths[num].fileNameWithoutExtension++".json", {"free server".postln; server.free});
		});
	}
}
)

{(0..39).do{|i|~bigExtract.value(i); 1.wait}}.fork
13

You can get rid of the manual speccing of the first 20 channels for your processing:

stats = FluidBufStats.kr(mfccbuf, 0, -1, 1, 19, stats:statsbuf,trig:Done.kr(mfcc));

which I assume you are doing because you are getting 40 channels (20 blank, 20 with sane data). This is because maxNumCoeffs (or whatever its called) has to be set explicitly otherwise it always returns a 40 channel buffer. Absolute terrible nitpick on my part, but it might also make your code more understandable when you come back to it later.

I also present a Python implementation using my wrapper which is multithreaded in 28 loc :wink:

from flucoma import fluid
from flucoma.dataset import pack
from flucoma.utils import get_buffer
from ftis.common.io import write_json
from pathlib import Path
from multiprocessing import Manager
from concurrent.futures import ThreadPoolExecutor, as_completed


multi_threaded_storage = Manager.dict()
path = Path("/Volumes/StretchFiles/SynthAnalysis/3_soundsAnalysis/sounds/").iterdir()
sounds = [x for x in path if x.suffx == ".wav"]

def process(item):
    slices = get_buffer(fluid.noveltyslice(item))
    for i, (start, end) in enumerate(zip(slices, slices[1:])):
        x = get_buffer(
            fluid.stats(
                fluid.mfcc(item, startframe=start, numframes=end-start)
            ), "numpy"
        ).flatten()
        multi_threaded_storage[f"{i}_{item}"] = x
    
with ThreadPoolExecutor() as pool:
    futures = [pool.submit(process, item) for item in sounds]

dataset = pack(dict(multi_threaded_storage))
write_json("data.json", dataset)
14

The python looks real pretty.

Are you skipping the first value of the mfcc?

15

Yes you would just have to pass startframe to stats and it would cleave the first band out.

x = get_buffer(fluid.stats(fluid.mfcc(item, startframe=start, numframes=end-start), startchan=1
            ), "numpy"
        ).flatten()
16

Can you explain more? As in, you’re getting exactly the same badness you were with the previous version, or similar, or something new?

The number of threads and the crash are different issues, though the former increase(d / es) the probability of the latter. The crash was coming about because sometimes the synchronisation between threads when stuff gets destroyed was getting muddled (between the scsynth audio thread and the command thread, our custom thread is dead and gone by this point), but having more of them increased the likelihood that the Bad Thing would happen.

But, yes, more than one thread or process per core and you’re going to see performance degrade.

17

I experienced the same issue. The analysis got going and then crashed. I went down to 1 task and had to add a little SystemClock pause into the FluidProcessSlices and all was good.

But the Multicore approach is super dope if there are multiple files. I have made almost 1 million MFCC analyses with no crash.

Sam

18

How aggravating! @tremblap and I both felt like this was behaving for us. I shall revisit in due course, but perhaps will settle on having the breathing space you added in the language side as a work around temporarily.

19

Well, I am also on an old Mac Pro, which has a bunch o’ cores, but each core is kind of wimpy. So it can’t handle the sauce of this process.

20

Even then, it should just take longer, rather than crashing. The crash that I found was due to the horrors of the different SC threads treading on each other’s toes after a job was complete. I think the basic problem is that I don’t have a great deal of control over when the synth decides to die (on the one hand) but also that scsynth seems to (rarely) start trying to process unit commands on a node it hasn’t finished building :sob:

The whole SC wrapper is getting pretty unwieldy at the moment, so I need to pick some of this apart and review.

1 Like