FluidProcessSlices crashing

Bug Reports and Issues Toolbox2 Related Bugs
21

Hello! Popping back into this thread here because I was again getting crashes with FluidProcessSlices in Alpha 07–more specifically, I believe it would get through all the slices (I had 793) but then stop right at the end when it’s done. It wouldn’t run the action function. Here’s what the post window looks like when it stops.

Processing 790/793
FluidProcessSlices: 787/793
Processing 791/793
FluidProcessSlices: 788/793
Processing 792/793
FluidProcessSlices: 789/793
Processing 793/793
FluidProcessSlices: 790/793
FluidProcessSlices: 791/793

These two changes again solved it for me. The Alpha 07 code didn’t have these changes to the class so I had to re-add them. For whatever it’s worth.

22

But was the server also crashing? Is this the same sequence of jobs as at the top of this thread?

23

Here I processed 2700 buffers, and 750 items, and other combinations in the high hundred and low thousand, without issues (except one ghost which I’m still chasing but that never happens in processBlocking)

So my question: if you replace all your process with processBlocking, does it still happen? If yes, then we are not chasing the same ghost. If no, then it might be the same as mine and I am near finding a way to reproduce)

24

@tremblap has supplied me with some test code of running masses of threaded jobs where, sooner or later, we can see it stop processing. It’s definitely a concurrency based issue, but trouble is narrowing it down. I’m wondering if dropped packets might be a problem: if I run the test code with the internal server, it always seems to work, even if things get occasionally choked and slow down.

@tedmoore if you have the time / inclination, could you see what happens with your failing case (without the added wait) on the internal server and / or a TCP connection? This might help narrow it down. If it always works, then the problem is possibly due to flooding the UDP queue at one end or the other.

25

Man, a UDP packet loss makes total sense! Good to know the internal server might work… I might get in trouble with my 2 gigs of sound though, but hey, that is for next year :smiley:

26

The server was not crashing. And, no–it’s different code than what is at the top of the thread, but the same idea. I’ll paste it below.

Do you mean rather than running at ServerProcessSlices, running at FluidBuf*.processBlocking() on everything?

Without the if(jobs.size > 0){SystemClock.sched(0.1, perf.value(jobID))}; that Sam suggested above? Or without the new .wait in Alpha 07 (which I’m not using yet)? Either way, I reverted my FluidProcessSlices class to not have the added things above to make sure I was testing on the same code. I tried it once today (using the default 4 tasks) to double check and it failed in the same way.

On the internal server, it worked great!
SuperCollider is not letting me boot at TCP server, so I don’t about that one. It’s giving me:

Requested notification messages from server 'test'
too many users
test - could not register, too many users.

// ========================== here’s the code: ===================

(
var folder;
//=============================== FOLDER WITH THE STUFF WE'LL BE USING... =========================

folder = "/Users/ted/Desktop/SCD/sample_space_maker/outputs/201228_222352/";

//==================================================================================================
Routine{
	var buf_path, index_path, buffer, index, files, dataSet;
	var mfccBuf, pitchBuf, specBuf, loudBuf, mfcc_stats, pitch_stats, spec_stats, loud_stats;
	var mfcc_flat, pitch_flat, spec_flat, loud_flat, master_buf, extractor;
	var mfcc_chans, pitch_chans, spec_chans, loud_chans, n_mfccs = 40;
	var stats_order = ["mean","stddev","skewness","kurtosis","min","median","max"];
	var names_order;
	var column_headers;

	files = PathName(folder).files;

	buf_path = files.select({
		arg pn;
		pn.fileName[0..1] == "01";
	})[0];

	index_path = files.select({
		arg pn;
		pn.fileName[0..1] == "02";
	})[0];

	buffer = Buffer.read(s,buf_path.fullPath);
	index = Object.readArchive(index_path.fullPath);

	mfccBuf = 4.collect{Buffer(s)};
	pitchBuf = 4.collect{Buffer(s)};
	specBuf = 4.collect{Buffer(s)};
	loudBuf = 4.collect{Buffer(s)};

	mfcc_stats = 4.collect{Buffer(s)};
	pitch_stats = 4.collect{Buffer(s)};
	spec_stats = 4.collect{Buffer(s)};
	loud_stats = 4.collect{Buffer(s)};

	mfcc_flat = 4.collect{Buffer(s)};
	pitch_flat = 4.collect{Buffer(s)};
	spec_flat = 4.collect{Buffer(s)};
	loud_flat = 4.collect{Buffer(s)};

	mfcc_chans = n_mfccs * 7 * 2; // 40 mfccs * 7 stats * 2 derivatives (no derivative & 1st derivative)
	pitch_chans = 2 * 7 * 2;
	spec_chans = 7 * 7 * 2;
	loud_chans = 2 * 7 * 2;

	master_buf = 4.collect{Buffer(s,mfcc_chans + pitch_chans + spec_chans + loud_chans)};

	dataSet = FluidDataSet(s);
	s.sync;

	extractor = FluidProcessSlices({|src,start,num,data|
		var label = data.key;
		var voice = data.value[\voice];
		var t00 = FluidBufMFCC.kr(src,start,num,features:mfccBuf[voice],numCoeffs:n_mfccs);
		var t01 = FluidBufPitch.kr(src,start,num,0,1,pitchBuf[voice],trig:Done.kr(t00));
		var t02 = FluidBufSpectralShape.kr(src,start,num,0,1,specBuf[voice],trig:Done.kr(t01));
		var t03 = FluidBufLoudness.kr(src,start,num,features:loudBuf[voice],trig:Done.kr(t02));

		var t04 = FluidBufStats.kr(mfccBuf[voice],stats:mfcc_stats[voice],numDerivs:1,trig:Done.kr(t03));
		var t05 = FluidBufStats.kr(pitchBuf[voice],stats:pitch_stats[voice],numDerivs:1,trig:Done.kr(t04));
		var t06 = FluidBufStats.kr(specBuf[voice],stats:spec_stats[voice],numDerivs:1,trig:Done.kr(t05));
		var t07 = FluidBufStats.kr(loudBuf[voice],stats:loud_stats[voice],numDerivs:1,trig:Done.kr(t06));

		var t08 = FluidBufFlatten.kr(mfcc_stats[voice],mfcc_flat[voice],trig:Done.kr(t07));
		var t09 = FluidBufFlatten.kr(pitch_stats[voice],pitch_flat[voice],trig:Done.kr(t08));
		var t10 = FluidBufFlatten.kr(spec_stats[voice],spec_flat[voice],trig:Done.kr(t09));
		var t11 = FluidBufFlatten.kr(loud_stats[voice],loud_flat[voice],trig:Done.kr(t10));

		var t12 = FluidBufCompose.kr( mfcc_flat[voice],0,-1,0,-1,1,master_buf[voice],0,0,0,Done.kr(t11));
		var t13 = FluidBufCompose.kr(pitch_flat[voice],0,-1,0,-1,1,master_buf[voice],mfcc_chans,0,0,Done.kr(t12));
		var t14 = FluidBufCompose.kr( spec_flat[voice],0,-1,0,-1,1,master_buf[voice],mfcc_chans + pitch_chans,0,0,Done.kr(t13));
		var t15 = FluidBufCompose.kr(loud_flat[voice], 0,-1,0,-1,1,master_buf[voice],mfcc_chans + pitch_chans + spec_chans,0,0,Done.kr(t14));

		FluidDataSetWr.kr(dataSet,label,-1, master_buf[voice], Done.kr(t15));
	});

	extractor.play(s,buffer,index,{
		"feature extraction done".postln;
		dataSet.write(folder+/+"03_full_dataset.fl");
		names_order = Array.fill(n_mfccs,{arg i; "mfcc%".format(i.asString.padLeft(2,"0"))}) ++
		["pitch","pitchConf"] ++
		["specCent","specSpread","specSkew","specKurtosis","specRolloff","specFlatness","specCrest"] ++
		["loudness","truePeak"];
		column_headers = Array.fill(master_buf[0].numFrames,{
			arg i;
			var stat = stats_order[(i/2).floor.asInteger % stats_order.size];
			var name = names_order[(i / 14).floor.asInteger];
			var deriv = "deriv-";
			if((i % 2) == 0,{deriv = ""});
			name = "%-%%".format(name,deriv,stat);
			[i,name];
		});
		ArrayToCSV(column_headers,folder+/+"04_full_dataset_column_headers.csv");
	});
}.play;
)
1 Like
27

ok I’ve tried with s.options.protocol_(\tcp); and then s.reboot and I still get the issue of ‘lost functions’. Now, there was an old bug in sc where not all com was going through TCP so I wonder if there is a way to spy on this on our side.

(fun trivia: found another ‘feature’ in SC: the local server does not support changing the number of buffer numbers…) I’ve tried with smaller numbers but it is hard to reproduce with only 333 threads in parallel :smiley:

28

I have the same problem with TCP as @tedmoore

Booting server 'localhost' on address 127.0.0.1:57110.
NetAddr-Connect failed with exception: connect: Connection refused
Found 0 LADSPA plugins
NetAddr-Connect failed with exception: connect: Connection refused
NetAddr-Connect failed with exception: connect: Connection refused
...
Requested notification messages from server 'localhost'
too many users

The internal server or the default (local, but not internal) server? Both seem to work for me:

(
s = Server.internal;
s.options.numBuffers_(2048); 
s.waitForBoot{
    Routine{
        ~bufs = 2048.collect{Buffer.alloc(s,1,1)}; 
        s.sync; 
        ~bufs[2047].query;
        s.quit;
    }.play; 
};
)


(
s = Server.default;
s.options.numBuffers_(2048); 
s.waitForBoot{
    Routine{
        ~bufs = 2048.collect{Buffer.alloc(s,1,1)}; 
        s.sync; 
        ~bufs[2047].query;
        s.quit;
    }.play; 
};
)
29

I may be mistaken, but I think the internal server is a legacy of when the local server couldn’t access the same memory space as the language. But now it can. I wouldn’t be surprised if it really is no longer supported.

30

ok I cannot get TCP to work anymore. I swear it was… but I must have messed it up and started under udp…

as for internal server max, your code works, but try this code:

s = Server.internal;
s.options.numBuffers = 4000; 
s.reboot

~statsbuffers = 1000.collect({Buffer.new;});
~flattenbuffers = 1000.collect({Buffer.new;});
31

Ok the problem was me… Buffer.new has the default server, not s.
Sorry for the noise.

Let me know if you get TCP to work. I’ll try my 1k buffers now on internal

32

on internal, I still lose stuff with the code I sent you @weefuzzy
now time for calorie-intake

33

I’ve not yet found the complete documentation for all this, but AFAICT the shared memory interface seems only to be about being able to write synchronously and directly to control busses, whereas (my understanding) of the internal server was that any OSC commands to/from the synth are no longer mediated by a socket (hence my hope that it would help narrow down whether this problem is dropped packets or something else).

34

sadly not… I’m trying to get TCP to work now (at least trying to confirm it is a known bug)

35

I still can’t reproduce with the internal server, so I’ll try and make a more compact reproducing case to chase down.

36

ok so it seems that TCP actually works after a few refusal:

so if I can trust it is in TCP after the few expected failures (IIUC if you don’t get 20 refusal as per tryConnectTCP maxAttempt declaration then you’re in!) the sad news is that it does not solve our problem.

37

Doesn’t work for me at all. NetAddr throws primitive exceptions about a broken pipe after I get all that garbage.

1 Like
38

IAC, if it’s not dropped packets, I’ll need to spend some time making a debuggable test case.

39

I just get 3 x

NetAddr-Connect failed with exception: connect: Connection refused

then I get

exception in GraphDef_Load: UGen ‘FluidBufPitch’ not installed.

which is strange but everything else seem to work. I wish there was a way to know it is really in TCP… maybe I’ll start the terminal scsynth in TCP and that way I’ll be certain…

40

ok this is helpful a bit

TCP mode in SuperCollider | TAI Studio

but I cannot manage to make the new server to not freak with

FAILURE IN SERVER /g_new negative node IDs are reserved

when it connects to SC. but with

./scsynth -t 57115 -b 4000

and the following code

(
q = q ? ();
q.netAddr = NetAddr("localhost", 57115);
q.netAddr.tryConnectTCP;

q.server = Server(
    \test, 
    q.netAddr, 
	ServerOptions.new.protocol_(\tcp).numBuffers_(4000);
);
q.server.startAliveThread(0);
q.server.doWhenBooted({ 
    "Yo".postln; 
    q.server.notify; 
    q.server.initTree 
});
CmdPeriod.add({q.server.freeAll;});
q.server.makeWindow;
)

(which is his but corrected with the notify and initTree with the right server)

but I cannot get it to deal with 4000 buffers. So I will park this for now as I now hate SC.