Does SC work asynchronously?

Are there benefits to using SC for batch processing over Max for asynchronous processing or is it all locked into one thread right now? I only ask because I want to do NMF on a big database but I cannot in good conscience run a python script on my macbook for a whole day at 70 degrees. I am however willing to abuse my desktop which can VM macOS (but won’t run Max within a VM).

Furthermore could CLI be compiled to work on windows alongside the CCE externals/packages?

:slight_smile:

this code should reliably parallel process a folder with up to 800 wav files in it, running on 8 parallel servers and write out 3 aiff files with the nmf data in them. it should actually do more, but if it is going too fast it could conceivably (though very unlikely) wrap around and reuse one of the analysis buffers before the aiff files are written. the .wait is only necessary if the analysis is somehow processing over 100 at a time.

~numServers = 8;

~servers = Array.fill(~numServers, {arg i; Server.new(“s”++i.asString,NetAddr(“127.0.0.1”,57110+i)).boot});

(
~files = PathName("/Users/spluta/Documents/SC/FluCoMA/Mahler/Mozart/Mozart - Symphonies Nos. 35, 38 & 39/").files.select({|file| file.fullPath.postln; file.fullPath.contains(".wav")});

~files = ~files.clump(~numServers).flop;

~destBuffers = Array.fill(~numServers, {arg i; Pseq(Array.fill(100, {Buffer.new(~servers[i])}), inf).asStream});
~basesBuffers = Array.fill(~numServers, {arg i; Pseq(Array.fill(100, {Buffer.new(~servers[i])}), inf).asStream});
~activsBuffers = Array.fill(~numServers, {arg i; Pseq(Array.fill(100, {Buffer.new(~servers[i])}), inf).asStream});
)

(
~servers.do{|server, i|
Routine {
var file;
i.postln;
~files[i].do{arg file;
file.postln;
if(file!=nil,{
Buffer.read(server, file.fullPath, action:{arg source;
FluidBufNMF.process(server, source, destination:~destBuffers[i].next, bases:~basesBuffers[i].next, activations: ~activsBuffers[i].next, action:{|dest, bases, activs|
[server, dest, bases, activs].postln;
dest.write(file.pathOnly++file.fileNameWithoutExtension++"_destBuf.aiff", “aiff”);
bases.write(file.pathOnly++file.fileNameWithoutExtension++"_basesBuf.aiff", “aiff”);
activs.write(file.pathOnly++file.fileNameWithoutExtension++"_activsBuf.aiff", “aiff”);
});
});
server.sync;
1.wait;
Main.elapsedTime.postln;
});
};
}.play;
}
)

aaaaaannnnnd, you probably want to tell it the correct number of components or the data will be kind of useless

increasing the server mem size may also help…i don’t know:

(
~numServers = 8;
Server.local.options.memSize_(800000);
~servers = Array.fill(~numServers, {arg i; Server.new(“s”++i.asString,NetAddr(“127.0.0.1”,57110+i), Server.local.options).boot});
)

(
~files = PathName("/Users/spluta/Documents/SC/FluCoMA/Mahler/Mozart/Mozart - Symphonies Nos. 35, 38 & 39/").files.select({|file| file.fullPath.postln; file.fullPath.contains(".wav")});

~files = ~files.clump(~numServers).flop;

~destBuffers = Array.fill(~numServers, {arg i; Pseq(Array.fill(100, {Buffer.new(~servers[i])}), inf).asStream});
~basesBuffers = Array.fill(~numServers, {arg i; Pseq(Array.fill(100, {Buffer.new(~servers[i])}), inf).asStream});
~activsBuffers = Array.fill(~numServers, {arg i; Pseq(Array.fill(100, {Buffer.new(~servers[i])}), inf).asStream});
)

(
~servers.do{|server, i|
Routine {
var file;
i.postln;
~files[i].do{arg file;
file.postln;
if(file!=nil,{
Buffer.read(server, file.fullPath, action:{arg source;
FluidBufNMF.process(server, source, destination:~destBuffers[i].next, bases:~basesBuffers[i].next, activations: ~activsBuffers[i].next, components: 10, action:{|dest, bases, activs|
[server, dest, bases, activs].postln;
dest.write(file.pathOnly++file.fileNameWithoutExtension++"_destBuf.aiff", “aiff”);
bases.write(file.pathOnly++file.fileNameWithoutExtension++"_basesBuf.aiff", “aiff”);
activs.write(file.pathOnly++file.fileNameWithoutExtension++"_activsBuf.aiff", “aiff”);
});
});
server.sync;
1.wait;
Main.elapsedTime.postln;
});
};
}.play;
}
)

kernal task isn’t a fan of 8 parallel servers…

This is really great. Thanks for the code - I’ll try get it to run my end. Does it make your fan try and take off?

yes. fan is going bonkers.

let me know if it doesn’t work for some reason. it does take a second for the servers to all boot, and you will not get a visual that the servers are running on the lower right of the IDE. but you should see the files all magically appearing in the finder.

1 Like

I think this will be far superior to running Max instances through Xcode for parallelisation… thanks for your help.

1 Like

@spluta will convert us all, one at a time…

1 Like

If only the bracing wasn’t so icky

I found a bit of a tricky “feature” of SC. By default, it will save buffers as 24 bit integers. Fine for audio files, but not so good for things like MFCC or Spectral Shapes. Anyhow, to save 32bit floats, you just add the “float” tag after the header type:

dest.write(file.pathOnly++file.fileNameWithoutExtension++"_destBuf.aiff", “aiff”, “float”);

This messed me up for some time.

Sam

2 Likes

What is the default format of a .wav file?

The default for either is an int24. This is fine for anything sticking between -1 and 1, so NMF is fine. But for values that go above or below -1 to 1, it will just clip it unless you specify float.

It’s interesting that SC records live as float32 and dumps its buffers as int24…