Better way to deal with long string of arguments

In the Training for real-time NMF thread, @jamesbradbury made a comment about how tricky it can be to use fluid.bufcompose~, which I echoed:

I understand that this approach is computationally efficient, but it is a massive eyesore, means nothing when reading the message, and always requires returning to the help file.

Man, just looking at the helpfile for fluid.bufcompose~ again, and the message format is crazy long (18 items!), with many of them being either 0, 1, or -1.

process srcBufA <startAtA> <nFramesA> <startChanA> <nChansA> <srcGainA> <dstStartAtA> <dstStartChanA> srcBufB <startAtB><nFramesB><startChanB><nChansB> <srcGainB> <dstStartAtB> <dstStartChanB> dstBuf

Maybe, as @jamesbradbury suggested, something like the MuBu syntax:

That could get a bit message as it can get confusing with @attribute names, especially since you would be often changing things that may also be attributes.

At minimum, maybe using words instead of numbers, to at least add some legibility:

process something 0 all

I figured this was worth mentioning in the ‘early days’ of the project, before this approach becomes entrenched/solidified.

1 Like

I understand. I’m told by @weefuzzy that I’m the only one using it so fluently (or at all)… I should create a Discourse badge ‘BufCompose Ninja’ :wink:

This is more a sort of extension of jitter, and I think we should stick to more Max native stuff, but I take your point that we need a simpler syntax. In the meantime, 14 of these 15 are redundant:

sourcebuf with the usual 4 target defining a range, then gain and offset of both dim
same thing again for the 2nd source

I am really openned to a suggestion, but I don’t think there are many use cases when one does not have to define a lot of those, so adding a lot of words make it more readable, but longer.

I like SuperCollider, which has both: you either use them in order without label, or with label in whichever order you like. What do you think of that?

The game: : send me your bufcompose impossibilities and I’ll show you how to do it. It could be good to build up other people’s use case since I am the poweruser but also the interface designer so it is simple to me - the primer above should help make sense of the actual 3 arguments sourceAwithboundaries sourceBwithboundaries dest

The game is not about adding feature like blurring, cross-fading, reversing, etc. These are other compositing options that we do not cover for now. Just assembling buffers as they come is this object’s agenda.

Go!

2 Likes

Well you’re a couple toolboxes in of the same (illegible) syntax, so one would hope!

Doesn’t matter if it’s conceptually redundant, seeing a list of 1 0 0 1 1 -1 0 1 is a list of eight illegible arguments, regardless of “four of them refer to the same thing”.

If there are redundant messages required, then the problem is in the format of the message required…

At minimum, that’s a start. But I would propose a rethinking of the paradigm. For example, for my use cases 99% of what I would want to do could be satisfied by having keywords (concat, merge, stack, etc…).

For example:

  • concat bufferA bufferB = would concatenate one (entire) buffer at the end of the other (entire) one
  • concat bufferA bufferB bufferC = the same, but it should take an arbitrary amount of buffers, which would be concatenated in order
  • merge bufferA bufferB = write both buffers in place (at the start, no gain modification)
  • stack bufferA bufferB bufferC = create a multichannel buffer out of individual buffers

Beyond things like that, if you want to deal with offsets, gain multipliers, etc… then you can get into having to declare everything (using what you’ve suggested from SC).

I could be wrong, but I think the exact opposite. Most of what needs to be done (in a straight FluCoMa context) would be building up of whole buffers, or creating aggregate dictionary buffers etc… where “gain” isn’t a thing, and neither is length/offset as each buffer would have been created as a specific length.

In the case where you’re creating “custom” buffers (for the helpfiles etc…), then you’re playing with gain and offsets, but I honestly don’t see that as being the core use case of the object.

Putting this as a separate reply to decouple the feature-request-y bit from the specific use cases.

  1. So in the case from the other thread, wanting to create a single multichannel buffer from an arbitrary amount of individual buffers (with no manipulation of gains/offsets/etc…)

I’ll go back to trying to fix what I had, but this is what I posted in the other thread:


----------begin_max5_patcher----------
927.3ocuWtsiaxCD.95jmBKK06XQ1FLG9eUpph.hSpq3j.S61V09r2wClbXW
RVRW5etvAlX6Y9lS14ma2PyadV0SI+G4ijMa941MaPQVAabuugVk8bQYVONM
ZQSUkp1P8F+Mi5YCJOqpYn1PZNP5K0Evd5lfdO9yM4e4IQxjv5gJccoxfaI+
rvlAyjTlSZalo3y55i65TElQCUvX9LORnzNlhiBPB4S1U7qsasCdKDGPo4pt
4L1H5IKnKqRYTc6T0Y4kpKstE.hvIcTj46spQJnTOBMOq9Hk7oayJOLDYkaG
kH2BweKqpuAj8pH2SD9b7GLavRLKi7awnFxTtKdAg9BoGgKRQzR8kuC.yGLl
l54fQrRAyaB5aGH4BjzwHoHbZbMijC+PSDyFLYOPvL3tL5c962L35PNI9bJ7
Zm7ZHZhdFh4ORuFw8Re8dfr3X1JjEeCRa6zma6dIowqSW0HmsK84QgQwhDIW
FYoJP7uHvAmijqqUD.Jxt5pC6NnKMy1IlG8.Iu2sa6hBhBNlnN0QZkwtGCiG
HscMvgj8jOzSXjm3v.mXG+vdxYmwt9gpW75bE2qQi5k4ZjA9blLIUllFwRCh
sYGR4+B+zgxA8d+7gCPdRaSu52y.dfj9+Rm6I5ecoAmkLK731QKg76Wb2Jb6
sxu1iz2LzULYPtXJjLLYT6U8FcclQCGtcdNV7uXRMc6gxGaM9rQg0Ty1VNyo
Y96Ry1V1PMv8UsXbNuODWBi1iLIr2olPjdSuYzZnonkvzJnHzuHdKEErFZJX
INuj0PSQKMgfOSYdVa6WUc8tYiJAZw8kFrjHwCeUWO9Jd9KsS8U8z7w9zzrN
nqiAZ4LzgFF84nP53Rafhq5Asq9BvCTYu46kur0RMbuVboP60HaOWjF.lCYC
klq8.4GKZJGsHn+Fdb2oAlu7jmv1NFm4NcskQmSiMNuoAt+YeWVQA7uCuX2Y
9I1okDjJ3Q1mhYoIrP2pFcldtnUGrTDMjCG.Uf0qm5qcYz8Jfw1+2C4qLoPQ
fTjZslfXgLDeRDGJ4qsYUAmwmcTcOC6w8w4Gg6CTdBnMm1KXybmqQO1ksWe9
+o63meJlDAeh8l8oKTkaYhokEmj.dNuYe50KaZUhTFOEi8AtXu8IPD+EqBNI
dzIKvKR6jBWUpsoapZDVZ5o4OXZNA5zIeiEm2L7cc0wxhhi2YXwYWXLKDRqX
B7IfYvl+ayrFaxfWXxZT8sYi8tv6Us8Wa+i.8Iwg
-----------end_max5_patcher-----------

It doesn’t work (but the patch in context is biggish and a bit confusing in terms of structure since there’s dynamically created objects and externals)

So that’s a super simple, seemingly super obvious use case, which requires a pretty messy argument strand (sprintf process %s 0 -1 0 1 1 0 %d nmf_filter_sum nmf_filter_sum).

  1. This one was solved by @weefuzzy just after the plenary, but it seems like concatenating a buffer from lots of individual files (a folder specifically) shouldn’t be as (relatively) complex as it is.

  2. Splitting a big multichannel buffer into individual (poly?)buffers. So being able to take a higher @rank resynthbuf’s output and then splitting it out into individually addressable buffers for easy play in the mc. universe (and/or easier individual treatment/processing).

YES

Predefined routines with parameters makes more sense to me. It’s higher level, but to me it would satisfy a multitude of common use cases.

2 Likes

Scenarios

  1. Take n-buffers and copy their contents into successive channels in another buffer. If the source buffers are stereo, then it copies to channels 1, 2 - the next buffer would copy into 3, 4.

  2. Extract channels from a buffer and separate into n-buffers with a defined amount of channels. 60 channels in master buffer could be converted to 2 30 channel buffers, or 60 1 channel buffers.

  3. Concatenate the contents of many buffers into one long linear buffer.

1 Like

that is true. but also, SC and Max are fighting a good fight in my brains now…

I know. I was trying to be useful. I struggle too. but I’m afraid of the overhead of flagged args.

they can become abstractions - that makes the paradigm much simpler and much more flexible I think. This list is useful, let’s grow it and I’ll do the calls - and the comments to explain them so they become tutorial too!

that does not surprise me - this is why you’re here :wink:
I think that you can see in the many, many use cases of the tutorials that they are used in many different ways. A MS filter is the geakiest so far :wink:

OK you two gave me a few minutes of full brain coding. I will reply tomorrow afternoon - now I need to recoup the 3.5h night, and I corrupt the youth for 4h tomorrow morning. I’ll do that as a matter of priority, so we get the ball rolling.

Also, keep sending an expending case use for keywords, and that will help on many fronts here (and I’ll code them for you if that helps, although I reckon you can code most of them already :wink:

In terms of ‘style’, and I could be wrong on this but, I don’t know of any objects (specifically modern ones) which work via this super long string/flag argument message style… except HISS/HIRT/FluCoMa. So I don’t know if it’s very Max-like at its core.

That would definitely be useful, but if the underlying “message” is still a long list of arguments, it doesn’t become any more flexible, as it is always confusing. For example, I’m sure you could also solve those same problems in C++, but that does little for a general purpose toolbox you are sharing with others. If every time someone wants to use the object they have to open the helpfile and squint at a list of arguments, then touch their finger to their laptop screen to count which inlet of pack (or whatever) they need to connect to. I don’t think I’m alone in having to use the object in this way…

Are you conceptually/fundamentally opposed to having a ‘keyword’ system for commands? (this is something that would/could apply to all buffer-based objects which expect long strings of arguments, not just fluid.bufcompose~.

Yeah, in cases like those, all the arguments should be used. But those cases are not the ‘norm’, and the current messaging syntax requires behavior from everyone, that is only required/desired by a few.

In terms of another model there aren’t really any objects that use this many arguments as it is. Something like peek~ which can theoretically address the functions of bufcompose~ doesn’t even come close because bufcompose~ works at a more abstract level. I really think that keeping the old syntax (with some more nuancing) in conjunction with some compound functions would be really nice e.g. stack, split, lineup

2 Likes

check the set message to oscilbank, or line~ - it is in the paradigm but not super convenient, and these latter two are pairwise so easier to remember the syntax (still very prone to error!)

the more we black-box commands, the less we share knowledge and flexibility - hence a bpatcher to make these message a reality for the user but keep the underlying code clean. Check the Bach abstractions, they do that all the time, and this allows a user to learn the power uses under the (accessible) hood… because there is always a function that almost does what you want and then the list becomes endless.

I was very ill yesterday after our exchange, so the coding promised for this afternoon will have to be tomorrow morning, but it’ll be soon

It’s not really a black box though. Something like concat bufA bufB literally does what it says, the whole string of arguments that would be required to do that with fluid.bufcompose~ now have nothing to do know knowledge, or flexibility. They just add needless confusion.

What I’m suggesting doesn’t make things any less powerful, it just simplifies common use cases.

It seems like you’re not interested in changing the messaging syntax, which is your call. I’m just giving my feedback as a user that it’s confusing to the point of uselessness, so if that’s the cost of “flexibility”, so be it…

(I’m also not a huge fan of having tons of small abstractions as makes it more likely there things will not share/copy/paste easily as it’s easy to forget what dependencies you have, if they are global (i.e. part of a package))

Hello everyone,

My 2 pence: tl;dr I’m not a fan of these big long argument lists either, but I see BufCompose as being a seperate problem, because it’s fundamentally unlike the other things. So, in two parts, here’s where my thinking is going as I’m refactoring the code.
Non-Realtime Things that aren’t BufCompose
It’s never encouraging to be confused by one’s own design. The current alphas of the go through quite a lot of acrobatics in order to treat NRT object’s properties simultaneously as attributes and arguments to the process message, and I don’t find I like the results. My suggestion would be to simplify things greatly (but not everyone might like the simplification):
Proposition 1: insofar as NRT objects are (kind of) just real-time objects that use buffers rather than signals, the only arguments to process will be source and destination buffers (with optional offsets and extents, still, probably), and everything else will be a named attribute, and only a named attribute
Proposition 2: the whole business of instantiation-only properties is also confusing, and – arguably – just an implementatation detail; wherever possible, I plan to get rid of these, BUT that means changing some things will cause a (documented) fart/dropout, and in some cases it will fall to you to specify maxima in advance (like with [delay~])

BufCompose
The balance here seems to be between having something hugely flexible, almost like a domain-specific language for buffer manipulation, and having things you can reach for to accomplish the common things. So, another two parter

  1. Similar to @jamesbradbury’s suggestion, but I’d been thinking of packaging common operations into abstractions rather than adding more messages to the object. I think this would make more better self-documenting code at your end, as well as embodying some robust usage examples.
  2. To try and get a better trade off between scalability and legibility, one thought is to approach the interface to BufCompose more like [fffb~], in that it would expose messages taking arrays of arguments, e.g. sources <buf 1><buf 2>...<buf n>; startframes <start 1><start2> and so forth. Upside: we add scalability, and some help in making code clearer. Downside: the object is now stateful, and people still have to keep proper track of supplying the right numbers of offsets, destination channels etc. We could, I guess, go further, and look at how the user-side structured data representations in Max and SC could help, but the more of that we do, the more we risk baking things into narrower ranges of environments.
2 Likes

This seems like a good middle ground of flexibility and clarity in the messages. I like the idea of possibly being able to separate out the messages too. One scenario might be…

sources <buf1> <buf2>, startframes <start1><start2>

where are arguments can be comma separated and be treated as individual entities. This would make the actual processing of building your bufcompose~ process up more modular, segmented and readable.

Sorry to bust balls especially as I’m not a composer on the project, but its fun pushing the envelope with these things :slight_smile:

1 Like

Yeah, I agree with the separating of the parameters that are being controlled. Also having words in there (sources, startframes, etc… is massively helpful for legibility).

I presume that much of that could be optional? (so if you don’t specify startframes at all, that it would start at 0., etc…) Otherwise it still ends up being a hugely long message, just in a different (albeit more legible) order.

For the NRT stuff, I think I prefer Proposition 1, as it’s a step towards simplicity (without removing flexibility) and I think having to specify maxima can be a bit offputting, especially if you’re not sure what or how much you’re expecting.

As a “composer” on the project, I definitely appreciate (someone else also) busting balls, but then again, it’s not my balls being busted…

Such is the miracle of Max message boxes, that you get this for free! As ever, we’d need to make sure the SC equivalent was also non-horrible, but IIRC arrays can be expressed as [stuff, stuff, stuff], so – again – it should be possible to highlight the different, chunks(?), of the overall message.

This isn’t ball-busting, it’s exactly the engaged, critical and thoughtful feedback-through-experimentation we need. I mean, now that I’ve gone off Max and intend only ever to compose using abstruse C++ techniques, it’s more important than ever :grimacing:

3 Likes

If you’re not composing in Fortran you’re doing it entirely wrong.

1 Like

Ah, that would explain my decline in musical quality, right enough.

1 Like

Another one for the list.

So based on @weefuzzy’s feedback in the “other thread”, part of creating a training set would be to sum a bunch of dicts to create an aggregate dict.

So this use case would be having an arbitrary amount of buffer slices (delineated by fluid.buftranslientslice~) which need to be written somewhere, then summed together, before then being stacked up into a multi-channel dict.

This is easy enough to do with individual buffers for everything… if you know how many transients you will get. If I’m feeding it some training audio, it may contain any number of hits. So I guess stacking them end-to-end in a single buffer, but keeping track of them (either with fixed durations or if the buftransient~ objects ever start spitting out “transient offsets”, then variable durations too), then once that process is complete, summing all of those together into a single buffer.

I guess this would require three individual buffers? One to concat all the transients into, another to sum them to (or can this happen “in place”?), then finally the actual multi-channel buffer that they will be stacked into, to feed fluid.nmfmatch~.

I have to look at the patch, but I think you can get the handles from transientslice and process bufnmfon that slice (without copying it, since we can do that with the process message) and add in place in the right channel of the dest buffer the dict you are accumulating.

I might read you wrong though.