Another noob question on scaling non-overlaying corpora

[agustinissidoro]

Hi all there! I am glad to be participating for the first time in this forum, and as it couldn’t be other way, I have a very noob question I hope someone could help me understand and tackle a better solution.
I have scanned through the related topics and I know there have been many approaches and discussions regarding the topic of scaling non-overlaying corpora. From reading most of them I have come to the idea that there’s probably not a straight forward solution nor a universal one for every project. Unfortunately, I understand very little of maths and only the basics of AI/machine learning solutions to be able to work around a solution by myself. So here are some of the characteristics of my project just in case someone can think of a possible approach:

I am writing a piece for doublebass-flute and live electronics. I intend to do some live mosaicking with concatenative synthesis. The corpus I am using is pretty varied, all sounds coming from playing around, hitting, bowing and scratching a cheap aluminium pizza plate (and captured with a contact and condenser mic).

To ensure that most of the corpus is available for playing despite a direct match, I intend to scale it.
In order to do this I was thinking of:

1. putting the analyzed corpus (descriptors and mfcc) into fluid.robustscale~ (as I read this would be better for non uniformly distributed data)
2. doing the same with a analyzed target audio from the flute
3. training a regression engine to fit the scaled flute descriptors with the raw flute descriptors
4. feeding the live input analyzed descriptors to the regression engine and finally using this to find a match with fluid.kdtree~

There’s probably more than one conceptual mistake here so I apologize for the very noob question. I also apologize for duplicating topics. I felt my question was way simpler and steeper than all the exchanges that took place here and so I thought of begging a new one not to mess up the other ones.

I hope someone could give me a hand here!

Thank you very much and greets!

[tremblap]

Hello and welcome

This is definitely not a noob question! It is actually one of the core questions of the whole project! So I will try to unfold a few ideas if you don’t mind - some of it might sound obvious, some of it cryptic - come back to me with what is not clear and I’ll try to help.

The workflow you describe is far from trivial: overlapping such different sonic material for musaiking implies a lot of very interesting questions and challenges.

1. describing sound - what you care about for a musaik to be ‘right’ depends on how to describe it - this is contextual and very much dependent on your tastes and musical intention.

2. how often you query for that is also not trivial: A full gesture? a granular stream? every attack? when it is loud?

Just talking about this could be really fun and would help me help you on how to get going with audio query… if that all sounds too complicated, let me know too, and I might have another approach to your quest.

Looking forward to hearing from you

p

[rodrigo.constanzo]

I can offer some of my experiences trying to do similar things.

As @tremblap mentioned, the descriptors (and stats) will have a big impact on what you find. So picking descriptors (and stats) that are conceptually (and musically) meaningful to what you want to differentiate. For example, pitch may be important in your bass/flute, but perhaps quite static in the corpus of pizza plate sounds. So you may get a bunch of bad/junk info in the corpus which the matching will try to make sense of, which in my experience leads to a “stuck note”-type sound since the bad pitch data tends to all come back as a single value (0 or -999 or whatever).

Similarly the amount of each type of descriptor/statistic matters too. Say if you take MFCCs and stats, you’re looking at hundreds of values right there. Whereas you may have only 1-8 (depending on settings/stats) for loudness, meaning that in a nearest-neighbor matching sense, the loudness is almost irrelevant in the sea of MFCC numbers.

Towards that end, dimensionality reduction can be useful (e.g. making a reduced “timbre” space out of the MFCC/stats), but this can be a whole rabbit hole in and of itself. There’s some experiments/tests in this thread here.

A simpler solution/approach is to try just using descriptors/stats at the start as those are, roughly, in similar amounts. And depending on the units you use, can also be in similar scale (dB and MIDI pitch for example where 1 “unit” is roughly/equally perceptually different). That will let you get an analysis/process pipeline going and test other parameters like the playback speed, grain size, and other things that @tremblap mentions.

As perhaps a not so immediately useful aside, in my experience I found I get better (audible) results by doing no scaling of the data spaces at all, even if they are vastly different. I tried with robust scale, standardization, and normalization too. Again, this is likely specific to my approach/descriptors/corpora/input, so take that with a grain of salt. But I did want to say that “getting it working without fancy stuff” is usually a good place to start.

You probably don’t need (or want) a regressor at all in what you’re doing. You can just scale your corpus, then create a scaling for your input space (then transformpoint individual descriptor chunks) and fit a kdtree with the corpus and just find the nearest point from your scaled inputs.

[tremblap]

@agustinissidoro I’ll make a little patch later, as simple as possible, to showcase the moving parts. That should get the discussion going with some sounds

[tremblap]

@agustinissidoro let me know if you are on Max SC or Pd so I can use the right one…

[agustinissidoro]

Hi! @tremblap and @rodrigo.constanzo Thank you both for your answers, they have been of great value. Also, sorry for the delay in the reply, I am in Argentina, some hours earlier.

Dear @tremblap, I am using Max. Also, your questions really got me thinking about what I am looking for as regards the electronics in my piece and particularly what I want to achieve with the mosaicking. I will try to answer as clearly as I can what I have thought so far:

1. A good or right mosaiking to my purposes should be one that makes available the most varied sounds in the corpus. I like to think of a corpus as a whole complex identity, each sample being a particular, yet related, expression of that. With this in mind, a good mosaicing should be able to render this complexity audible as much as possible. This pseudo-spinozian explanation is the best I can do to describe my purpose. For the technical part, I am not so interested in direct matching, perhaps a “rigorous mosaiking”, as much as in having a consistent matching. This is perhaps the main reason I thought of scaling in the first place, but as @rodrigo.constanzo mentioned, I am probably wrong at this.

However, this poses a problem: the actions of the flute for the mosaiking part are mainly characterized by noise-air sounds combined with frulatti, something very “gestural”, but with little variety in terms of audio characteristics (or so I guess…). The “musical target” here is to have a smooth complex texture composed by the gestural flute actions plus the timbrically-colouring granual mosaiking. @rodrigo.constanzo 's contributions resonated to me here: what descriptors should work better here to grasp meaningful changes? Almost certainly not pitch.

2. This kind of mosaiking I am describring should play for full gestures, almost non-stop and ongoing for the purpose of the smooth texture. Nonetheless, I have planned to include other kinds of electronics behaviour for other events, for example, the kind of rodrigo’s corpus playing in the sp library for loud attacks, but this should be trigger by an entierly different engine.

I am probably biased by the “getting it working with a lot fancy stuff” idea you mention @rodrigo.constanzo and so I will definetly try a more straight forward approach; I was giving too much importance to the preconception that playing the most varied sounds out of the corpus with small overlapping space would not be very effective, but I will follow your advice and try things as you suggested. Thank your as well for the explanation on scaling and discarding the need of a regression part. Also, I am learning tons with your sp library and it sounds amazing!

Thank you both for your kind answers and I hope I was clear enough while explaining the characteristics of the project.

Greets!

[rodrigo.constanzo]

With what you’re describing, I would try different combinations of “vanilla” descriptors (e.g. centroid, loudness, flatness, etc…). MFCCs are great for timbre, but aren’t super great for material that is noise based (as in noisy sounding).

That’s great that you’re finding SP Tools useful. sp.concatsynth~ is essentially my solution to this using FluCoMa, though I prefer the overall sound and performance I get with C-C-Combine, which is completely different under the hood. The abstractions related to concat stuff in SP Tools are a bit more complex than necessary since there are loads of arguments/abstractions up and down the chain, but the core architecture is probably ok to look at in terms of structure.

It would make an easy jumping point to experiment with the playback parameters though. To see what kind of window size/density you want, etc…

[tremblap]

@agustinissidoro this is interesting - but you still forgot to tell me when you want to match: as a continuous flow aka a granular process? every attack?

as for the matching, I would start with something simple, to hear what it yields, and then post here again with what is working and what isn’t in line with your musical explorations. This is exactly what @rodrigo.constanzo did and it worked out ok

As a start point, let’s modify the patch from the browsing tutorial. Incoming.

[tremblap]

This is ugly code, but there are a few variations from the 4th step of the tutorial

1. I changed the centroid to a log centroid because I like that descriptor more
2. I removed the default audio in and I analyse as a stream of descriptor. Way too fast but that is just to show one dirty way. playback can be done via a metro, or attacks, or delays after attacks, etc.
3. I robust-scale 10-90 instead of full range normalisation

if you go through the numbers it should work, and start creative question - what you will get and how disapointed you are will enable to clarify the questions (how to trigger, how to describe, how to playback)

I hope this helps. I’ll use that in a session tomorrow in the same workflow

p

----------begin_max5_patcher----------
5680.3oc68z1bixijeNyuBNWyGta27jE8BucUcUk62wtWMEwF6vyP.u.dlIy
Va9seB0BarCuHLRXkYHSUIi.an6VsZ0cq9k+0mta0SY+HpXk0+s0e25t69We
5t63Wp5B2IFe2pWB+w5jvB9Ga05rWdIJsb08v8Ji9QI+5dOXUTFlWZU9bjU3
gMwYV+yCQ4uV+ASO7R1gxjnR9iwVb03M7ub1S+4ePw0ez8gkqeNNc2WxiVWB
vlqC8A66snXR0ePN7+fsev15+6zyONs9wipt1+9Sep5W2KIhwd.OEk2N3haA
bQm.27vWhJix+RTZ3SIQMwO3YT959H.OVs5dqUOElta0Q.uEr0A4TgeNNA7+
XyQVrRQ1nuyPh2MItKrLpcJ.58T.h2pNwx9vNWBG6ndb7hf6G6vWA1UlsaWx
HPDmqZpLlsJnW7DyQMJhOKho0+V6yh64q7hGAyLAupOF19QSafY01k+G2AXV
a9fXPHfbBr6tUaiSh9VTdQbVZiO8cqB2uuwkuqwWohj7mY7Gj+8GuTbJbIxw
KkG8s3KerrmaNCcJY3xgbNBu5GtzUmdLYahxSODeb9gO4H.I9zPJicoXe3Z3
KWMaUe6SDHjMBUQTB7q9sKky1S8aH7hMOuKIa8Wi1zfgiMWrOJMNcedTASfa
Xo.3Od6MQaCOjT9ksYokEw+jCBnJVtVt+VAH15MqPBN7++lGGlbDC1kGuIKs
BHNapn5x0uNFx4.BjahL7OQZ39V9xLlMFcoiax1+n7PwSg4UyThUe35aVlkk
b9sN98Rh1VJt893zzKnhkY669l4w6dtmu6SYra9ReOa9cJ9xgT3tegwTT9kh
vucN0tLLIQrX87G+OBSiegIzsLFlBv1GuIHA54h04YIImguvc9VK2YCiIecz
2i2T9L+E0jYf8wi2WyDs53r7l3cQEkmesxvcEmekhxWAhdiKc3Iwh3uTF8x9
jpsNN6CvVdDWTV7b12KDevZFslDfSZezbQcSIemc89j.dtTvblnrHqulFElW
gfM9HsJN7RQh9AM+J8KVrKQiHDrEGhKTz24RQicra2c06DHD3bkzF.l6Fwsa
GwQnURgUTPpFwt92ciUnF2nV+wlrJbHHcSzOZHnUIjfdXOXhUYxe1X8LSBPR
kTftITn1ITAN8ygLH2gOeCABnkGaWhgYOlIZy17rWJd8kmxRFOQgNQhhXICP
T7lYZxUtjAiWI0TstVxfmI1hjCwad3qaJyihdyJIK6qG1OZ4pHWUHWELZHfS
HQHx7xkvT4qHbWTqzn58arJVGlDsooIvxuHxepqg.xiCnaFlLu66zCGTo0Si
lXzuT1yrkdHBxIFECQP6SG1tMJ+slLKVOVD9x9BK7nWY0uj2sIYgkuy6CcQw
nf0bHOXuIOGyYEVYdXZw1r7W1mwr.GbxzjVsQTxpMDgKhG46YLq1.404YOcf
oFdEA5Mqxv7cQvfbqGSx9NSuCqGqzAxJvdzbbApSVNxM3AGFcyalkkyepilk
AoDVFOOY0FvdpJPex37j3ztL4hCoU2uc5VQ1g7003X8V4VmC0LyLKiSO5qh+
9Iaop9fRMuMVnn9gOHTTou9Eevr7MfOnruwfFsaPCoEPqk2X6fVk4fZatqko
j1ghJMr0GGjrjBWMBDxRI704zgrKnczHPPjcMiNmMPRBDjNgAwEq8W7pJeBt
4Kf+O+RXYYd7SGJAIwMc.9n7S2tjrmBStv4Zs4FuOcB3F4QLz0w849.nyE+3
9.sJrXJlsKR5y6C62y484E.VJiCz148wH+kLhrrGRD95NuuAOqOOWRCLcVOk
n+wpTlZzgIwEb1Z9DJC39GqjdND06bnKWKGhKLGRBVNYnKOYHBmK2yNX4jgV
NYnkSFRkmLjNLrSJWm.1yAG4yrXam17bvlvxvhnx2rp2lHZCrS+3c5qMY5dJ
.6CTVaZkmBPdFjWe2FWdzsT09XIjw4GMBRWWGnxDcFLvBBJYfIzEmSMFVNRS
Ntq12TZw4KA9ejrSJv+WcCkFYLmgzULmoaqINWLXivsnx3Oq+.YgjmFXecAP
HwFTTl50PzlRCfvNMYJLML4UlxKxGagtqtZSCEAPHwkakf6PAJ4ue1O4Gvsw
.zKvANqJuEymVLeZw7ooZ9z0FIL9CDILDvIGttNejCdrKrOpoF+iVOUrJNDU
wgO6ga5tzYyznyz8QdSavWDogcpGzv5B0IgABNcJpc0gtYA9Qok838MQ+Qjp
LjCBDUGtdFVbvzHuSFA8vcZVFCpJ6DPaOMTtYVFWDkXMdycQRDmTxHJAVp3C
lQXNtK3mIVqetw4KMBBCUAtdxl9fmCSNhCXxg87RW5y0SeFM9ENSL7UvNvgy
45CQgI24HeTIFdSjVH3LtQDiA0QII6vlTFE6swm.DpXYC2jLBbLW9AlS.tyo
MUlEUXgqbwXQYV9qVndD6R5XsjDN1dTRXfrDAQ79HDGb3dQ9h3coLydGbGGL
CkY+fsIvOAtUD.sb9IFbbNeNy2UFnyAPRXRwFlBdBgQr0Y3mNr05wFdIVDAu
OFdnLqx8GWglw392hGhxZoWB5XKbdZfwEm3SLBwwDEFh3Bs.b.iIPTaigZ8m
YwoimGBOQEA.G1IBbdiSOfh8LHMOLo34v8QuY83gz3RKj0i6y9dTtEZ7mtlB
NbMDsg1AXTf4bdt6yiSG+tgDWkvB4BFYXPVjtN6PZ4wxIPKDDZGDD7fdr39S
5HIiCL73qtbQPpzKQ1A5LSbLga1.4ixmQVWWZnfmnkHBRCv7H76kAI.RQNK0
Y5xc.BkG08l3qzd0ZDYsNIJLuOsG6zNDhLq03Odo2h2GAIPHU1UaFPl4fmrS
RqP2Yxbj4KUcrkMQK7m6jjAKclnXqy.YQ1zNnFb0DTTAFXYR7.L7A0HXfjgZ
3pcpwMOybp7rtTDiivqdVm3IKG56SknWhAsSDRS7cprNw0GKJKI0CzUF83pL
HGi4ghmGw8DnyGnKhtrA3FRmoDEJvXVGHETnyz8DgMhYDYYoI5LwEkdWcN3h
ly7KEI8t5ZUlozLK5bWc73RsyYLIfMgb.1yDRAXDZLPA5FmHxdF.yZvbam.Q
VHqEEkZW+GJog9OUC9UN9z6HjlC2r9sQDx1DTmgy7Em1WqAzLD0eX3OTG0WW
a6Jckcdvp36wL.xJK8dq8IguxSb4sGRRfzV1JaayTYtLy5kJ3+9iessaqenU
7rbOh1vcLRjtyd8jprTHHjvPIwU3SppvaW+TF5CVeOtJ78rBS2.Tl+GFcIU9
731sGDCCNaSTIiQhRYr8bfXDFh8bTp0yg62+JCGKe1J8opo4hj30QE2Khk+n
VmVwRi8N8f8hYUn3uIPdBYNPd7C.ZZEWZcrHsIA1P6qFb6d5XIl0YRzCVIYg
ardkskf05r78GJjGi5qphSbEK6zFJMxhL.xUOEY.XpC66pqrBpqoNnnpeZUm
7UV.7vqqrcz17VWY4D1w19AomN8ux7axoA5MT0eGqPrq747nhmyR1X8Y4UIv
85Pxpjp7TYgvyQ8X41jL1yP5Zf9w8mqRh2PNT5psJ7eM1a6oqJ7+Hk73nGAO
DwoXSl4haRUJPsINLIam0VF+7HJE9nUSZhUfvN3NpoupFi+V3I6EfCZccV55
vRdG3nJm+JdGso8O14wjUe6x1B2SPmDsMwqqdfg4uJCYCAY1IlPlsVdAPMNQ
GhGQIA5JEuC3JnYb.c1vz880PVZYkvwnE6ZZJDNODv+wU7iOyDaH3DoK0AnK
yi05J.t6RY.ZIOVWxiUCHOVGpGHHrkG6szBDlTgvECYMCn9+RKPnAukfl3YN
jDCngpTanD4VzOUjpr+ekE7ehJXYD1RCtazHB8SEk2B9HEl2BHZy5QoIk2BC
ljLimuxVEIFif6BrMSzR.LHB1DaHIddpRtDBxZ5Yuejbk4tGRExbn1FecNTa
M0GP4EcoDHd1pjbFQQXW9BRug1KAtHFCbfL.ChZ9pz4sdzGkFPP63Cj0DMFL
mwqjurwCrmNiPFeYi5NecFZaddlPy2P53i029W8B.YOkY8MGdYeiJAqUUtPU
EhI6SxJKix6pfq2lKmuxRlXa9ikRgyfJXok8t3P1EGxt3PVyni8VIrbzFw4Z
qfhnGT0LPhxJaf43co9IIc02DmZSJCBLCHwGCL0rI9ztpiueIphN7LrQAzWW
QT7GhTj0WELFyXFwN2M.6fUxInHfLam9iVL7201DRnKGYMgfpSnf+vkIIOc0
ZVHKaFf3f9M1bpswkVfSeGgkSHOEZ4jM9zdiKFNsX3zhgSKFNYpUj8UC0Uk.
tZpwGIK2jV0rH.UL+S4Zf1SkTFJgtpRKsjzPvfZeaCyRxId5wHWUzXy.iHoS
5ziuss3arQT3ILgVcUO0che8UMeTMT11zRmdkZo61nSIszpYWZ0rKZlunY9G
bMyIMC10e6zLeoW6ZDcY249HMjtK6V2IFmhgMpr25h77L4dqq41UcoKcU22A
EFgUWKs1Wk2MXQjUSrFdPfl5wPMC1eGO5AnmGSglyERzHcVZGrKl3rXhiNiZ
qaUuNRTYwb+H0pi3ATe3lM80KMzZONRPzvMqvPFUGN5DExTZyQPRcIHYMZcp
+d1li.FHpO0H6xQLtHb0JOFuCWcVqp1dzev6yeiWFU+4EX0qY3bjK.zIQDvX
9HCyDw2Qu17z0RsbmN0RvaIHVtAFWZLKC2TWAapBEIEHpep1jZ6nMq89XjKN
Q5syVFx67Xrqva5lmnjrqDkvPMqWhGuGuTQp6JVhBDy5JBoow0wIdJtEtNxz
lWzRRqOiMaOCjq6hlbmf6CJlYVOtMJrx.zhyXGAlu0LK+xyh2XDMFOAKofR6
PLu9P4vaSzoKpUofPJTI0DqcCvFubv5twbsXvZ53jDBhrsUUSQqVJHEabRAq
IcieAosJ43fhqhXoYfAJC7XC+taweM3BAttql15qr1wmfhRmaSROqLoNhjq5
7ytsyRl5j8uAAggSI4xRmpdIKZproAnA5CvQZuzcXW5NrpiioY2gcJ9.TosH
VBxTSpyKNWsQ34OksejnX0hvdFT6g8I1+FrAwRutFDKeOo6O82w0oXAA6dhH
gvbbq7+weXcrd8OBmYgkwx4goIPg0PjlXliuRqXjRFuYFCr3plwQxEXXgMFR
PVlq80YlhkWtsRWtS8+fwvzLw59oHrkEZ74YP0e.1KkSVJtN5xDKtkh0QB5h
u4zXyYFNY8yDgfxwtZBMrihpe1xsfBAwgliqwH78mIOTTlGE9hE15welv9ZU
wyP0YuaaeEDLks2NFReWjG1TONFNI5j2yqZcNhZw1nW8MvQsOhClQX1FFMyR
lzRifWZsZ9kqSvKe1Tg0ZGzEIaEW.gzObXa.vgzUiC81ekkt2ueDhuo8lVsV
oGkdohssAzgbQ5jV3b08E1Va9pDJ3jFa+iilyBiprXiuijninzqeZvbVWTIR
Gu3ChNXLOp7cCPM5MtAnGzEa0spq9pfRepmxH5Bf0G2.x8w5CxktTKaaB8lc
rN6M6HOSnweKeZ1i09V+DI.C9pVberzHeticP19MRQplCbqidIWN6ts9VnBn
EVIqTEEx6dPq.6lBNs0LVcqKX5bOkJo0DDsBGxtitjBiwPf91XfVgboz2tEZ
8EvtnTNKBRYml.+7oa0XvHpjXDDoEHJWaQ9n4T+Jjz6XNn1uBT3Fq9q7HzvJ
.eNF0P694cFhnpYHwxla9LDQYyPmiQ2rYHY2nfn0MJrsMhhIjrPQK1G059UN
9M1uhO32xLxceR3qOEt9qxmQtc2umKh2kFlbJmbaO+OaMA2EQfhC2HZ2k9qv
k4pqGTtAgfozApbBNANKYp6Rl5tjotZKSc+YxCqeNLcmpSjD4pT9NPxL.JUY
LG7ZcOvkG6GpNGAkjnPgfLyfZrhUQ1vUDgP1phMgvkZaQ7m2fCR4ARVfBBiL
Bsgx6FTTjcUoerRVuP7MslRxUQKnpfVfEwlgu4PLh9w9bq+yOuEY82r97V7+
k0eop2wZq5TJdXBjHTwHhnWwbBbr8QQe8soEVODEHYQPfPBVHY595jOVrPtJ
YIl4w.EmtM6MQhc0MQIPhbWp67giGYX223Bx7ejMSMvv4Kej0iXLp9T4AggI
sXIJomc4PAoy3ZP4P3Tg8l47PPSI.VsU.leBf0CKxe0Bo5ZnuLqYpEWSvFW9
DDy9WOojBoCZBUlD.S7mQrjATOjZaNxTl798JY69a5t8WYSpWIFLf79XGyti
IVHQ8cVAPG+k.YjDELrzSeQdgzwF.0fhdxdNGMNAiZKxeoiCzDfKaLAQkCtw
Pu4pwfabjh5ND2Jw1F1iCByFxoQy4AWNplOxHvmlilySVdrA+2P7UDwNImFn
qPEcD7UR.2nfFqG3Ct0AJpujBf7aH+wWef83BurAgZnao1XvMOhNkDvcZvey
GbqCBT5PBZPAMk32bzMuCMQFLbh.nEIJRAMFoKXmJYzgNXb1P4AeABN7eg3c
9HcB4REDIRJQrolMHMpYChJa.BNHMm.J3Cz7ZtEcRyIiL7gFlSOn4pTctQDe
GcjJhQNgcDPEArwfaczRIoJ7B.uwfeKCVpJGRDWUzFjLVoHSo2rgbgf8YowE
bYvP4hP012rzZ1VhFpkngRgQC0UVd+N+3U0S88C5ZYht95bWe+F5rW+7Vr0e
ielqV+E1fwedGSryZIHNPASGg7LrSdUJW4GLoJAtVO9UA4kZaZm95wZdsLD3
tJ05NprfuBEyL3PXQFTCy3XEiIKsf8lqnWuY8HSLUd7Zq.qGKeloWvyYIaXR
4cdWsfMNcC6KTLM5LUAg6m6IpK0YlOi6aVaqD4dpm0XzssRM0YSE3OAOaQAf
VNVNeCnK8Qoizya2zyTvWyvfqjv.5V6cHpAz3Foze0abi+YAy17V8tA98d2f
dz8F8nuO2ADGAVXO38ILq.OVMeGXivVcPhyCA7ebE+3WY+uv+23i+ocGlfUm
Gg1Dt9M1RCb6jL62SxbbVMXRv4C9Zywoe+9bMnQR72hdXWXb5a0fw2BOwI89
6dN93zB9P6iE.MPhBdey+2YpkeVQ1quDFDZqVvA7gHtcRrDWrxYBUYzSZTBu
Hb2bZBVxdbsZOqZ+VXxgnrsu2wYmHBIYo65lvd1G8E1TT0Gy8nuEN+twbQC+
gmcq2mowZdojuo5Yf1dNUcwki9jf9olx2tJIIm4Ggg8TJ0SFYIWaOfUj9Tuy
kAsnorZDLTaU1YQE6vRTIzNkI1tQXcz2gNgy9CHEoINy+DmqtGPBtb+TAgn8
RS0wiKoqMQanV06H1x9lnmskcmkaOE7lNOcx6BmTxaxQh2DwSEuIOYdSXU7l
bkEmPS7MgkhivWA3DRp4IWUP8Px9llJ0SFFBUfQRwNnf2iCUhWjy4SjWdN4X
dgqxW3zgFiN9MNON5lAfEMDvRfSV1kbbz6AV6oArxrLSER38lIVEJQhWDVEu
IYdQ9pPTHYDaC2N+TkeaoWZgkGzu3wPyz77QTHWr8Z4qwukJPIY2uucTxqaL
h.MDzyG4RcZrvmOZpHgsLpRnBNMYzYIPEaEJyhTrsJdSxryAREzNjLRiQpPM
LjTL0TU7ljQQBGk7llMdbYPo2qBKXiyEA+R0q3hfd4h.d48A6R2A5xkA4B2m
fcEbKe5e+o+e.k8SzRA
-----------end_max5_patcher-----------

[rodrigo.constanzo]

Cool, that’s useful to see.

A couple differences and things to note (for @agustinissidoro) with the approach in SP Tools (and C-C-Combine).

In my approach I tend to load a single file (which is still possible in this example, but just replace-ing the buffer~ corpus instead of loading a folder) and then slice at a fixed interval, as opposed to onsets/slices. This is a large point of diversion as I end up with (usually (significantly)) more segments this way, as well as not necessarily having slices happening on “onsets”.

This approach also has the side effect of each grain/segment being the same size, which has ramifications further down the line for playback.

Another big point of divergence is in the playback/matching. Although it’s a relatively simple edit in the patch that @tremblap shared, this example is monophonic in that when a new match is found, it stops playing the current grain and immediately starts playing the next one. So even though the slices you are creating upstream are generally going to be of different sizes, each one is only played for amount of time based on your matching rate (which in this case looks like the signal vector size and/or @history amount in fluid.stats~).

I would drop my playback bit from SP Tools in, but it’s fundamentally different in that the example @tremblap posted presumes a single long (concatenated) buffer with variable start/end points. That small playback bit can be wrapped in a poly~ or made to use mc.play~ instead without much work.

Similarly to this last point, the match/playback rate is fixed based on the analysis pipeline. So you can also experiment with having periodic/aperiodic options here, as well as having different rates. A simple hack there would be to use zl reg and a metro to only send a match when you want:

This is kind of hacky as you’d ideally have your rate correlated with a similarly lengthed analysis length, but this could give you a different flavor of playback.

[tremblap]

More ugly code, or as @jacob.hart so kindly captured, “garage code”

In this, after playing with the guitarist, I have:

• added 13 MFCC of spectral descriptor (we’ll see how it fares tomorrow)
• added a gate to my query - it only queries when there is sound coming in
• removed all the temporary buffers using the generated buffers and the buffer message instead
• used a simple dimension reduction algo (MDS is not as powerful as UMAP but it is much faster)

All of that in a few minutes, just for experimenting. So many more moving parts… and I reckon I could remove the first 2 (loudness and logcentroid) but I’ll try with the performer first to see how they feel…

I hope this helps

---

----------begin_max5_patcher----------
6226.3oc68rsbaqjiOm7UvUUdXmY8wC6K71T0tk2ef8GXloRQKQYyyghTKIU
RblZ729zrQSIJadARtapN1xoJ6PQIQ.z.nAPiK+yO+oE2W7ijpEN+Um+lym9
z+7ye5SxWp4E9j55OsXS7OVlEWIeaKVVrYSRd8haf6Um7iZ4qGdqS7tUoENO
DWmrx4+eWR4SN+0+T66aab8xGSye3qkIKqgmWPf+st23v78Z9Cwi07Gp6stN
+C0GJe2lz7rjZ4Slb3EK1U29ptpWMckDJJt+2+Ml6hlW6e84O27qaPhW4IeW
7geEZ0fMigCbITycoRTQ9aJcHTf1KJPdMJvaejvaq9osIvyawBm+wYfb0EO7
PFB7f4SjX.u82mxRQO3AI5vyrLdSRcR4WSxiuOKo6Z2qPxTA+0YgmCrH9+37
aDW2QPeefGjEFIWMYZZYjFsP6n3PxeA25TUGWV6T+XhRRTJCNBV6ERAzUyBf
b5YI.tay8IkiAtDIb54IWj7bGeQpenk1CzRNKdzE23r3937GzJi5DZa7BfEL
+.spsgEL2pa7fUONSupaXdVg5lsRIuzQYl8cAlYW4dfd9ZhYlQWLFC6h8e8R
PR.g.xovtOsXcZVx2RJqRKx67b+zh3sa67xepyGogj76Exunva1+Ro4vKw1+
RkIeK8keshu2RAgoVPU1UJgyE+vucuululhUIk46RkeF3EEKNJPRtLjKVkq1
FuD9vMqVs29.ol3RjrYQgM+1mCrdgczyIVmeHqX4ejrpCehfDtMIOMeaYRkP
gabsB32e6UIqi2kU+00E40Uo+TBBjFYxdt+ZEH16MaPBI7++VlFmsGCdnLcU
QdCPbzRQyK293DHmR2cWjQ9Nxi21yGVviHnKCbSw9G06ptOtrYkRIzPauYcQ
Q1w2Z+mKKYcs51aSyyeAUrtX6v2rL8gGG4ydeg3laF66VdmputKGt6WELE0e
sJ9aGSsqiyxTBqG+0+i37zMBkt0ovR.0c+MAEGOVsrrHK6H7Ety254NqDL4K
S9d5p5GkOntLCh2d51VlnE6WkWk9PRU8wuVc7CUG+JU0OAD8Nuzt6UBwesNY
y1rlsNN5MHDORqpqdr36Up2XKiVWBvAq+6JT2Uy2Qu9XZ.OVKXoPCThyejmD
W1ffcdK8nNjPjRkLhTQXn2KUGN3taCqV7kpFCi5BCCodr8cH2IPov4LoMvi.
GhyAUTL21eOLhSFDwc6GwIjtvPq8icYUjef7UI+nihVsPBFg8PnVUn+YkyiB
M.YMZAlhPEJUey.q+D5zmlEYXJEoeJUj23rHZk8XDZy5xhMUOs49hLjhM.QI
vHzD9LRSvIx.bBlTjgROIQF5LwVjsKc0s+wp5xjjmcxJJ9icamjAAbRHRRkH
D1agCY.EqD+YVwpvjup3GR5kF0teiS0x3rjUccAdJZjGXJEk8V17Y.onP6Pw
Rsy8HIFG3TztFkwUx1wmZySOte250IkO2kYw4tp3MaqDlTOAkhCttQBfMjBo
FPzZbcuqyJhqeQbHlAQr5x37p0Eka1VH7bVEo2SVbivBA5VfAD2XVx93RE1k
E2uSXGdCA5Ym53xGRfKJctKq36BSYbtqwHHmHWjTN+na8Djr.SnLOZtUlKgU
bHdP.1M7cOUFFxTLLuUKnO3cdVZ9P9bIQ8l62OcqpXW4xVXpcwx4XxfvOy5z
78Aq3usWkay6C0x1ICDdHAhFipLGTvQBEMdCYLnnwGSTPQODshxUP73buvfF
eXPiXFPC6RmuAW5vxEGZPXfgcIxjDBBVkJsbTlAJPBDrAgA0K1Fv3EMAEb0W
g.f90355xz62UCZh6FA7SJPcOjUbeb1KhtVeww6yG.NMcde92B1bIOuOvpBG
ggYOL5o75wACHX58.+ngmyA9IH+0Bh7HfK0uC3psSIhddG3m1OrusN+8E4By
niyRqjr0xETAI3uuXThBDbTWXMjEcNmbTOqgjvEe.OZH3.HCbitdzPWOZnqG
MjNOZnwBgv1xz7ob5iAQTIzs+bc3WrnDfwMWvyV3Hvt.d4p1fyBOqHHNJqhq
iqRpe1ocSyjUfcOSQV8fHM4xaBaBwHgMg3xrnffuNsdeP5Zi3TrPOPxIS5jb
gJ6uXbCHCFE9tLTcJQYfkyxhTmQBEEVWWkBJlKJPgVgGzXAhv26NuNYh.5Cm
1FiFo27N1VRDv373rmD1ZMJIf1I0q8Oqj5ruLAzewndx9QMa.C4PLPfjd1CR
0nvPuqt7c0kuqt7YrrADruZy5kKe14NYgJrrHY8ZGxDFR4oJLEHPMDNw.VRw
rwDCjSU0hhu4RLvKVRNghBvfvw0VaNFIOuBWX+t610AtoNnbURU.0IRa.90q
zBctyPfiLhbPLGpmB9.EThFRaI5KTUMnckSYa4bjlWS5ep5jUBLVVdQFOcqm
SxQmppZH5A3FhWD2XAabprjbtHFUIYSxc.RQgvNubCjK9DD4.3MyHQ4mYNKe
rygmNHahK+1.OgZFOofiuqInMbKJRheYJ6yndPVX6GB4YrLdWZVxgLiRNuMh
gh8vjTi.aJdoYE6VkKnXOOIgIpiU7gQFvrjvYVpYRZSi+yMoQbSHiqpKJeRX
iBV0KPTIH72j8arADmP3tyEIKO4MY1f3GpKC9IxuINMF6zvniRGpReHON6DO
OLp0jF+QPcEyoly.OW6IO9UZiDxYz62s14tNwaWke52EuqtnIPYNXEB8bUwj
Nx.DO53axC0kvrWFDmVAPn152CbhfvCLgylrKUEPLB452KRycXX0kSUcHloo
NrSlKhZSFCTsUf7kwYUOFuM4Ym61kmV6Pbtaaw2SJmLPeLBuiIBThIrQHxhr
rFSJvPg3466EXtbfwRbKcYwt758cLiAS9Q4tZ9DnSNfHVvdCRO3CPOnSF1ha
NXjj1ihwXbLwqVAUb0WHNXJzJEwBXdTwGSybOTqxajyIho.QJf6av.l5YOtl
znPdYVRb4jVPFBG3BDJUehQbDggQVSBtlYOdLdkzP.lQmRHTDgM0JKFM1oTF
XTSk5MjSAJHFBJZqzsoyFJtASCofSrt7luB9hdZE2y7UjbDzLwlbgiftVFMZ
tzYEETIRfHvBjiHyM2JCak5QecwMtIELlTo7mRkYpZ.qs+K0dgopzQrp.nAS
A5.v5CwJsyElR2df1H5JfUUmCGtvTPNZ0JtFTZhhcGZhIKZcrreQ15VigCCX
TyroTjMTk1nKNYlIa1.DrEKNIxzPAAkYBlzNA3amhcMgZTv.C4f5abpwkWNg
ZCZYa2HY5kjvY2Re2SYYhfeKf2IUtw.UzP7pkO6PltSpSg+v8NmpSuuN6LYv
pYX+A0pwlqf2sNUeOUfbNE423rMK9IYaVX8trLnIK3TrtaiWntvYSCs3l8er
0qa+RaXJkQ2tSnv583ugjOiF31ITjMGp1as4LDrPiTF9sNeOsIeLchyWATl+
aAcYr13.EhwppusSTMtcMz0I70IhwDH1iI4NOFuc6SBbr9Qm76aVlqxRWlTc
ipTdR5cYkNL1qVUgdVoB4Yr2Nx6oSjmdKflNo0N66oj8Kde3jnz3JIWmHC4V
mrh3UNOIz46rrnb6twJ+JluRrSynD0HsDEf9SC80a4x4OycDkgV5fY.wAotE
SJW45o20MxYstc7w10Ynb3e6XrddcPgolmEnmlCgZdXNLDtU+XYR0iEYqb9x
XFDzTCzGZTMAdZBK80LVtNqP7LQfGtAmxzGooB7ikeC9XKbyfKv3HYR8NLUx
Gn2NwjmczHlZJ8wUowYEO3rVvOO5b6PQH7nCzExOy41AYwbLmY9V7AuEfiLe
YQ9x3Z43BpoXeqdEso+21wYX2z6wpLXvEjZXLM4UvviWoUoKa.s3xmzJmBPM
NPGRQHy.V+Fw0DNS0rhuQmgMiM8nDd3EI+wW8S3MN9PZmxizjXw9z+653r4P
EwpF.B9WaYYWqe8q0utNqecTkkrxEdZfkTW1157ZgB0.EXw+040RGdGEM4W+
w0htl9SsdSwd+L7mPMiRvjEu.sQ49LDEQcOaRrjokPuUgBcJ0LfslJ5CASYV
b5kVHwBKBkIK4Ib7UfKYpgXhl0EE5ZSE4TeSOI5JbTInT3ITeCvcEDXgSbC0
7Y00bEeowm3FlcFjAV2XRq.0vLHyH44cPfMLe.BwllEAAWm8IuHCUstQexKx
KROnuI.EcBgEr+peUlWJ8iOPMF04hYcFqfMeWBY1PxeF59du0nNxPgX0tMa6
zolcZpqslTLYaVQccR4niGh9hJKW0xKi70ULncGKHze7BIaDU4Dh60XxdMlr
WiI6r1SQ2W0uzUoa9+lpZNAAUgPTyn.fDZ.+47rnHM0rSBJJBzvHYQFH7RVU
.IQPPHpdlCL3Q7iLPzH8cswgroZOLyUZyg+Z6xOvWvnFrg7FsvJKsarEZhuI
KbLB1ZA0jklK2BF3p9tu8pO58uSTqSqcff8d59K4BItitRSIRvU2ktlAKW8V
5p2RWzgtWiFQL9HQftWKX4qm+u3CdOcehfJpCAZWcl4HA413.ofppfIt4Nwq
KVdOgpQPE1VhIWjoy3Ez8noTpbHT0nHgfhkPWSzj88rWMK3DuTpUHFYHUZUS
LPhs1DnXVfeln6yE1QatvFllh8.sefKT+ya3xqJu21SdORWN.yu5.7vicdhK
TZgWcA9pKvWcA1jGXXsy8S12aYpvQgsiae546IpQezEHWhUaW7yjmwMbqijc
rtHSXopusMLWNoCYlqF+dAdeLNk4iF27MDHTTIEWjGPjX9ueO54dXgPxAw4.
Gj66fodJZNnNY9FBREvFERjDJeSDTAOhMwFsYU0yN2I.bAoIIuw90oKRFknF
Ql5oj.SnwlGYiwrj0svx9.FyR0Bu+EJlkJiXr+QrK5HX1pXlvjhRbSXfHw0Z
UMC8.szpSHjuf2fJ8yAAFP+bTnMoetr39cBWbVFmk7rLlKIkN2kU7cga9MSB
vGdzIxEGMiz1scLg95neIiEroay6XRxonPKnWy6YxR4wFh8L1DsxjQdlGYAm
DfGVYCtIK2tHrIeWj+G1bNqcyQDsOSoYC9TcEd8g6RRGYA1GpbLCxqcN3QHQ
MQcCtFh8qgX+ZH1MdM4zzZGDfVcR9TwRNHR5R.GJj9HSDnKRjsEMYA4g1z4N
PRafI3XfIF+miOs4jv3rSZjCN8mctqJISPJb1jDmijP4BsoCelApYGlExDsY
8xkMDJo0Zp1vo3x53x5kEIqWO4P2UQ3TG3OkG999rbldtoqZU89WGa5xFXP7
pUHmIudztMy5ODSK8CTHziLcBMpC+EIfecjo+JNIdnZjyxd2OwzQZG.CNwFN
DsYiXG.IvBMD.kMR9Fmz3+KfMRoqPxCoLQxiZ.SjBrPSjp1JHAkwYUOFuMoG
ak.52Rgq9kEoqbtaWdZyLx9tsEeWnWmfipprehShLQj5sQp5ox9oDSMI2Gw0
0BoTYE6VkmTUMHqW6a.EAr0P8HOSzlKm6bJ5noNwDHNzwNd8jmPCVW9Bi1Fb
RTLWFriqZr3cNhv2Ycg5wNN5sko403N+TeHtILSXT.y2NriTNAxP5hlOjHh9
HZ6zdCRO3CPOF2tZ4ZlRExMvkyVBLH7PS5olyWHmpSsJNHU55q6PuEXkYm3K
NEqIHPPKkiPCLgirdVUjiD+aYVRb4jwPBTMCVFRBdS6TOfzlbHyhMIxE+UB2
ytO++G+li6sSRqf79UkShPuwTy9uQCQDPj4jQJCkrEUkbYHnIm9QhLgrUKiy
7okVFz50M1dcXbxMXTzfBSSYSXfIZZYD5bl63iQXDfqjrTMMcQILonKus4nv
fzEKo2MHbbx4mYJEaHjlTGJjmIZWCDDmvXKrNWzmelcaUcYR7FGpyc+LS7wZ
xdflS510Em5GWULpMhu5DOa8LYARzdu0kygTUybFEelJoMIFQqzDGdj9iEIp
FigvdGwOTWF7SjuYaifz4HmyMVez6DRpwg6n3gxhlhwCuk1no20+vkyYmRv+
zZgClo8.PY5hpR8Tj0fiIqAyLYUZ7gdPIhfmvJPoHKHmhYu89lnFZA9mxDif
dgSybhQGdFVPR+iW6QDZsGPoiGR2e0bNmGvq6HBstiKJB4oKzIzFVdBQ2Rcb
csB19PuSiuuCUte5J8By1OMBcLe+EBg7zF5nP.0.D7vEy5xiqqUzJoBNkMeI
W7d6kIMFhh1PDSVcSLzzho0CQkMKA+HxAFc4ElBzQS.CvIh5yYcf7lKL0Ref
1H5JfMj1AxColCxQOoyL5tmMG+hCCiiuJi3GYylPnoL3BmMKbnQcuvWkyLfm
axqLJZQ0BegZpsMBZAk5QqXpqgwJLE1KYFXZtzSoO4gggCJnlMpMdm1jTXJU
PT+fC7SxKLJjihepGZ8Kfc0DYCfckgzJfeNC2DdLhiDif5JQYIs7p40IXrL4
LixjiNpWDhmwgChE.Gn89kZV3.s2HFMTfnAidL0rWkfdgcTBJu3CYg3uMK9o
6iW9GiTH9LUtv4IcVvWWCFSBYvJwuJ8g73rC0he+0O9GoRzO.Fk.gxUBOn0.
4E4cs.8uVf9WKPeiUf9+L61kOFmeb1F0WRVI4f8T9fZfb.Izdx.DgtDgP2JG
YRnghtn5WAAQlnBgCrmTrJCE0fQfQiXnIRUQKppdvjQqLdmCOvHIzZjEkNqM
00KNVjPlwlvrVh7BZZAUkpXguiG2tI+Xaoy+4WVSb9KNeYM8O47malyASk5g
DURqxT4Rm27WOuy2lNII+wymP9EpnMDEyiKw.kGOgYQ5VNSVHyyAYILPo4qK
dVUNoSNjuIGw67l5nnQmSGoeeam.po+Nu.l+iQJZrwjMyheBIoUYOHbBXAFo
NnFkvphqvMWrJQs0Qg2iUh5HrH+WS1b.ZUXynlq1l7lrrKmOuERE+6dTLKgp
dQwaQXgM.EgioPTU+YVKGtSd+dyuc+ruaOlxInEuMVADLcg2ZkEOvojtrjwN
q.NLAWaapPR2UCLWZEf9.m4VPe0lLYt5.YYD2UUil6u3B2Kt43faJL606bgo
Za0mPELLJ2Jy0E13DxgD1gqrwrPlbh3S2qlyy7+Tyipo3qXp8mNbwEtFN7wA
2jnNxCxKtz4bWHREPgcz+Dd6ktZUHHgZ0zh6vEW7L8EIf60g+VdwkN4f4Son
gD0Uie2qt3iKX1joOH.sDU+WoyUlB14HS8wISGMd2g9mR8t7JSB4m2bzteMh
csrgXPKaHbrIkzjzbF3x.Pya4VLIMmchUyvzb5QckRM4FQxczI5nfpT9QvBT
9QzdwkNaoPZBuBv6bwGxjkpIfDoMc6kIFZIDeHOcz0PKgccjf2MYn7IjV+aZ
R3Bt60IB90rg5Z1PokrgBUeFEFRlDx9NLp16ynDlk0nQm5rW+xZpyeQdlqN+
YwEDbTvPPQFwDi5wPa5jWOgP4CTFPotYN8U96nSesc.Kfm.yg1rHbHrDWSzc
j4dV3XUXeqqpHuR.9MzqmctSnVoLcoSjyc0OJrK3whrUBs7dupCTmluR7ApN
YFYfLqFKQ5lLym4z8C0QR4eXbv7waLJiKK..RDiZwSZaibrbg1vvRkehQd6h
dlBgFFF7QBCjKczgL4AdhdpwxeuOuV+8Jgu4iDcCuaij+3q9IrwuaUbmo6+S
+Q6f1VLWoYI6gdXS4sYB2B22mw6eyui13ic.HGzyfQ2PTOQDZU7xmEhFzoKd
tPH.ZddiGQHZuT.2WSA77VbFnQV52Rt8g3z8SZxEeK9.mzqu6HECHL3ofSVi
v7mBgZ7huoTZxSxjsg+tn0wHqWOHKerkaxDUQ3Mc+eGYqtpEfdjH6dY0QjZ+
Vb1tjh0uNvYG.urh7GFlvdzaciXIp4s4uO1BGe2TopgeKvs26KrXsrF4Spk1
z22Syr7YeLI3etq9syRSxQwQX3HkppVpWEBgyLPo7fyRwPG+PzihgVuxNJqX
60TPRmhFKz6bhWbO5GY7ACXbe9dovY4C8Xy8.RvK2OUQHdchebjUMCcnfG0.
EdQta7Jpu9ezGdSu7fwnx1vRnx0kNW8Zfk7l.VFAAvx7OX10Y+jvPUXcZe.m
8ChFg3Iw0.FQv9fdqXT.hmiuFdN9XVhzvywCwygngmCOXlPHNCwChpkmDF4U
hNHdLTDOpNdRn36BzfDK6nz7an0oHcfSdXwo27Shh.m3Lc8jlTemV38P7fB0
.+.kcBzt92ptIp07W5HZ.Lufo9dv3Ms6Ubn3yC54iIukNPIrJH5GkBFFiXt7
NXj5JetWGyTjW8VQBTZ4B0gwBtX3o0gzCFgGcn4gfQuM0UGOIL5sI5f1Q7vZ
k5a9IgwLUhNzaSPInpCyGIXjl7zxSBCq2qdNfyduHKfZdDuH6edQl+75r9Y3
L94kY6iL3nCkkOe9e84+MMMvBCB
-----------end_max5_patcher-----------