Fluid.audiotransport~ not "null summing"?

Experimenting with the Audio Transport stuff again after seeing a friend’s gig that made heavy use of it and am finding that when interpolation is at either extreme, the audio null summing. Or rather, having interpolation at either extreme does not present unprocessed audio.

Here’s a quick example:


----------begin_max5_patcher----------
1241.3oc6Xs0iahCE94jeEHz9ztIQ1fg.8oo6ppUqTm8o8spUUFvIwsN3Hvj
YRqZ9s2isgDxLjaSXlmZjlA7E747cNemKv2GNvMQ9Hqz04cNexYvfuObv.yT
5IFTOdf6R5ioBZoYatB9Z1jxT4J1V2Q10yqVJqTBlxrCb8r1oTaVwrGuaBMe
tqy+Wu7JpJcAOe9mKXoJ6N7hQSPibBsWvQDynnIncODHIddif7pmjmYTLYxW
FiCc0y8igC0+azEBob1CvC2fFE6Qi93lwDzMacHDLB4fQdjqDuk744TwoQL1
.0.MHADii0W77uFDG0iH9O19J.wZmJwbwKdRvUhPOTOhveeqyXb+CReBokez
O7pcidduDPpgURkRIyafDnWVf1MF8qmMYdpTHKrJOVqsnISih77Cz2QlhBhh
FoWXG..afDvqfm90SZrbGY+ynGMO7LYtJmtzrG22WvoBm+TJxbOtAEaoKAdd
F9hk6fZoO67n67lExp7LlwjFp2eGF9FsVvlSS2nOhclAbqysksAMwmDi7F0w
MGXc16Ii2ipB.xJVwmY4zDgA7nWfSNUtbICrlOE3uuJiKc165OzSiNtsMXp0
zZLpSMC7BNFWswtjTkTiAbGIgBb6QfceUIOEHHOd0HyKxfI+.TSkC.Z3yAsm
4xCP33PzzHhO1epmIj3Rl5HDBLtGyeIjzrkrxRs7uxzXmNAVPc5YCcH5LIv5
fBzqLf+IGBaVIETEyIgodfwxcTKXNpGjNY7YyXEvS4TJqJRY5K4YkMmgfmyR
gYTsSz1MAxRoK4YryPqHgVZkkcgmZy0eagLduHVwxJghWJ.UtnaueChSgLtY
TE874p2gAYAGrpTEGRobZ6QMYwm3aSKWGz0xdTxTfAAT8lPJzggGmiMMsuSh
piYnyYOiow2wz.X67a3WgXJ+XCewFYcUME7hZlk8sLZ51Sm5THS0MzrGXcUA
1V.FENIz7KNhDDDgQPhNRfdIRvU.EROlCblnhmMgpK+oJn4kqjEpsN2cnqDx
O5b2rYJfIp.fUZZfG5BEnqjnSG6brF.GcF+s0h0jrHFe09a+WhQJwnI6yGrl
ydXMujmvEb0lCXuylAViVQjs6mBnDekkkUPmWlVHEh1zESWepEPCVyWzddaH
4hm+.1EV+7ErRoj+sCR4lLeo7vjvIxBcJtVyziuPBw9xVj3I9leHb.7NdQw5
FM8LozvPam6MMzh40ugr6cy3PNHfH7ubnigweTJW8gw2O4A5Z8j2sVpYIH8s
BXI3d7dmeS+uV5aFaozT4ZB3MAszsKNBpitI9EG4MfiXCkuMJx+UvVlHnaF+
We3ii+avPmtXio0Vny16YB4aJq4f7ulS2zozS9xOFKgd9C4R1FspwW8wAJ7N
kLiUB4X204vm18kIZuo8dqNIvWpn8u.ICuL8qfjexodDQirtuaRROwxcBIgt
QI0bHm1QF1CRZ5EHn9PNgWfbNjCsjmsRBMOT1z0l4STQHl90v6GbiJVvEnXS
6ACvkDfP5N9v6MQv3mKX7sEtPtZedeI4KTv3iQ17Bl1hro6YrS1lMgMc0p0r
hx5i2nTPY9uX+tEQiLC441gglgELcUd69M8aBEpfRcJnNWUgstxig1Oqsqt7
ZQdEu1p.lig5p5PKD5ZPkqnVja5zX3OF9Svs8o1K
-----------end_max5_patcher-----------

If you only play in one stream of audio then the input is the same as the output, regardless of where the interpolation value is at.

Is this meant to be the case and/or a limitation of the algorithm?

It’s obviously possible to just crossfade to clean at the extremes, but it can create a bit of phasing/funny-business when moving to the edges like that.

My guess is that interpolation being at either extremes makes it so the object still computes the optimal transport plan for each FFT frame in the morph rather than just resynthesizing the input. Maybe this adds some error?

1 Like

Interesting. I’m not super familiar with this algorithm so not sure what to expect under those circumstances.

Curious what @tremblap thinks as I he know was really big on all the objects “null summing” (though this is not exactly that kind of algorithm).

Indeed, this was a ‘test’ - we implemented that 4 years ago and it is the first time someone flags it :slight_smile:

Seriously, in this case, it made no sense to null-sum - it is super experimental and work so hard under the hood. using this at 0 or 100% is not really efficient. also, using it at 99.9 gives good results, so the liminal space became an overcomplicated question.

if you need a bypass, I recommend poly~ and killing the voice if 0 or 100, your CPU will thank you.

2 Likes

Good to know, will experiment further.

As it is, it works pretty poorly with time-sensitive material (i.e. drum/percussive hits) as it just smears into FFT mush, so will see if there’s some use case(s) where it works/sounds good for the kind of thing I have in mind and go from there.

If that works, I’ll explore properly bypassing audio when at the extremes.

1 Like

actually I remembered - there is a bypass in there somehow. just silence the input you don’t use. this nullsums:


----------begin_max5_patcher----------
1035.3ocuX0ziihCD8bxuBDG2MMxFLIvdpmY0p8xN21aqZMx.lDOiCNx.oSl
QS+aeKaCIjtIIzcnajBeXa7yupdUUl7yoSbSj6XktN+gy+4LYxOmNYhoIcCS
Zddh6Z5tTAszLL2B1ixju4Ny1UEaWko4R9xmZabCsJcEuX4WUrzJ6bGPB7Py
bBB8MWv5y9v8NOz7NE0q4EBVkAD7wFk0UOuUdlAQXUbGF0hocbU62vr.pWPE
TgqyC59+0zo5SytMN9aO4fQliKvTLITSt4wH8kEnKyT+gxzn2GhVuNgoN33x
kEUEz0lY18SJNLsM8rVlw5tdOOwWfizWhCGFw0PlSSMSNpWig+KMF9WyXLyw
MWHovJ6gN3Tx+gYbXsHz1LLdCxdn2f0Sv2x7JSkaXWR56aUBMBBbDw7TzMqH
vyOqUHgVrbTU9YLAc+SNDBn+gX.exk3K1Pzv4Fc.FGORQ.3nOxX8eeHtThM5
N1KbTXn+Gc1r6vWLkMoieLX9H4F88eWHolVI0UUxh1oGniknmO6lymkhrKlK
2JkAZq8vVuM5ZUsTx5hLlgwyQGxzjrLUJjplIVOSHuEQQ9Ag56HKPgQQyzcz
cx6XJCZMkKKjvJTvS+dWP07uC.Hu.RLxeVO2zEhN9k3i1AEXjpXpuxJnIhSx
LKXKoo6OAK747l5jv5eF2vCOW8gceCN4T450LX5d9j8o5Ltz4nquGWY3Bqmz
VU17fO40s+CTO4jBagLoNowVgeCDi8iLZ5gLNBYpNL335nOooUYhl6M2bDGQ
BCivHPMYqCqO+ZBU6gcD2wKaStnlm4Q0NpJEsnbiTU8jy8f3fo1HEzJtrvA4
beddUIHi.xVZJz.IYgPNRj60LEDSrV6NLesIp5YSFACXSFusbUIFRvTs.rky
dbKujmvE7p8mDTkmCVi1PZSNiizBjIemkkonKKSURgnqWzjenZEjJZ4ptsaC
oW8xWv1w1W1gEk1sNcLaV6NBOzhTkwTca4rtJLI1KvbfvgvdJhhM0OMeg.97
dNzEKwbHytUokwVKK0Ih8.SOrRb6I3kb8JQMCfpV17QRt2myg3bv2+uJ15DX
aQ28m+0+b2eCdtzU6+RcIO8yxcegIjdOR2pG28akZsBReqPJ2.2iOQ1XPA1L
Ywy+PLiAT29oZIfVpz1UZ6WA4bznkwJg3GSDUmAE7Bbm8pAJZ.33aGyMgi+P
HzoKlCpO7sgr+PnHZDrkACAH+94H5cGYR+.6+g.Ldr8pXx.CPFe8zPTx5uja
7cy3ngJku0v0AYdwQWmj1LgzMa1xTkMutAUn942ra2LZl4Qdg8wPyiJlt7oc
71VnJnzSET2oVYqJratMau4exPUTya7s.empKWB0l0UPJ2z7eQXJgO8WS+e.
6n+Fa
-----------end_max5_patcher-----------

I presume I could add in the client at hard bypass - the algo does it deep now as an optimisation (computes the whole stft and magnitude and if either is 0 it doesn’t assign)… so we could check at client level if you are 0 or 100 and then bypass there. if you think that is logical, put a gh feature request.

now before you do so, let’s think about what null-summing is. it is a sum of the outputs that nulls to the input. in this case, there is only one output. so we could argue for ever if that is the right concept there…

What is that pop-corn meme again? I could tag here a few people with strong opinions and sit and watch :slight_smile:

I’ll pop a github issue in for this. I think that would be ideal, that at either 0/100 it does the same thing and doesn’t do anything. “null summing” is not the ideal term here as it’s not that, but more that at either extreme (of interpolation), nothing is done to the signal (other than delay it).

1 Like

that is where the stuffing hits the fan.

Heh, in a perfect world this would bypass to zero-delay as well, but that obviously becomes really really messy. Hence just keeping delay consistent (like your example above).

it’s actually easier to have no delay at all, but imagine the mess in your code. suddenly no latency.

1 Like

Totally, though phase can get really nasty then too.

phase will always be a problem - there are no real phases once you start messing with fft interpolations, hence us proposing two different models :slight_smile:

1 Like