Rate of output values of “fluid.onsetfeature~”

I am trying to understand the rate at which the “fluid.onsetfeature~” generates output values. I was assuming that the output rate was determined by the Hop Size. I made this small MAX patch to analyze this point.

Screenshot 2023-01-26 at 16.44.14

With the default setting: “fluid.onsetfeature~ @fftsettings 1024 512 -1”
I was expecting an onset feature value each 11.6 ms. But this seems to depend on the size of Signal and I/O vector size. If I set both vector sizes to 512 as the Hop size everything is fine. The print I am getting from the patch is:
print: 11.609977 241.88843
print: 11.609977 499.457685
print: 11.609977 135.080762
print: 11.609977 0.
print: 11.609977 -88.049114
print: 11.609977 -158.878539
print: 11.609977 -124.43916
print: 11.609977 -89.999782
print: 11.609977 -55.542309
print: 11.609977 -22.809686
print: 11.609977 -3.669806
print: 11.609977 -0.012383
The first value is the time interval and the second, the onset feature value.

But if I lower both vector sizes to 256 but keeping the Hop size to 512, I got different timings, sometimes with repetition and sometimes without.
print: 11.609977 0.
print: 11.609977 -88.049114
print: 11.609977 -88.049114
print: 5.804989 -132.999661
print: 5.804989 -132.999661
print: 5.804989 255.860709
print: 5.804989 255.860709
print: 5.804989 164.360588
print: 11.609977 0.
print: 11.609977 -96.658959
print: 11.609977 -176.098228
print: 5.804989 -176.098228
print: 5.804989 -141.658849
print: 5.804989 -141.658849
print: 11.609977 -72.780093
print: 11.609977 -38.332619
print: 11.609977 -11.084429
print: 11.609977 -11.084429
print: 5.804989 -0.532012
print: 5.804989 -0.532012

I was hoping that if I set the Hop size to 256 as well as the vector sizes to 256, everything was going to run smoothly creating a value at each 5.80 ms. But this is not what I get.
print: 5.804989 16.37757
print: 5.804989 241.88843
print: 5.804989 494.113026
print: 5.804989 0.
print: 11.609977 -113.878648
print: 11.609977 -96.658959
print: 11.609977 -88.049114
print: 5.804989 -79.439269
print: 5.804989 -70.829425
print: 5.804989 -62.21958
print: 5.804989 -53.609736
print: 5.804989 -36.390046
print: 11.609977 -19.152263
print: 11.609977 -3.657423
print: 11.609977 -0.532012

For my specific application, I would like to have low latency. So, I was willing to use vector sizes as low as 64. So here the results are even more chaotic (Hop size: 256, vector sizes 64)
print: 1.451247 -60.067119
print: 4.353741 -60.067119
print: 2.902494 -51.457274
print: 4.353741 -51.457274
print: 1.451247 -42.84743
print: 4.353741 -42.84743
print: 1.451247 -34.237585
print: 5.804989 -25.627741
print: 2.902494 -16.976876
print: 4.353741 -16.976876
print: 2.902494 -8.559195
print: 2.902494 -8.559195
print: 2.902494 -2.504547
print: 4.353741 -2.504547

I have the feeling that I am missing something and any help on understanding these timing and repetition issues would be very welcome.

Many thanks in advance.

With all our descriptors objects that turn signals to Max lists (or values), the output rate is indeed related to the signal vector size. Things can’t be updated any more often than the signal vector size (because this is how often the audio calculations are refreshed). So

  • if hop size < signal vector size, all calculations get done but you miss some outputs
  • if hop size > signal vector size, outputs will be repeated (which can be filtered using zl.change)

With some re-engineering the second of these cases could be avoided, I think.

As for jitter:

  • there will be some, unavoidably, because (unless you have scheduler in audio thread), there is a thread change involved in getting the list out, and that always introduces timing jitter
  • Using cpuclock might be better than timer for this job

What’s the use case here? You would get more accurate timing from fluid.onsetslice~ (if it’s the onsets you want), because that doesn’t involve a change from signal to message domains.

Thanks a lot for the feedback. In my use case: hop size (256) > signal vector size (64). It is good to know that I do not have to worry about repetitions and that I can filter them.

In fact, I am trying to estimate the live BPM of a piano track. In another thread (BPM estimation from onset or onsetfeatures), I raised the issue and Rodrigo Constanzo provided me with some interesting hints filtering onset timings but it does not work very well with the piano tracks I am dealing with.

So, I have made some preliminary experiments using the tempogram approach and it seems to work much better in my case. In order to compute the tempogram, I have to compute the FFT of the onsetfeatures values. In order to deal with the low sampling rate of the onsetfeatures values, I write them in a buffer and then call the fluid.bufstft~ to compute the FFT.

So the point of the current thread is to be sure to write properly the onsetfeatures values into the buffer. I want to exactly write a single value per Hop. So my understanding is that I do not have to worry about the jitter and filter the repetitions. That is simple;)

Thanks again.

1 Like

feel free to share your code once you have it working - I’m super curious about it!

Great. Thanks for your interest.

I will have to work on it a little bit more as I rapidly realized that filtering the repetitions with zl.change is not a very good option… In case of silence, the zl.change filter the 0.0 repetitions and therefore completely looses track of the one sample per Hop sampling rate.
I think I will transform the output of the fluid.onsetfeature~ into a signal and then subsample this signal at a rate of one sample per Hop to write into the buffer (unless you have a better advice;)). Let us see how this develop…

One possibility is to use a vector size of 256, which would simplify the output to match exactly with the hop size. If you need less latency elsewhere in the patch, then you could do this bit in a poly~ with the @vs attribute set.

Another possibility, given that you know the number of hops per signal vector is just to use a counter (or similar) just to take every nth output.

Hi,

I have posted the current solution I am working on the following thread BPM estimation from onset or onsetfeatures (which seems to be more appropriate than the current one to discuss BPM estimation).

Thanks for your interest and advice.

1 Like