Aaaaand here’s a video of it working.
(I’m using the Sensory Percussion pickup for the onset detection (as per the other thread) and DPA for the audio analysis, and have the DPA audio panned left, and synthetic drums panned right)
I did around 40-50 hits for each point (total of 144 if I send a size message to fluild.database~ and fluid.labelset~).
Here’s the latency:
(that 85ms is obviously an outlier given the average vs the min)
I’m really surprised how well it was able to differentiate between the center and the edge of the drum with only 256 samples worth of MFCCs. There are a couple of false positives in there (particularly when I play really fast), but overall really good.
I’ll also try doing all audio coming from the Sensory Percussion pickup since the MFCCs (and knn) likely don’t care that the audio sounds worse from it directly.