What may work is to try different descriptor recipes. It may be worthwhile including pitch as well, and/or trying different combinations of descriptors etc…
Here’s a thread that may be of interest too:
It goes into a bunch of detail about other things, but you can see how similar data has very different representations depending on the descriptors you feed in.
Makes a bit of sense. I guess its worth remembering that the means of the reduction are relative to that newly created space, and not necessarily the original space. You can calculate the means of each “cluster” on the original data, and then run that through the same UMAP projection, and then plot that so that way the means are plotting in the same projection. It gets a bit brain puzzley thing about stuff like this.