A dimensionality reduction visualisation

Code Sharing
1

happy new year everyone,

At the recent plenary dimensionality reduction was a topic of discussion and something that I discussed in my presentation.

Something I have wanted to do for a while is create an efficient visualisation tool for looking at results of dimension reduction in order to share the results with others, but also as documentation (for what will inevitably crystallise in my PhD thesis).

As a result, I spent this weekend committing to my idea and whipped up some three.js code to render my dimension reduction work in a navigable 3d space. You can see the results here and also navigate the space with your mouse. I would encourage you to look around the various clusters which are organised by colour. With UMAP, the preservation of local and global features is quite good, and so sometimes geographic clusters contain some coloured sub-clusters.

If anyone is interested in the code that did all the heavy lifting, that is in a git repo linked at the example although its a total mess right now. I’d be happy to go into more detail if that is something anyone is interested in.

disclaimer: this example might not work in Safari. In the more recent versions auto-play for media is disabled by default. Try it out in Firefox or Chrome. Also, playback might be a little slow as it has to download each sample every time you mouseover, but it’s still relatively fast.

Interactive Dimension Reduction Scatterplot

4 Likes
2

Amazing! I am working on similar stuff, but including visualization of the sounds themselves, also on browser. I looked at three but I was not clear about how easy it would be to render a canvas for each sound as I am doing. There seems to be an object for that, so I’ll try… I presented at WAC 2019 comparing three algorithms (including UMAP), plus panzoom for navigation. I’ll share the video when it is online.

1 Like
3

So a kind of ‘zoom’ but you start to unpack more information about individual sounds? That sounds interesting, or perhaps I’m missing the mark here :stuck_out_tongue:

I would be interested to see that! Do let me know

4

Has there been any progress on this topic? I would be very interested.
Hans

1 Like
5

I took the website down because the provider complained I was hosting too much audio on their free tier, something about fair use.

I haven’t touched dimension reduction stuff for a while but I have finished the works that were composed and use dim-reduc to assist in my composition workflow. I need to mix them but I don’t want to do any final releases till I can go back to a well calibrated studio to check detail work :slight_smile: stay tuned

2 Likes
6

I’ve fixed the repo and the link here and it uses substantially less bandwidth (by compressing the life out of the audio).

https://umapaudio.onrender.com/src/interactive_scatter.html

2 Likes
7

Hi James, this looks great. But I was not able to get sound playback. Tried with Safari and Chrome on Mac. Do I need to do anything other than hovering the mouse over a point?

8

Hovering should do the trick. It actually won’t work at all on safari, so try again on Chrome. If the sound is not playing then It’s either not downloaded or very very very quiet. The quieteness is a consequence of the corpus being totally unsanitised (on purpose). It’s meant to be quite messy and unforgiving.

Let me know if you have still have problems on Chrome as this suggests something else is going wrong that I can’t reproduce here.

9

Insisting a second time with Chrome made it work. Thanks