NIME 2015 Scientific Program - MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration

MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration

Much of the challenge and appeal in remixing music comes from manipulating samples. Typically, identifying distinct samples of a song requires expertise in music production software. Additionally, it is difficult to visualize similarities and differences between all samples of a song simultaneously and use this to select samples.
MusicMapper is a web application that allows nonexpert users to find and visualize distinctive samples from a song without any manual intervention, and enables remixing by having users play back clusterings of such samples. This is accomplished by splitting audio from the Soundcloud API into appropriately-sized spectrograms, and applying the t-SNE algorithm to visualize these spectrograms in two dimensions. Next, we apply k-means to guide the user's eye toward related clusters and set $k=26$ to enable playback of the clusters by pressing keys on an ordinary keyboard. We present the source code (https://github.com/fatsmcgee/MusicMappr) and a demo video (http://youtu.be/mvD6e1uiO8k) of the MusicMapper web application that can be run in most modern browsers.