ata visualization has driven the innovation of how we use and digest data over the past decades. A lesser-known technique of this is data sonification, the transformation of data into sound. Listeners are given a different perspective to identify patterns and relationships in data sets that can be better heard than seen. The most common uses for data sonification are building data exploration tools for the visually impaired and creating informal learning environments in museums and zoos. Still, the subject's uses and projects continue to grow.
Experimentation with Max MSP
I explored data sonification using Max MSP, a visual programming language used for sound, graphics, and interactivity. In the first experiment, I turned visual data from my laptop camera into sounds. The program would analyze the most dominant color, convert the color data to a frequency, and then convert it to audio. As the user moved about the camera, the pitch would change, resulting in changes in tone and complexity of the piece. In this interactive audiovisual space, the user uses the tools to create a synesthetic stimulation to hear colors.
I became more interested in learning more about sonification and other artists that have also explored this. It was then when I stumbled upon an article that described how Aphex Twin hid his face in one of his tracks. The image was only viewable with a spectrograph program, a tool that can view and analyze the contents of a music or audio file. In his example, the hidden face was easily masked as ambient music on the track and wouldn't suspect a listener.
I was determined to 'sonify' my face into sound using Max MSP with this concept in mind. After I uploaded an image to the program, I was able to break the image down pixel by pixel and follow a similar concept from the previous project where I converted that pixel data into a sound. After the first pixel was sonified, the program would move onto the next pixel and then repeat. While the overall project was successful, the noise that results is not an easy listening experience. Many listeners were confused since the sounds are abstract or ambient and require a trained ear to decode.
These ambient sounds have real-life applications and work well in the monitoring of processes. In a project by LARAS Sonification Laboratory, listeners can monitor network activity and detect unusual behavior. The differences in noise alert the listener of hacking or abuse disturbing the network. Another tool called Glide Radio allows you to monitor your own back-end systems this way.
Even though the popularity of data visualization is still on the rise, the tool is limited. While data visualization has to be manipulated to fit a screen, the sound always sticks to the data. Even though data sonification is still an emerging tool for pattern recognization and explorations in data, its possibilities are endless.