The Creative Technologist Blog

Data visualization has driven the innovation of using and digesting data over the past decades. A lesser-known technique of this is data sonification, transforming data into sound. Listeners are given a different perspective to identify patterns and relationships in data sets that can be better heard than seen. The most common uses for data sonification are building data exploration tools for the visually impaired and creating informal learning environments in museums and zoos. Still, the subject's uses and projects continue to grow.‍

Experimentation with Max MSP

I explored data sonification using Max MSP, a visual programming language used for sound, graphics, and interactivity. In the first experiment, I turned visual data from my laptop camera into sounds. The program would analyze the most dominant color, convert the color data to a frequency, and convert it to audio. As the user moved about the camera, the pitch would change, resulting in changes in tone and complexity of the piece. In this interactive audiovisual space, the user uses the tools to create a synesthetic stimulation to hear colors.

I became more interested in learning more about sonification and other artists exploring this. It was then that I stumbled upon an article that described how Aphex Twin hid his face in one of his tracks. The image was only viewable with a spectrograph program, a tool that can view and analyze the contents of a music or audio file. In his example, the hidden face was easily masked as ambient music on the track and wouldn't suspect a listener.

The Aphex Face: Visualizing The Sound Spectrum From '#2' - Magnetic Magazine

I was determined to 'sonify' my face into sound using Max MSP with this concept in mind. After I uploaded an image to the program, I was able to break the idea down pixel by pixel and follow a similar concept from the previous project where I converted that pixel data into a sound. After the first pixel was sonified, the program would move onto the next pixel and then repeat. While the overall project was successful, the results' noise is not an easy listening experience. Many listeners were confused since the sounds are abstract or ambient and require a trained ear to decode.

These ambient sounds have real-life applications and work well in monitoring processes. In a project by LARAS Sonification Laboratory, listeners can monitor network activity and detect unusual behavior. The differences in noise alert the listener of hacking or abuse disturbing the network. Another tool called Glide Radio allows you to monitor your back-end systems this way.

Even though the popularity of data visualization is still on the rise, the tool is limited. While a developer must manipulate data visualization to fit a screen, the sound always sticks to the data. Even though data sonification is still an emerging tool for pattern recognition and data exploration, its possibilities are endless.

You’ve successfully subscribed to Augmented
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Your link has expired
Success! Check your email for magic link to sign-in.