Musification of Urban Data
Problem
Data sonification is the process of turning data into sounds, in the same way data visualization turns data into visuals. One deals with our eyes, the other with our ears. There are many good reasons for sonifying data: firstly, for accessibility reasons, as visually impaired users cannot enjoy the benefits of visualization, but also to provide an additional channel of information transmission, which could provide enhanced potential for performing data-based tasks. Urban data, which is mostly geospatial, is much harder to be sonified than visualized, because it lacks an extra dimension that allows it to be easily mapped to a plane. However, the large availability of GeoJSON data on platforms such as OpenStreetMap provides an interesting opportunity to explore solutions for this problem.
Musification is a kind of sonification where musical language and musical elements such as rhythm, harmony and melody are used to structure the information. This can be advantageous, as humans are not as well trained to do cognitive tasks with their ears as with their eyes. Our education includes chars and graphs, which have their own visual language, from an early age, but there is no analogous language for sonification. Music comes close at least in terms of exposition, but our perception of music is largely subjective, presenting challenges for musification.
Aim
This project aims to explore the potential of the auditory channel to complement or substitute the visual channel in analyzing urban data. There are three main dimensions of questioning involved:
- Psychological / Music Theoretic: What are the cognitive processes involved in processing sound and music, and how can they justify design choices in visualization and sonification? Can musical theoretical frameworks and elements be translated to a visualization context?
- Aesthetic: both music and visualization should be engaging to the subject, and therefore aesthetics are a central question in their design and production. However, they can also get in the way of objectively performing tasks. How to deal with this problem?
- Technical: What are good technical solutions to design and implement such applications?
Other information
This miro board presents an introductory research into the subject and is filled with resources such as videos, papers, and explanations on this topic.