Cacophonography: A Community-Generated Map of Sound Powered by Pervasive Computing
Note: slide #17 makes a reference to a flash demo, currently unavailable. I plan to host multimedia on a new website in the near future.
Pushpins riddle a tattered world map in an office, each pin representing the travel destinations of co-workers. A smartphone user calls up a map of coastal Somalia on Google Earth and begins exploring the Wikipedia entries posted there. A person listens to a recording of a gurgling stream, bird song, and wind rustling the leaves of trees, and is reminded of a summer picnic enjoyed years before. These experiences might seem only tenuously related at first glance, but a closer look reveals an opportunity to link these experiences to both create and listen to our locally situated sounds in a web map application, distributed across platforms, including desktop computers and smartphones. What would a map sound like if people placed their recorded sounds on it, like pushpins? What if we could hear those pushpins, or sound-points, playing simultaneously, just as the map allows us to view those locations simultaneously? The result might be a riot of sounds, or a mild chorus of disparate murmurs; in any case, each portion of the map would render its own mix based on what previous visitors left there for us to discover. We might consider this a geography of cacophony, or a Cacophonography. What sorts of potential sonic experiences, cultural knowledges, and geographic perspectives might result from the unique mix of sounds presented by each locality? And if the din proves too unruly, how can we interact with the map and the data to focus on specific types of sounds, or specific places?
Cacophonography is a conceptual project that seeks to imagine the possibilities of a community-generated, web-based map of sound. Underpinning this map application are the ever-expanding capabilities of the latest mobile computing technologies, chief among them smartphones, both for collecting sound recordings associated with specific map coordinates, and for viewing/listening to maps associated with the user’s location. It should be noted, however, that ownership of a smartphone or other mobile computer is not required to use the application. The application is designed to be by and for the community; it will be freely available to anyone with an internet connection regardless of their cell-phone usage, it will be based on open-source geographic data, and its content will be community-monitored. This paper will discuss related research in associating sound with place, efforts to map sound, design and interface considerations, anticipated challenges and constraints, and at the close, speculations on the sorts of geographic perspectives the system could provide if implemented.
To map sound is to situate it in space. Just as places vary in terms of local climate, population density, architectural styles, among countless others, so too do they vary in how they sound, or in the types, frequencies, and intensities of sounds that occur there. Much of the literature on the relationship of sound and place has been spearheaded by landscape architects, environmental designers, and urban planners who have sought to improve urban spaces by understanding how ambient sounds either enhance or diminish their quality. Designers understand sound to be an important component of the information field of urban environments—the means by which we perceive a place to be either relatively inviting or unappealing (Salingaros 1999). Inviting places will be well-used and hostile environments avoided; the ambient sounds permeating these places help to shape this perception. Urban ambient sounds combine to create the urban soundscape, an idea first put forward in 1969 by R.M. Schafer in The New Soundscape (Botteldooren 2006, Guastavino 2006). Since this early work, research on soundscapes has proliferated, including analyses based on sound data collected in the field, at times referred to as noise surveying (Mydlarz, Drumm, & Cox 2008). Once the soundscape of a particular area is understood, the ultimate goal of sonically-oriented planners is to compose new, less stressful soundscapes (Raimbault & Dubois 2005). Contemporary examples have made an explicit connection between sounds and their mapped locations for the purpose of urban planning and design. Architect J. Cohen touches on the efforts of M.J. Shiffer to build multimedia GIS applications that can “accurately simulate traffic noise and model the effect of sound-screening devices or that simulate aircraft sound from various locations, accounting for wind and other variables” (2004, n. p.). A project at the University of Salford, Manchester, UK, is building a web-map of sound by compiling participants’ recordings, captured using a mobile phone, and asking the participants to describe how that sound made them feel (Mydlarz 2009). The work of these urban planners and designers begins to suggest what sorts of information we can glean from a sound map: both the locations and distribution of sounds and their qualitative associations.
Others are interested in what sound can tell us about place in a more broad sense. Sound is complex and evocative; the opportunities afforded by the internet to share sound and map it as a means of documenting the rich variety and diversity of urban experience have inspired a number of interesting applications to date. According to the authors of NYSoundmap.org, “Maps are tools for understanding the world from different points of view—political, cultural, personal, and historical…(NYSoundmap) is at once a historical record and a subjective representation of the city. It is what each user wishes it to be and it is ever growing, ever changing and totally interactive” (n.d., n.p.). Produced by members of the New York Society for Acoustical Ecology, NYSoundmap.org provides a variety of sound locations peppered throughout the city, using Google Maps as a base map. When an icon is clicked, a marker appears with a small amount of metadata, and the map user can press play on the default media player to listen to the associated mp3 or aiff file. In addition to the NYSoundmap, a basic internet search using the terms “sound map” and “sonic map” produces several hits; among them is a sonic map of the island of Capri (radioanacapri.com), a compilation of field recordings produced by the artist Diego Cortez and the Architectural Association Independent Radio, London (n.d.). A particularly useful feature of this web application is a pre-defined list of tags that the user can select to filter for certain types of recordings, i.e. “transport,” and “night/evening.” The map itself is quite minimal, composed of horizontal lines that mimic music notation and refer merely to the island’s coastal outline; the sound-points become “notes” distributed across the island based on their relative locations. A graduate student in music at Queen’s University, Belfast, has created Soundpoints: Belfast, an immersive sound experience, or “locative media piece,” in which sounds collected at various locations throughout Belfast are triggered to play through a mobile phone based on the user’s current location as he or she strolls through a park; the path taken determines the sequence and juxtaposition of sounds experienced (Drury 2006). The application perhaps most removed from a tangible map is the SoundTransit project (soundtransit.nl), a collaboration of three Netherlands-based artists. Initially exhibited at the Museum De Paviljoens (Almere, NL), visitors to the website can “book a transit” by selecting an origin point, destination point, and up to five stopovers. The application then produces an “itinerary:” an mp3 comprised of sounds drawn from locations along the route, aligned linearly. Contributors from across the globe continuously add more field recordings with location details and metadata, expanding the application’s capabilities. Each of these web-based applications expresses an exuberance for the myriad sounds that permeate our world, coupled with a desire to relate them in some way to places, and in doing so, promote knowledge and interpretation of the nature of those places. Cacophonography shares this aim and seeks to build upon these applications by better collecting, sharing, and distributing those sounds in a location-based service context.
Cacophonography creates a web map application that passes sound data and associated coordinates and metadata tags to and from smartphones and desktop/laptop computers. Audio recorded in the field using a smartphone’s built-in microphone is then directly uploaded via a WiFi or cellular connection to the server, along with a GPS coordinate or cell-tower locational fix and any descriptive tags. Those using more conventional field recording equipment, or uploading other types of audio, can post audio to the map using a desktop or laptop computer connected to the internet, specifying the sound-point’s location and descriptive tags in the process. When a portion the map is then viewed at a particular zoom level, the server creates a mix of all the audio available in the map’s viewable extent, panning individual audio sources left and right based on the location of the sound-point relative to the extent, and streams this combined audio out to the user. The user can also choose to call up a descriptive tag cloud available in the extent and filter for specific sounds. Selecting an individual sound-point isolates that audio. Any changes to the map extent, by panning or zooming, has the potential to change the sound-points visible and their position relative to the extent frame, which also modifies the mix of sounds. Smartphone users see their current location on the map relative to nearby sound sources; moving from one place to another has the approximate effect of panning the map and changing the extent. With each change, the server recalculates the mix to send to the user. This aspect of the design may pose the greatest technical challenge, and would rely on a particularly robust cellular connection. Early implementations would likely experience interruptions in the audio stream while re-buffering occurs. Third-generation networks (“3G”) are increasingly able to handle larger data rates while expanding coverage areas; once released, 4G networks should be more than able to handle the Cacophonography interface (International Telecommunication Union 2005).
The smartphone application, or “app,” is an important component to the Cacophonography project. The high cost of mobile devices is an unfortunate reality of implementing pervasive computing applications, as is the exclusive control of cellular networks by a handful of private corporations. Earlier sound map projects with mobile functionality have relied on partnerships with software developers and mobile-phone providers to develop and distribute the applications that collect their participants’ multimedia and locative data. In the case of the Urban Tapestries project, which began in 2002, partners included Hewlett Packard Research labs, France Telecom R&D UK, and the Ordnance Survey (Proboscis 2008). Similarly, the Soundpoints: Belfast project was built upon the Mobile Bristol Toolkit developed by Hewlett Packard (Drury 2006). The Sound Around You project is developing Java-based software for low-end mobile phones with audio capture capabilities and a parallel app that can run on Windows Mobile 5.0+ (Mydlarz 2009). In contrast, Cacophonography would utilize the software development kits (SDKs) that have been released since 2008 for Apple’s iPhone and the Google Android mobile operating system. Recent analysis points to the increasing popularity and market share of these mobile operating systems over their competitors (Hansell 2009). Most promising is the rapidity with which apps for these operating systems can be developed, distributed, and updated, free of charge and directly to the user’s device. The hope is that more and more mobile device providers will follow the example these operating systems and similarly make apps built with SDKs freely available for download. Cacophonography could then operate on a larger number of mobile platforms.
Cacophonography utilizes OpenStreetMap.org as its base map data with a custom map style akin to those currently available and editable through the service provided by CloudMade (2009). Using open-source geographic data assures that the data will always be freely available and sharable. An added benefit in using this base map is the incredible flexibility in manipulating the cartographic appearance of the data. The Cacophonography interface would highlight those sound-points as bright, star-like points on a dark background. The “brighter” the map, the more data the user can anticipate handling in a particular area. When the selection of a particular sound-point, or a tag filter is applied, those points no longer audible would then dim (but not black out entirely). In this way, the relationship between the audio and the visual is further reinforced. Yet since Cacophonography is a map of sound, a manipulation of sound variables is central to the overall experience.
Just as visual variables like hue, saturation, and texture are manipulated in visual design, sound varies in a multitude of ways, offering opportunities for those who produce and manipulate sound, from musicians to acoustic engineers, to contribute to the infinite diversity of sonic experiences available to us. The addition of sound to a map presents specific design opportunities and constraints. Much of the cartographic research on the role of sound in maps and data visualization has focused on the difficult cognitive leap of making sound represent abstract data, with particular emphasis on making data visualization accessible for the sight-impaired (Zhao et al, 2008). Krygier (1994) first formally enumerated sound variables with regards to their application in multi-media maps, which were just beginning to be designed in larger numbers owing to advances in personal computing technology in the mid-1990s (Harrower 2004). Krygier’s approach to the problem is cartographic in nature, assessing the suitability of sound variables for handling either nominal or ordinal data, much like Bertin (1983) had earlier with visual variables. For example, the variable of pitch (the relative frequency of a sound) can be manipulated to express ordinal data, with low sounds representing low values, and high sounds, high values. In contrast, Cacophonography does not represent abstract data, and handles nominal data exclusively: each uploaded audio file is unique to a particular time and place.
Two sound variables in particular, however, can be manipulated to enhance the experience and create correlations between the map extent and the resulting audio stream. The map zoom level is one of the most important interactive features in the map interface. Zooming out from the map greatly increases the number of sound-points in the map extent, adding more and more audio to the stream. While this may be an interesting experience within a certain range of zoom levels, at a certain point the din could become a sort of white noise. Zoom levels are ordinal by nature and present an opportunity to determine the loudness of the audio stream based on zoom, loudness being one of the sound variables enumerated by Krygier (1994). Loudness increases as the user zooms in. Fully zoomed out to a worldwide extent, none of the sound-points visible are audible. The effect is analogous to the silence of near-earth orbit. Sound-points only begin to be barely audible at approximately the regional scale, in keeping with the analogy above, and from a practical standpoint. One can imagine the cacophony of the five boroughs of New York, with many audio sources simultaneously audible. The bird soaring above it would hear quite a range of sounds, all at a reduced level of loudness (the analogy works best if we subtract out the rushing winds of high altitudes). At the neighborhood zoom level, individual sound-points are clearly audible without becoming deafening. Some audio compression at the server end will be required to handle certain neighborhoods that have become dense with sound-points.
The location of sound can vary in two or three dimensional sound space, depending on the complexity of the system emitting the sound (ibid.). It is another variable that can be manipulated in the interface, though it requires (at minimum) stereo speakers, and even a simple left/right stereo experience presents severe limits to the type of variability attainable. In such a set-up, sound can be shifted from the left speaker to the right, but up/down and forward/back shifts are impossible. The interface will allow the user to perceive a sound-point shifting left to right as the map extent is similarly panned left to right, further reinforcing the relationship between the audio stream and the visual map.
Cacophonography is a project that bases its audio data capture and distribution upon pervasive computing technologies—principle among them, smartphones and desktop/laptop computers connected to the internet. Simultaneously, it is a project that seeks to provide a broad forum for sound, from as many voices as possible. Technology-oriented projects must be cognizant of the deep divides that remain in our society in terms of access to technology. Cellular phones are becoming ubiquitous, but smartphones capable of handling an app like Cacophonography are not. Similarly, reliable access to the internet, and the acquired skills to use it, pose major challenges for disadvantaged communities (Thompson 2007). A full discussion of these challenges is beyond the scope of this paper, but future partnerships between the online community, community groups, libraries, and arts and education organizations may be able to bridge the divide by creating opportunities to both experience the map and to capture field audio using community-shared mobile recording equipment, GPS receivers, and/or mobile computers.
A question posed in the beginning of the paper asked: what sorts of potential sonic experiences, cultural knowledges, and geographic perspectives might result from the unique mix of sounds presented by each locality? What would it be like to “listen to” our communities beyond the confines of privately-owned mass-media channels? We can begin to answer that question by imagining a number of potential uses were such an application to become widely implemented. Local radio stations could mark their studio locations with a sound-point, making their contribution to the community mix. Spoken word and live music performances could be mapped and searchable. A band could provide a link on their website to their sound map: all locations associated with their recorded live performances, tracing connections to the sound, their set lists, and their tour itinerary. Audio blogs could trace the paths taken by their authors while recording the ambient soundscapes captured along the way. The number of descriptive tags for audio is potentially epic in its proportions, and could help us associate places with anything from blaring train horns, jackhammers, and gunshots to bustling markets, Saturday recreation league games, and flamenco dancing. Political action and social justice efforts could find geographic as well as sonic representation and a mass audience. These examples only begin to scratch the surface.
Another potential definition of Cacophonography is “dissonance writing.” When the web and the emerging technologies of pervasive computing deliver on a promise to democratize information and information access, the result will be a chaotic and vibrant mirror of our world. Too often, a map presents the voice of one or a few over the voices of the many. Cacophonography provides a forum to celebrate the multiple voices and perspectives of our urban spaces, while relating those voices geographically to their neighborhoods, regions, and the world.
Architectural Association Independent Radio. (n.d.). Radio Anacapri — sonic map of the Island of Capri. Retrieved November 20, 2009, from http://www.radioanacapri.com/
Bertin, J. (1983) Semiology of Graphics. University of Wisconsin Press, Madison.
Botteldooren, D., De Coensel, B., & De Muer, T. (2006). The temporal structure of urban soundscapes. Journal of Sound and Vibration, 292(1-2), 105–123.
CloudMade. (2009). CloudMade Maps. Retrieved November 23, 2009, from http://maps.cloudmade.com/#
Cohen, J. (2004). GIS and planning visualization. Retrieved November 20, 2009, from http://www.jcarchitects.com/GISvisualization.html
Cohen, N. (2009, November 13). Wikipedia – The New York Times. Retrieved November 20, 2009, from http://www.nytimes.com/info/wikipedia/
Drury, D. (2006). Soundpoints: Phonographic practice in the context of locative media. M.A. Thesis, Queen’s University Belfast, Ireland. Retrieved October 29, 2009, from http://www.dubline.com/papers/thesis.pdf
Guastavino, C. (2006). The ideal urban soundscape: Investigating the sound quality of French cities. Acta Acustica United with Acustica, 92(6), 945.
Hansell, S. (2009, October 26). Big Cellphone Makers Shifting to Android System. The New York Times. Retrieved November 20, 2009, from http://www.nytimes.com/2009/10/26/technology/26android.html
Harrower, M. (2004). A Look at the History and Future of Animated Maps. Cartographica: The International Journal for Geographic Information and Geovisualization, 39(3), 33-42.
International Telecommunication Union. (2005). About mobile technology and IMT-2000. Retrieved November 20, 2009, from http://www.itu.int/osg/spu/imt-2000/technology.html
Krygier, J. B. (1994). Sound and geographic visualization. In: Visualization in Modern Cartography. A. M. MacEachren and D. R. F. Taylor (eds.). Oxford: Pergamon, pp. 149-166.
Mydlarz, C., Drumm, I., & Cox, T. J. (2008). Internet and mobile technologies for a public role in noise surveying. Journal of the Acoustical Society of America, 123(5), 3680.
Mydlarz, C. (2009). soundaroundyou. Retrieved November 20, 2009, from http://soundaroundyou.com/#1
nysoundmap.org. (n.d.). Retrieved November 2, 2009, from http://nysoundmap.org/
Proboscis (2008). Proboscis | SoMa | projects | urban tapestries. Retrieved November 20, 2009, from http://urbantapestries.net/
Raimbault, M., & Dubois, D. (2005). Urban soundscapes: Experiences and knowledge. Cities, 22(5), 339–350.
Salingaros, N. A. (1999). Urban space and its information field. Journal of Urban Design. 4(1), 29-49.
soundtransit.nl. (n.d.). . Retrieved November 2, 2009, from http://www.soundtransit.nl/
Thompson, K. M. (2007). Furthering Understanding of Information Literacy through the Social Study of Information Poverty. Canadian Journal of Information & Library Sciences, 31(1), 87-115.
Zhao, H., Plaisant, C., Shneiderman, B., & Lazar, J. (2008). Data Sonification for Users with Visual Impairment: A Case Study with Georeferenced Data. ACM Trans. Comput.-Hum. Interact., 15(1), 1-28.