Sonic Browsing

browsing with your ears

Sonic Browser version 01
Sonic Browser version 02

Sonic Browser v2

Finding a sound or music on your computer - if you don't know the title or some other text-based attribute - is not easy. It may involve scrolling, clicking, double-clicking, listening and it can be quite a tedious process.

With the Sonic Browser, I tried to enable ears to do what they are best at - listening. Facilitating multiple stream audio in realtime in a tightly coupled interaction solved the problem. Each sound object (an arbitrary sound file, a piece of music, etc.) was shown a visual symbol on screen. A collection of files could be displayed for example in a grid or as a star field visualisation.
The cursor is not a single point device anymore, it has an aura - an area around the cursor that activates sound objects. If for example 4 sound objects are covered by the aura, all four will play simultaneously, panned out in stereo to help support tightly coupled interaction between ears and mouse actions - direct sonification.

I found that users could locate a target sound 28% faster than with single-stream audio. With this interaction style users experience an increased level of engagement using both visual and aural senses.

Sonic Browser version 03

Sonic browser v3

Together with my graduate students, we developed more and improved versions of the Sonic Browser. Adding interaction for zooming, selection and filtering to the search and selection critera enabled users to navigate really large collections of sound (or music) files.

We continued to use and develop the Sonic Browers in the Sounding Object project.

Sonic Browser demo

This is a simple demo with 16 sounds, written in javascript

Javascript and embedded mixed media content using http has to be enabled on your device to run the demo.

Publications

  • Fernström, M. & Bannon, L., 1997. Explorations in Sonic Browsing. In H. Thimbleby, B. O’Conaill, & P. Thomas, eds. HCI on People and Computers XII. Bristol, UK: Springer Verlag, London, pp. 117–132.

  • Fernström, M. & McNamara, C., 1998. After Direct Manipulation - Direct Sonification. In ICAD. Glasgow, Scotland: Springer-Verlag, pp. 1–5.

  • O Maidin, D. & Fernström, M., 2000. The Best of Two Worlds: Retrieving and Browsing. In D. Rocchesso & M. Signoretto, eds. DAFx. Verona: Universita degli Studi Verona, pp. 131–134.

  • Fernström, M. & Brazil, E., 2001. Sonic Browsing: an Auditory Tool for Multimedia Asset Management. In J. Hiipakka, N. Zacharov, & T. Takala, eds. ICAD. Helsinki, Finland: Laboratory of Acoustics and Audio Signal Processing and the Telecommunications Software and Multimedia Laboratory, Univeristy of Helsinki, Espoo, Finland, pp. 132–135.

  • Brazil, E. et al., 2002. Enhancing Sonic Browsing Using Audio Information Retrieval. In R. Nakatsu & H. Kawahara, eds. ICAD. Kyoto, Japan: ATR, Kyoto, Japan, pp. 113–118.

  • Ottaviani, L. & Fernström, M., 2003. Investigations with the sonic browser on two of the perceptual auditory dimensions of sound objects: elasticity and force. In Proceedings of Digital Audio Effects DAFx-03. London, UK

  • Ottaviani, L., Brazil, E. & Fernström, M., 2003. Psychoacoustic experiments for validating sound objects in a 2-D space using the sonic browser. In Firenze, Italy

  • Brazil, E. & Fernström, M., 2003. Where’s that sound? Exploring arbitrary user classifications of sounds for audio collection management. In B. Shinn-Cunningham, ed. ICAD. Boston, MA: Boston University Publications, pp. 66–70.

  • Brazil, E., Fernström, M. & Ottaviani, L., 2003. A new experimental technique for gathering similarity ratings for sounds. In B. Shinn-Cunningham, ed. ICAD. Boston, MA: Boston University Publications, pp. 238–242.

  • Brazil, E., Fernström, M. & Ottaviani, L., 2003. Psychoacoustic validation and cataloguing of sonic objects: 2D browsing. In D. Rocchesso & F. Fontana, eds. Sounding Object. Firenze: Mondo Estremo, pp. 297–339.

  • Rocchesso, D., Bresin, R. & Fernström, M., 2003. Sounding objects. IEEE Multimedia, 10(2), pp.42–52.

  • Fernström, M. & McNamara, C., 2005. After direct manipulation - direct sonification. ACM Transaction on Applied Perception, 2(4), pp.495–499.

  • Fernström, M., 2005. Reflections on sonic browsing: Comments on Fernström and McNamara, ICAD 1998. ACM Transaction on Applied Perception, 2(4), pp.500–504.

  • Brazil, E. & Fernström, M., 2011. Navigation of data. In T. Hermann, A. Hunt, & J. G. Neuhoff, eds. The Sonification Handbook. Berlin, Germany: Logos Verlag, pp. 509–524.