Seeing Sound: A Multi-Sensory Design Experiment

Hearing the Landscape

Written by Yue Wu.

In 2016, I took a composition class at Cornell’s music department with Dr. Marianthi Papalexandri-Alexandri. The class was about contemporary composition and exploration of ways to create “music” in non-traditional ways: you pick a piece of material, which will become your instrument, and invent a way to play with it, the performance. This hands-on way of creating music helped me realize that music is as tangible as audible. Further, all types of sensory experiences are inherently fluid: seeing a smooth pebble can prompt one to imagine the smooth touch of its surface, or likewise, the clattering sound it makes when we kick it across the pavement.

This caused me to wonder if design can create new “pathways” for different senses to connect, and in that way, help amplify the experience in multiple senses. In the exercise “Seeing Sound,” I designed an interactive sculpture that is in the shape of a hand, saying “put on hearing-aid” in American Sign Language.

The concept is that the sculpture’s façade panels will change color according to the surrounding sound. On a color wheel, colors closer to red imply human or human activities-related sound, colors closer to black imply industry-related sound, and colors closer to green imply nature-related sound. In theory, the changing colors of the sculpture in real time would illicit the sound of the built environment from background to foreground. It also provides a new way to re-experience the urban soundscape both in visual and auditory ways.

Morning.

Midday.

Night.


Yue joined Landworks Studio in July 2020 as an entry-level landscape designer. She received a Bachelor of Science in Urban and Regional Studies and a Bachelor of Arts in Music from Cornell University, and later a Master of Landscape Architecture from Harvard Graduate School of Design. With her training in auditory and visual design, Yue is interested in the interplay of sensory stimuli and how visual, auditory, and tactile sensations influence the experience of a landscape.