This material is part of a larger project that explores perception and imagination in interactive media, and how they are created by sound, image, and action. It marks an early stage in extending my previous work on the virtual kinoeye in virtual worlds and video games to consider aural, haptic, and kinetic aspects of embodiment and interactivity. In particular, the argument is that innovative sound has the potential to bridge the gaps in experiencing embodiment caused by disconnections between the perceived body in physical, representational, and imagined contexts. To explore this, I draw on Walter Murch’s spectrum of encoded and embodied sound, interviews with sound designers and composers, ways of thinking about the body from phenomenology, and analyze examples of the relationship between sound effects and music in video games.
A 20-minute presentation for the Society for Cinema and Media Studies conference in Chicago is here. The longer version (about 35 minute presentation, is here.)
Clips (including examples ommited from the SCMS presentation) are below.