Sound Design

Pedro Rebelo 2006

URL: http://www.sarc.qub.ac.uk/~prebelo/teaching/sd

Sound in Cinema

Sound in Cinema

We gestate in Sound, and are born into Sight

Cinema gestated in Sight and was born into Sound
Walter Murch (in Michel Chions Audio Vision)

From sonic acuteness to the explosion of senses

Cinema: Mechanistic Reversal

Sound in Cinema

Silent Cinema (1892 - 1927)

1927: Synchresis (Synchronism + Synthesis) (Chion)

Sonic environment composed of live music (piano, organ), audience,

theatre sounds etc...

The spontaneous and irresistible mental fusion, completely free of any

logic, that happens between a sound and a visual when these occur at
exactly the same time (Chion, 1994 p. xviii).

Sound in Cinema

Fabricating reality: actress on screen with different voices

From dubbed voice to hyper-real sound
Rendered Sound : more real than real (Chion)
Sound sources often do not correspond to actions on screen

Both Marlene Dietrich and Greta Garbo did not

speak a word of English when they began to work in
Hollywood

Sound Design

Separation (by virtue of film practice) between what we see and

what we hear

A layer of sound fertilises the image layer (not always by

mimicking it)

Sound communicates physical qualities (material, weight, mass,

volume, tension, force...)

Sound Design

Complex set of relationships in a synchronised audio-visual environment:

Visual stimuli focusing listening

Audio stimuli defining visual time
Discrepancy between audio and visual stimuli providing dual modality perception

Sound Design

Visual stimuli focusing listening

The panning of a video camera is able to focus our listening

to a particular layer (e.g. orchestral concert broadcast)

Example: Elis Regina & Tom Jobim Aguas de Maro

Sound Design

Visual stimuli focusing listening

The panning of a video camera is able to focus our listening

to a particular layer (e.g. orchestral concert broadcast)

Example: Elis Regina & Tom Jobim Aguas de Maro

Sound Design

Audio stimuli defining visual time

Sound conveys time information more accurately than visuals

(e.g. the moment the gun is shot is primarily a sonic event)

Sonic acceleration and deceleration can modulate visual

rhythm

Rdn#06 Underworld (onedotzero_select dvd - 07)

Visual movement/speed is perceived in two different modes by virtue of two contrasting sound environments.

Sound Design

Discrepancy between audio and visual stimuli providing dual

modality perception

Non-linear interaction between audio and visuals

Sound can be used to occlude expected events (e.g. parts of
a dialogue). (Jean Luc Goddards Weekend)

8 1/2 Women, Peter Greenaway1999

Chapter 1
Opening credit sequence features a shot of nigh-time urban cityscape
with the sound of birdsong.

Designing in Context

Designing in Context

Sound Design for screen or visual media is context specific: Design

of connections as much as design of materials

John Cage / Merce Cunningham collaboration: releasing accidental

connectivity

Contextual elements such as performance situation and particularly

timing (duration) are agreed, whereas micro-structure events are
left to chance

Designing in Context

Excerpt from Variations V

John Cage / Merce Cunningham
John Cage made Variations V in 1965 for the Merce Cunningham Dance Company. He and David Tudor settled on two systems for the sound to be affected by movement.
For the first, Billy Klver and his colleagues set up a system of directional photocells aimed at the stage lights, so that the dancers triggered sounds as they cut the light
beams with their movements. A second system used a series of antennas. When a dancer came within four feet of an antenna a sound would result. Ten photocells were
wired to activate tape-recorders and short-wave radios. Cecil Coker designed a control circuit, which was built by my assistant Witt Wittnebert. Film footage by Stan
VanDerBeek and Nam June Paik's manipulated television images were projected on screens behind the dancers.
The score was created by flipping coins to determine each element and consisted of thirty-five remarks outlining the structure, components, and methodology. The
specific sound score would change at each performance as it was created by radio antennas responding to the dancers' movements.
http://www.medienkunstnetz.de/works/variations-v/video/1/

Designing in Context

Sound design for the screen, in its most inventive moments,

suggests an approach in which image and sound interact by a
counterpoint of editing and a process of layering that brings
forward both sound and image at different times.

The exploration of provocative and unusual audio-visual situations

is a primary concern in a wide range of film and new media
practice. Mimicking is not enough...

Three Listening Modes

(Chion, 1990)

Understanding the relationship between sound, its source and its perception...

Causal Listening
Semantic Listening
Reduced Listening

Causal Listening
Listening to a sound in order to gather information about its cause or source
The most common listening mode

how much water in the bottle?

We recognise a persons voice
Without it necessarily corresponding to an image (radio presenter)
CATEGORIES of sources: animal, human, mechanic
MORPHOLOGIES of sound (causal history)
Scraping = acceleration, changes of pressure, speed, amplitude, timbre
We recognise scraping as a sound producing action
(without knowing the precise nature of materials)
In cinema, the identification of cause is dependent on:
the audio-visual contract and the phenomenon of synchresis

Semantic Listening
Listening as referring to a code of language
The interpretation of a message

what does that mean?

At the centre of linguistic studies
Not about the acoustic properties of an individual event (phoneme)
but rather its contribution to a system of differences and oppositions
e.g. consonants, vowels, diphthongs...

Reduced Listening
Focusing on sound itself, independent of cause or meaning
Pierre Schaeffer

how does that sound?

Rooted in fixed sounds - recording
Infinite repetition directs our attention away from cause or meaning
Ear Training
Similar to traditional music ear training (identifying an interval between two notes)
Pierre Schaeffer Trait des object musicaux (1966)
System of classification that suggests a solfeggio for recorded sound objects
Reduced listening,
as a discipline allows for the sound designer to engage with the media
and explore the perceptual and acoustic richness in any sound

Reduced Listening
and Acousmatic Sound
In an acousmatic listening situation the sound source is not visible.

how does that sound?

The Acousmatics
Pythagoras disciples promoting lectures behind the screen
Obscuring the source
would focus the audience on sound - the message
without the distractions of bodily presence
Sound as primordial mode of communication (also theorised by Marshall McLuhan)
Acousmatic Music is defined by composer Francois Bayle
as shot and developed in the studio, and projected in a hall, like a film (Bayle)
sound composed to be played over an array of loudspeakers

Phythagoras Footsteps
Vision is typically enclosed by a stable frame (painting, photography, cinema, computer screen...)
it directs/crops our gaze
In sound there is no frame...
Analysis of sound in cinema refers to the role of the frame:
diegetic sound - belongs to actions in the frame
characters speaking, objects being moved etc...
non-diegetic - sounds outside the frame
background music

Phythagoras Footsteps
diegetic sound - belongs to actions in the frame
characters speaking, objects being moved etc...
non-diegetic - sounds outside the frame
background music

http://www.stanford.edu/~gdegroat/PF/pictures.htm

Phythagoras Footsteps
Numerous examples problematise this categorisation.
In particular, technological presence in film:
radio, computer voices, recorded conversations...
Chions terminology:
Offscreen - not visible but part of the frame
Onscreen - visible action for sound
Non-diegetic - non visible

Michel Chion Audio Vision

Phythagoras Footsteps
Chions extended terminology:
Ambient Sound (Territory-Sound)
sound that envelops a scene and inhabits its space without raising the question of the identification or visual embodiment of its
source (Chion, p.75)
sound as a way of identifying a particular space or situation (church bells, birdsong, etc...)

Internal Sound
Situated in the present action, physical and mental interior of a character
internal voices, bodily sounds...

On-the-Air Sound
Transmitted electronically onto the scene

Phythagoras Footsteps
Dimensions and Oppositions in film sound:
Opposition between acousmatic and visualised
Opposition between objective and subjective or real and imagined
Difference between past, present and future

Michel Chion Audio Vision

Elements in a Soundtrack
Ambience
Triggers of motion
Evocation of space or situation through sound
Dependent on references (non-narrative)
e.g. voice whispers, whistling wind...
Earcons
Imply an immediate response (e.g. Carl Stalling)
User interface sounds... not spatially defined, somewhat arbitrary

No Soundtrack
Sound element subsidiary to narrative (or film structure)
Integrated with visuals to produce one single effect
The relationship between sound and image often changes throughout a film; they become so
intertwined that it becomes impossible to separate them and still treat either element as a
coherent entity.

Forced Marriage

Exercise: Source Library

In preparation for the next two assignments the class will work as a group
in recording and editing sound sources

Sound Object Groups:

A:
B:
C:
D:
E:

Footsteps
Movement
Environments (indoors and outdoors)
Impact and Collision
Sound Events from Analysis Files

Exercise: Source Library

1. Record as a group (Experiment!)

2. Divide up Editing Tasks
3. Catalogue all edited files as appropriate
4. Present Sound Library to Class

Audio-Visual Analysis
Excerpts for Analysis:
Artificial Intelligence (S. Spielberg 2001)
Delicatessen (Jeunet & Caro, 1991)
Russian Ark (A. Sokurov, 2002)

Itemize the different audio elements

which is dominant and what point?
Characterise consistency
how do the different elements interact?
balance, reverberation, masking
Points of Synchronisation
points that are crucial for meaning and dynamics
Sound-Image Comparison
formal relationships in speed, materials, definition, distance, scale
technical comparison between camera movement and the soundtrack
compare by observing audio and visual elements separately
What do I see of what I hear?
What do I hear of what I see?
Metaphor through Sound
Are there any metaphorical relationships articulated specifically through sound?