Towards Real-time Interactive Performance

Environments

Project I: Multiple Players

 Interactivity

“Interaction is a two-way street.” - Winkler (1998)

“Interaction means mutual influence.” - Pressing (1990)

 Project Aims:

• Develop real-time performance analysis strategies through segmentation of an incoming

audio signal;

• Develop musically related yet unpredictable algorithmic responses derived from real-
time performance analysis;

• Explore notions of performer and system control and influence over algorithmic
responses and interactive mappings;

• Provide a balance between fixed and user modifiable interactive mappings.

 Defining Interactive Music Systems:

“Interactive computer music systems are those whose behaviour changes in response to
musical input. Such responsiveness allows these systems to participate in live
performance of both notated and improvised music.” - Rowe (1993)

“Interactive music is (…) a music composition or improvisation where software interprets a

live performance to affect music generated or modified by computers. Usually this
involves a performer playing an instrument while a computer creates music that is in
some way shaped by the performance.” - Winkler (1998)

“An interactive system has the potential for variation and unpredictability in its response,
and depending on the context may well be considered more in terms of a composition or
structured improvisation than an instrument.” - Drummond (2009)
 Interactive Models and Classifications:
Rowe and Winkler

Robert Rowe’s system classifications:

• score-driven vs. performance-driven systems

• transformative, generative or sequenced response types
• instrument paradigm vs. player paradigm

Todd Winkler’s interactive models:

• conducted classical orchestra

• string quartet
• jazz combo
• free improvisation ensemble
 System Anatomies:
Rowe and Winkler

Rowe’s System Anatomy:

Sensing Processing Response

Winkler’s System Anatomy:

Human Input Listening Interpretation Composition Sound

 “Multiple Players”
interactive performance environment for
live instrumentalist and three virtual players

System classification:

• a performance-driven interactive system;

• makes use of generative algorithms in responding to performer input;
• conceptually linked to Rowe’s player-paradigm classification.

System anatomy:

Influence

Human Input Passive Sensing Active Sensing Composition/ Sound

Output
System Input (Hearing) (Listening) Improvisation

Control and Influence

 Performance analysis strategies:
Segmentation and MIDI Representation

• Pitch analysis - when above user specified amplitude threshold, pitch of incoming audio
signal extracted and represented as MIDI note number using FFT analysis techniques - MSP
fiddle~ object;

• Discrete amplitude analysis - amplitude of individual pitches above threshold converted to

MIDI velocity values and packed with corresponding pitches - MSP fiddle~ object and
amplitude-velocity scaling;

• Rhythm - time interval (ms) between successive pitch onsets calculated - Delta-time/Inter-
Onset-Interval (IOI) calculation;

• Articulation - time interval (ms) between pitch onset and pitch end calculated (positive
threshold crossing = pitch onset/MIDI note-on, negative threshold crossing = pitch end/MIDI
note-off).
• Legato movement calculated as equaling Delta-time/IOI (no negative threshold detected
before new pitch onset - MIDI note-off triggered before new MIDI note-on).
 System Response:
Option 1) Probability - Normal Distribution

• Normal distribution treats each event as a discrete event, making output

oblivious to what came before

• Number of occurrences of chromatic pitches in this example:

C3 = 1 C#3 = 0 D3 = 0 D#3 = 0 E3 = 2 F3 = 0
F#3 = 0 G3 = 1 G# = 0 A3 = 0 A#3 = 0 B3 = 0
C4 = 3

• Probability weightings: C4 - 3, E - 2, C3 - 1, G - 1
 System Response:
Option 2) Probability - 1st Order Markov Chain

• Markov chains take transitions from one state to another into consideration

• Transition weightings for current example (after one repetition):

C3 - C3 = 0 E3 - C3 = 0 G3 - C3 = 0 C4 - C3 = 1

C3 - E3 = 1 E3 - E3 = 1 G3 - E3 = 0 C4 - E3 = 0

C3 - G3 = 0 E3 - G3 = 1 G3 - G3 = 0 C4 - G3 = 0

C3 - C4 = 0 E3 - C4 = 0 G3 - C4 = 1 C4 - C4 = 2
 Levels and Directions of Control:

• Start/Stop Performance:
• live performer → all players; (held note trigger) - predictable, user-defined
• live performer → one player; (note count after rest) - semi-predictable, user-defined
• one player → one player; (note count) - semi-predictable, user-defined

• Memory Loss:
• live performer → one player; (note count after rest) - semi-predictable, user-defined
• one player → one player; (note count) - semi predictable, user-defined

• Synchronised Playback:
• live performer → players 1 & 2; (held note trigger) - predictable, user-defined
• live performer → one player; (time threshold) - predictable, fixed
• one player → one player; (time threshold) - predictable, fixed
 Directions of Influence:

• Influence is defined as the possibility that the output of either the live performer or any
virtual player will directly or indirectly affect the potential future output of another

• live performer output → one player output; semi-predictable, user defined

• one player output → one player output; semi-predictable, user defined
• any/all players outputs → live performer output; unpredictable