NUNTIUS: A Computer System for the Interactive Composition and Analysis of Music and

Dance
Author(s): Tamas Ungvary, Simon Waters, Peter Rajka
Source: Leonardo, Vol. 25, No. 1 (1992), pp. 59-68
Published by: The MIT Press
Stable URL: http://www.jstor.org/stable/1575623
Accessed: 06/03/2009 08:37

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=mitpress.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.

JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the
scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that
promotes the discovery and use of these resources. For more information about JSTOR, please contact support@jstor.org.

The MIT Press is collaborating with JSTOR to digitize, preserve and extend access to Leonardo.

http://www.jstor.org
TECHNICAL ARTICLE

NUNTIUS: A Computer System

for the Interactive Compositionand
Analysis of Music and Dance

Tamas Ungvary,
Simon Watersand
PeterRajka

ABSTRACT

examines
Thisinvestigation
THE RELATIONSHIP ture and cross reference (the thedegreetowhich notational
BETWEENMUSIC AND DANCE possibility of accurate referral ofthetwomediaof
integration
forward or backward through a musicanddanceis possible. Adis-
The nature of the relationship between music and dance is madebetween
tinction theinter-
given musical/choreographical actionofmusicanddance,andthe
has frequently been investigated, but nearly alwaysfrom the form). This foundation con- issuesofthecomposer/
practical
perspective of only one of the two media [1]. Attempts to tributed crucially to the move- choreographerrelationship,witha
integrate the two media have often been frustrated by the ment of Western music toward tentative ofboth.The
classification
lack of a two-sided approach to understanding the nature the harmonic and rhythmic authorsthendescribe theircom-
puter-basedcompositionaland
of their interaction. Their integration defines an area complexities exhibited in the choreographicsystem,NUNTIUS,
where, for primarily historical reasons, theory lags behind early twentieth century. which directdatatransfer
provides
practice. Whereas music has developed to a great extent as The transmissionand storage andinterpretation
(structural
pat-
a notated medium, dance notation has remained a relatively of experience is not, however, ternsorprocesses) between the
diverse and scattered phenomenon, without a comparable restricted to notational systems: twomedia.
continuous tradition of development and use. As analysis the medium of internalized I
has historically been based on documentary (i. e. notated) storage-the human biological
rather than primary (i. e. performed) evidence, a significant system-occupies as important
body of work defining the compositional aspects of dance a historical and cultural position as any representational
has emerged only since the beginning of this century. tradition. Oral and physical transmissions of information
Moreover, music has traditionally been seen as the vehicle have been more important for dance, as a medium with less
of spiritual and abstract expression, while dance has gener- developed notational traditions [6] than those of Western
ally been regarded as a representation of corporeal and music. Indeed, it is arguable that without kinetic memory,
mimetic or narrative aspects of expression [2]. or the body's ability to store motor functions, the medium
As most analytical or explanatory systems within a given of dance could not have developed at all. In current prac-
medium tend to rely on some degree of 'parameterisation', tice, choreography is still dependent to a large extent on
a potential pitfall is the temptation to base attempts at dancers' internalized storage of kinetic and motor func-
integration on a naive mapping of parameters [3] from one tions. Therefore practicalities of structuring and cross-
medium to another (e. g. pitch to movement height). This referencing within a complex dance composition grow or-
approach fails to recognise that more convincing analogies ganicallyout of a dancer's abilities to 'store' such instructions
between the two media of music and dance exist at a higher accurately and to provide feedback to the choreographer,
level in the compositional process, in the areas of structur- not only of the movements but also of the ideas and situa-
ing devices and the manipulation and processing of materi- tions out of which they arose. That the structural complexity
als (the strategies used by composers in any medium to of dance composition began to parallel that of music only
determine the coherence of formal structure) [4]. in the twentieth century is nevertheless explicable only
partly in terms of the lack of a continuous notational tradi-
tion. The sociocultural position occupied by Western 'artis-
THE ROLE OF NOTATION tic' dance has historically been rather different from that of
music, playing at certain points a primarily 'decorative' role.
The impulse to represent or record experience is charac-
teristically human, and the development of increasingly
sophisticated systems of representation (or notation) both
reflects and reinforces an increasing historical conscious- Tamas Ungvary (composer, researcher), Royal Institute of Technology, Department of
ness. Musical notations, which initially played a descriptive Speech Communication and Music Acoustics, Box 70014, S-100 44 Stockholm, Sweden.
role, have over time come to occupy an increasingly pre- Simon Waters (composer, researcher), Music Dept., Bath College of Higher Education,
Newton Park, Newton St. Loe, Bath BA2 9BN, United Kingdom.
scriptive role that serves to define possibilities as much as Peter Rajka (choreographer, researcher), Royal Institute of Technology, Department of
it does to preserve or record known experiences [5]. The Speech Communication and Music Acoustics, Box 70014, S-100 44 Stockholm, Sweden.
musical notation system that emerged in the fifteenth cen- Received 27 February 1989.
tury provided the foundation for formal compositional struc-

? 1992 ISAST
PergamonPresspic.PrintedinGreatBritain.
0024-094X/92$3.00+0.00 LEONARDO, Vol. 25, No. 1, pp. 59-68, 1992 59

Fig. 1. Schematic model of creative and communicative processes within NUNTIUS.
MOVEMENTNOTATION
Movement can most easily be recorded
with some degree of accuracy by re-
lating the positions of different body
parts. This results in a large number of
potential parameters, but without con-
sidering other essential features such as
kinetic energy, timing, speed, weight of
movement and interrelationships be-
Fig. 2. The schematic representation of the existing internal software and data-communication structure of both Macrosticon (on the left)
and Motographicon (on the right).
60 Ungvary,Watersand Rajka:NUNTIUS
joint interaction have only recently been
tween different bodies. The phenom-
enological problem posed by the
immense complexity and variability of
body movement, even at its most basic,
must have acted as a deterrent to poten-
tial developers of notational systems.
Initial attempts at symbolic or alpha-
betic dance notation were gradually
replaced by simple sketches of dance
movements. The earliest methodical sys-
tem of a symbolic type was developed by
Beauchamp and published by Feuillet
[7] in 1700 at Louis XIV's Academy of
Dance-the first institution to deal with
the 'science of dancing'. Comparably
rigorous study only re-emerged at the
beginning of the twentieth century.
The joints and fingers of the human
hand can be used to illustrate the prob-
lems of notating [8] or modelling [9]
complex movement interactions. How-
ever, there are limitations. We cannot
use sign-language animation on a com-
puter screen to communicate with deaf
people. The biomechanics of finger-
modelled and animated to a degree of
subtlety and understanding in three
dimensions. (The problems of finger-
movement representation and dynamic
control have been described by several
authors [10].) It is no accident that
music notations rarely refer directly to
specifics of human movement and that
those that have done so (e. g. lute tab-
lature) have fallen into disuse.
MUSIC/DANCE,
COMPOSER/
CHOREOGRAPHER
At this point it is important to distin-
guish between the interaction of music
and dance (which involves external vari-
ables such as an audience's reaction) Labanotation
AL
and the practical relationship between { c

composer and choreographer. On a 3 a

E
cn o o c
theoretical level, four types of dance/
0
music interaction can be defined-ana- v
ala
E
0o
logue, dialogue, structural interdepen-
c
0
._
(r ^o en n o
dence and total independence. These L- o
c,

represent points on a continuum, and, aX

in practice, dance may operate with

music on various levels spanning several time
of these theoretical categories [11]. the left side | the right side

(Horizontal) direction symbols Duration

Dance/Music Interaction forward

Analogue. Dancers do not necessarily n 5
respond to the rhythmic impulse or
melodic line of music, nor do they nec-
essarily interpret the music in any sub- left < D [> right
jective or stylised way. However, when
this type of response and interpretation D place
tDie l
occurs, and when the music adds weight
to the dancer'smovement through rhyth-
back
back time

mic or phrasic reinforcement, the sim-

plest and most obvious relationship ex-
ists: a parallel, or analogue, one. This is
the principle that operates in most folk,
popular and narrative dance. Analogue
relationships can be seen in their most
literal form in popular animated car-
toons, such as Tom and Jerryand Mickey
Mouse.This direct correspondence be-
tween musical rhythm and movement
rhythm also invariably operates at
higher structural levels (musical 'move-
ments', large phrase units, and so
forth) [12].
Dialogue. This is an attempt to liber-
ate the two media from the task of
merely reinforcing or duplicating each
other. Dialogue does not preclude such
reinforcements; indeed, the essence
of dialogue is its use of contradiction,
divergence and displaced imitation
between visual and musical elements,
which presumes some degree of an-
alogical interaction as a reference.
Naturally, this 'contradiction' might
occur at various compositional levels
(rhythmic, phrasal, gestural and so
forth) [13].
Structural Interdependence. Conven-
tionally, dance and music have been
mutually dependent in that one of the
elements (usually dance) has relied
on the other for its form. Most music
for dance can be performed indepen-
dently of the dance and still make
'sense' (although this is not necessarily
the same 'sense' that is made when it
is combined with dance). An example
is the (usually) programmatic music
that has occupied the role of accom-
paniment to (semimimetic) classical
ballet [14], which becomes somewhat
Notation of Motographicon
Symbolic
k
time
Horizontal direction symbols Duration
forward
E
n
zero tim
I
left I Nright
direction
back
i ELe
time
Fig. 3. Comparison of the principles of score layout, and the symbols for horizontal direc-
tion and duration, for (top) Labanotation and (bottom) Symbolic Notation.
more abstract when performed alone. cesses and methodsofmanipulatingmate-
In a situation in which either element rial.Common procedures might include
is inadequate without the other, a state retrograde/inversion, fragmentation, re-
of structural interdependence exists. petition/accumulation, dynamic/speed
This may be causal, as in some African change and stochastic, aleatoric or
and some contemporary Western other rule-based processes [16].
dance in which music is generated by Total Independence. Practicallyspeak-
the dancers (e. g. with wrist and ankle ing, this category can be discounted for
bells) and forms part of the kinesthetic its failure to take into account the per-
experience [15]. A state of structural ceptual mechanisms of the audience. Hu-
interdependence can be more abstract, man beings inevitablytend to construct
as occurs when choreographer and connections between any experiences
composer work with common compo- that are coincident in time [17]. This
sitional procedures, structuring pro-

Unguary,Watersand Rajka:NUNTIUS 61
 phenomenon is referred to as the may simultaneously develop material, completion and final revision of a work.
'Gestaltprinciple of perception' [18]. adjusting or updating it after consul- The notational systems used are usually
tation. (3) The composer writes music Laban [20], Benesh [21] or Eshkol-
for pre-existing choreography. This is a Wachmann [22], all of which were
Composer/Choreographer relatively infrequent relationship-few developed in the middle of this century
Interaction composers are able to read dance nota- and continue to be refined. These sys-
The relationship between composer and tion-but has become less rare since tems have allowed the development
choreographer is more easily defined the advent of video recording [19]. of analytical thinking within the dance
than that between music and dance, This relationship effectively gives the world; however, they are still used al-
and usually takes one of three forms: composer an additional choreographic most exclusively as descriptive rather
(1) The choreographer workswith a pre- responsibility. than prescriptive tools.
existing composition. (2) The choreog- An additional element in the chore-
rapher and composer are involved ographer/composer relationship is the
collaboratively.This may simply entail a dance notator, whose role is necessi- NOTATIONALPURPOSE
choreographer commissioning a musical tated by the fact that many choreog- Since the three most common dance-
work, with some discussion taking place raphers and dancers do not read or notation systems have functioned pri-
beforehand, or it may involve the com- write dance notation. The notator may
marily as recording systems rather than
poser in all choreographic rehearsals. record the dance movements either as creative compositional tools, it is
Or the composer and choreographer during choreography or at the time of not surprising that their primaryinade-
quacy is in the area of originating and
Fig. 4. Comparison of Symbolic Notation, ANIMA and Labanotation. (top) Notations of a shaping large structures. A flexible
vertical-direction body-part movement. (middle) Notations of a subordinate body-part notation system should provide the
movement. (bottom) Notations of flexed body-part position. opportunity for structure-building and
a) manipulation of materials, while re-
Symbolic Notation Angle Illustration Labanotatlon
maining efficient at storing and trans-
mitting information. (The systemshould
be both accurate and easy to learn
and read.)
The disparity of purposes between
i 135,'
! existing music notations and dance no-
tations (the former both prescriptive
TB| d 1125''
125 and descriptive, the latter mainly
descriptive) has contributed to an in-
a adequate appreciation of the common
. . 67.5'
EN
45' compositional (structural and manipu-
lative) concepts underlying the two
media. This lack of appreciation is
E A 22.5A
related to an inability to reflect para-
metric development in time as recog-
00 90?
0? 180? nizable visual patterns that allow con-
trol over the spatial-temporal design of
b)
the formal elements of a dance com-
position. The creation of a common
means of relating and comparing ele-
ments at any compositional level-
from the smallest parameter to the
largest formal structure-would illumi-
.... 112.5.......... nate this common compositional ground
S r and allow choreographer and compo-
(j 90. v / ser to communicate their processes and
I. A6751
~
............ _
6
structures with a degree of accuracy
previously unattainable. Dance struc-
tures and processes could be applied to
musical material, and vice versa, and
such structures and processes could be
0' 90' 180?
developed in the abstract and applied
to both media. These new methods of
c) exchange would naturally supplement,
Symbolic Notation
Symbolic Notation Labanotation
Labanotation rather than supplant, existing modes
of communication between composer
^
I:;
^ ^i and choreographer.

El6 J

62 Ungvary, Waters and Rajka: NUNTIUS

THE NUNTIUS SYSTEM SymbolicNotation Illustration Labanotation

Hopeful of achieving this new level of

interaction between music and dance,
we developed a crossmedia computer
system, NUNTIUS (messenger) [23],
which allows choreographer and com- h
poser to communicate structure and
process in a convenient manner. NUN-
TIUS is the generic name of a 0
computer-software system that ulti-
mately will allow such communication Lii
with additional media, particularly I
video, although at present it consists of
two medium-specific computer sys-
jb
tems: Macrosticon (the musical system)
[24] and Motographicon (the dance sys- I
time left right left right
tem) [25]. These two systems communi- arm leg leg arm arm leg leg arm
cate with each other by means of com-
mon data, which describe shapes and
Fig. 5. Comparison of notations for a specific dance movement in (left) Symbolic
contours-i. e. parameter evolutions in Notation, (middle) ANIMA and (right) Labanotation.
time-numerically. These common data
are stored for access by any medium,
through a reversal of the interpretative working processes within NUNTIUS is columns are complementary lines for
process (conversion back to medium- as follows: the choreographer creates a supplementary information, indicating,
specific data). Data can also be created dance score and stores it as common for example, support and contact be-
or edited at the common, non-medium- data, then the composer applies the tween body parts. The layout of the
specific level, either by choreographer data to create the music score or a dig- body parts and the direction of time
and composer or by users wishing itally generated music, and vice versa. flow are similar to those in the Laban
to embrace both roles or to 'design' The revelation of 'concealed' rela- system, but Symbolic Notation gives a
at the conceptual level. Figure 1 shows tionships between music and dance [27] more symmetrical and visually continu-
a schematic representation of the cre- is already possible to some extent with ous structure in its standard scoreform,
ative and communicative processes conventional notation (Benesh and in addition to being methodologically
within NUNTIUS. musical notation, for example, are consistent. In special cases the score
A central advantage of using a com- oftenjuxtaposed), but the difference in can be expanded or reduced and the
puter system is the flexibility with which the degree of conscious examination of columns can be redefined.
information can be depicted. Within structures afforded by computer use is Central to the concept of Labano-
Motographicon, for example, move- akin to the difference between a pho- tation-both graphically and in move-
ment may be represented by symbols tograph and an X-ray. This increased ment analysis-is the importance of the
(Symbolic Notation), as a group of capacity for structural analysis is ob- body part that supports the weight of
number sequences standing for the viously as advantageous for dance and the body. This type of notation is par-
evolution of parameter values in time, music research as it is for the composer ticularly suitable for most folk dance,
as curves (Function Notation) or, and choreographer. where the body parts (usually the legs)
through the ANIMA animation pack- that support the weight of the body also
age [26], as animated figures on a perform the central movement or ac-
screen. This multifaceted approach is Motographicon: tion. If body support is performed by
important for both the exchange and The Dance System some other body part, it is necessary
processing of information in that it il- An overview of the use of computers in with Labanotation to use different body
luminates otherwise hidden elements dance is beyond the scope of this paper; signs in the 'support' column of the
of compositional and choreographic information on this subject has been notation, which can make the system
processes. This approach also facilitates documented in Leonardo by George problematic for structural movement
the quick manipulation of large num- Politis [28]. Likewise, we cannot fully representation.
bers of different structural elements. explain the technical and notational In both Motographicon and Laban-
The exchange of information through specifics of Motographicon here; how- otation systems, the duration of a
common data suggests an extension of ever, Fig. 2 shows the schematic repre- movement element is indicated by a
the communicative possibilities of the sentation of its internal structure. The length-of-movement sign. In Labanota-
traditional working procedure. The principles of score layoutand the symbols tion horizontal direction is represented
idealized conventional processesfor both for horizontal direction and duration by the shape of the whole movement
dance and music are as follows: the of both Motographicon and Labanota- sign, whereas in Motographicon a sep-
choreographer creates a dance, the tion are shown in Fig. 3 [29,30]. arate symbol, positioned exactly at the
dancer learns and performs it, and the Symbolic Notation in Motographi- destination point of the movement, in-
notator notates it afterwards.The com- con is based on a uniform scale system. dicates horizontal direction and adds to
poser writes the music score, and the The scale system (shown in columns) the movement duration symbol.
musicians perform it. One of the possi- represents different body parts or Figure 4a shows a vertical direction of
bilities of the inherent extension of the groups of body parts. Connected to the a body part as represented in Symbolic

Ungvary, Waters and Rajka: NUNTIUS 63

 Symbolic Notation program can display
three subordinate body parts in the
time axis Function Notation Symbolic Notation same scale-system. The signs for sec-
ondary and tertiary body parts repre-
Ai sent the angles between these and the
primary body part. If a subordinate
body part has its own independent hori-
zontal direction, it is allocated a modi-
fier. In Labanotation it is necessary to
use a separate series of signs for each
subordinate body part; this involves
dividing or expanding the vertical col-
umn system. Various alternative repre-
sentational forms are used.
Figure 4c shows three positions with
bent right arm, one of them also with
flexed hand, as notated in Symbolic
Notation and Labanotation. Figure 5
compares a specific dance movement as
depicted byANIMA with the equivalent
Symbolic Notation and Laban score.
The course of a parameter in time,
notated with Symbolic Notation, may
be transformed into Function Nota-
tion [31] and vice versa. They are inter-
related and compatible. Visual compati-
bilitybetween the two kinds of notation
is achieved by using the same time- and
graphic-scale systems in both notations.
From a technical point ofview, the com-
munication between Symbolic Nota-
tion and Function Notation is based on
normalized function files [32]. Func-
tion Notation displays the evolution of
a parameter as a function of time, in the
form of a curve. Figure 6 displays a
movement with two parameters in both
Function and Symbolic Notations.
Additional features allow the displays
of any number of overlapping func-
tions or of several curves in parallel.
Comparison of functions is supported,
which allows patterns to be detected,
even in the case of delayed motion. New
functions, representing the difference,
In Function Notation: sum or average of compared existing
functions, also may be computed.
-- vertical parameters of the main body part (upper arm)

the relationship of angles between main and subordinate Macrosticon: The Music System
body part (lower arm) NUNTIUS will eventually be suffi-
ciently flexible to operate with various
computer-music systems, but Macrosti-
con, developed concurrently with and
Fig. 6. The evolution of two parameters (vertical parameters of the upper arm and with specific reference to Motographi-
relationship of angles between upper and lower arm) in Function and Symbolic Notations.
con, is currently the musical system
most appropriate for use with Moto-
Notation, Labanotation and as levels: low, medium and high. These graphicon [33].
illustrated by frames ofANIMA. In Sym- are indicated by shading of the hori- Macrosticon, the musical system cur-
bolic Notation the vertical direction of zontal direction sign. Any repre- rently in use at the Institute for Electro-
a body part is indicated by the position sentation of intermediate positions ne- Acoustic Music (EMS) in Stockholm
of the horizontal direction-sign in the cessitates the use of modification signs. [34], is an open-ended software/hard-
scale system. The scale is divided into Figure 4b shows the notation of ware network with intermediate com-
22.5? units. In Labanotation every posi- the relationship between primary and munication between different modules.
tion is analysed in terms of three main subordinate body-partmovements. The This approach has been determined by

64 Ungvoay,Watersand Rajka:NUNTIUS
 >_ _w _s__ _U __ _ _

left arm movement

'
24 - I
23 * L . s
22
21 ~~~~~~~~~~~~I I;,I _r * r I __ __

2a _ w_ I II * I tII r < *;S__-t-->

19 z__-_ i~r
rrivrrr _ ~ll
v_ _ _ _ _ t*i w_ _- 7
18 3 > ~
_ __I t_ __1 ; _ _ b __
17 I_ __I r_ _ I z _ _ _ * l __ t_- _ * *__
16 (high frequency) (ow frequency) (top __ampl__itu
15
14 .I7--
---.--
---._I
_rr .I n. ....
13
12
11
1a -------
a:--
r-LI
9 _

8 _~~I t_ ll _ _ ) _ - _ _ t _
__?_tt_,jI- __* _*__

7 -E;z;f I.t .?_b_.r .r rbII ~?C_b_

6
5
4
3
2
1
(high frequency) (low frequency ) (top amplitude )

vertical horizontal angles between

directions directions lower and upper
arm

24
23
22
21
z20
19
18
17
16
15
14
13
12
11
10
9
9
7
6
5
4
3
2
1
(low frequency ) (high frequency)(middle frequency)

right arm movement

Fig. 7. (left) Function files transferred from the music system and applied on movement parameters displayed in Function Notation.
(right) The same movement parameters in Symbolic Notation.

Unguary,Watersand Rajka:NUNTIUS 65
 practical compositional needs. Our soft-
(5) ware environment consists partly of
Fig. 8. The final score.
stand-alone programs, partly of special-
iw -.,... ........ .. ......... . .. ...... .. .
......... Music data was inter-
(Cs
preted by the choreog- ized integrated software packages (see
52W..*..
"' ..-[,......., ... .., ,
'
.. rapher and transcribed in
Symbolic Notation.
Fig. 1). We have not been able to afford
the time to develop a general solution
for the integration of different formats
and input lists used by different pro-
...........
,....................... . .... .....
............... grams.
. ...... . ;.. .......
..... ..........
...... ....
.. ....
Since the operating system [35] we
~
(4)
51
..... ... ...... ..
...............
...........
..
.....
......... use does not offer a UNIX 'pipe'
(3) !
C')
....- ...I................. . ..
...... ......... U
,. ............. facility, we have created a software net-
work to establish intermediate data-
communication between the different
(0) 0
programs. The hierarchy of this soft-
ware network is based on normalized
(I) .. ..... .. I. function files [36], which can repre-
.. ... -
r
. F .. . ___. . . . .. . . ... . .. . .. __
sent, steer or modulate the evolution of
. .. -.... ... . . ............. ... ....,^
(7)
any sonic, musical or technical parame-
(4)
.' .... ..... .. ...... .
ter. Function files can be seen and used,
.... ...... ...... .................. ............
for example, as wavetables [37], or en-
velopes [38], and are easily tested, ana-
lysed, treated, rescaled and read into
o)'i~ ........................... ............ ..............

(3)
i. .. ...................
...........................
^.............,.j i
the appropriate columns of a complex
score list or extracted out of one. Pro-
?~/.....-.P
(3) ... .......... ........... .. grams have been developed to assist
composers to generate, test and treat
these files.
;(. ... . . ... .... .......... The network offers the use of mixed
(2)
.I .. ... ........ ....
...
a(?. I . composition techniques, providing a
'Bi~-l' ** ****
. ***- -- -^ * -
..... *-, - ?- . -... ...
...... ...... .
(3
2 wide scope in both compositional
strategy and sonic results. This design
philosophy is similar to that of Decker,
Kendall, Schmidt, Ludwig and Freed,
33C": :...... . ... .......... .. ......
(4)
(4) 1*-- -***-a -*******- -- . . who have written that "no single mu-
(3) . .. . sical input-language or user-interface
(2) -.- 7"....! can adequately accommodate a wide
....................... ..--...'-I
....'""~`~""~'~;)~??
.^ .I?
.... ...........................
t ?r.. -...1- . B ......,.

range of compositional styles and inten-

tions" [39].

+?~?r. ..... ..... ....

....i.+....- .. .... ................

.........'~
'?" '"";" * "* * -'/ ......
. ..........r?-..
?.... .. i ...... .. . .......
PRESENT
l&
'
... ...
. ..... COMMUNICATION
... ... . ..... . . .......
j.......- . POSSIBILITIES
We have carried out a series of com-
I?jj i?~~)I
?... .? .... .. . ^....
(^ .p ........
..... .
munication experiments to test this
(i) ?. I m I*
system. As a first step, we obtained the
j .,... . . .... ..... .........-.....
.. .. ... ....
.w. .... .. outline of an experimental choreog-
.-" ......~ ......? '-.. ... " . -.... T-
..............
.... .........
raphy, which was created with Symbolic
Notation, with its exact segment dura-
tions defined. We used this movement
. . . .

information (presented to us in both

.

- ...
. .

...
... .;
.. . .. .......... .. .

written and oral forms) to compose a

(4)''' "' B'''--?' ?. .l ?lr?I.. r .r~.
short electroacoustic piece. The result-
"T
.... ,""".' .I ........ . ............... '
..................~, ing 1-min dance/music work, TEMA
.--"'' t f-
.O
itt ... .. .... .. . ... ... ... No. 1, has been used several times to
demonstrate the movements of an ani-
(4) ,
mated figure on the computer screen,
accompanied by music. We have also
recorded TEMANo. 1, with live-action
. .
t I.1I ........... .....::
I ....... ....... .......
)..... ........ .... .... .. dance performed by A. Henriksson, on
<0t ? I .I... videotape.
As the second step of the experi-
ment, we analysed the music of TEMA

66 Ungvary,Watersand Rajka:NUNTIUS
No. 1with signal-processingsoftware.The
obtained data (e. g. the overall ampli-
tude curve of the piece or the motion
characteristics of the fundamental fre-
quency) was transferred to Moto-
graphicon and, without any treatment,
has been applied to different body-
motion parameters controlling their
evolution in time. Figure 7a shows func-
tions obtained and transferred from
the music system, as the Function Nota-
tion program displays them. Figure 7b
shows the application of the functions,
left- and right-arm movements, in
Symbolic Notation.
In the next phase of the experiment,
the data were interpreted by the chore-
ographer, who tested manipulative pro-
cesses such as rescaling (altering the
magnitude of the parameters), retro-
grade (arranging the parameters back-
wardsin time), augmentation (enlarging
the duration), diminution (decreasing
the duration) and mirror image (re-
versing left and right sides). A further
manipulation consisted of modulating
the existing movement by adjusting
parametric values with reference to an
additional function file. Finally the re-
sulting material was adjusted freely by
the choreographer using the pro-
gram's 'symbolic notation editor'. Fig-
ure 8 presents the obtained dance score
in Symbolic Notation of Motographi-
con.
CONCLUSION
In its present form, the primary advan-
tage of NUNTIUS is that it allows direct
communication at either parametric
or structural levels. However, this is
merely the first condition for complete
interactive communication. The ulti-
mate goal of the project is the develop-
ment of two 'artistic interfaces' capable
not only of data transmission, but also
of autonomous evaluationand utilisation
of that data. We regard the develop-
ment of the necessarymusical and chor-
eographic expert systems as the next
essential phase of the development of
NUNTIUS. Naturally, the possibilities
offered cannot replace human dial-
ogue or other direct means of informa-
tion exchange. In addition, considera-
tions such as stylistic detail and the
microstructural events communicated
by the performer remain beyond the
immediate scope of the system.
Our next aim is to create a com-
mon database to access these systems,
with comparable manipulative, struc-
tural and transformative tools [40].
These in themselves do not guarantee
any degree of structural coherence in a
completed work. Nonetheless, there is
potential for a more conscious inter-
change of concepts, at various composi-
tional levels, than was possible before
the advent of computers.
Acknowledgments
This paper was supported by the Bank of Sweden
Tercentenary Foundation and by the National Col-
lege of Dance, Stockholm.
References and Notes
1. See B. Lamerz, Musik und Tanz (Overath bei
Koln: Ulrich Steiner Verlag, 1987); H. Searle, Ballet
Music (New York: Dover, 1973) (Originally pub-
lished in 1958); P. Gilbert and A. Lockhart, Music
forModermDance (Iowa City, IA:Univ. of Iowa Press,
1961).
2. Paraphrase of R. Mulder and K. Allik, "Geinte-
greerde Media/What Is Integrated Media?",Logos-
Blad 10, No. 10, 9-10 (Gent: Stichting Logos, 1988).
3. A parameter is a quantifiable characteristic or
feature (which also may control an attribute).
In the case of music, the pitch, amplitude and
duration-the sound colour-are the main charac-
teristics of one or more notes. Parameters of move-
ment characteristics could include the height (of a
jump), the duration, the horizontal and vertical
positions of the body parts.
4. The terms 'structure forming' and 'manipula-
tion of materials', or their equivalents, are used
throughout to refer to the basic functions of the
compositional process (irrespective of medium), to
the controlling of formal shape and coherence and
to the compositional strategies by which the rela-
tionship between the basic materials (e. g. sounds
or movements) are formed and changed.
5. The term 'prescriptive' is used throughout to
define a notation system used as a compositional
tool, preceding any notional performance. Most
descriptive score types (recording a pre-existing
work) can be used prescriptively in the sense that
they can serve to re-create that work in perform-
ance.
6. C. Jeschke, Tanzschrifte, ihre Geschichteund
Methode(Bad Reichenhall: Comes Verlag, 1983).
7. R. A. Feuillet, Choregraphie, ou l'art de d'ecrirela
dancepar caracteres,
figureset signesdimonstratifs,avec
lesquelleson apprendfacilement desoy-miime toutessortes
de dances. Ouvragetresutile aux maitresd danceret a
toutesles personnesqui s'appliquenti la dance (Paris,
1700).
8. D. Herbison-Evans, "Dance, Video, Notation,
Computers", Leonardo21 No. 1, 45-50 (1988);
W. C. Reynolds, "Human Movement Notation:
The Analysis and Written Recording of Human
Movement", paper presented at The International
Congress on Movement Notation (Tel-Aviv,Israel,
1984).
9. D. Zeltzer, "Towards an Integrated View of
3-D Computer Animation", The Visual Computer4
(1958) pp. 249-259.
10. N. Magnenat-Thalmann,'The Makingof a Film
with Synthetic Actors", ElectronicArtSupplemental
Issue, Leonardo(1988) pp. 58-59; N. Magnenat-
Thalmann, D. Thalmann, eds., "State-of-the-Artin
Computer Animation" (Tokyo: Springer-Verlag,
1989).
11. R. Smith, "TheForms of Relationship Between
Music and Dance", unpublished undergraduate
thesis, University of Nottingham, 1981.
12. Examples of analogue interaction include
Martha Graham's Clytemnestra;Lucinda Childs's
collaborations with Philip Glass; and Richard
Alston's Rainbow Ripples, which was set to the
speech rhythms of Charles Amirkhanian's text-
sound.
13. Merce Cunningham's choreography Diversions
(1948) with John Cage's Suitefor ToyPiano (1948)
exemplifies dialogue embedded in the very struc-
ture of a work. In this collaboration, both dance
and music are built on a hierarchy of relatively
regular cycles subdivided into 'irrational' phrasic
units. The structure is thus emphasised by regular
reinforcement of its relatively longer durational
units, but it incorporates various levels of rhythmic
and phrasic contradiction within this long-term
regularity-a sort of intermedia counterpoint.
14. M. Smith, "Borrowings and Original Music:
A Dilemma for the Ballet-Pantomime Composer",
DanceResearch6, No. 2, 3-29 (1988).
15. Another example of 'causal' interdependence
is the Cage and Cunningham collaboration Varia-
tionsV, in which the dancers generate the music by
triggering photo-electric cells around the stage,
which in turn control various elements of a sound-
synthesis system.
16. A particularly elegant example of structural
interdependence occurs in Trisha Brown's Accumu-
lation with Talking (1973), in which the dancer's
exploration of the structuring technique of accu-
mulation is mirrored in her spoken text, which acts
both as a reinforcement of the accumulative struc-
ture and, simultaneously, as a (literal) commentary
on the making of the work: 'This dance has a very
obvious structure- / I make one movement and
do it for a while / And then I add another. / When
I made the dance every movement that I added /
Had to mesh with the beginning. / I also had to
choose movements that were comfortable to do /
Because I repeat them a thousand times each.... /
While I was making the dance my father died /
Somewhere between these two movements".
(Text transcribed by S. Waters during perform-
ance, 1973.)
17. The theoretically independent works of Cage
and Cunningham are a case in point. The per-
ceived structural coherence achieved in perform-
ance is often remarkable, and conceptual connec-
tions exist even at the compositional level. Both
Cage and Cunningham can be seen to exhibit a
'durational' or 'proportional' approach to placing
events in time, rather than a 'rhythmic' or
'counted' approach. See D. Daniels in BalletReview
(Summer 1985) pp. 29-35.
18. J. A. Sloboda, The Musical Mind: The Cognitive
Psychologyof Music (Oxford: Oxford Univ. Press,
1988) pp. 154-156.
19. An example is Stephen Petronio's Walk In
(1986) with music by David Linton ("Diasde Fuego
yMuerte").
20. R. von Laban, Principlesof Dance and Movement
Notation(Plymouth: Macdonald and Evans, 1956);
R. von Laban, Laban'sPrinciplesof Dance and Move-
mentNotation(Boston, MA:PLAYS,1975); A. Knust,
Abriss der KinetographieLaban (Munich: 1942); A.
Knust,A Dictionaryof KinetographyLaban (Labanota-
tion) (Plymouth: Macdonald and Evans, 1979);
M. Szentpal, Lehrbuchder Kinetographie,(Leipzig:
Abt. des Zentralhauses ffir Volkskunst, 1958);
M. Szentpal, Tdnqelircs:Laban-Kinetogrifia,Vol. 1
(Budapest: Nepmfvelesi Intezet, 1964), Vols. 2-3
(Budapest: Nepmivelesi Propaganda Iroda, 1969
and 1973); V. Preston-Dunlop, PracticalKinetogra-
phy Laban (London: Macdonald and Evans, 1969);
A. Hutchinson, Labanotationor KinetographyLaban,
2nd Ed. (London: Oxford Univ. Press, 1974).
21. R. Benesh and J. Benesh, An Introductionto
BeneshMovementNotation (London: A. and C. Black,
1956); R. Benesh andJ. Benesh, ReadingDance:The
Birthof Choreology(London: Souvenir Press, 1983);
M. Causley, An Introductionto BeneshMovementNo-
tation, Its GeneralPrinciplesand Its Use in Physical
Education(London: Parrish, 1967).
22. N. Eshkol and A. Wachmann, MovementNota-
tion (London: Weidenfeld and Nicolson, 1958).
23. Nuntiusis Latin for envoy, or 'messenger'. The
main task of an intermedial system, which con-
sists of at least two autonomous media systems
(music and dance), is to transfer information, or
messages. However, a true envoy (e. g. a nuncio
Ungvary,Watersand Rajka:NUNTIUS 67
 papal ambassador) does more. He interprets the
messages as a diplomat always does. Interpreted
transmission, as described in this article, is our goal.
That is why we have named our system NUNTIUS.
See also T. Ungvary and P. Rajka, "Nuntius",Inter-
national Conferenceon CoordinationMethod:Dance
NotationandApplication,Digest of Papers (Nanjing,
China: Institute of Technology, 1988).
24. The name Macrosticon comes from macro,
meaning 'large scale', and con, meaning 'a collec-
tion of. In Macrosticon, items of software pro-
grams are used for signal- and data-generation and
treatments. Together the name means 'an interre-
lated collection of large-scale items'.
25. The name Motographicon comes from motus
meaning motion, and graph meaning 'forming
nouns and verbs denoting something written in
specified records' (e. g. Symbolic Notation, Func-
tion Notation, animation), and con meaning 'a
collection of.
26. Anima means soul, spirit and breath. Anima-
tion is a technique of displaying successive draw-
ings or positions of items to create illusion of move-
ments. ANIMAis a program that 'breathes life' into
computer drawings by giving the illusion of move-
ments on the screen. The basic structure of the
program has been developed by a group of stu-
dents led by P. Rajka at the Royal Institute of
Technology, Department of Computer Sciences
(NADA), Stockholm, in 1988. Further develop-
ment was carried out by the Kineto-AuditoryCom-
munication Research Group (KACOR), P. Rajka,
Magnus Lundin and Gustaf Nyman, led by Tamas
Ungvary.
27. Concealed relationships are regarded as those
not immediately apparent on first apprehension of
a music/dance performance, but which can be
elucidated by a study of theirjuxtaposed notations,
or by repeated viewing/listening.
28. G. Politis, "Computers and Dance: A Bibliog-
raphy", Leonardo23, No. 1, 87-90 (1990).
29. Since a few but fundamental features are very
68 Ungvary,Watersand Rajka:NUNTIUS
similar in both Motographicon and Labanotation,
it is necessary to clarify the degree of similarityand
difference between the two notation systems. Mo-
tographicon aims to continuously develop and op-
timize both structural expressiveness and the read-
ing and writing of long movement sequences and
processes. This aim is realized by experimentation
with new solutions for movement notation; full
utilization of the possibilities of computer-based
application; and modification and integration of
solutions and ideas of existing notation systems
(mainly of music notation and Labanotation).
30. The figures of a dancer that illustrate the nota-
tion are extracted from the animation program
ANIMA and arranged by a drawing program. By
opening and displaying the Symbolic Notation,
Function Notation and animation programs on the
same screen in parallel, the user is provided a visual
comparison of different representations of move-
ment sequences.
31. Function is a fundamental concept of mathe-
matics. It is used whenever some quantity (value)
depends on other quantities. In both music and
dance compositions, the parameters change values
in time-that is they are always time dependent.
We can therefore see them as functions of time.
These functions are displayed as graphs, and in
everyday terminology as curves.
32. A file is an organized collection of data units
or, in this case, numbers. A function is a relation-
ship between variables. Since time is a constant
variable in both music and dance, the use of the
term 'function' in this context implies the mapping
of a series of values into the time axis with equi-
distant time differences. One obtains a normalized
function by multiplying the function values with a
constant coefficient so that the values will lie be-
tween 0 and 1.
33. The integration of the music- and dance-nota-
tion systems within a microcomputer environment
(based on a Macintosh IIfx) is under development
at the Institute of Speech Communication and
Music Acoustics of the Royal Institute of Tech-
nology (KTH), Stockholm, by the Kineto-AuditorD
Communication Research Group (KACOR).
34. For more information, contact the Institute for
Electro-Acoustic Music in Sweden (EMS), S6der
Malarstrand61, S-11725 Stockholm, Sweden.
35. We used VMS, an operating system that runs
on aVAX 11/750 computer. VMS is a trademark of
Digital Equipment Corporation (DEC). The soft-
ware-systemon which we use Macrosticon includes
an FPS 120B arrayprocessor.
36. See [32].
37. A wavetable is an array in which a set of values
generally define 1 cycle of a sound wave. An array
is a collection of values stored in the computer as
a table. It may have one or more dimensions.
38. 'Envelope' is the shape of the variation of a
characteristic during the course of an item. In the
case of a tone it usually denotes the overall ampli-
tude shape that in its simplest form consists of three
segments: attack (rise), steady state and decay.
39. S. L. Decker, G. S. Kendall, B. L. Schmidt, M.
D. Ludwig and D.J. Freed, "A Modular Environ-
ment for Sound Synthesis and Composition", Com-
puterMusicJournal10, No. 4, pp. 28-41.
40. We are also working on a two-wayconversion
program that will allow communication between
our symbolic notation editor and the animation
program COMPOSE,developed by T. Calvertet al,
Simon Fraser University, Canada. See S. Gaudet, T.
Calvert, S. Welman and C. Lee, "Composition of
Multiple-Figure Sequences for Dance and Anima-
tion", New Advancesin ComputerGraphics,Proceed-
ings of Computer Graphics International (1989).
The first regular course in using Motographicon
was held in 1990 at the National College of Dance,
Stockholm. Development is ongoing to implement
an object-oriented interactive-programming envi-
ronment for computer-aided composition and to
integrate different time codes and three-dimen-
sional animation into Motographicon.