Appeared in the Proceedings of Creativity and Cognition 4, 2002
Supporting Musical Composition
by Externalizing the Composer's Mental Space
Shigeki Amitani
Department of Advanced
Interdisciplinary Studies
University of Tokyo
+81 3 5452 5289
amitani@ai.rcast.u-tokyo.ac.jp
ABSTRACT
In the field of design and creativity support,
"externalization of mental space" has been recognized as an
important challenge. In this paper, we tackle this challenge
on musical composition, which is one of the important
human creative activities. In our research, we focus on the
analysis of the cognitive process in musical composition.
That is, we analyze what the cognitive process in musical
composition is like and how the process is affected when
the representation of information is changed. For this
research, we propose a musical composition supporting
system named "MACSS (MAcroscopic Composition
Supporting System)" which offers a spatial representation
of music to the composer to support his/her composition
process. Comparing with ordinary musical editors, which
give chronological (1-dimensional, score-metaphored)
representation, this system is expected to provide a
macroscopic view by locating phrases on a 2-dimensional
space. We have investigated how the cognitive process
changes when the spatial representation is introduced, i.e.,
the representation of information is changed. Moreover, we
have analyzed the observed change microscopically,
especially the process of "mental fixation" and "mental
leap". We have found how the spatial representation
triggers mental leap escaping from mental fixation in
musical composition.
Keywords
Creativity support, musical composition, externalization,
mental space, cognitive process, spatial representation
INTRODUCTION
In the field of creativity support, it is important to
investigate how the change of external representation of
information gives effects on human cognitive processes in
creative activities. In this paper musical composition
process is selected as one of the important creative
activities.
Norman [1] claims that cognitive artifact should support
our activities by amplifying our reflective thinking. It
means that tools are required to provide the user with
proper information in a proper way of representation when
the user is engaged in a creative activity.
In our research, we focus on the analysis of the cognitive
process in musical composition. Tanaka [2] points out that
fewer analysis on musical composition have been
Koichi Hori
Department of Advanced
Interdisciplinary Studies
University of Tokyo
hori@ai.rcast.u-tokyo.ac.jp
conducted compared to those on the cognitive processes of
musical listening and performance, and that most of them
are macroscopic analysis through flashing an idea to
completion of the musical piece. That is the reason we
focus on detailed analysis of composition process.
• How composition process proceeds with an ordinary
musical editor
• How the human cognitive process in musical
composition is affected when the representation of
information changed.
For this research, we propose a musical composition
supporting system named "MACSS (MAcroscopic
Composition Supporting System)" which offers a spatial
representation to the composer to support his/her
composition process. Comparing with ordinary musical
editors, which provide chronological (1-dimensional, score-
metaphored) representation, this system is expected to
provide a macroscopic view of pieces by locating phrases
on a 2-dimensional space. In the context of musical
composition, the term "macroscopic view" means to grab
the whole of the musical piece being composed or to grab
the relationship among the pieces (phrases) the composer
has composed. The observed change was analyzed in detail
and described, especially the process of "mental fixation"
which means the deadlock of the thought and "mental leap"
which means to escape from the deadlock of the thought.
We observed how the spatial representation triggers mental
leap escaping from mental fixation in musical composition
more frequently than chronological representation.
RELATED WORKS
Schoen [3] explained the design process as "Seeing-
Drawing-Seeing Cycle". Design process starts from
"Seeing", proceeds to "Drawing", or generating
(externalizing) the partial solution to the design problem,
returns to "Seeing" and repeats this cycle, which is shown
in Figure 1. This process can be regarded as a "dialog"
between the designer and his/her material. Usually this
process is ambiguous and dealt in the designer's mind. It is
possible to put this design process into concrete by
"externalizing the designer's mental space", which enables
the designer to grasp his/her whole of mental space. It is
expected that this method can support the creative activities
[4][5]. This process is shown in Figure 2.
Zhang [6] proposed a theoretical framework for the
Appeared in the Proceedings of Creativity and Cognition 4, 2002
problem-solution process based on the external
representation.
Figure 1 Seeing-Drawing-Seeing Cycle
Figure 2 A Cycle of Design Process with Externalizing the
Designer's Mental Space
In Zhang's research, he conducted his experiment with a
game called Tic-Tac-Toe and its isomorph. The experiment
demonstrated that the actions in the problem-solution
process were determined by the information directly
available to human cognition or perception. Based on this
experiment, Zhang suggested the representational
determinism, that is, the form of representation determines
what information can be perceived, what processes can be
activated, and what structures can be discovered from the
specific representation.
Suwa et al. [7] devised a scheme for coding designer's
cognitive actions from video/audio design protocols.
Designer's actions were coded into four cognitive levels
(i.e., physical, perceptual, functional and conceptual). The
relations between each cognitive process were
systematically described and analyzed. Through this
analysis, Suwa concluded that the interaction between the
designer and his/her sketches provides the designer with
such effects as re-interpretation of the design problems and
unexpected discoveries. This method is useful to code and
to analyze human cognitive process.
The Spatial Representation as a Way of Externalizing
Mental Space
In this research, the spatial representation is adopted as a
way of externalizing mental space. Several studies have
been conducted which employed the spatial representation
for externalization.
Shipman et. al. [8] investigated the significance of the
spatial representation. He concluded that the significance of
the spatial representation is to represent mainly the
similarity among information with the proximity of them.
Their system aimed to support the process "incremental
formalization", that is, the user gradually structures the
concept.
Hori 5[ ] conducted the experiment for the effect of the
spatial representation on the conceptual design. The system
named AA1 presents words, which represent the concept
the user vaguely conceives, on the two -dimensional space.
The user can change the location of the words on the sp ace.
This action helps the user clarify his/her concept gradually
and such phenomenon was observed as the user came to
generate new concept by looking at the blank area on the
space.
Yamamoto et al. [9] presented the theoretical framework
for the method of support with an interactive system at the
early stage of the creation of information. The interactive
system aimed to support the process to create information,
which is still ambiguous in the user's mind through trial-
and-error. They concluded that it is not "the representation
for the design solution" but "the representation for
understanding the design solution" that the designer needs
at the early stage of the creation of information. That is, the
spatial representation can support to clarify gradually the
designer's concept that was ambiguous through the
interaction between the designer and the system in the
divergent thinking process.
From these studies, the spatial representation can support to
structure and clarify the designer's ambiguous concept
gradually through the interaction. Furthermore, it can help
generate new idea.
Information with "Linearity" and "Gestalt"
Reybook [10] examined the limitations and possibilities of
application of Gestalt concepts in relation to music. The
research was conducted mainly on the cognitive process of
listening music, but the assertion that music analysis has to
be broadened from a structural description to a description
in terms of processes must be taken into consideration to
analyze the cognitive process of mu sical composition.
Yamamoto et al. [9] also referred to "linearity" and
"Gestalt" in the semiotics and they attempted to present
with the system Gestalt to the linear information, i.e.,
document in the case of their research, simultaneously.
They concluded that this approach allows the designer
(writer):
• To inspect the information in detail with the linearity
Appeared in the Proceedings of Creativity and Cognition 4, 2002
• To grasp the information from the bird's-eye view There are composers who accumulate small phrases they
with Gestalt hit upon and develop or combine them to compose a
This semiotics approach can be applied to music that is also musical piece. Though ordinary musical editors are useful
information with linearit y. As well as the document for detailing a phrase, it lacks a global view of the whole
composition, the spatial representation that might provide piece. MACSS provides the flexibility to reconstruct the
Gestalt allows the designer (composer) to grasp the whole piece by its spatial representation. It cooperates with the
structure of the music piece, and moreover, this ordinary editor and they complement with each other.
representation can support the musical composition process.
In our research, it is attempted to introduce Gestalt concept
in musical composition process with a spatial
representation, which is appropriate to grasp the relation
among the objects. Introducing this representation, it is
expected that the composer use both spatial and score -
metaphored representation, i.e. he/she uses a spatial
representation (Gestalt) to grasp the information from
bird's-eye view and uses score -metaphored representation
(linearity) to inspect the information in detail.
Research on Musical Composition Process
A few researches on the composition process have been
conducted in the music psychology. Sloboda [11] described
the composition process in Figure 3.
Though Umemoto [12] also presented macroscopic analysis
like Sloboda's, as Tanaka [2] mentioned in his paper, the
Figure 3 Sloboda's Diagram of Musical Composition Process
problem in this field is that almost no research has been
The composer is expected to proceed his/her composition
conducted to investigate what the each process is like and
by exploring the "whole" and "a part" of the piece. That is ,
what the transition among processes is like. In this paper,
MACSS allows the composer to rearrange the phrase
we investigate the cognitive process in the musical
objects on the space and to listen to the phrases in the order
composition microscopically with protocol analysis
the composer specifies so that he/she can compose the
method. It is expected that the analysis provide us with the
"whole structure of the piece". On the other hand, the "Edit
rich knowledge for designing a creativity support system in
Phrase" command allows the composer to edit the each
the future. In the field of creativity support, as few practical
phrase, i.e. "a part of the piece" on the ordinary
researches have been conducted it is necessary to analyze a
chronological editor. When the composer rearranges the
lot of cognitive processes through various instances. As our
phrase objects, MACSS recalculates the similarity of the
approach to this challenge, the musical composition
phrases from the distances among the phrase objects and
process, which is one of the representative and important
tunes the weight of the three musical characteristics used
human creative activities, is selected.
for similarity. It is expected to become easier for the
The "mental space" in this research refers to the part named composer to grasp the phrases in his/her long-term
"A: idea" and "B: theme" in "CONSCIOUS" area in Figure composition that can provide the composition process with
3. It is expected that the spatial representation decrease the the nature of Gestalt to. Furthermore, it is also expected
cognitive load when the composer gra sps his/her unfinished such interaction that the composer come to generate new
phrases and offers macroscopic view that the ordinary concept by looking at the blank area on the space. To
chronological editor does not provide. enhance the nature of Gestalt, the aspect of each phrase
MACSS (MAcroscopic Composition Supporting System) object is editable with "Edit Picture" command (right-click
MACSS aims at providing the composer with the ability to menu). It opens the picture with the external picture editor
grasp whole of his/her piece. That is, MACSS provides the and the composer can put the image on the object. For
"macroscopic view". Figure 4 shows the screen shot of example, "This color expresses the atmosphere of this
MACSS. MACSS arranges the phrase objects (MIDI files phrase", "This picture can be the jacket of this piece", etc.
only) on the two-dimensional space by Multi-Dimensional This function was added because one of the composers who
Scaling method (MDS) based on the calculated similarity participated in this experiment wished.
among the phrases. In the music psychology field, it is said The Representation of Music on a Computer
that the similarity of phrases is grasped mainly based on the Hiraga [15] proposed Functional Representation of Music
three musical characteristics, such as melody contour, key (FRM) as a representation of music on a computer. We
and rhythm [13]. We adopted these criteria for the have partly adopted this representation to implement the
calculation of similarity among music phrases on MACSS, system. Hiraga also suggested the representation of
though there is room for further discussion to define the similarity of musical phrases based on such formal
similarity [14]. MACSS was developed with JDK ver.1.3. elements as melody contour, contrary motion, retrograde
Appeared in the Proceedings of Creativity and Cognition 4, 2002
motion, repercussa, rhythm and chord.
Figure 4 A Snapshot of MACSS
The Similarity Calculation
Melody Contour and Rhythm Similarity
The melody contour is represented as the array of the
musical intervals, that is:
Interval =currentMIDINoteNumber - previousMIDINoteNumber
The rhythm is also represented as the array of the time
between the Note-on times, that is:
Rhythm = currentMIDINoteOnTime - previousMIDINoteOnTime
To define the similarity of the phrases, Direct Pattern
Matching (DPM) method is adopted [16]. This method is to
count how many elements in two groups with discreet
values are correspondent with each other.
Key Similarity
Several key recognition methods are proposed and the
model of Yoshino and Abe [17] is adopted for MACSS
because it offers "the most appropriate key name", while
the others do not unless the input phrase completely
conforms to the key rule.
The Total Similarity
The total similarity is defined as:
Similarity = w 1 * similarityOfInterval
+ w2 * similarityOfKey + w 3 * similarityOfRhythm
These values of w1 , w2 and w3 change according to the
changes of the object positions on the space. They are
recalculated with the difference of the distances before and
after the position changes. Though there is a room for
discussion about the method of similarity calculation, it is
useful to show the simila rity among phrases from these
viewpoints.
The Spatial Representation
Multi-Dimensional Scaling (MDS) is a set of data analysis
techniques that display the structure of distance-like data as
a geometrical picture. In this research, the adopted method
for MDS is Torgerson's method [18].
The Expected Interactions
Using MACSS, the following interactions are expected.
As a Macroscopic Editor
MACSS facilitates for the composer to grasp the
characteristics of phrases, and their relations by arranging
the phrases in accordance with the similarity the composer
decides . It is expected that it promote to think how to
compose the whole structure of the piece. With only a
chronological editor, it is sometimes difficult to grasp the
whole of the piece. As MACSS provides the composer with
the macroscopic viewpoint, it is helpful to develop the
piece macroscopically.
As a Macroscopic Database
In the long-term composition, as the number of phrase
increases, it gradually becomes difficult to grasp what each
phrase is like with the ordinary "list" representation on the
computer system. In MACSS, as the arrangement is defined
by the composer, it is easier for the composer to know what
each phrase is like.
EXPERIMENT
In this experiment, we are going to investigate how the
cognitive process in musical composition changes when the
spatial space representation is presented during the
composition. Table 1 shows the five composers who
participated in the experiment. They all have experience of
musical composition. The place of the experiment was up
to the subjects. If they do not specify any particular place,
experiments were held at our laboratory. The hardware and
software used in the experiment are shown in Table 2. EOS
provides almost same functions as Cakewalk, that is, it is
possible to edit various musical characteristics and it has
the chronological representation. As the subject E used to
composition of musical pieces on EOS, he used EOS in this
experiment instead of Cakewalk.
Procedures
The process of the composition was recorded by digital
video camera (SONY DCR-TRV20). The visual and audio
protocol data were analyzed. The procedure of the
experiment is based on the reference [19]. The subjects
were presented with a picture and asked to compose a
music piece/music pieces about it or whatever it suggested.
Figure 5 shows the picture presented to the subjects as a
theme. The subjects were asked to answer the questionnaire
based on the reference [19].
Preliminary Experiment
A preliminary experiment was held to establish the
procedure of the experiment. The subject was a graduate
student (the subject A) in our laboratory. In this preliminary
experiment, the whole process was recorded on VTR and
analyzed to grasp the process roughly. In this paper, the
words "microscopic action" and "macroscopic action" are
used. The former means "the action to detail the phrase",
such as adjusting the velocity of a note, changing the length
of a note, etc. The latter means "the editing action on
plural phrases", such as thinking the structure of the piece,
etc. It was observed that the composition proceeds
chronologically even at the early stage of divergent
thinking process when the subject composed only with the
Appeared in the Proceedings of Creativity and Cognition 4, 2002
Table 1 List of the Subjects
Subject Sex Age Musical exp. Exp. for composition
A M 24 5 years in chorus circle Some pieces on a computer
Arrangement at a brass band
B M 23 Piano for 18 years
circle, music for a lyrics
4 years. Mainly composition
C F 24 4 years in some bands
of the vocal melody
Compose when he hits upon
D M 25 Electone, Synthesizer
a melody
E M 25 8 years in some bands 7 years
Table 2 Equipment
Equipment
PC SONY VAIO PCV-R63K or Gateway SOLO
Keyboard Roland SK-88 Pro or YAMAHA EOS B-900
Musical Editor Roland Cakewalk Audio Pro 9 and MACSS
Figure 5 The Theme Picture
ordinary score-metaphored musical editor. Note that "the
composition proceeds chronologically" means that the
composer develops the next phrase to the phrase which
he/she has composed just now. In this case the subject
seemed to regard "the next phrase to the phrase which
he/she has composed just now" as one and only solution at
that point and the mental fixation was often observed. It is
often said that a part of a piece does not consistent to the
other part of the same piece, which can happen even to a
professional composer. It is necessary for the composer to
have means to grasp the whole musical information in
some way because music is classified to "Temporal art"
where it is impossible to grasp the piece until it is played.
In this research, we introduce the spatial representation that
may promote the composer's understanding of the
information. From the observation of this preliminary
experiment, the following hypothesis was established to
investigate the change of cognitive process.
Though the musical composition tend to proceed
chronologically when the composer uses a musical
editor with a chronological score-representation, the
composition process will proceed unchronologically with
spatial representation musical editor MACSS.
Main Experiment
Based on the preliminary experiment, the procedure was
modified for ease of understanding and the main
experiment was designed. The main experiment was held
four times for each subject under the following conditions.
As the composition does not finish at one time, the rest of
the composition was continued on the other day.
1st & 2nd Chronological editor only
3rd & 4th Chronological editor and MACSS
The subjects were presented with a picture and asked to
compose a music piece/music pieces about it or whatever it
suggested. The process of the composition was recorded
and was analyzed in the same way as preliminary
experiment. The time of the experiment was completely up
to each subject and it continued by the time when the
subject declared the end. So, the experiment time ranges
from 40 minutes to 3 hours for each session. The
instruction was presented to the subjects before the
experiment.
Analysis Method
In this experiment, the composition process was analyzed
based on the scheme that Suwa [7] proposed to analyze
architectural design. The unit cognitive processes were
defined in Table 3, which consists of three cognitive levels
such as "Thought", "Perception" and "Action". The
composition process was analyzed along with these unit
cognitive processes, so that the processes are analyzed
systematically. In this experiment, we focused on:
1. The "mental fixation " and "mental leap" in the
development of the phrase
2. The change of the cognitive processes
before/after the spatial representation
These processes were analyzed microscopically and the
transitions of the cognitive processes were described.
RESULT AND DISCUSSION
The cognitive processes were analyzed especially the
process where a subject hit upon a new development or
failed in developing the phrase after he/she said, "What
should I do next?" Figure 6 to 8 shows the transition of the
cognitive processes. These diagrams indicate the transition
of the cognitive processes at three levels that we defined:
"What did you see? (=Perception)", "What did you think?
(=Thought)" and "What did you do? (=Action)". The unit
cognitive processes on the nodes in the figures are defined
in Table 3.
The following two cases were compared:
Case 1 Composition process with the chronological
editor only
Case 2 Composition process with the chronological
editor and MACSS
The main observed difference between the two cases is:
• In the case 1, the subject composed the piece in a
chronological way.
• In the case 2, the subject composed the piece by
combining some phrases he/she composed in their
own order.
Appeared in the Proceedings of Creativity and Cognition 4, 2002
Table 3 Defined Unit Cognition Processes chronological. It was observed that the spatial
representation offered a macroscopic viewpoint to the
Category Names Description Examples subject. The subject composed not in the chronological
Thought Compare Compareobjects Compare one combination with another way, but in such a way as "The subject composes some
Confirm Confirm object Listen to whole phrase phrases and then examines the structure of the piece from
Analyze Analyze object Analyze how 2 phrases are different the macroscopic view point".
Remember Remember object Remember which phrase is like what The subjects commented as follows in the interview after
Plan Plan the next action Plan to change the timbre the experiment:
Decide Decide the next action Decide to extend the phrase At the second time (without the spatial representation), it
Perception Listen Trial listening of object Listen to the phrase just editted was the failure that I tried to decide the phrase only in
Look Look at object Look at objects on the mental space, score my mind. That's why I came to the mental fixation. I
Read Read object Read the sentences attatched to the phrase
wanted to change the base line vastly (though he did not
do it indeed). (At the third time with the spatial
Action Play Play object Play the phrases
representation) as the change did not go along with my
Explore Explore to find something Look for a timbre
taste, I composed three candidates. Then I tried to
Adjust Adjust attributes of object Adjust the length, pitch, timing choose which were suitable for this part. I wanted to
Enter Enter object Enter a note, or notes compare with each other. While I moved the phrases and
Delete Delete object Delete a note, notes or a phrase did trial listenings (with the spatial representation), it
Combine Combineobjects Combine 2 or 3 objects went well though it was difficult when I did it only in my
Move Move objects on the space Move to group the phrases mind. Visual representation helped a lot.
Extend Extend the object Extend the phrase By comparing phrases on the space, one of the subject
Select Select the object(s) Select a new phrase found that he unconsciously composed the completely
Make Make a new object Make a new file
same phrases. Another subject talked about the positions of
the phrases on the space to analyze his own piece.
Figure 6 shows the composition process where the subject
could not develop the phrase in the case 1. In this diagram, Table 4 shows the ratio of the generated bars before and
the loop 1 shows that the subject looked at and wandered after MACSS was introduced. As to the subject E, he was
some windows on the screen trying to hit upon a new very accustomed to composition so he could complete
development. The process changed to the loop 2 (transition composition of a piece at one time. It can be seen that the
arrow A) where the subject repeated microscopic actions. ratios of the subject A and B increases while those of the
Eventually, the subject did not extended the phrase and said, subject C and D decreases. Though the subject A and B
"What should I do next?" again (transition arrow B and C). used MACSS, C and D did not. It is interesting that the
After this process, the subject repeated trial listenings and users ' ratio increased and non-users ' decreased. Of course it
many of the actions the subject took were only microscopic is possible to judge that the subject C and D already
ones such as adjusting the tempo. finished developing their phrase by the time when MACSS
Figure 7 shows the composition process where the subject was introduced. However, the subject C developed the
could develop the phrase in the case 2. In this diagram, the phrase chronologically and there were some clues for
same loop 1 as in the figure 6 was observed. The repeated mental fixations.
microscopic actions were also observed (loop 2). Then the In the middle of the second experiments the subject C
subject did a trial listening through whole phrase to hit referred to the further development of the piece: "A
upon a phrase that comes to the right next to the phrase the dramatic development is coming soon"; "Main phrase is
subject has just listened. As mentioned, in the two cases of coming after this phrase. In spite of these statements, there
the case 1, composition processes proceeded in a was no further development. Though the subject D
chronological way. composed some phrases, he did not come to the
Figure 8 shows the composition process where the subject organization of the whole of the piece. That is, the
could develop the phrase in the case 2. In this diagram, composition of subject C was restricted by the
such new processes as "PlanMakePhrase", "MakeNewFile" "chronological limitation", i.e., "The feeling the
and (transition A), the new process "PlanCombination", representation lets the composer compose in a
where the subject plans how to combine the phrases into a chronological way" and that of D lacked the "macroscopic
piece, were observed. This process is, what we call, a viewpoint". From these results of the analysis, there are two
macroscopic composition that did not appear in the case 1. patterns of escaping from the mental fixation:
In the loop 2 in figure 8, the subject selected phrases for
1. Develop the phrase chronologically
trial listenings (SelectPhrase), did trial listenings
(PlayOnScreen) and moved phrases to decide how to 2. Develop some phrases unchronologically and try
organize the piece. After some repetition of the loop 2, the to organize their combination
subject finally could decide the s tructure of the piece The second pattern can be triggered by the spatial
(transition B). In this third case the process was not representation because:
Appeared in the Proceedings of Creativity and Cognition 4, 2002
Figure 6 Cognitive Processes without MACSS
(Mental Fixation)
Figure 7 Cognitive Processes without MACSS
(Chronological Development)
Figure 8 Cognitive Processes with MACSS (Mental Leap)
Table 4 The Ratio of the Number of the Generated Bars
before/after Using MACSS
Before After
Subject A 4.92% 95.08%
Subject B 42.75% 57.25%
Subject C 76.58% 23.42%
Subject D 56.86% 43.14%
Subject E 100.00% 100.00%
1. It allows the composer to edit the piece not in
"note level" but "phrase level".
2. It facilitates for the composer to grasp the
relationship among phrases.
3. It does not show chronologicality with which
ordinary musical editors usually provides the
composer.
As these patterns seem to be dependent on the
representation, the result should be taken into consideration
when the musical composition supporting system is
designed. Though further statistical analysis is necessary,
this is one of the researches to tackle the real-world
analysis of creative activities. In the research field of
creative activities, it is necessary to investigate various
types of real creative activities in detail.
Nakakoji et al. [20] refers the support for the reflection-in-
action and reflection-on-action in the early phase of the
design, especially writing process. In their system, the
writer can move the document objects on the two-
dimensional space and it supports the reflection-in-action
by facilitating the comparison between the document
objects. In our research the subject reflected on
himself/herself by comparing some patterns of organization
which the subject composed by moving the phrases and
changing the order of the phrases. Therefore, MACSS
offers a macroscopic view with the spatial representation
and facilitates for the composer to grasp all the phrases and
the relations between them, which supports the reflection-
on-action phase on the musical composition. In addition, by
allowing the composer to listen to the plural phrases in the
sequence in which the composer would like to listen to,
MACSS supports the reflection-in-action phase. We
believe that MACSS facilitates for a composer to change
the viewpoint from microscopic to macroscopic and vice
versa, which enables him/her to compose a piece with
consistency that he/she would like to realize.
CONCLUSION
Though there has been a lot of research on cognitive
processes of listening and music al performance, few
researches on those of musical composition have been
conducted [2]. In the research field of creativity support, as
few microscopic analysis have been conducted it is one of
the important challenge to detail the knowledge on control
and measurement of cognitive processes through various
instances. As for the application of a spatial representation
to musical composition, Nishimoto et al. [21] proposed and
implemented a supporting system for improvisation in jazz
session. Our research focused on the microscopic analysis
Appeared in the Proceedings of Creativity and Cognition 4, 2002
of the effects on cognitive processes in the musical
composition caused by the change of the representation of
the information. The analysis of behaviour difference in
terms of the degree of expertize in composition is also
interesting topic, but that is beyond this paper. With such
analyses on various aspects of the musical composition
process, it is expected to obtain knowledge about the design
theory to construct a supporting environment for musicians
that enables them to be more creative. And it is also
expected to generalize the theory to the various systems for
creativity support.
REFERENCES
[1] Donald A. Norman: "Things That Make Us Smart:
Defending Human Attributes in the Age of the
Machine", Addison Wesley Publishing Company,
1994
[2] Tanaka, Y.: "Musical Composition as a Creative
Cognition Process", Report of Cultural Sciences
Faculty of Tokyo Metropolitan University, Vol.307,
No.41, pp.51-71, 2000
[3] Schoen, D A: "The Reflective Practitioner: How
Professionals Think in Action", Basic Books, NY,
1983
[4] Hori, K: A Model to Explain and Predict the Effect of
Human-Computer Interaction in the Articulation
Process for Concept Formation", Information
Modelling and Knowledge Bases, Vol.7, pp.36-43,
IOS press, 1996
[5] Hori, K.: "A System for Aiding Creative Concept
Formation", IEEE Trans. Systems, Man, and
Cybernetics, Vol.24, No.6, pp.882-894, 1994
[6] Zhang J: "The Nature of External Representations in
Problem Solving", Cognitive Science, Cognitive
Science Society, vol.21 (2), pp.179-217, 1997
[7] M. Suwa and T. Purcell and J. Gero: "Macroscopic
analysis of design processes based on a scheme for
coding designers' cognitive actions", Design Studies,
Vol.19, No.4, pp.455-483, 1998
[8] Shipman, F M, Marshall, C. C., and Moran, T P:
"Finding and Using Implicit Structure in Human-
Organized Spatial Layouts of Information",
Proceedings of CHI'95, ACM, pp.346-353, 1995
[9] Yamamoto, Y.: "Amplifying Representational
Talkback: Interactive Systems Using Spatial
Positioning to Support Early Stages of Information
Design", Ph.D thesis at Graduate School of
Information Science Nara Institute of Science and
Technology, 2000
[10] Reybrook: "Gestalt concepts and music: limitations
and possibilities" in: M. Leman (ed.) Music, Gestalt
and computing. Lecture notes in artificial intelligence,
no.1317 (Berlin: Springer-Verlag), 1997
[11] Sloboda, J.A: "Composition and Improvisation", The
Musical Mind, chapter 4, Oxford, pp.102--150, 1985
[12] Umemoto, T.: "The Psychology of Musical
Composition", Musical Psychology, Chapter 6,
pp.377-413, Seishin Shobo, 1972
[13] Abe, J.: "How is melody processed", Music and
Cognition, Cognitive Science Library, Vol.12, pp.41-
68, University of Tokyo Press, 1993
[14] Walter B. Hewlett, Eleanor Selfridge-Field: "Melodic
Similarity": Concepts, Procedures, and Applications",
Computing in Musicology 11, MIT Press, 1998
[15] Hiraga, Y.: "Knowledge Representation for Music
Cognition", Music and Cognition, Cognitive Science
Library, Vol.12, pp.97-130, University of Tokyo
Press, 1993
[16] Yanase, T.: "Melody Searching System by Feature
Excerption from Play Information", Ph.D thesis, the
Department of Electric Engineering, Graduate School
of University of Tokyo, 1999
[17] Yoshino, I. and Abe, J.: "Key Recognition:
Calculation Model to Interpret the Key of the
Melody", The World of Co mputer and Music, pp.117-
131, Kyoritsu Press, 1998
[18] Saito, T.: "Non-measuring Multi-Dimensional Scaling
Method", Multi-Dimensional Scaling Method,
Chapter 5, pp.70-86, Asakura Books, 1983
[19] Patrick, C.: "Creative Thought in Artists", Journal of
Psychology, Vol.4, pp.35-73, 1937
[20] Nakakoji, K., Yamamoto, Y., Reeves, B.N., Takada,
S.: "Two -dimensional Positioning as a Means for
Reflection in Design", Proceedings of DIS 2000,
ACM, pp.145-154, August, 2001
[21] Nishimoto, K. Mase, K. Nakatsu, R.: "Melody
Creation Support by Visualization of Relationships
among Phrases and Musical Primitives", Journal of
Information Processing Society of Japan, Vol.40,
No.2, pp.687-697, 1999