Professional Documents
Culture Documents
research-article2015
UPDXXX10.1177/8755123314567782UpdateBrunkan
Article
Update
2016, Vol. 34(3) 5662
2015 National Association for
Music Education
Reprints and permissions:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/8755123314567782
update.sagepub.com
Melissa C. Brunkan1
Abstract
The purpose of this study was to validate previous research that suggests using movement in conjunction with singing
tasks can affect intonation and perception of the task. Singers (N = 49) were video and audio recorded, using a motion
capture system, while singing a phrase from a familiar song, first with no motion, and then while doing a low, circular
arm gesture. Analysis of relationships between a circular singer arm gesture and changes in intonation indicated
most singers (67.3%, n = 33) were closer to the target pitch when doing the low, circular gesture. Additionally,
significant correlations were found between motion of the hand and face. Participant perceptions of singing with
motion included fuller tone and more breath with the lower motion and singing without motion viewed as
easy and comfortable. Results of this study suggest that singing with motion can affect intonation, other bodily
movements, and perception of singing.
Keywords
acoustics, gesture, motion capture, perception, singing
Corresponding Author:
Melissa C. Brunkan, School of Music, Louisiana State University, 102
New Music Building, Baton Rouge, LA 70803, USA.
Email: mbrunkan@lsu.edu
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016
57
Brunkan
The use of motion capture technology in research on
music listening or performance has been somewhat limited. In 2008, Toiviainen used a motion capture system to
examine the kinematic and kinetic characteristics of
spontaneous movement to music. Results indicated that
musical beat was most clearly represented by movements
in the vertical direction and the beat tended to be associated with bursts of instantaneous muscular power.
Schoonderwaldt (2009) examined the bowing performance of violin and viola players (N = 4) using an optical
motion capture system that measured velocity, force, and
distance. He found that the players adapted the bowing
parameters to the physical properties of the string and
instrument because of the playable control parameter
space. Other motion capture studies in music performance have included the guitar (ElKoura, 2003; Norton,
2008), drums (Kawakami, Mito, & Watanuma, 2008),
and piano (Palmer & Bella, 2004).
In recent years, vocal and choral music have emerged
as areas of interest examined via motion capture technology. Livingstone, Thompson, and Russo (2009) reported
results of two experiments on facial expressions during
perception, planning, production, and postproduction of
emotional singing. Participants (N = 7) were recorded
with motion capture as they watched and imitated emotional singing as indicated by facial expression. Facial
expressions were monitored before their imitation, during
their imitation, and after their imitation. Results indicated
a role of facial expressions in the perception, planning,
production, and postproduction of emotional singing. In
another study, Kun (2004) used a responsive/interactive
system able to capture a conductors performance in three
dimensions. Finally, Manternach (2012) used motion
capture to examine mimicking effects of singers observing a videotaped conductor. Results indicated that participants imitated the conductors rounded lips, but most did
not notice changes in conductor eyebrow lifting.
The present investigation aligns with Brunkans
(2013) study of acoustic and psychoacoustic measurements of singing taken during three conducting conditions, singer gestural training, and singer gestural
movement on individual singers (N = 49) performances
of a sung /u/ vowel1 in the context of a familiar song.
Results indicated beneficial effects on intonation when
singers performed a low, circular gesture while singing an
/u/ vowel.
Method
Participants (N = 49) were male (n = 14, 29%) and female
(n = 35, 71%) singers ranging in age from 19 to 67 years
(M = 30 years, SD = 11), with 27 participants (55.1%) in
a choir at the time of the study. A majority of participants
had choral experience (40; 81.60%), as an adolescent (36;
73.42%), in high school (38; 77.65%), and in college (40;
81.60%). This convenience sample was obtained on a
volunteer basis from university classes. All participants
completed consent forms and stated that they were familiar with the song excerpt.
The musical excerpt used consisted of the final phrase
of the melody line of Happy Birthday to You in the key
of D major. This phrase was chosen because it was a part
of a well-known composition and it ended on a sustained
/u/ vowel on the word you. The /u/ vowel was chosen
for examination as its clear enunciation by singers or
speakers occasions exaggerating the acoustic effect of
backness through some degree of forward lip rounding,
which serves to lengthen the vocal tract. This increase in
vocal tract length allows clear analysis of vowel sound.
I gave participants an information form on entering the
research room, as well as a code that identified their number, group, and ordering of conditions. Participants then
completed a 12-item questionnaire on demographic information, such as singing experience, age, and training.
Next, the musical excerpt was played once on a keyboard
(MM = 85) in the key of D major. Participants rehearsed
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016
58
Update 34(3)
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016
59
Brunkan
Using the computer software (Arena Motion Capture
software, Version 1.4.0) developed to accompany the
motion capture system, all the participant sensors, as
recorded by six of the seven infrared cameras, were trajectorized on the X (horizontal), Y (vertical), and Z (depth)
planes. The seventh camera was used for audio and video
recording. The audio recording was opened with
WaveSurfer acoustical software (Version 1.8.5), where
labels were manually placed for each of the measurement
windows. A temporal subset of the motion data was analyzed on each trial to account for a variable asynchrony
between the motion and acoustic data inherent in the
OptiTrack recording system.
Each marker (four on the headband, one over each
eyebrow, one on the bridge of the nose, four around the
lips, and four on the finger) was checked for numerical
labeling in order to correlate the number of the marker
with the row of data in the output file. Next, using Virtual
Dub (Version 1.9.11), the audio and video files were separated and saved under the participant code (number,
take, condition). Then, using Tcltk (Version 1), the output
of the c3d file was converted into a text file. A custom
program was used to merge the motion and derived auditory/acoustic measurement files.
Text files of the numbers corresponding to the motion
data in three dimensions (x [side to side], y [up and down],
and z [depth]) were loaded into Excel. The numbers were
labeled corresponding to the placement of the marker, participant take/number, and the dimension recorded (i.e.,
head X [head movement on horizontal plane], nose Z [nose
movement on depth plane], and hand Y [hand movement in
vertical plane]). Those numbers were then used to create
correlation matrices by each marker location and dimension in order to assess possible relationships. Weak (<.30),
moderate (.30.58), and strong (>.58) correlations were
noted (Stevens, 2009, p. 251). Absolute numbers were then
charted in order to assess strong relationships, positive or
negative, of the various markers and dimensions. All matrices were then reevaluated in terms of relationships
observed (i.e., nose x and head x). These expected relationships were recorded. Other correlations were recorded as
well (i.e., eyebrow y and hand x). Movements were measured from the time the singer began singing the phrase
until the end of the phrase (5.97 seconds).
The motion capture system requires that a majority of
the cameras track each sensor in order to trajectorize participant movement. Depending on the height, facial structure, and body movements of each participant, the sensors
occasionally moved out of view of the required number of
cameras. Therefore, the analysis used case-wise deletion,
thus eliminating participants who were not deemed to have
reliable measurements (i.e., at least 80% of data points for
each marker recording window) for all the conducted conditions in a particular comparison. All participants (N = 49)
were included for at least one of the following tests.
17 cents
812 cents
1319 cents
20 cents
Percentage
6.06
30.30
27.27
36.36
2
10
9
12
Results
The first research question asked whether there were differences in measures of fundamental frequency (intonation) when singers (N = 49) performed a low, circular arm
gesture compared with singing with no gesture. Most singers (67.37%, n = 33/49) were closer to the target fundamental frequency when doing the low, circular motion (M
= 19.01 cents, SD = 47.53 cents) compared with baseline
measures. Findings also indicated that 93.94% (n = 31/33)
of participants who came closer to the target fundamental
frequency when doing movement made an audible change
(>7 cents). Difference in measures of deviation in cents
from target fundamental frequency is shown in Table 1.
Results of a paired sample t test (p < .05) of mean
deviation in cents from target fundamental frequency by
condition were statistically significant, t(49) = 2.77,
p = .01. Measures of deviation in cents from target fundamental frequency showed that overall, participant deviation from the target fundamental frequency was lowest
(most in tune) when doing the low, circular gesture
(M = 18.97, SD = 44.94) versus doing no motion (M =
29.01, SD = 52.69). The mean difference across participants was 10.06 cents, an audible difference.
The second research question inquired as to the extent
of correlations between motion data (direction of body
movement measured by sensors) and frequency contours
(direction of sung pitch during the sung phrase). Results
of Pearson productmoment correlation coefficients indicated no significant correlations between frequency contour and the hand moving in the x (side to side) dimension,
r(20) = .20, p = .39; the z (depth) dimension,
r(20) = .25, p = .29; and y (vertical) dimension,
r(20) = .33, p = .15, of the hand markers.
The third research question addressed correlations
between movement of the head, nose, eyebrows, or mouth
with that of the hand as measured by tracked markers.
Moderate correlations were found between some directions
of movement. Results of Pearson productmoment correlation coefficients indicated significant relationships
between head markers in the z (depth) dimension with hand
markers in the x (side to side) dimension, r(20) = .48, p =
.03 as well as hand markers in the z dimension, r(20) = .46,
p = .04. These results indicate that the forward or backward
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016
60
Update 34(3)
Response category
Breath
Volume
Vibrato
Tone color
Rhythm/beat
No motion
Attention/focus
None
Physical/movement
Breath
Vibrato
Number
Percentage
19
11
7
7
6
39
22
14
14
12
8
5
4
2
1
16
10
8
4
2
Discussion
movement of the head is associated with similar and sideways movement of the hand. This finding suggests that
singers may move their heads forward and backward when
their hands move.
Moderate, positive correlations were seen between a
variety of markers. Significant correlations were found
between the eyebrow marker in the y dimension and the
hand marker in the y dimension, r(20) = .47, p = .04.
Similar to the head marker in the y (vertical) dimension,
when the hand moved vertically, so did the eyebrow.
There was also a significant relationship between the bottom lip marker in the x dimension and hand marker in the
x dimension, r(20) = .50, p = .02. Again, similar to the y
(vertical) dimension, this finding may indicate that when
the hand moved in the x (horizontal) dimension, so did
the bottom lip. Moderate, significant correlations between
the bottom lip marker in the z dimension and hand marker
in the x dimension, r(20) = .51, p = .02 and z dimension,
r(20) = .55, p = .01, were also found.
Finally, the fourth research question inquired as to
participant perceptual responses of singing with the low,
circular arm movement or no movement. Participants
wrote responses to the following prompt: What
difference(s), if any, did you notice in your singing when
doing (a) low arm circles and (b) no motion? Table 2
reports the five most frequent responses in order of
decreasing frequency according to type of gesture (low
arm circles, no motion). A research assistant independently analyzed participant responses. Reliability was
calculated using the formula (agreements/agreements +
disagreements) 100. Observer agreement was found to
be 87% for response analysis.
The most frequent responses from participants with
respect to the low, circular gesture had to do with breath
and volume. Those who mentioned breath (n = 19/49,
39%) wrote that the influence of low circles was more
air production/support, lower breath support, and the
Singer gesture has historically been used in vocal pedagogy to evoke changes in sound. Findings of this study
seem to support previously reported evidence on the beneficial effects of singer gesture. Participants perceived the
low, circular arm gesture positively, acoustical measures
of fundamental frequency indicated significant differences in intonation while using the circular gesture, and
the majority of participants (67%) came closer to the target fundamental frequency when doing the low, circular
arm gesture. Thus, the findings of this study indicated
that the low, circular arm gesture appears to contribute
overall to in tune singing as found in previous research
(Brunkan, 2013). It is also important to note that the magnitude of the differences would be noticeable to the
human ear (>7 cents). With this finding in mind, future
research might examine the plane of this gesture in relation to pitch. Perhaps doing the same size and direction of
a circular motion at a higher plane would affect acoustical
measures, particularly intonation, in a different manner.
Findings about frequency contour and movement
markers of the motion capture system indicate some
interesting trends. Motion markers of the nose, eyes,
head, and lips showed similar relationships in the three
dimensions. Overall, in the x (vertical) and y (horizontal)
dimensions, most mean correlations were negative, possibly indicating that bodily motion and direction of frequency change do not move similarly in time or direction.
Depth of movement, however, seems to indicate the
opposite. This finding may be of particular interest to
voice educators. Singer movement of the face and head
indicated a variety of idiosyncratic movements while participants sang. Often, teachers and conductors indicate
direction of pitch with a gesture; however, it may be useful to use a variety of gestures when communicating pitch
direction to singers.
Participant movements were tracked by a variety of
markers in this study. Correlations between markers of the
hand, lips, head, and eyebrow offer interesting insights into
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016
61
Brunkan
the movements of singers. Findings indicated that horizontal movement of the hand accompanies similar movement
of the lips and head for some singers. Participants in this
study also moved their heads in the depth dimension (forward and backward) when the hand moved on a similar
plane. As voice educators, awareness of the effect of singer
movement is essential to fostering efficient vocal production, as secondary movements may affect vocal technique.
It is important, therefore, to observe the effects of movement on other areas of the body.
Singer perception of sound and the sensation of singing
are often important to the longevity and enjoyment of the
task. Singer responses, overall, suggest perceptions of
singing louder, taking a deeper breath, and producing a
fuller tone while performing the low, circular gesture. As
breath is essential to each of these observations, it follows
that the low gesture might enable singers to focus more on
their lower abdominal muscles, and thus, feel like they are
getting a deeper, fuller breath that could support a louder,
fuller tone. Singers with choral singing experience may
find no movement most comfortable and familiar as this
posture would normally be employed in a choral rehearsal.
Therefore, future research could examine movement in
relation to singer experience, familiarity with movement
tasks, participation in choir, or voice lesson experience.
However, the positive effects on intonation found in this
investigation supports use of gesture with singers even if
there is a level of discomfort or unfamiliarity.
Several confounding variables may have affected the
results of this study. Although participants were afforded
gestural training and gesture usage was controlled as
much as possible through instruction, participant performance varied depending on factors such as singer familiarity with movement, singer arm and hand shape, singer
energy level, and other physical qualities of the singers.
Furthermore, this research aimed to create a naturalistic
setting similar to that of a voice studio or choral rehearsal;
however, idiosyncratic singer movement could have contributed to slight differences in motion and subsequent
variability in acoustic, motion, and/or perceptual measures. Such restrictions may also inhibit singers natural
physical response and thus, their vocal production. Future
researchers may well decide to assess level of familiarity
with movement or explore singer-chosen movement
instead of conductor-prescribed gestures. Although not
included in this investigation, motion capture data on
velocity and distance of movement may be examined in
future research to further investigate the relationship of
movement to sound.
The movement of the arms may have caused movement in other parts of the body, such as the torso.
Movement of the torso could affect aspects of singing
such as breath and therefore intonation. Although movement of the torso was not measured as a part of this
investigation, measurement of breath and muscle activation may be areas of interest to future investigations.
Future research is needed to examine the effects of
movement in a choral setting. The results of this solo
singing study may be of interest to choral directors.
Conglomerate choral sound as created by individuals in
the choir is often more than a simple sum of each of its
individual parts. If sufficient numbers of individual choristers with similar proclivities evidence desirable
nuances in vocal production behaviors before singing in a
group, the acoustical chorusing that occurs in choirsinging contexts may result in more robust differences in
group sound than would be the case in solo sound.
Finally, singer perceptions of breath and tone quality
in relation to gesture may suggest that future research in
this area use dependent measures other than those that
focus exclusively on intonation. Researchers could complete additional studies to examine the effects of singer
gesture with technologies such as electromyography.
The primary contribution of this investigation is that a
low, circular arm gesture may produce audible improvement in sung intonation. Moreover, participants perceived
beneficial effects on their singing when the low, circular
gesture was present. These findings on the use of singer
gesture may encourage teachers and students in voice studios, as well as choral rehearsals, to explore movement as
a tool to evoke certain sounds in the process of singer
education.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
Note
1. The International Phonetic Alphabet nomenclature (/u/)
represents the sound produced by singers within the final
word of the phrase, you. The sung /u/ vowel was selected
for this initial exploration of potential acoustical effects for
two reasons. First, /u/ can be one of the most readily visible vowels. Although /u/ is typically described as a high,
back vowel, its clear enunciation by singers or speakers occasions exaggerating the acoustic effect of backness through some degree of forward lip rounding, which
serves both to lengthen the vocal tract and lower the first
three formants.
References
Boersma, P., & Weenink, D. (2014). Praat: Doing phonetics by
computer (Version 5.4) [Computer software]. Retrieved
from http://www.praat.org/
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016
62
Update 34(3)
Nair, S. P., Gibbs, S., Arnold, G., Abboud, R., & Wang, W.
(2010). A method to calculate the centre of the ankle joint.
Clinical Biomechanics, 2, 582587.
Norton, J. C. (2008). Motion capture to build a foundation for
a computer-controlled instrument by study of classical guitar performance (Doctoral dissertation). Retrieved from
ProQuest Dissertations and Theses database. (ATT No.
3332997)
OptiTrack (Model V100 3-D infrared motion capture system) [Apparatus and software]. (1996). Corvallis, OR:
NaturalPoint.
Palmer, C., & Bella, S. D. (2004). Movement amplitude and
tempo change in piano performance. Journal of the
Acoustical Society of America, 115, 2590.
Peterson, C. W. (2000). Moving musical experience in chorus: Movement that is explored and utilized in the choral
rehearsal can be a powerful tool for music expression.
Music Educators Journal, 18(3), 2528. Retrieved from
http://gateway.proquest.com.www2.lib.ku.edu:2048/
openurl?url_ver=Z39.88-2004&res_dat=xri:iimp:&rft_dat
=xri:iimp:article:citation:iimp00163540
Pierce, A. (2007). Deepening musical performance through
movement. Indianapolis: Indiana University Press.
Racic, V., Brownjohn, J. M. W., & Pavic, A. (2010).
Reproduction and application of human bouncing and
jumping forces from visual marker data. Journal of Sound
and Vibration, 329, 33973416.
Samaan, M. (2009). Development of a three dimensional
kinematic model of lip and jaw coordination during
speech (Doctoral dissertation). Retrieved from ProQuest
Dissertations and Theses database. (ATT No. 1472919)
Schoonderwaldt, E. (2009). The player and the bowed string:
Coordination of bowing parameters in violin and viola performance. Journal of the Acoustical Society of America,
126, 27092720.
Stevens, J. P. (2009). Multivariate statistics for the social sciences. New York, NY: Routledge.
Suk, H., Sin, B., & Lee, S. (2010). Hand gesture recognition
based on dynamic Bayesian network framework. Pattern
Recognition, 43, 30593072.
Thurman, L., & Welch, G. (2000). Forewords to this book:
Sunsets, elephants, vocal self- expression, and life-long
learning. In L. Thurman & G. Welch (Eds.), Bodymind
and voice: Foundations of voice education (pp. xi-xxiv).
Collegeville, MN: VoiceCare Network, National Center for
Voice and Speech, Fairview Voice Center.
Toiviainen, P. (2008). Kinematics and kinetics of musicinduced movement. Journal of the Acoustical Society of
America, 124, 2447.
Wis, R. M. (1993). Gesture and body movement as physical
metaphor to facilitate learning and to enhance musical
experience in the choral rehearsal (Doctoral dissertation).
Retrieved from ProQuest Dissertations and Theses database. (AAT No. 9327319)
Yehia, H., Kuratate, T., & Varikiotis-Bateson, E. (2002).
Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 30, 555568.
Downloaded from upd.sagepub.com at Universitatea de Arte "George Enescu" din Iasi on May 6, 2016