Mood Lighting System Reflecting

Music Mood
Chang Bae Moon,1 HyunSoo Kim,1 Dong Won Lee,2
Byeong Man Kim1*
1

Department of Computer Software Engineering, Kumoh National Institute of Technology, Gumi, Gyeongbuk, South Korea

Development Team, Hanwha Corporation, South Korea

Received 2 September 2013; revised 12 November 2013; accepted 13 November 2013

Abstract: The emotional impact of music or color can be

maximized if they are used together. This article presents
a mood-lighting system that automatically detects the
mood of a piece of music and expresses the mood via
synchronized lighting. To do this, the relationship
between mood words and colors was analyzed via a web
questionnaire (n 5 202) on moods associated with music
and colors. Data analysis generated lighting scenarios
reflecting changes in the mood of music. Each piece of
music was divided into several segments using structural
analysis, with the mood of each segment detected by the
mood classification module using the neural network. A
matching color was then assigned. The best performance
of our mood classification module was <70%, which
is not sufficient for commercial use; however, this
figure is high enough to show the potential of this
approach. VC 2013 Wiley Periodicals, Inc. Col Res Appl, 40, 201
212, 2015; Published Online 11 December 2013 in Wiley Online
Library (wileyonlinelibrary.com). DOI 10.1002/col.21864

Key words: music mood classification; mood color; mood

lighting system; neural network; mood color map; lighting scenario

INTRODUCTION

In settings such as theaters, cafes, concerts, and nightclubs,

music can maximize a persons mood. For example, playing sad music during a tragic scene in a film, or fearful
music during a frightening scene maximizes the intended
mood. Like music, lighting can be a useful tool for express*Correspondence to: Byeong Man Kim (e-mail: bmkim@kumoh.ac.kr)
Contract grant sponsor: Research Fund, Kumoh National Institute of
Technology.
C 2013 Wiley Periodicals, Inc.
V

Volume 40, Number 2, April 2015

ing mood. When the lighting in a cafe is integrated with the

overall design, this can produce a much better mood in
patrons. Lighting that works together with music is likely
to produce a much stronger mood. For instance, in a concert, emotional feelings are heightened if lighting corresponds to the music that the audience is enjoying.
As combinations of music and lighting promote a
stronger emotional impact, we have developed a system
to produce illumination synchronized with music. To construct such a system, the correlation between music and
colors had to be understood. Therefore, in the article, we
aimed to correlate colors with music indirectly, based on
correlations between mood and color (rather than a direct
correlation between music and color); this system first
identifies the mood of a piece of music, and then chooses
a color that matches the mood.
A number of researchers15 have sought to develop
methods to detect the mood of music. These studies have
been based on well-known mood models, such as those
developed by Russell,1 Hevner,2 or Thayer.3 The method
used in this article is based on Thayers mood model,
employing a similar approach to existing Thayer-based
models. Our data are unique due to their large volume
and the use of a Korean sample.
Some researchers, including Manav,6 Barbiere et al.,7
Bresin,8 Odbert et al.,9 Spence,10 Wright and Rainwater,11
Valdez and Mehrabian,12 DAndrade and Egan,13 and Ou
et al.,14 have investigated the correlation between mood
and color. While the results of these studies have
informed our system, we have based our model on our
own correlations between mood and color obtained
through an analysis of a Korean sample, as we suspect
correlations between mood and color may be vary based
on nationality.14 To understand this correlation, the relationship between mood words and colors was analyzed
via a web questionnaire (n 5 202) on moods associated
201

Fig. 1.

Mood models. (a) Russell Model. (b) Hevner Model. (c) Thayers Two-Dimensional Model.

with music and colors. Data analysis generated a lighting

scenario reflecting changes in the mood of music. As we
assumed each musical piece to consist of several parts,
and therefore various moods, the music was divided into
several segments using structural analysis,15 with the
mood of each segment detected by the mood classification module using the neural network. For the neural network to learn, 391 acoustic features (e.g., tempo,
chromagram, MFCC, and so forth) were extracted from
each musical segment. We found that the mood classification performance of our system declined when all 391
features were used, so we used the regression coefficient
to reduce the number of features used.16 This allowed us
to develop a mood lighting device that displays color
matching the mood of the music being played.
The remainder of this article is organized as follows.
Related Studies gives an overview of the pertinent litera202

ture, and Mapping Music to Mood describes the methods

used to automatically map the moods associated with a
music segment and analyzes the collected data on moods.
Mapping Mood to Color describes the method of mapping mood words to colors through an analysis of the collected data on colors associated with mood words, and
Implementation of Mood Lighting System describes an
implementation of the system proposed in this article.
Finally, Conclusion and Future Studies presents the
conclusion, limitations of the study, and ideas for future
studies.

RELATED STUDIES

Existing emotion models include those by Russell1 (Fig.

1a), Hevner2 (Fig. 1b), and Thayer.3 The Russell and
COLOR research and application

Hevner models use adjectives to describe emotions, which

can result in ambiguity when adjectives have duplicate
meanings. For this reason, in the article, we used
Thayers two-dimensional model, in which mood or emotion are expressed by a combination of arousal and
valence (AV) values. Arousal refers to the strength of
stimulation that listeners feel (i.e., weak or powerful) and
valence refers to the intrinsic attractiveness (positive
valence) or aversiveness (negative valence). Figure 1c
depicts Thayers two-dimensional mood model, as well as
the relationships among 12 adjectives used to describe
moods and emotions.
Liu et al.17 has presented a music mood recognition
system that used a fuzzy classifier to categorize a Strauss
waltz into five classes. Features such as tempo, strength,
pitch change, note density, and timbre are used. Katayose
et al.4 proposes a sentimentextraction system for pop
music; in this system, monody sound data are first converted into music code, from which melody, rhythm, harmony, and form are extracted. These two systems are
useful in themselves, but they use musical instrument digital interface or symbolic expression, as it is difficult to
extract useful features from sound data. However, much
of the sound present in the real world cannot be
expressed through symbols; no system exists that can correctly translate such sound data into symbolic expressions.18 This limitation necessitates a system that can
directly detect mood from sound data.
Feng et al.19 proposed a method of classifying moods
into four groupshappiness, sadness, anger, and fear
using tempo and articulation features. Li and Ogihara20
proposed a method of detecting mood using timbre, texture, rhythm, and pitch features, using 13 adjective groups
based on checklists by Hevner21 and Farnsworth22 as
mood classes. Yang et al.23 used a fuzzy-based method to
solve the ambiguity of expression that can occur when
only a single mood is allowed; they expressed musical
mood as a mix of several moods, denoted as separate
numerical values. However, Yang et al.24,25 noted that
this method might fail to take into account subjective
individual preferences, which would be necessary for personalized service. To solve this problem, rather than using
a single mood class, the authors used AV values composed of two real number values between 21 and 1 on
each axis of Thayers two-dimensional mood model.
They used two regressors to model AV values collected
from subjects, and suggested a personalized detection
method that characterizes users into professional or
non-professional groups depending on their degree of
understanding of music.
A number of studies5,79,26 have investigated the association between music and color. Barbiere et al.7 asked
subjects to assign 11 basic colors according to music with
two types of mood, happy or sad. Brighter colors
were usually associated with happy songs, while more
muted colors were usually assigned to sad songs. Bresin8
asked subjects to rate how well each of 8 colors in each
of their 3 shades corresponded with 12 music pieces
Volume 40, Number 2, April 2015

expressing different emotions. Different hues were associated with different emotions; furthermore, dark colors
were associated with minor tonality and light colors with
major tonality. Odbert et al.9 surveyed the mood of music
using Hevners model and asked subjects what color they
associated with the music. Through analyzing the relationship between the mood of music and the colors suggested by that music, They demonstrated that subjects
who disagree on the mood of a piece of music tend to
also disagree on the colors associated with that music.
These results were very similar to those obtained by studies in which subjects were asked to name the color best
fitting certain mood words. Palmer et al.26 provided
experimental evidence that music and colors were mediated by emotional association in two cultures, US and
Mexico, by showing that there were strong correlations
between the emotional associations of the music and
those of the colors chosen to go with the music. They
showed that faster music in the major mode made participants choose more saturated lighter colors and yellower
colors whereas slower. They also showed that minor
music produced the opposite pattern, that is, desaturated,
darker, and bluer colors were chosen for minor music.
Some researchers6,1114 have investigated the relationship between color and mood. Manav6 defined this relationship using adjective mood words. In this study,
subjects were provided with 41 colors and were asked to
select which of 30 provided adjectives (e.g., vivid, boring, cold, warm, exciting, fearful, mysterious, peaceful,
and relaxing) best matched them. Manav further analyzed responses for 10 colors and looked at associations
between subjects education level, age, and gender.
Based on these findings, he recommended certain colors
be used for bedrooms, bathrooms, and childrens room.
Valdez and Mehrabian12 investigated the relationship
between mood and color using the PleasureArousal
Dominance (PAD) emotion model, providing PAD-value
prediction equations with the parameters of hue, saturation, and brightness. Valdez and Mehrabian12 demonstrated strong and reliable relationships between
emotional reactions and each of brightness and saturation, but only a weak relationship with hue. Ou et al.14
demonstrated that color preference can be determined
using three color-emotion scales (cleandirty, tense
relaxed, and heavylight), with cleandirty being the
predominant scale. They also showed that color preference can be determined by the three color-appearance
attributes of hue, lightness, and chroma; the most disliked color was found to be at the hue angle of 105
with the chroma value of 31.
Previously,27 we investigated the relationship between
mood and color. However, the findings of our prior study
were not useful for informing our lighting system, as the
relationship focused on users individual musical preferences. As our mood-lighting system is intended to be
installed in public spaces such as parks, plazas, and
squares, we needed a model that mapped mood to color
without considering individual users preferences.
203

Fig. 2.

Music mood mapping process.

MAPPING MUSIC TO MOOD

In this article, to find a color that matched the music, first

the mood of the music was determined, and then the
color corresponding to that mood was chosen. This section describes the method of mapping music to mood; the
section after this describes the method of mapping mood
to color. As shown in Fig. 2, the mood mapping process
of music consisted of two phases: one was mood training
phase and the other mood identification phase. In the
mood training phase, features of each music segment of
Moon et al.s27 were extracted by MIRtoolbox28 and then
were reduced by the R2 method (see "Mapping Music
Segments to Moods" section); using the reduced feature
sets of the music segments and their mood information,
the relationship between mood and music (or mood mapping model or shortly mood model) was modeled via a
neural network. The music segments of Moon et als27
had been obtained by dividing the music into several segments via Levy et al.15 method and their mood information had been also collected from several subjects in our
previous work.27 In the mood identification phase, a new
music piece was divided into several segments via Levy
et al.15 method; their features were extracted and reduced;
the mood of each segment was determined by inputting
its reduced features into the mood model built in the
training phase.
Collecting the Mood of Music Segments from Subjects
To understand the relationship between mood and
music, we used Moon et als27 mood data set. This data
set had been collected for 3 days in a room with only one
dark glass window on one side, between the hours of
approximately 10:00 am to 5:00 pm. The subjects were
189 general participants. A total of 281 musical segments
(each of approximately 12 s) were extracted from 101
music pieces and were used for mood collection. From
this set of 281 segments, 47 were selected randomly and
played for the 189 general participants. All participants
used headsets to prevent interference from non-studyrelated noises.
Figure 3 shows the mood distribution of some sample
music pieces: note that approximately three music seg204

ments were extracted from each music piece, and that the
moods of these segments may differ. Distributions of 101
pieces of music can be summarized and classified into six
types. The first type is a piece of music which has one
predominant mood (Fig. 3a); the second type is for a
piece in which all moods are similar (Fig. 3b); the third
type contains a wide distribution of moods covering two
quadrants (Fig. 3c); the fourth type contains different
moods, all of which are in one quadrant (Fig. 3d); the
fifth type contains different moods with a wide range of
distribution (Fig. 3e); and the sixth type has high frequencies of several moods (Fig. 3f).
In Fig. 3, different colors denote different segments of
the same piece of music; there were an average of three
segments per piece. This analysis supports our assumption
that individual music pieces contain several moods, making it necessary to change the lighting color when playing
music.
Mapping Music Segments to Moods
To map a music segment to a mood automatically, it is
necessary to learn the relationship between its musical
features and its mood. In this article, features of music
segments were extracted using MIRtoolbox.28 These features can be divided into five categories: dynamics,
rhythm, timbre, pitch, and tonality. Dynamics includes
root mean square (RMS) energy, and rhythm includes
fluctuation summary, tempo, and attack times. Timbre
includes zero cross rate, brightness, and roll-off, as well
as spectral centroid, spread, skewness, kurtosis, entropy,
and flatness. Pitch refers to pitch and chromagram, and
tonality includes the clarity, mode, and harmonic change
detection function. We used the 391-dimensional vector
produced by the mirfeatures function of MIRtoolbox,
which calculates statistics such as the mean, standard
deviation, and slope, as well as the frequency, amplitude,
and entropy of periods, instead of the complete features
themselves.
If all 391 features of the feature vector are used, mapping performance may decrease due to the effect of noise.
For this reason, we chose some noise features using the
well-known dimensional reduction method, R2 reduction.29 In our experiments, we used the 50 features with
COLOR research and application

Fig. 3. Mood distribution of music. (a) Case 1: Music 37. (b) Case 2: Music 13. (c) Case 3: Music 95. (d) Case 4: Music
74 (e) Case 5: Music 1. (f) Case 6: Music 28.

the largest r2 values [see Eq. (1)] (Table I). The performances of these selected features are given in the section

Mood Mapping Performance. The term r2 was calculated as follows:

12
n
  n  n 
P
P
P
B
C
n
xi yi 2
xi
yi
B
C
i51
i51
i51
2 B
C
r 5Bs
C
 n   n 2s
n  n

2
n
@
A
P
P 2P
P 2
P 2
n
xi 2
xi
yi 2
yi
yi
n
i51

i51

where r2 is the regression coefficient, xi is the ith feature

vector, and yi is the class number of the ith feature vector.
In this article, the model used to map the musical features to mood was automatically generated using the neural network approach, a well-known machine-learning
technique, and based on a training dataset consisting of
the musical features of music segments and their associated moods. Subjects might assign different moods to the
same music segment, but it is necessary to define one representative mood per segment; in the training dataset,
only the representative mood of each music segment is
given. In this article, the representative mood of each segment was calculated using the definition in Moon et al.s
study.27
Volume 40, Number 2, April 2015

i51

i51

(1)

i51

Figure 4 shows the structure of the neural network

used in this study. The neural network consisted of 50
nodes in the input layer and 4 nodes in the output layer.
The feature vector consisting of 50 musical features, as
described above, was used as the input for the neural network, and a four-dimensional vector was the output. The
output vector (1, 0, 0, 0) indicates the first quadrant of
the AV model, (0, 1, 0, 0) indicates the second quadrant,
(0, 0, 1, 0) indicates the third quadrant, and (0, 0, 0, 1)
indicates the fourth quadrant. Note that the number of
output nodes should be 12 rather than 4, as there were 12
possible moods for participants to choose from. However,
our sample of 281 musical segments meant that there was
only an average of 23 segments per mood, with fewer
205

TABLE I. Top 50 musical features used in the experiments

Category
DYNAMICS
RHYTHM
TIMBER

TONAL

Subcategory

Sub-sub category

RMS energy
Attack slope
Attack time
Tempo
Spectrum
Spectral irregularity
Spectral kurtosis
Zero crossingrate
Brightness
Entropy of spectrum
Rolloff (85%)
Rolloff (95%)
Spectral centroid
Spectral flatness
Spectral flux
Roughness
Spectral skewness
Mel-frequency cepstral
coefficients (MFCC)
Delta MFCC (DMFCC)
Delta Delta MFCC (DDMFCC)
Centroid of chromagram
Chromagram
Harmonic change detection Function
Keyclarity
Mode
Peak chromagram

SD
Period entropy
Mean, period entropy
SD
Mean, SD
Mean, SD
Mean, SD
Mean, SD
Mean
Mean
Mean
Mean
Mean
Mean
Mean
Mean, Period Amp
SD
Mean (1, 8, 9, 10), Period Freq
(9, 11, 12, 13)SD (6, 7, 8, 9, 10, 11, 12, 13)
Period Amp (2), Std (4)
Std (4)
Mean
Mean
Mean, Period Amp
Mean, Std
Period Amp
Peak Pos Mean

than 10 segments for some moods, an insufficient number

to learn neural networks. For this reason, the 12 moods
were categorized into four groups corresponding with the
quadrants of Fig. 1c.
Mood Mapping Performance
To evaluate the performance of the mapping model,
leave-one-out cross-validation (LOOCV)30 was used. In
LOOCV, a single example from a set of examples is used as
the validation data, and the remaining examples are used as
the training data. This is repeated such that each example in
the set is used once as the validation data. In this article, we
chose 164 music segments covering all moods evenly as
examples among the total 281 samples. Precision which is
the number of correctly classified positive examples divided
by the number of examples labeled by the system as positive
was used as the measure of classification performance.
The performance of the mapping (or classification)
model might be dependent on the number of hidden

Fig. 4. Structure of neural network to learn and determine

the mood of music segments.

206

nodes or the number of iterations of learning. So, we conducted experiments varying the number of hidden nodes
from 2 to 25 and employing two different iterations
(3000 times and 4000 times) in the learning stage of the
neural network. As shown in the previous section, for
performance reasons, we did not use 391 features, but
used only the top 50 features after sorting the features
with the regression coefficient calculated by Eq. (1). The
experimental results are shown in Fig. 5. With 3000
learning iterations, we obtained the best performance
(66.46%) using 17 hidden nodes (Fig. 5b). With 4000
learning iterations, we obtained the best performance
(65.24%) using 7 hidden nodes (Fig. 5a). From the
results, we could conclude that the number of hidden
nodes and the number of iterations of learning did not
contribute to the performance improvement of our model
impressively. The performance we got is not suitable for
commercial use, but we believe it confirms the promise

Fig. 5.

Performance of mapping music to mood.

COLOR research and application

Fig. 6.

Mood color map of Nervous.

of our approach: our future research will focus on

improving the mood mapping performance.
MAPPING MOOD TO COLOR

To map mood to color, the relationship between mood

and color was analyzed using Moon et als27 color data
set. This data set was collected by asking subjects to
select a color matching a mood word, under the same
conditions described in the section on collecting moods
of music. Based on subsequent analysis, a representative
color was chosen for each mood word, which was used to
generate mood lighting scenarios.
Representative Color of Mood
Since different subjects might associate different colors
with given mood words, we needed to define the representative color of a mood word. For this, a mood color
map representing frequencies of colors that subjects
selected as the color matching the given mood was constructed for each mood word (e.g., Fig. 6 presents the
color map of Nervous). To create a lighting scenario,
the representative color (i.e., that with the highest selection frequency) was first selected from the mood color
map. In Fig. 6, for example, the representative color is
dark red, selected by 11 subjects.
The representative colors for the 12 mood words are
shown in Table II. The mood words belonging to Group
1 (Excited, Happy, and Pleased) are mapped with bright
red, bright red, and bright yellow, respectively; Group 2
(Angry, Annoying, and Nervous) with red, red, and dark
red, respectively; Group 3 (Sad, Bored, and Sleepy) with
dark yellow, bright blue, and bright yellow, respectively;
and Group 4 (Calm, Peaceful, and Relaxed) with bright
green, bright light green, and bright green, respectively.
Comparisons with results from related studies are given
in Table III.
The colors defined in Table II should be used when
mapping mood words to colors; however, the music was
mapped to 4 mood groups in our experiments, as opposed
to 12 moods. To accommodate this methodological
change, the color map for each mood group was reconVolume 40, Number 2, April 2015

structed by adding the color maps of moods belonging to

each group; from this, the color with the maximum frequency was selected as the representative color of that
mood group. Group 1 was yellow (RGB: 255-255-51,
HSV: 42204-255), Group 2 was red (RGB: 204-0-0,
HSV: 0255-204), Group 3 was bright blue (RGB: 204204-255, HSV: 170-51-255), and Group 4 was bright
green (RGB: 153255-153, HSV: 85102-255).
Analyzing the Color Distribution of Mood Words
In Moon et al.s27 color data set, the average number
of colors per mood was 154, for a total of 1848 colors.
For our analysis, we summarized the color of each mood
word as a percentage (Fig. 7). As was described in the
section Mapping Music Segments to Moods, 12 mood
words were grouped into 4 groups: Group 1 (Excited,
Happy and Pleased); Group 2 (Annoying, Angry and
Nervous); Group 3 (Sad, Bored and Sleepy); and Group 4
(Calm, Peaceful and Relaxed). Figure 7 summarizes the
color distribution for each group. After creating this color
distribution, we examined the relationship between colors
and mood words. For simplicity, we grouped colors into
six groups by their hue values, such that colors with hues
of 23029 were mapped onto Red, 3089 onto Yellow,
90149 onto Green, 150209 onto Cyan, 210269 onto
Blue, and 270329 onto Magenta.

TABLE II. Representative colors of 12 moods

No

Mood

Color

1
2
3
4
5
6
7
8
9
10
11

Excited
Happy
Pleased
Angry
Annoying
Nervous
Sad
Bored
Sleepy
Calm
Peaceful

255
255
255
204
255
153
102
204
255
153
204

51
153
255
0
0
0
102
204
255
255
255

51
153
51
0
0
0
0
255
204
153
153

0
0
42
0
0
0
42
170
42
85
63

204
102
204
255
255
255
255
51
51
102
102

255
255
255
204
255
253
102
255
255
255
255

12

Relaxed

Bright red
Bright red
Bright yellow
Red
Red
Dark red
Dark yellow
Bright blue
Bright yellow
Bright green
Bright
light green
Bright green

153

255

153

85

102

255

207

TABLE III. Comparisons of color mapping

Color
Red
Bright red
Dark red
Blue
Bright blue
(light blue)
Yellow
Bright yellow
(light yellow)
Dark yellow
Green
Bright green
Bright light green
Black
Gray
Near white
Cyan
Orange
Pink

Proposed

Manav6

Angry, annoying
Excited, happy
Nervous
Bored
Pleased, sleepy

Barbiere et al.7

Bresin et al.8

Odbert et al.9

Happy
Happy

Anger, jealousy

Exciting

Hygiene, pure
Calm, peaceful,
modern, relaxing
Hygiene, pure
Simple, classic, plain

Happy
Happy

Love, fear

Tender

Happy
Happy

Happiness, pride

Playful

Hygiene, pure

Happy
Happy

Sad
Calm, relaxed
Peaceful

Leisurely
Sad

Sad
Hygiene, pure, plain,
simple
Curiosity
Contentment, shame

Gay

Warm, romantic,
enjoying, cheerful,
striking

Purple
Violet

As shown in Fig. 7, red was most often selected for

Group 1 mood words; however, yellow was also highly
selected for happy and pleased, and blue was frequently
associated with excited. Red was also most often selected
for Group 2 mood words; the mood word with the highest
percentage of Red among the three words in this group
was angry. Blue and yellow were the main colors for
Group 3; Sleepy was frequently associated with yellow
while sad and bored had relatively equal associations
between blue and yellow. The main colors of Group 4
were yellow and green.
We also examined the relationship between the brightness and saturation of colors and the associated mood
words. Figure 8 represents the brightness and saturation
of mood words as a percentage: the x-axis indicates the

Solemn
Sadness, love,
tenderness, disgust

numerical value of brightness and saturation obtained

from the default color table given by Java31 (top row5 saturation; bottom row 5 brightness). Group 1 mood
words were associated with brighter and more highly saturated colors (Fig. 8a); Group 2 mood words with less
bright but highly saturated colors (Fig. 8b); Group 4 with
brighter but less saturated colors (Fig. 8d). For Group 3,
no pattern of association between brightness or saturation
and mood words was detected (Fig. 8c).

IMPLEMENTATION OF MOOD LIGHTING SYSTEM

The configuration of the emotional lighting system representing the mood of music is shown in Fig. 9. A PC

Fig. 7. Color distribution according to AV model area. (a) Color Distribution of Group 1. (b) Color Distribution of Group 2.
(c) Color Distribution of Group 3. (d) Color Distribution of Group 4.

208

COLOR research and application

Fig. 8. Brightness and saturation distribution for groups of mood words. (a) Group 1 brightness and saturation distribution. (b) Group 2 brightness and saturation distribution. (c) Group 3 brightness and saturation distribution. (d) Group 4
brightness and saturation distribution.

sends the lighting scenario through a network to a control

board, which in turn controls the lighting in accordance
with the scenario received. At the same time, the PC
plays the music (Fig. 9b). The system consists of a mood
color DB (Database) and five modules: music segmentation, music playing, mood extraction, the creation of
mood lighting scenarios, and the mood lighting module.
The mood lighting module is installed on the control
board, and the other components are installed on the PC.
Figure 9a shows the structure of the system that collects the moods of music and the colors of moods. The
server selects music segments by analyzing the musics
structure and sends these structures or mood words to clients. On the client end, a user is required to input the
moods of music segments or colors of moods. The input

data are saved in the music mood or mood color database

and used later, when needed.
Music Segmentation Module
To acquire music segments, the structure of the music
is first analyzed. Then, using the information about the
musics structure obtained through structured analysis,24
the music is segmented into several pieces, each with
similar acoustic features. In music structure analysis, similar segment clustering is used based on the state
sequence.15 Levy et al.15 extracted a music feature vector
and calculated a timbre-type sequence; finally, the music
structure information is identified through a timbre-type
soft k-means clustering algorithm. The music is then segmented according to its structure.

Fig. 9. Mood lighting system reflecting the mood of music. (a) Mood collection system configuration. (b) Mood lighting
system configuration. (c) Internal configuration of system.

Volume 40, Number 2, April 2015

209

Fig. 10.

Process of learning and determining the mood of a music segment.

Fig. 11. Process of creating a mood lighting scenario. (a)

Lighting scenario. (b) Scenario saving vector.

For example, if the mood changes from Angry to Annoying (Fig. 12a), HSV color vectors (Fig. 12b) are generated
to display the colors located along the connecting dotted
line, with the time interval calculated by considering the
music playing time and the number of colors to be displayed during that interval.
Music Playing Module

Mood Extraction Module

To determine the mood of a music segment (Fig. 10),
391 features of a music segment were extracted and
reduced using the R2 reduction method.29 In this module,
a mood model is constructed by the neural network, using
data consisting of the reduced features of music segments
and their representative moods, by which the mood of a
new music segment is determined.
Mood Lighting Scenario Creation Module
Figure 11 shows the process of generating a music
lighting scenario. When a piece of music is entered into
the system, its structure is analyzed and it is separated
into segments by the music segmentation module. Mood
segments are then determined by the mood extraction
module. The lighting scenario for a music segment is
generated using its mood information and the color map
of the mood obtained from the mood color information
database. The overall scenario of the selected piece of
music is obtained by combining the scenarios of each
music segment sequentially, and is saved in the lighting
scenario file.
The lighting scenario of a music segment is generated
by taking into account the mood of the current and
upcoming music segments. A sequence of HSV color vectors is generated so that the color changes smoothly from
the representative color of one music segment to the next.

In the PC, a lighting scenario file is sent to the control

board before the music is played. The user interface for
playing music is shown in Fig. 13a, and the music play
list is shown in Fig. 13b. Music is played in the order of
the play list and, once determined, the order cannot be
modified. In this article, numbers are used for identifying
music instead of file names due to copyright restrictions.
The user interface (Fig. 13c) enables administrators to
check the status of the lighting device. If there is no problem with a lighting device, the color of the bulb is indicated in white. If there is a problem, the bulb is black.
The number on the lower part of the bulb (963 in Fig.
13d) is the identifier of the lighting module. The user
interface (Fig. 13d) allows users to check the communication status of the network, which is shown to the user in
real time. The communication status is also recorded in
the log file for reviewing later.
Mood Lighting Module
The mood lighting module consists of an RGB lightemitting diode (LED) matrix and a control board (Fig.
14). The RGB LED matrix expresses color by receiving
the RGB signal from the control board. The control board
sends data to and from the host system, in addition to
controlling the RGB LED matrix.
The control board consists of an micro controller unit
(MCU), pulse width modulation (PWM) GENERATOR,

Fig. 12. Example of creating a lighting scenario. (a) Music player. (b) Music playlist. (c) Status of lighting devices.
(d) Communication status.

210

COLOR research and application

Fig. 13. Hostuser interface.

 RTC: provides a real-time clock

 Wired LAN, 802.11: communicates with the host system

Fig. 14. Mood lighting module. (a) Rear. (b) Front.

static random access memory (SRAM), real-time clock

(RTC), wired LAN and 802.11. Each component is
described below:
 MCU: controls all devices
 PWM GENERATOR: controls the LED of the RGB
LED matrix and expresses color
 SRAM: saves RGB data (lighting scenario)

The control board is shown in Fig. 15. Figure 15a

shows the back of the control board; the RGB LED
MATRIX (Fig. 15b) is mounted on the front of the control board using the supporters. For data communication
with the host system, 10 Base-T or 802.11.b/g is used,
and the color of lighting is controlled by the LED driver.
The cross shape at the center of the RGB LED matrix
allows heat generated from the LED to be expelled using
the cooling fan mounted at the back.

CONCLUSION AND FUTURE STUDIES

Both music and lighting can be used to express mood;

moods can therefore be maximized by combining music
and lighting. This article proposed a system that automatically segments music into several pieces, extracts their
moods, and displays the color matching each mood via an

Fig. 15. Hardware of the mood lighting system.

Volume 40, Number 2, April 2015

211

LED lighting device. To do this, we analyzed the associations between mood words and colors using Moon
et al.s27 mood and color datasets. We observed the color
distribution associated with each mood and set a representative color for each mood word.
In this article, we mapped music to color indirectly,
first determining the mood of a music segment and then
selecting a color that matched that mood. To determine
the mood of a music segment, we took a similar approach
with prior approaches based on the Thayer model. However, the data used in the article differed from their dataset in that music segments were automatically generated
through a structural analysis, not manually. Furthermore,
our mood data set was much larger and was collected
from Korea. Finally, prior studies have treated musical
pieces as unchanging in mood, while our model separated
music into short segments that could have different
moods.
We obtained our own data on the correlations between
mood and color based on a large sample of Korean participants instead of using the correlations previously
determined by researchers such as Manav,6 Barbiere
et al.,7 Bresin,8 Odbert et al.,9 Spence,10 Wright and
Rainwater,11 Valdez and Mehrabian,12 DAndrade and
Egan,13 and Ou et al,14 whose results differed greatly
from ours. This discrepancy may reflect cultural differences in samples of different nationalities; however, our
findings should not be generalized to the Korean population as a whole, as our sample was mostly university
students.
The prototype proposed in this article lays the foundation for a commercial system; however, before such a
system can be developed, mood classification performance must be improved. The model also needs to be
expanded to cover 12 distinct moods, as opposed to 4
mood groups, as was done in the current study. We
assumed that the algorithm successfully extracted the
mood from the music, and a color was chosen to match
that mood. So, in the near future, we need to show the
evidence that participants agree that the color matches the
music. Furthermore, we need to get a more continuous
map from color to music through mood by allowing
music to represent different moods at once and colors to
represent the multiple moods as well.
1. Russell JA. A circumplex model of affect. J Personality Social Psychol
1980;39(6):11611178.
2. Hevner K. Experimental studies of the elements of expression in music.
Am J Psychol 1936;48(2):246268.
3. Thayer RE. The Biopsychology of Mood and Arousal. New York:
Oxford University Press; 1989.
4. Katayose H, Imai M, Inokuchi S. Sentiment extraction in music. International Conference on Pattern Recognition, 1988. p 10831087.
5. Lee JI, Yeo D-G, Kim BM, Lee H-Y. Automatic Music Mood Detection through Musical Structure Analysis. International Conference on
Computer Science and its Application CSA, jeju, Korea, 2009. p 510
515.

212

6. Manav B. Color-emotion associations and color preferences: a case

study for residences. Color Res Appl 2007;32(2):144150.
7. Barbiere JM, Vidal A, Zellner DA. The color of music: correspondence
through emotion. Emp Stud Arts 2007;25(2):193208.
8. Bresin R. What is the color of that music performance? Proceedings of
the International Computer Music Conference, 2005. p 367370.
9. Odbert HS, Karwoski TF, Eckerson AB. Studies in synesthetic thinking: I. Musical and verbal associations of color and mood. J Gen Psychol 1942;26:153173.
10. Spence C. Crossmodal correspondences: A tutorial review. Attent Percept Psychophys 2011;2011(73):971995.
11. Wright B, Rainwater L. The meanings of color. J Gen Psychol 1962;
67:8999.
12. Valdez P, Mehrabian A. Effects of color on emotions. J Exp Psychol
Gen 1994:123(4):394409.
13. DAndrade R, Egan M. The colors of emotion. Am Ethnolog 1974;
1(1):4963.
14. Ou L-C, Luo MR, Woodcock A, Wright A. A study of colour emotion
and colour preference. Part III: colour preference modeling. Color Res
Appl 2004;29(5).
15. Levy M, Sandier M, Casey M. Extraction of high-level musical structure from audio data and its application to thumbnail generation. Proceedings IEEE International Conference Acoust, Speech, Signal Process
(ICASSP), Vol. 5, Toulouse, France, May 2006. p 1316.
16. Kim JW, Kim HJ, Kim BM. Determination of usenet news groups by
fuzzy inference and neural network. Proceedings of the Korea Fuzzy
Logic and Intelligent Systems Society Conference, 2004.
17. Liu D, Zhang NY, Zhu HC. Form and mood recognition of Johann
Strausss waltz centos. Chin J Electron 2003;12(4):587593.
18. Scheirer ED. Music-Listening Systems, Ph.D. Thesis, MIT Media Lab,
2000.
19. Feng Y, Zhuang Y, Pan Y. Popular music retrieval by detecting mood.
Proceedings of the 26th International ACM SIGIR Conference on
Research and Development in Information Retrieval, Toronto, Canada,
2003. p 375376.
20. Li T, Ogihara M. Detecting emotion in music. Proc. ISMIR 2003,
2003.
21. Hevner K. Expression in music: a discussion of experimental studies
and theories. Psychol Rev 1935;42:186204.
22. Farnsworth PR. The Social Psychology of Music. New York: Dryden
Press; 1958.
23. Yang YH, Liu CC, Chen HH. Music Emotion Classification: A Fuzzy
Approach. Proceedings of ACM Multimedia 2006 (ACM MM06),
Santa Barbara, CA, USA, 2006. p 8184.
24. Yang YH, Su YF, Lin YC, Chen HH. Music emotion recognition: The
role of individuality. Proceedings of ACM International Workshop on
Human-centered Multimedia (ACM HCM), 2007. p 1321.
25. Yang YH, Lin YC, Su YF, Chen HH. A regression approach to music
emotion recognition. IEEE Trans Audio Speech Lang Process 2008;
16(2):448457.
26. Palmer SE, Schloss KB, Xu Z, Rrado-Leon LR. Music-color associations are mediated by emotion. Proc Natl Acad Sci 2013;110(22):8836
8841.
27. Moon CB, Kim HS, Lee HA, Kim BM. Analysis of relationships
between mood and color for different musical preferences. Color Res
Appl 2013.
28. Lartillot O. MIRtoolbox 1.2.4, Finnish Centre of Excellence in Interdisciplinary Music Research, March, 18th, 2010.
29. William Mendenhall M, Beaver RJ, Beaver BM. Introduction to Probability and Statistics. Independence, KY: Cengage Learning; 2008.
30. Croft B, Metzler D, Strohman T. Search Engine: Information Retrieval
in Practice. Upper Saddle River, NJ: Pearson Education, 2010.
31. Oracle, JAVA API. Available at: http://docs.oracle.com/javase/1.4.2/
docs/api/java/awt/Color.html. Last accessed 27 February 2013.

COLOR research and application