THE JOURNAL OF GENETIC PSYCHOLOGY

, VOL. , NO. , –

https://doi.org/./..

Integrating Dimensional and Discrete Theories of Emotions: A New

Set of Anger- and Fear-Eliciting Stimuli for Children
Simona Scaini a,b , Paola M. V. Rancoitac , Riccardo M. Martonid , Micol Omerob , Anna Ogliarib,d ,
and Chiara Brombinc
a
Department of Psychology, Sigmund Freud University, Milan, Italy; b Developmental Psychopathology Unit, Vita-Salute
San Raffaele University, Milan, Italy; c University Centre of Statistics in the Biomedical Sciences (CUSSB), Vita-Salute San
Raffaele University, Milan, Italy; d Department of Clinical Neuroscience, San Raffaele Scientific Institute, Milan, Italy

ABSTRACT ARTICLE HISTORY

The selection of appropriate stimuli for inducing specific emotional states Received  September 
has become one of the most challenging topics in psychological research. In Accepted  June 
the literature there is a lack of affective picture database specifically suited
KEYWORDS
to investigate emotional response in children. Here the authors present Affective visual stimuli;
the methodology that led us to create a new database (called Anger- and discrete emotions;
Fear-Eliciting Stimuli for Children) of affective stimuli inducing experiences dimensional theory of
of 3 target emotions (neutral, anger, and fear) to use in experimental session emotions; emotion
involving children. A total of 84 children were asked to (a) indicate the per- elicitation in children;
ceived emotion and its intensity and (b) rate the three affective dimensions of multivariate techniques
the Self-Assessment Manikin (SAM). Based on concordance between labeled
and expected target emotion, the authors decided to select 15 stimuli to be
included in Multivariate modeling techniques were applied to evaluate the
association between expected target emotion and SAM ratings. The authors
found that the hit rate for the neutral pictures was good (greater than 81%), for
fear-eliciting pictures it was greater than 64%, and for anger-eliciting pictures
it was moderate (between 45% and 56%). The study results reveal also an age
effect only in the arousal scale. However, the authors did not find significant
gender-related differences in SAM ratings.

In the last decades, a lot of studies have aimed at studying the mechanisms underlying emotional pro-
cesses and assessing the influence of emotions on cognitive processes. This goal may be accomplished by
efficiently modulating the environment to help understanding the vast spectrum of emotional reactions
experienced in affectively charged situations (Dan-Glauser & Scherer, 2011). In the majority of studies in
this field, assessment of emotional reactivity is performed immediately after the presentation of a salient
stimulus. For these reasons, the selection of appropriate emotional stimuli has become one of the most
challenging topics in psychological research.
In particular, the development of adequate stimuli able to trigger emotions in children is still at its
preliminary stages. One of the most common practices is to use static pictures. Among stimuli sets avail-
able in the current literature, we mention four affective pictures databases: the International Affective
Picture System (Lang, Bradley, & Cuthbert, 1997), the Geneva Affective Picture Database (Dan-Glauser
& Scherer, 2011), the Emotional Picture System (Wessa et al., 2010), and the Nencki Affective Picture
System (Marchewka, Żurawski, Jednoróg, & Grabowska, 2014).
However, to our knowledge, only some pictures from the International Affective Picture System (Lang,
Bradley, & Cuthbert, 2008) have been selected to be rated by children, and only a study published in

CONTACT Simona Scaini s.scaini@milano-sfu.it Department of Psychology, Sigmund Freud University, Ripa di Porta Ticinese
  Milano, Italy.
Supplemental data for this article can be accessed on the publisher’s website.
©  Taylor & Francis
254 S. SCAINI ET AL.

German tested a number of these pictures in children between 6 and 12 years old (Müller et al., 2004). In
addition, some limitations related to the use of this database have been pointed out (Colden, Bruder, &
Manstead, 2008; Dan-Glauser & Scherer, 2011; Grabowska et al., 2011; Mikels et al., 2005). For example,
an important limitation encountered from previous studies in this field (Mikels et al., 2005) concerns the
difficulty to differentiate representative numbers of stimuli that induce clear basic emotions. In addition,
to our knowledge there is need of database, which has been specifically realized for both clinical and
nonclinical childhood samples.
All these issues call for the realization of a new database to elicit basic emotions in experiments involv-
ing children. In our work, we focused specifically on two negative emotions (i.e., anger and fear), which
are connected with relatively specific deviant pattern of brain activation in children affected by internaliz-
ing symptoms such as anxiety (Battaglia et al., 2005; Battaglia et al., 2004; Battaglia et al., 2010; Simonian,
Beidel, Turner, Berkes, & Long, 2001). From recent studies (Garner, Baldwin, Bradley, & Mogg, 2009;
Kessler, Roth, von Wietersheim, Deighton, & Traue, 2007) it has emerged that recognition of emotions
in patients with anxiety disorders was significantly worse than that in controls, specifically, showing a ten-
dency to interpret nonanger emotion as anger (Kessler et al., 2007). The study of emotional dysfunctions
is noteworthy in children affected by anxiety since these deficits could lead to poor social functioning
which, in turn, is linked with psychopathology (Keltner & Kring, 1998).
In this study, 60 stimuli in total, 20 pictures for each target emotional category (e.g., anger, fear, neu-
tral) were selected and administered through an online survey.
We measure the agreement or concordance between elicited and expected emotions using the hit rate
of the expected target emotions (i.e., the percentage of raters declaring to have felt the target emotion).
Based on the obtained hit rates, we decided to select 15 stimuli to be included in the final set.
Multivariate and modeling statistical approaches have been then applied to collected data aiming
at integrating discrete and dimensional models of emotions using information on the Self-Assessment
Manikin (SAM; Lang, 1980) ratings.
Specifically, in the current work we:
1) design an online questionnaire to select stimuli for children based on discrete categorical theory;
2) create a new database of affective stimuli to use in experimental session involving children (called
the Anger- and Fear-Eliciting Stimuli for Children), emphasizing hit rate of target emotions; and
3) apply multivariate techniques on the selected pictures to evaluate picture distribution in a two
dimensional map, realized using affective ratings obtained through SAM.
Stimuli are available to the scientific community, for noncommercial use only, on request and the
request form is available at www.cussb.unisr.it/FIRB2012.

Method

Stimuli presentation and rating scales

Sixty visual stimuli in total, gathered through an extensive online web search, were selected as likely
to elicit 3 specific target emotional states (indicated in the following as “expected emotions”): neutral,
fear, and anger. Images have been selected by three experts in the field of developmental psychology
and psychopathology: a child psychiatrist, professor of developmental psychology and psychopathology
(Anna Ogliari), a clinical child psychologist, PhD in developmental psychopathology, tenured lecturer
in developmental psychology (Simona Scaini), a psychologist trainer in a child and adolescent mental
health service (Micol Omero).
Criteria adopted in choosing stimuli were the following:
1) the depicted objects had to be known by children,
2) the frightening stimuli had to be linked to childhood specific fears,
3) the anger stimuli had to represent life situations typical of childhood inducing anger, and
4) images had to comply with ethics recommendation and not cause harm in children.
All disagreements on eligibility were resolved by consensus. Actually 20 pictures were chosen for each
target emotion. These stimuli were presented through an online questionnaire. The questionnaire was
 THE JOURNAL OF GENETIC PSYCHOLOGY 255

developed, with the help of System Administrators of Vita-Salute San Raffaele University on a webpage
and a web link was then provided to allow online access. The link was promoted by word of mouth,
in schools and recreation centers, and on social networks, the university website, and the project web-
site www.cussb.unisr.it/FIRB2012. To inform parents of our research project, a brief description of the
research goals was provided in the webpage. Since attention span in children is limited, 40 of the 60 pic-
tures were randomly presented in each session. The rating task was proposed as a game based on a story
about a witch mismatching emotions in a village. The child was then involved in an emotion-labeling
task to help villagers to dissolve the spell. In the questionnaire, each picture was displayed for 4 s, and
after the picture left the screen, the rater was asked to do the following:
r indicate the perceived emotion, choosing from a list of six emotions (anger, fear, disgust, joy, sad-
ness, neutral);
r rate only pictures perceived as emotionally charged indicating the intensity of the felt emotion using
a 5-point Likert-type scale with responses ranging from 1 (minimal) to 5 (maximal intensity of the
emotion). To simplify the task, whenever a stimulus was not perceived as emotion inducing (e.g., it
was labeled as neutral), we decided to not ask to rate the level of that emotional experience.
r assess the three dimensions of pleasure, arousal, and dominance, using the SAM, a picture-oriented
instrument. Actually, we used a modified version of the original SAM, where manikins in each
dimension convey emotional contents ranging from negative to positive. In detail, the pleasure scale
(SAM1) varies from unpleasant to pleasant, the arousal scale (SAM2) ranges from excited to calm
and the last dimension, dominance (SAM3), ranges from out of control to in control. This reversion
has been motivated by the need to simplify the rating task for children. In fact, by showing manikins
that for each dimension convey emotional contents always ranging from negative to positive, the
rater can apply the same cognitive process to evaluate the three dimensions. As in the original SAM,
ratings were given in a 9-point scale for each dimension.
The primary goal of this online survey was to select a smaller sample of stimuli that consistently
elicit, at least at the subjective level, one of the target emotions. The selected stimuli could be used in
experimental studies, for example they could be presented while monitoring the physiological activation,
to assess both the subjective and the objective emotional responses to the presented visual stimuli. Due to
the limited attention span, the laboratory sessions with children will have short duration thus requiring
few stimuli. For this reason, we have decided to select 15 stimuli in total, five for each target emotion.

Participants
A total of 160 Italian children from the general population filled in the questionnaire. Out of these
84 children (52.5%, 35 girls and 49 boys, M age = 10.21 years, age range = 7–14 years) evaluated
more than 30 pictures (completed at least 75% of the questionnaire) and on average 55 ratings (range
40–68) were collected for each picture. Almost all the participants were Italian, most of them attend-
ing elementary school. A t test analysis showed no differences for age in boys and girls. Two age groups
have been defined to investigate emotion recognition issues in different ages: childhood (7–9 years old;
n = 33) versus preadolescence (10–14 years old; n = 51).
Moreover, an independent validation sample of 83 children from general population, comparable
with the previous sample (e.g., no significantly different in terms of raters’ age and gender) was recruited
to assess the robustness and stability of the subjective emotional response both in terms of hit-rate and
intensity level of the declared felt emotion.

Statistical analyses
Following the considerations provided in the Method section, 15 stimuli of the initial 60, character-
ized by the highest hit rate of the target emotion, were selected and examined. In particular, the sub-
jective ratings collected through the SAM associated with these pictures, were analyzed using different
multivariate techniques to evaluate relative position of stimuli on affective dimensions. In particular, to
highlight groupings of pictures, an agglomerative hierarchical cluster analysis was performed on dis-
tances obtained from the matrix containing for each picture (in rows, representing our cases) average
256 S. SCAINI ET AL.

SAM ratings (three columns [i.e., our variables]). In agglomerative methods, the procedure starts with
each observation being in its own separate cluster (i.e., at the beginning the number of clusters coin-
cides with the number of observations) and then proceeds to combine clusters until all observations
belong to a unique cluster. Euclidean distance was used to calculate dissimilarities and the average link-
age method was chosen, using the information provided by the cophenetic correlation index, to calculate
the distance between two clusters. The dendrogram was cut on the basis of two criteria. The agglomer-
ation schedule was examined to identify the (relative) change in agglomeration coefficient (i.e., larger
jumps in the agglomeration process). Moreover, the silhouette index, introduced by Rousseeuw (1987),
was used to determine the optimal number of clusters in the data (Kaufman & Rousseeuw, 2009). The
maximum value of the index is associated with the optimal number of clusters in the data. Instead of
traditional analysis of variance (ANOVA) analysis, latent class mixed models (LCMMs) were fitted to
assess whether pictures belonging to different emotional categories showed differences in SAM ratings,
by properly accounting for repeated measures issue arising from multiple evaluations of the same subject
(Proust-Lima, Philipps, & Liquet, 2017). The key advantage of this procedure is that different homoge-
neous groups of subjects with similar observed responses (patterns of affective ratings) may be identified.
The optimal number of latent classes was determined using the Bayesian Information Criterion fit index
(Schwarz, 1978), thus following mixture modeling approaches (Hawkins, Allen, & Stromberg, 2001). In
particular, target emotion was included as a discrete fixed effect in the linear mixed model, as random-
effect in the latent process mixed model and, when the number of latent classes was greater than 1, as
class-specific fixed effect in the latent process mixed model. Subject-specific random effects were also
considered into the model. We included age and sex as fixed effects in the linear mixed model or as
covariates in the class-membership multinomial logistic model. All the analyses were performed using
R Statistical Software version 3.2.1 (R Project for Statistical Computing, Vienna, Austria). In particular
the lcmm (Proust-Lima et al., 2017) and hclust (Murtagh, 1985) functions in the stats package and the
NbClust (Charrad, Ghazzali, Boiteau, Niknafs, & Charrad, 2014) package were used respectively to esti-
mate LCMMs, to perform agglomerative hierarchical cluster analysis and to choose the optimal number
of clusters.

Results and discussion

In this study we propose a new set of affective stimuli, specifically suited for children. We focused on
two negative emotions (i.e., anger and fear), which are connected with relatively specific deviant pattern
of brain activation in children affected by internalizing symptoms in childhood (Battaglia et al., 2005;
Battaglia et al., 2004; Battaglia et al., 2010; Simonian et al., 2001).
To achieve this goal, we developed a questionnaire where 60 stimuli were presented and assessed
by children. Based on the degree of agreement between self-report and expected emotion and previous
considerations on children attention span during experimental sessions, 15 of the 60 stimuli (five pictures
for each target emotion) were selected to be included in our database. We observed that, for the selected
neutral pictures, the hit rate was good (greater than 81%), for fear-eliciting pictures it was substantial
(greater than 64%), and for anger-eliciting pictures it was moderate (between 45% and 56%).
In the validation sample, the hit rate for these pictures was not significantly different (by using Fisher’s
exact test). These results suggest the stability of the labels assigned to the affective pictures and emphasize
the consistency of the emotional responses at least at the subjective level.
Moreover, with reference to the original sample of 84 children, to avoid a possible criticism for privi-
leging agreement between a priori labeled emotions and self-report emotions, we also evaluated whether
pictures, expected to elicit a specific emotional state, were indicated by raters as evoking another emo-
tion. We found that 18 of 60 pictures were labeled differently from expected emotion and in most of the
cases the selected emotion was not among the target emotional states. In particular, several mismatches
were observed for pictures expected to elicit anger and fear, which were indicated as inducing sadness
(12 of 18) or disgust (3 of 18). However, both disgust and sadness were not among target emotions of
interest, hence these pictures were excluded. Only two pictures expected to elicit respectively fear and
anger were assigned to the neutral category and the hit rate for these pictures was lower than the one
found for the selected neutral pictures included in the final database. Finally, one neutral picture was
 THE JOURNAL OF GENETIC PSYCHOLOGY 257

Table . Hit rate of expected target emotions and descriptive statistics for the entire sample.

Picture ID Content Hit rate Descriptive statistics Valence Arousal Dominance

M (SD) . (.) . (.) . (.)

.jpg kitchen sink / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg car parking / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg piece fabric / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg fork / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg light bulb / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg pulling hair / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg bullism scene / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg bullism scene / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg pulling hair / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg fighting shoppingcart / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg dracula / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg horror picture girl / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg lightning storm / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg bogeyman / (.%) Range – – –
Raters   
M (SD) . (.) . (.) . (.)
.jpg monster under bed / (.%) Range – – –
Raters   

assigned to joy category, another nontarget emotion. When considering information about gender and
age, for the selected stimuli, we did not find a significant difference in terms of hit rate between younger
vs. older children or girls vs. boys. The hit rate and the descriptive statistics on SAM dimensions (arousal,
valence, dominance) are reported in Table 1 for the overall sample. In Supplemental Tables S1 and S2,
the same statistics are reported stratified for sex and age.
As discussed in the Method section, children were also asked to rate the intensity of the felt emotion
using a 5-point Likert-type scale ranging from 1 (minimal intensity of the emotion) to 5 (maximal intensity
of the emotion) whenever they select a non-neutral label for describing the experienced emotion.
When examining the intensity level declared by the children who identified the target emotion, for
the selected anger-eliciting pictures the average was 3.26, with a median of 3, whereas for the fear elic-
iting pictures it was 3.63, with a median of 4. For the sake of completeness, in Table 2, for the selected
emotionally charged pictures, we report the proportion of correctly classified pictures (i.e., the declared
felt emotion was equal to the expected target emotion) with a low (rating ࣘ 2), medium (rating equal
258 S. SCAINI ET AL.

Table . Proportion of correctly classified pictures with a low (rating ࣘ ), medium (rating equal to ), and high (rating ࣙ ) declared
emotional intensity.

Picture ID propH propM propL

.jpg . . .

.jpg . . .
.jpg . . .
.jpg . . .
.jpg . . .
.jpg . . .
.jpg . . .
.jpg . . .
.jpg . . .
.jpg . . .

Note. This evaluation was done on the entire sample. propH = high emotional intensity; propL = low emotional intensity; propM =
medium emotional intensity.

to 3) and high (rating ࣙ 4) declared emotional intensity. This evaluation has been done on the entire
sample. We would like to highlight that, in the validation sample, the reported intensity levels for the
selected pictures did not significantly differ from that observed in the present study (by using Mann-
Whitney’s test): this result strengthens the idea that the subjective emotional responses remain constant
even changing the sample of raters. We found that the hit rate was lower for fear- and anger-eliciting
pictures than for neutral ones and the average intensity was medium or almost high. This result should
be interpreted considering well-known methodological difficulties in inducing anger and fear. Several
studies reported success in eliciting differential emotion self-reports for amusement, sadness, and a neu-
tral state, but had less success in eliciting anger, disgust, and fear (Gerrards-Hesse, Spies, & Hesse, 1994;
Gross & Levenson, 1995).
Anger is contingent on appraisals of extreme unpleasantness, high effort, high certainty, and strong
human agency (Smith & Ellsworth, 1985; Smith & Ellsworth, 1987) or, more simply, is appraised as
involving a demeaning offense against the self (Lazarus, 1991). Such prerequisite conditions are difficult
to achieve with the passive and essentially effortless viewing of static images. In the recent work by Mikels
et al. (2005), of the 510 pictures used in the study, about 72% visual stimuli were matched to specific
basic emotions, while the other pictures were classified as blended, meaning that they eliciting similar
amounts of four, five, or six basic emotions. Moreover, only two were classified as anger eliciting and
11 were classified as fear eliciting, thus highlighting the difficulty in eliciting these negative and strong
emotions using simple static pictures even when considering a larger affective picture database.
Affective ratings collected for the 15 stimuli were then analyzed using multivariate techniques to eval-
uate picture perception and distribution in a lower dimensional space. Analyses were carried out on
the entire sample and then separately for boys and girls and for two groups of ages (7–9 years old and
10–14 years old). Pleasure-arousal scatterplots in Figures 1 and Supplemental Figures S1 and S2, and
cluster analysis, carried out on all the three SAM dimensions, highlight two main groups of pictures:
emotionally charged pictures and neutral pictures. In particular, fear- and anger-eliciting pictures are
characterized by lower levels of valence (low pleasure) and medium values in the arousal axis (low val-
ues in our reversed arousal dimension correspond to higher activation), whereas neutral pictures are
placed apart in the quadrant characterized by higher levels of valence (high pleasure) and higher values
of arousal (i.e., low activation). It seems that, stratifying by age, the distinction between neutral and non-
neutral pictures is more evident in younger children (see Supplemental Figure S2). In the hierarchical
cluster analysis, calculated on SAM average ratings, the analyses of agglomeration process and the sil-
houette index suggested the same results: neutral pictures and non-neutral emotionally charged pictures
are assigned to two different clusters (Figure 1). The same results hold also when considering analyses
stratified by gender or age. LCMMs were then fitted on each affective dimension. These models were
estimated for a number of latent classes varying from 1 to 4, to have an adequate sample size in each class
thus allowing for accurate parameter estimates. For SAM1 (valence dimension), we found that age and
sex had no significant effects in modulating the subjective response and that the model with only one
 THE JOURNAL OF GENETIC PSYCHOLOGY 259

Figure . Mean pleasure and arousal ratings of the  AFES-C pictures displayed in a scatterplot and results of cluster analysis, for all the
children. We recall that the arousal dimension in our SAM system should be interpreted in a reverse way: lower values indicate greater
activation. Analysis of agglomeration process and silhouette index suggested  main clusters. In the dendrogram, each picture is labelled
with its ID. Neutral pictures are denoted by N, anger-eliciting pictures are indicated by the letter A, and finally letter F is associated to
fear-inducing pictures.

latent class, including expected target emotion as fixed covariate, was the best in terms of model fit. Fear-
and anger-eliciting pictures were negatively associated with valence, significantly decreasing SAM1 rat-
ings (i.e., lowering pleasure) with respect to neutral pictures. For SAM2 (arousal dimension), again the
BIC favored one latent class model. Fear- and anger-eliciting pictures were negatively associated with the
SAM2 dimension, thus corresponding, in our reversed SAM system, in increasing arousal levels. More-
over, an overall significant age effect was found: older children tend to attribute higher SAM2 ratings
(lower arousal).
As for SAM1, for SAM3 (dominance dimension) we found only a significant effect of expected target
emotion with fear- and anger-eliciting pictures significantly decreasing SAM3 ratings, thus suggesting
that emotionally charged pictures reduce the subject’s feelings of control, with respect to neutral pictures.
One hypothesis for this pattern of results is that an increase of the age could be associated with a
reduction in arousal activation provided by negative emotions. The same effect was not found for domi-
nance. However, a previous research study suggested that SAM reflects the subject’s feelings of control in
the situation rather than an index of the pictured object (Bradley & Lang, 1994). In addition, the affective
dimension of dominance is typically regarded as less informative compared to the classic dimensions of
valence and arousal (Miccoli et al., 2014). This is particularly true in our set of data in which dominance
mean values are all above five. For the purpose of integrating dimensional and discrete theory of emotion,
the same models were fitted considering only the ratings provided by those children correctly declaring
the target emotions, and we obtained similar results.
To conclude, the validation of a small number of stimuli could be considered as a weak point of the
present research study.
However, as mentioned in the Method section, this choice was led by the idea to use stimuli that
consistently elicit, at least at the subjective level, one of the target emotions, in experimental settings
involving children. Due to limited attention span, the laboratory sessions with children will be short thus
requiring few stimuli. For this reason we have selected 15 stimuli in total, five for each target emotion. We
are aware that other studies (e.g., those relying/based on longitudinal design) may require more stimuli.

Funding
This research has been funded by the FIRB Project RBFR12VHR7 entitled / Interpreting emotions: a computational tool
integrating facial expressions and biosignals based on shape analysis and Bayesian networks” (Dr. Brombin recipient) and
by a NARSAD Young Investigator Grant from the Brain & Behavior Research Foundation, grant number: 23553, entitled
“Integrating multimodal affective information for emotion recognition in children with symptoms of anxiety disorders”
(Dr. Scaini recipient).
260 S. SCAINI ET AL.

ORCID
Simona Scaini http://orcid.org/0000-0002-5978-7165

References
Battaglia, M., Ogliari, A., Zanoni, A., Citterio, A., Pozzoli, U., Giorda, R., … Marino, C. (2005). Influence of the sero-
tonin transporter promoter gene and shyness on children’s cerebral responses to facial expressions. Archives of General
Psychiatry, 62, 85–94. doi:10.1001/archpsyc.62.1.85
Battaglia, M., Ogliari, A., Zanoni, A., Villa, F., Citterio, A., Binaghi, F., … Maffei, C. (2004). Children’s discrimination of
expressions of emotions: Relationship with indices of social anxiety and shyness. Journal of the American Academy of
Child and Adolescent Psychiatry, 43, 358–365.
Battaglia, M., Zanoni, A., Ogliari, A., Crevani, F., Falzone, L., Bertoletti, E., & Di Serio, C. (2010). Identification of gradually
changing emotional expressions in schoolchildren: The influence of the type of stimuli and of specific symptoms of
anxiety. Cognition and Emotion, 24, 1070–1079. https://doi.org/10.1080/02699930903112761-
Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal
of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59.
Charrad, M., Ghazzali, N., Boiteau, V., Niknafs, A., & Charrad, M. M. (2014). Package ‘NbClust’. Journal of Statistical
Software, 61, 1–36.
Colden, A., Bruder, M., & Manstead, A. S. (2008). Human content in affect-inducing stimuli: A secondary analysis of the
international affective picture system. Motivation and Emotion, 32, 260–269.
Dan-Glauser, E. S., & Scherer, K. R. (2011). The Geneva affective picture database (GAPED): A new 730-picture database
focusing on valence and normative significance. Behavior Research Methods, 43, 468.
Garner, M., Baldwin, D. S., Bradley, B. P., & Mogg, K. (2009). Impaired identification of fearful faces in generalised social
phobia. Journal of Affective Disorders, 115, 460–465. doi:10.1016/j.jad.2008.10.020; 10.1016/j.jad.2008.10.020
Gerrards-Hesse, A., Spies, K., & Hesse, F. W. (1994). Experimental inductions of emotional states and their effectiveness:
A review. British Journal of Psychology, 85, 55–78.
Grabowska, A., Marchewka, A., Seniów, J., Polanowska, K., Jednoróg, K., Królicki, L., … Członkowska, A. (2011). Emo-
tionally negative stimuli can overcome attentional deficits in patients with visuo-spatial hemineglect. Neuropsychologia,
49, 3327–3337.
Gross, J. J., & Levenson, R. W. (1995). Emotion elicitation using films. Cognition & Emotion, 9, 87–108.
Hawkins, D. S., Allen, D. M., & Stromberg, A. J. (2001). Determining the number of components in mixtures of linear
models. Computational Statistics & Data Analysis, 38, 15–48.
Kaufman, L., & Rousseeuw, P. J. (2009). Finding groups in data: An introduction to cluster analysis. New York, NY: Wiley.
Keltner, D., & Kring, A. M. (1998). Emotion, social function, and psychopathology. Review of General Psychology, 2,
320–342.
Kessler, H., Roth, J., von Wietersheim, J., Deighton, R. M., & Traue, H. C. (2007). Emotion recognition patterns in patients
with panic disorder. Depression and Anxiety, 24, 223–226.
Lang, P. J. (1980). Behavioral treatment and bio-behavioral assessment: Computer applications. In J. Sidowski, J. H. Johnson,
& T. A. Williams (Eds.), Technology in mental health care delivery systems (pp. 119–137). Norwood, NJ: Ablex.
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1997). International affective picture system (IAPS): Techni-
cal manual and affective ratings. NIMH Center for the Study of Emotion and Attention. Retrieved from
https://www2.unifesp.br/dpsicobio/adap/instructions.pdf
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures
and instruction manual. Technical Report A-8. Gainesville, FL: University of Florida.
Lazarus, R. S. (1991). Progress on a cognitive-motivational-relational theory of emotion. American Psychologist, 46,
819–834.
Marchewka, A., Żurawski, Ł., Jednoróg, K., & Grabowska, A. (2014). The nencki affective picture system (NAPS): Intro-
duction to a novel, standardized, wide-range, high-quality, realistic picture database. Behavior Research Methods, 46,
596–610.
Miccoli, L., Delgado, R., Rodríguez-Ruiz, S., Guerra, P., García-Mármol, E., & Fernández-Santaella, M. C. (2014). Meet
OLAF, a good friend of the IAPS! the open library of affective foods: A tool to investigate the emotional impact of food
in adolescents. PLoS One, 9, e114515.
Mikels, J. A., Fredrickson, B. L., Larkin, G. R., Lindberg, C. M., Maglio, S. J., & Reuter-Lorenz, P. A. (2005). Emotional
category data on images from the international affective picture system. Behavior Research Methods, 37, 626–630.
Müller, B., Winter, B., Schürkens, A., Herpertz-Dahlmann, B., and Herpertz, S. (2005). Rating and standardisation of pic-
tures from the International Affective Picture System appropriated for children. Examination of children in a field
sample aged six to twelve years,” Zeitschrift fur Kinder-und Jugendpsychiatrie und Psychotherapie, 32(4), 235–243.
Murtagh, F. (1985). Multidimensional clustering algorithms. Compstat lectures. Vienna, Austria: Physika Verlag.
Proust-Lima, C., Philipps, V., & Liquet, B. (2017). Estimation of extended mixed models using latent classes and latent
processes: The R package lcmm. Journal of Statistical Software, 78(2), 1–56.
 THE JOURNAL OF GENETIC PSYCHOLOGY 261

Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Com-
putational and Applied Mathematics, 20, 53–65.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
Simonian, S. J., Beidel, D. C., Turner, S. M., Berkes, J. L., & Long, J. H. (2001). Recognition of facial affect by children and
adolescents diagnosed with social phobia. Child Psychiatry and Human Development, 32, 137–145.
Smith, C. A., & Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychol-
ogy, 48, 813–838.
Smith, C. A., & Ellsworth, P. C. (1987). Patterns of appraisal and emotion related to taking an exam. Journal of Personality
and Social Psychology, 52, 475–488.
Wessa, M., Kanske, P., Neumeister, P., Bode, K., Heissler, J., & Schönfelder, S. (2010). EmoPicS: Subjective and psychophys-
iological evaluation of new imagery for clinical biopsychological research. Zeitschrift für Klinische Psychologie und
Psychotherapie, 1(11 Suppl), 1–77.
Copyright of Journal of Genetic Psychology is the property of Taylor & Francis Ltd and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.