Teaching Elija, a computational

model of an infant, to pronounce

his first words in three languages
Piers Messum and Ian Howard
Pronunciation Science Ltd and University of Plymouth
p.messum@pronsci.com
BAAP - Oxford, April 2014
2
Learning to pronounce and
learning the pronunciation of a word
horripilation
h r p len
Sensory
1
Motor
1
Sensory
2
Motor
2
Sensory
3
Motor
3
Horizontal
Vertical
Heyes -
Associative Sequence
Learning (ASL)
1
2
3
4 5
6
Acoustic matching
3
Solving the correspondence problem #1
!
1
2
3
4
5
Mirrored equivalence
4
Solving the correspondence problem #2
5
In this model:
The child discovers the actions (VMSs) for himself
The judgment of equivalence (not necessarily based on
similarity) is made by the expert (adult) not the novice (child)
No issue of normalisation arises
The fundamental association is
between action-and-sound, not
sound-and-sound.
This pushmi-pullyu
representation (Millikan 1996)
has a motor head and a
perceptual head.
(Cf. mirror neurones.)
Speech is neither gestures made audible nor an acoustic
code; the underlying, neural representation of speech is
inherently perceptuo-motor
Elija
No a priori articulatory or perceptual
knowledge of speech

Vocal tract: Maeda articulatory
synthesizer, scaled down
Motor system: modelled as a gestural
score
Sensory systems: touch and hearing
Reward system: for touch, various
acoustic effects (loudness, high and low
frequency energy, etc), diversity
Associative systems and memories:
for actions, rewards and sensory inputs
DTW (dynamic time warping) for
parsing speech

Howard and Messum (2011) Motor
Control
6
3 stage operation -

1.Babbling/sound discovery:
self supervised, 917 motor patterns
(vocal motor schemes)

2.Imitative interaction:
caregivers respond naturally to Elija,
4 x English, 2 x French and 2 x
German caregivers

3.Word learning, with Elija
imitating a caregiver:
Serial imitation, using speech sound
to motor pattern correspondences
learnt previously

Howard and Messum (2014) in
preparation
Word imitations accepted by caregivers
m = 55