From Inner Speech to Dialogic Semiosis:

A Semiotic Approach to Audiovisual

Multimedia Communication
Ulla Oksanen, Helsinki

Abstract
In the continuum of audiovisual narration from the early days of cinema until
today there has at times emerged the intention to study in depth this language
of images and sounds and to develop it radically. The inquiry into
these kinds of meanings, and the dynamics connected to them, has in particular
been approached from the semiotic point of view. The attempt to
comprehend and construct audiovisual narration as inner speech and dialogism
offers an attractive frame of reference for perceiving the nature of
communication in modern multimedia interaction as semiosis, the action of
signs. In this article the change in audiovisual narration, from the semiotic
point of view, can be reduced to the following tendencies: from monologism
to dialogism, from diachronic (successive) to synchronic (simultaneous) and
from syntagmatic (co-ordinative) to paradigmatic (associative). These tendencies
are exemplified by the views of Lev Vygotsky, Mikhail Bakhtin,
Sergei Eisenstein and Jean-Luc Godard.
In my argument C.S. Peirces concept of communicational dialogism
will be of particular importance. It is a central concept in audiovisual narration
as well as in hyper- and multimedial interactive communication, and in
the dynamic process of the function of signs, dialogic semiosis, it is a fundamental
dimension.
Keywords: audiovisual; multimedia; dialogism; synchrony; paradigm; semiotics;
dialogic semiosis.
194 Ulla Oksanen
1 INTRODUCTION
From the birth of audiovisual narration until today, there have
been attempts to reach reality or produce illusions of it through
real-like phenomena or more or less immersive experiences. This
kind of transition from magic lantern stories to multimedia1 can
also be traced as a continuum of audiovisual narration, where at
times other kinds of aspirations have also emerged, namely the
intentions of studying in depth the meaning of this language of
images and sounds and to develop it radically. The inquiry into
these kinds of meanings, and the dynamics connected to them, has
in particular been approached from the semiotic point of view.
The attempt to comprehend and construct audiovisual narration as
inner speech and dialogism offers an attractive frame of reference
for perceiving the nature of communication in modern multimedia
interaction as semiosis, the action of signs.
The top level of the multidimensional model by Tella &
Mononen-Aaltonen (2000), introduced in this volume, consists of
the background flows of communication and mediation, whose influences
are reflected to all levels of the model. These flows are
important in audiovisual communication as well. Along with
them, however, there are other unique flows of changes, which,
from the semiotic point of view, can be channelled to the following
tendencies: from monologism to dialogism, from diachronic
(successive) to synchronic (simultaneous) and from syntagmatic
(co-ordinative) to paradigmatic (associative). In this article these
tendencies are illustrated with the views of Lev Vygotsky, Mikhail
Bakhtin, Sergei Eisenstein and Jean-Luc Godard.
1 Hypermedia is here defined as an interactive manner of expression based on links. It may
consist of texts, images, moving images and sounds. By multimedia I mean digitally worked
up interactive programs, which may consist of texts, images, moving images and sounds and
which are distributed as CD-ROMs, DVDs or network products.
From Inner Speech to Dialogic Semiosis 195
In my argument, C.S. Peirces concept of communicational
dialogism in the communication background flow is of particular
importance. Communicational dialogism is a central concept in
audiovisual narration as well as in hyper- and multimedial1 interactive
communication. In the dynamic process of the function of
signs, dialogic semiosis, it is a fundamental dimension. Even the
context and the medium itself are always essentially present in this
semiotic process and appear as a cultural interaction, intermediation
(Lehtonen 1998) or to use Tella & Mononen-Aaltonens term,
as intermediality.
This article will now concentrate on certain instances of
change flows, which, together, seem to characterise the transition
towards hypermedial narration. The chapter Inner Speech and
Eisensteins Idea of Montage will deal with the concept of dialogism
as well as with its obvious counter poles: Eisensteins inner
monologue and Vygotskys inner speech. In the chapter From
Monologue to Dialogue: Bakhtins Inner Dialogism, dialogism
will be described in its Bakhtinian meaning. The chapter From
Diachrony to Synchrony, and Syntagm to Paradigm will examine
the change towards multimedial narration on these axes using the
ideas of Eisensteins and Godards cinematic theory. Finally, in
the chapter Total Mental Image and Dialogic Semiosis communicational
dialogism will be defined as a fundamental dimension
of dialogic semiosis in the light of total mental image and Peirces
pragmatic idea of signs.
2 INNER SPEECH AND EISENSTEINS IDEA OF
MONTAGE
At the beginning of the 1920s, Sergei Eisenstein, film theorist and
director, was fascinated by the attractions from the early age of
cinema, by the principles of showing and making perceptible. He
196 Ulla Oksanen
started to emphasise what he saw as the fundamental essence of
the appeal and impressiveness of the cinema, indeed all the arts,
namely the principle of conflict. In the intersection of nature and
industry there is art (Eisenstein 1978, 106). On the basis of this
kind of Hegelian thinking, the world is seen as a continuous development
of two conflicting opposites in interaction, in other
words, as a continuum of thesis, antithesis and synthesis. Eisenstein
even compared montage cutting to an efficient combustion
engine in a car, where the attraction shots, defined by cutting,
form a series of explosions, whose dynamic energy runs the
caror in his case the film. The same principle of conflict was
visualised, said Eisenstein, in Japanese pictography, where two
separate signs are summed up and explode into abstract ideas.
eye + water = to cry
mouth + bird = to sing
door + ear = to hear (Eisenstein 1978, 108)
Likewise, in Eisensteins thinking when two still pictures are
placed next to each other, they produce a new idea: ... because
successive things are not actually placed adjacent to each other,
but one upon another (Eisenstein 1978, 109). According to Juri
Lotman (1989, 67), Eisensteins montage, the so-called intellectual
montage, activates the boundary of thought and changes it
into a principal medium of meaning (e.g. Kerensky mounting the
stairs in the film October).
In 1927 a well-known formalist Boris Eikhenbaum presented
a view in which he emphasised the close connection of the language
of the cinema and the so-called inner speech. In his opinion,
montage, as well as common speech, consists of fragmentary
expressions rather than complicated and logical sentences. Many
scholars (among others Wollen 1977; Broms 1984; Hietala 1994)
see the decisive role of Eikhenbaum in the development of Eisenstein
s theory of montage and the inner monologue. However, the
From Inner Speech to Dialogic Semiosis 197
concept of inner speech has been analysed more profoundly by
Lev Vygotsky in his writings (1920) and later in his book
Thought and Language (1934). One of the basic principles of
Vygotsky in this work is that the genetic roots of language and
speech are separate (Vygotski 1982, 99). The decisive meaning in
the development of thought is in inner or silent speech, by which
Vygotsky refers to the junction where thinking is transformed into
the lingual, and language into thought. According to Vygotsky inner
speech is primarily thinking in terms of pure meanings, or it
functions on the semiotic level of speech. In other words, what in
Vygotskys ideas is expressed in language mainly as diachronic,
successive, appears in thought as synchronic, simultaneous. (Vygotski
1982, 243244)
The inner speech, hovering between word and thought,
also has according to Vygotsky its characteristic features. Inner
speech, by contrast with outer speech, essentially reflects a clearly
different, new and independent function of speech: poetry-like silent
speech to oneself. Inner speech is also simplified and compressed
as it opens up with difficulty to others and is hardly intelligible
without context. It consists of apparent fragmentariness,
which makes it elliptic, including open gaps. So inner speech
deviates by its syntax from written speech by being predicative
and often even idiomatic, like a dialect. (Vygotski 1982, 46-47,
178179, 230244)
After 1928 Eisenstein started to move away from strict antagonistic
dialectics and emphasised montage as an emerging
synthesis of the different senses. After familiarising himself with
James Joyces novel Ulysses, Eisensteins thinking began to
move towards a more organic, softer view of cinema, in which,
instead of conflicts, he started to stress harmony and holism.
(Malmberg 1974, 34) The filmic inner speech or inner monologue
now became the cornerstone of Eisensteins new idea of
cinema. This kind of monologue, which as a structure forms a
198 Ulla Oksanen
stream-of-consciousness-like reconstruction of the thinking process,
was a continuation of the earlier mentioned idea of intellectual
montage from the beginning of the 1920s. Though Eisensteinean
inner cinematic speech, the inner monologue, is fundamentally
thinking with pure meanings, the nature of it is described
as flexible, pictorial, non-logical and mythic. Even though
it is reminiscent of language, it uses, not only natural language as
its material, but also pictures, sounds and writing. (Willemen
1983, 155; cited in Hietala 1994, 49) Eisenstein, too, was convinced
of the fact that inner speech was, unlike outer verbal
speech, closer to sense- and image-based thinking (Wollen 1977,
29). As Eisensteins view of montage approached stream-ofconsciousness-
like speech, he also started to emphasise counterpoint
or polyphony (a term he was to use later) as a central
method and goal in cinema. Eisenstein compares harmonic polyphony
to a developing consciousness, which while creating contact
between separate phenomena of reality, experiences everything
as a simultaneous great unity (Jalander 1990, 79). On the
basis of psychoanalytic theory this kind of inner monologue can
also be understood as a meeting and negotiation point or intersection
of textual surfaces.
3 FROM MONOLOGUE TO DIALOGUE:
BAKHTINS INNER DIALOGISM
Unlike Vygotsky, his contemporary Mikhail Bakhtin considered
the nature of the whole of human consciousness as dialogic. A
word, a language or a whole culture can become dialogised. Because
of this it is at the same time an orientation of any living discourse
(Holqvist 1990, 427429). According to Bakhtin, the literature
of Dostoevsky broke the form of the established linear and
monologic novel in an epoch-making manner. Whereas Goethe
From Inner Speech to Dialogic Semiosis 199
strove to see the consciousnesses of the same person as successive
(diachronic), Dostoevskys work was characterised by a tendency
to describe those phases of consciousness as simultaneous (synchronic)
and parallel. Different persons seem to meet each other
in a spatial dimension. As the events are mainly placed in the
present, or as if in the eternity, his novels lack causality, allusions
to the past, the environment and education. The fact that
Dostoevsky viewed life as an interaction, as voices that sing in
different tones of the same theme, Bakhtin regarded this as polyphonic
and at same time thoroughly dialogic. Bakhtin also described
this kind of dialogue as an almost universal phenomenon
that covered all human speech and relations and all expressions of
human life, indeed, everything that matters. No idea, he said, is
simply able to survive in the isolated consciousness of a human
being, ...if it remains there only, it degenerates and dies (Bahtin
1991, 5171, 132; translated by C. Emerson). In other words, only
by bringing ideas into connection with other consciousnesses and
strange ideas can thoughts live and give birth to new ideas.
In Bakhtins views dialogisation can also occur in the human
consciousness in an internal monologue. This he calls inner dialogue2,
(microdialogue in the revised edition in 1963), a dialogue
with oneself. Dialogue can also be part of the whole structure
of words, and intervene at all semantic and expressive layers.
Dialogue penetrates into every single word and causes a struggle
and resonance between the voices (Bahtin 1991, 115, 279,
392397). The addresser, the author, and the addressee, the reader,
2 Unlike Bakhtin, Vygotsky considered the written language and inner speech as the monologic
form of language, and only oral speech was dialogic. Vygotsky saw dialogism as a
chain of reactions that presupposes that partners know the essence of the matter and whose
looks, gestures and tones add a zest to the conversation. (Vygotski 1982, 235) However, a
closer study of Vygotskys writings on inner speech suggests, according to Wertsch, that
Vygotsky in effect saw the dialogic (in Bakhtian terms) nature of speech. A more appropriate
term for inner speech would thus be inner dialogue. (Wertsch 1980, 151152; cited in Tella
& Mononen-Aaltonen 1998, 25) Accordingly, on the same grounds, Eisensteinean inner
monologue could also be interpreted as inner dialogue.
200 Ulla Oksanen
are both part of the dialogue. According to Sderbergh Widding
(1997, 14) the spectator of cinema is also an active partner in the
cinematic experience, because he or she has the decisive role as an
aesthetic subject, as an interpreter of meanings. Answering to the
impulses produced by the film (or novel) the addressee ultimately
creates the true essence of the film (or novel) and in a fundamental
way gives birth to his or her subjective experience.
Bakhtin understands the thoughts and voices of the parties to
the dialogue as a unity of intersecting levels, as a polyphonic system
that creates harmony and resonance or produces dissonances
(Bahtin 1991, 358). Similarly, Kristeva sees texts as multidimensional
intersections that endlessly bear traces of other texts. This
intersection is at the same time the dialogue among several writings,
where the author, the addressee and the characters, as well as
the present and the past cultural context meet (Kristeva 1993,
2226). So to Kristeva the Bakhtinian dialogism also means intertextuality,
the mutual communication and quotation which comprises
language itself. In the age of the Net these kinds of ideas,
from the cognitive semiotics viewpoint, can also be considered a
polysemic network of signification, where the signification itself is
a wholeness composed of several links and junctions, a synthesis
of meanings3. It seems that in the debate concerning the nature of
network-based media and multimedia narration, these kinds of
views of the network character of meaning will increase in the
future.
3 A polysemic network of meanings refers to the theory of network-based meaning and cognitive
semiotics (Norvig & Lakoff 1987, 197198; cited in Yl-Kotola 1998, 57).
From Inner Speech to Dialogic Semiosis 201
4 FROM DIACHRONY TO SYNCHRONY,
AND SYNTAGM TO PARADIGM
Inner speech or dialogue proceeding in the Eisensteinean streamof-
consciousness way, will in Broms ideas above all be manifested
as emotional and logical narration. This is, in his view,
characterised by repetition of lingual and cinematic expression,
psychic drumming. This kind of repetition refers, according to
Broms to mythic oriental consciousness, which, unlike its
counterpole, natural-science-like western consciousness, is
characterised by synchrony, a tendency to see the world as simultaneous
and repetitive. It also means a tendency to see the world
as devoid of linear continuity, a place where causality is abolished
and a new sense of time created: the overlapping of its levels and
cyclicality will gain power. (Broms 1984, 6775, 133) Juri Lotman
also views the form of Eisensteinean montage as a system of
jump-like transitions, which as such foregrounds rather the
structure of life than the logic of incidents. Rhythmic repetition
lessens the visible and emphasises the abstract, logical or associative
meanings. (Lotman 1989, 51, 69) Broms (1984, 146, 169),
referring to Lotman and Russian semioticians, also sees the
change of scientific paradigm in the long run leading towards unlinear
ways of thinking, to oriental consciousness, nocturnal
knowledge, where a deep-reaching change of course towards
pictorial language and symbolic knowledge has begun. Yl-
Kotola, too, holds that as an alternative to linearity the time exerience
can be converted into the fragmentary, the curvilinear and
the cyclic (e.g. Eisensteinean repetition). He accordingly suggests
that the fundamental questions of unlinearity be approached
rather by means of the concept of paradigm. Eisensteins experiments
at the beginning of 1920 (e.g. the God sequence and the
stair sequence in October which were based on repetition)
could, in his opinion, be considered early attempts to break away
202 Ulla Oksanen
from syntagmatic4, process-like thinking (e.g. classic cinematic
continuity) and direct itself towards paradigmatic, associative narration,
and towards structure. (Yl-Kotola 1998, 249)
The tendency towards a new kind of associative and paradigmatic
thinking did not begin to take its audiovisual form on a
large scale until the time of alternative or counter cinema and the
music videoand later with hyper and multimedia narration experiments.
Yl-Kotola suggests that especially in films directed by
Jean-Luc Godard after 1980 radical signs of change can be distinguished.
He considers the Histoire(s) du Cinma (produced in
the 1990s) in particular to be a kind of audiovisual hybrid and a
representative of a transition period from the traditional audiovisual
narration to hypermedial narration. Yl-Kotola, unlike Baudrillard,
considers the fragmentariness of texts (meaning widely
any significant element in culture) to be a key factor in interactiveness,
as it makes these texts suit the information network. The
abundance and richness of references in Godards texts, he argues,
function as built-in-encouragement to associative navigation.
(Yl-Kotola 1998, 214218, 281282) In contrast with classical
cinema in its temporal and spatial closeness and clear causal relations,
it is a typical tendency in Godards5 thinking to offer the
syntagmatic continuity to paradigmatic fragmentariness and associativeness
(Yl-Kotola 1998, 90).
Fragmentariness and interactivity have appeared as conscious
targets among other arts, too. In theatre for example, the tradi-
4 Syntagm can be defined as a combination of signs that fill a certain space: e.g. shoes, trousers,
jacket, cap... (Tarasti 1990, 17). A syntagm consists of all the elements that are placed
adjacently to each other (in bothand relation), elements that are present in a picture
(Kuusamo 1990, 49). Paradigm is defined as an associative, alternative relation (Tarasti
1990, 17) that combines the present sign with the store of other possible signs (an either
or relation). It characterises absent relations (Kuusamo 1990, 49), e.g. cap could be
substituted by following (absent) alternatives: fur cap, brimmed hat, swimming cap, etc.
5 In the fragmentary style of Godard this is expressed by following characteristics: the narration
is formed of jump cuts, monologues spoken to the camera, interviews, intertextual
loans, scratching, and radical separation of image and sound. Time is described in motion,
it is manipulated e.g. by slowing down. (Yl-Kotola 1998, 224)
From Inner Speech to Dialogic Semiosis 203
tional Brechtian methods of alienation were directed towards
pedagogic goals. Similarly, in the 1970s Gorin anticipated the
cognitive and constructivist phase, and rejected the idea of behaviourist
forced feeding in cinema. In his view a narration that
makes the spectator think for himself and offers versatile constituents
for this is progressive. (Yl-Kotola 2000) Also Eisenstein,
especially in his early attraction montages of the 1920s, developed
the theory and forms of fragmentary narration. Parallel to
Eisenstein, Godard also emphasised the activity of the spectator in
the cinematic experience. To Godard the blank association spaces
between the fragments (pictures, texts and other hints) have a particular
stimulating effect on thinking, helping the spectator to deal
with the film and to construct the whole. These kinds of intentions,
which connect together all these experiments viewing the
fragmentariness of texts as a new dimension in audiovisual narration,
mean at the same time an endeavour to see their connection
on the mental level, as dialogic. Yl-Kotola, too, refers to the dialogic
(or polylogic) nature of the spectators experience when he
emphasises the text itself, i.e. the film, as a place of study and selfreflection
where the film and the spectator on the basis of his or
her own consciousness and outlook on life converse (Yl-
Kotola 1998, 214215). When the spectators experience in this
way offers the viewer a possibility to observe critically his own
attitudes to the films and ultimately empowers the spectator to
choose, Yl-Kotola considers this kind of dialogism to be ethically
justified. Godard, as well as other counter cinema advocates, see
classical Hollywood films as narcotic, like a veil of fog that
often succeeds in immersing the spectator into the story, but reduces
the interaction to an automatic response. According to Bakhtin,
the essence of dialogic relation is integrally tied to the concept
of otherness. When a thought in a monologic world is either
approved or denied, in a true dialogic and conversational re204
Ulla Oksanen
lationship it also becomes possible to understand other kinds of
views.
5 TOTAL MENTAL IMAGE AND DIALOGIC SEMIOSIS
If an audiovisual experience is viewed as a multimedia interactive
wholeness, worked out by a subject, a spectator, reader, listener,
interpreter, writer, from fragmentary images, texts and sounds, it
is, according to constructivist ideas, revived only after the synthesis
which is produced by the subjects earlier experience and action.
In the dynamic interactive dialogue/polylogue of the texts
and the mental structures of the subject, the experience of the
subject is referred to as a mental image. Astrid Sderbergh Widding,
while actually engaged in studying the mental image in
Eisensteins cinematic ideas, comes near the fundamental questions
of the multimedia narration of today. She sees, like Eisenstein,
that the basis for cinematic experience is formed by fragments
which have presented themselves to be modified by the
consciousness of the subject and which offer a possibility of
building a total mental image on account of what is seen and experienced.
In this action the user-subject (or spectator) constructs
a total mental image or experience6 using his or her existing mental
(cognitive) structures. The user-subject thus becomes an active
partner in the dynamic drama, where he or she answers the impulses
of the message and fundamentally gives birth in a dialogic
relationship to his or her own mental image, and his or her own
experience. When the message in the constructivist process is
viewed as if it extended outside its own subject matter, it can
6 Sderbergh Widding does not define the concept of total mental image, but speculates at
the end of the article on the possibility of expanding the concept to mean cinematic experience
in totality. It is possible to assume that by total mental image she means implicitly
cinematic experience.
From Inner Speech to Dialogic Semiosis 205
never be considered final but instead indeterminate and open, always
capable of being converted into a new (total) mental image.
(Sderbergh Widding 1997, 11, 14) A multimedia message, too, is
always the result of a construction process where the subject on
both the physical and the mental level builds a synthesis out of
fragments. This kind of multimedial experience is also a labile
construction, which like a cinematic message is born and changed
in a dialogic relationship.
As said earlier, the concept of the mental image (viewed by
Eisenstein as dynamic and interactive), also enables one to see the
concept as dialogic. Gilles Deleuze described the relations created
and reflected by it as the concept of la tiercit, the triad, referring
to the pragmatist semiotician C.S. Peirce (Sderbergh
Widding 1997, 11). According to Peirce: Thinking always proceeds
in the form of a dialogue and it is essentially composed
of signs, as its matter. (Peirce 19311958, 4.6; cited in
Johansen 1993, ix, 189). A sign is for Peirce by its nature a mediating
relationship (see Figure 1: Peirces concept of sign), which
brings together three things: representamen (the sign-vehicle),
object (the object to which it refers) and interpretant (its influence
or the interpretation of it). For Peirce, the sign-vehicles, objects
and interpretants may as well be material as mental, as well a
thought as an action. (Veivo & Huttunen 1999, 41)
So a sign for Peirce is a general and wide concept, anything
that is related to another thing (Kuusamo 1990, 45): a picture,
word, piece of music, symptom that represents something and can
be interpreted somehow.
206 Ulla Oksanen
Figure 1 illustrates Peirces triadic conception of signs:
Figure 1. Peirces Triadic Conception of Signs (Huttunen & Veivo 1999,
41).
In Peirces pragmatist sign model, the topic of particular interest,
from the viewpoint of dialogism, is its processual nature, which
stresses the action of signs, as well as the contextuality of the
model, which emphasises the temporal and spatial connections.
The ability to extend beyond the apices of the triangle is also an
essential part of the dynamics of Peirces model. Any interpretation
may turn to a sign, which again can be interpreted (cf. Sderbergh
Widdings mental image). The process is continuous, as is
our whole culture, which, according to Octavio Paz, consists of an
endless chain of interpretations. If we were to define one sign, it
would be necessary to refer to another sign that again would refer
to a third one, etc. (Tarasti 1990, 29) Meaning expands continually
and is to be attained at the end points of these diverging paths.
Sderbergh Widding (1997) seeks for such a dialogic model
that would be capable of uniting audiovisual texts and multiple
elements of spectatorship and letting them interact and bring new
From Inner Speech to Dialogic Semiosis 207
kinds of information concerning the richness and complexity of
the cinematic experience. Jrgen Dines Johansen, too, seems to
aim at a similar goal as he has undertaken to develop consistently
Peirces triadic model and complete it on the basis of Peirces own
writings. This model will be presented in the following as an
uniting view of semiosis, in which the central theme of the article:
the communicational dialogism is realised and illustrated. Moreover,
this reconstruction, based on Peirces triad, offers to act as a
tentative visualisation of the dialogic semiosis which is active in
any audiovisual narrative process.
immediate
object
OBJECT
DYNAMICAL
DYNAMICAL INTERPRETANT
OBJECT
FINAL
INTERPRETANT
immediate
interpretant
UTTERER
addresser
INTERPRETER
addressee
SIGN
token
type
Figure 2. Johansens Dialogic Model of Semiosis, (The Semiotic Pyramid)
(Johansen 1993, 246).
Johansens new model is based essentially on C.S. Peirces dynamic
and expanding view of the apices of the triangle and Roman
Jacobsons concepts of communication (see Figure 2: Johansen
s Dialogic Model of Semiosis, The Semiotic Pyramid) (Johansen
1993, 246). The sign-vehicle (here: sign) presented in
this figure may dynamically appear as an occasional concrete token,
but it may also adopt the nature of a type, a rule. Similarly,
a sign may refer to an object applying two aspects: dy208
Ulla Oksanen
namic and immediate. By a dynamic object Peirce means the
real factor or reason affecting the sign, which is not usually seen
by the addressee of the sign. Concerning the dynamic object (e.g.
the reality of which a photograph is taken) affecting behind the
sign, some kind of idea is, however, mediated in the sign process
on the basis of the sign. This idea is called the immediate object
(e.g. the reality as such that the photograph presents). (Veivo &
Huttunen 1999, 4344)
Johansens view of Peirces interpretant is also dynamic so
that the immediate and dynamic objects also have their parallels:
the immediate interpretant is the immediate and potential effect of
the sign, whereas dynamic is its actual effect. In the former the
intention of the addresser/utterer is strongly manifested, whereas
the latter is dominated by the intention of the addressee/interpreter.
The final interpretant, where the interpretation in theory is
ended, includes the nucleus of Johansens interpretation process:
the communicational interpretation, the dialogic act, where both
the parties addresser/utterer and addressee/interpreter negotiate,
and where they can reach a mutual agreement. (Johansen 1993,
169174)
The worth of the model lies basically in its ability to illustrate
and in this way add to the development of Peirces pragmatic
semiotics and his view of semiosis. The three-dimensional pyramid
also visualises in the semiosis the dialogic or communicational
interpretation implied in Peirces thinking, which was made
more widely known by his admirer Roman Jacobson, the developer
of the classical communication model, in his own work (Johansen
1993, 190). It is also possible to see Sderbergh Widdings
audiovisual experience and the total mental image illustrated in
this model that unites the three parts: interpreter/addressee (e.g.
the receiving function of the user-subject), the utterer/addresser
(e.g. the writing function of the user-subject) and Peirces triadic
(Figure 1) idea of semiosis (e.g. the message).
From Inner Speech to Dialogic Semiosis 209
Similarly, her idea of the continual change, openness and
ambiguousness of the interpretation and its tendency to expand
outside itself is visualised by the arrows at the triangle apices
which refer to their dynamism.
6 CONCLUDING WORDS
If the future is viewed from the viewpoint of Greimass three dimensions:
temporality, spatiality and actoriality that determine the
basic nature of audiovisual narration, it is possible to arrive at the
following ideas: According to Yl-Kotola when the time and
space experience in the telematic society continues to become
weaker as principles that determine observing and representing the
world, correspondingly unlinear ways to analyse data will become
stronger (Yl-Kotola 1998, 321322, italics added). In this article
in fact the same basic idea has been outlined by three permeating
tendencies or threads, which are the changes (i) from monologism
to dialogism, (ii) from diachrony to synchrony and (iii) from
syntagm to paradigm. The concepts have been illustrated in separate
connections with the verbal and audiovisual ideas of Vygotsky,
Eisenstein, Bakhtin and Godard. These ideas can without
exaggeration also be seen as profound displacements in narrative
structures and as kinds of bridges to a new type of associative
audiovisual narration and polysemic multimedial expression.
Dialogue in its Bakhtinian and Vygotskyan sense is the undivided
origin of all knowledge, thought, and thus also learning
(Tella, Mononen-Aaltonen & Kynslahti 1998, 38). It is also the
central idea of the semiotic approach to audiovisual communication
in this article. Johansens model of dialogic semiosis or the
semiotic pyramid can be considered as such a pragmatic semiotic
view of how meanings come into existence and change in the
dialogic interaction and communication process, even in the
210 Ulla Oksanen
studying and learning process while working and reflecting with
an educational multimedia program.7 In the same chapter this
model is also presented as a dialogic total mental image, which the
subject constructs. In the frame of reference of media education,
the presented model could tentatively be considered a junction of
the communication and mediation flows that stream on the highest
level of the multidimensional model, a semiosis where the signs of
the background flows intersect and create new signs and new
meanings.
7 REFERENCES
Aarseth, E. 1999. Kyberteksti nkkulmia ergodiseen kirjallisuuteen (Cybertext:
Views on Ergodic Literature). Parnasso 3, 264. (in Finnish)
Bahtin, M. 1991. Dostojevskin poetiikan ongelma. (Problems of Dostoevsky
s poetics; translated by C. Emerson.) Helsinki: Orient Express.
(in Finnish)
Broms, H. 1984. Alkukuvien jljill: Kulttuurin semiotiikka. Porvoo: WSOY.
(in Finnish)
Eisenstein, S. 1978. Elokuvan muoto. Helsinki: Love. (in Finnish)
Hietala, V. 1994. Tunteesta teesiin. Helsinki: Kirjastopalvelu. (in Finnish)
Holquist, M. 1990. The Dialogic Imagination: Four Essays by M. M. Bakhtin.
Austin: University of Texas Press.
Jalander, Y. 1990. Eisensteinin elokuvateorioista. (On Eisensteins Cinematic
Theories.) Teoksessa Kinisjrvi, R., Lukkarila, M. & Malmberg, T.
(toim.) Elokuvateorian historia. (History of Cinematic Theory.)
Helsinki: Like, 5884. (in Finnish)
Johansen, J.D. 1993. Dialogic Semiosis. Bloomington & Indianapolis: Indiana
University Press.
Kristeva, Julia. 1993. Sana, dialogi ja romaani. (Word, Dialogue, and Novel.)
Teoksessa Puhuva subjekti. (The Speaking Subject.) Helsinki: Gaudeamus,
2150. (in Finnish)
7 The model also pays attention to the new possible user functions in cybertextual communication
allowing the subject to adopt e.g. a writing function. (Aarseth 1999).
From Inner Speech to Dialogic Semiosis 211
Kuusamo, A. 1990. Kuvien edess. Helsinki: Gaudeamus. (in Finnish)
Lehtonen, M. 1998. Merkitysten maailma. Tampere: Vastapaino. (in Finnish)
Lotman, J. Merkkien maailma. (The World of Signs.) 1989. Helsinki: SN-kirjat.
(in Finnish)
Malmberg, T. 1974. Vertov ja Eisenstein: Neuvostoelokuvan perinteiden
ajankohtaisuudesta. Helsinki: Suomen elokuvakerhojen liitto ry:n
julkaisusarja 3. (in Finnish)
Norvig, P. & Lakoff, G. 1987. Taking a Study in Lexical Network Theory. In
Proceedings of the Thirteenth Annual Meeting of the Berkeley Linguistic
Society. Berkeley California, 195206. (Cited in Yl-Kotola
1998)
Peirce, C. S. 19311958. Collected Papers IVIII. C. Hartshorne, P. Weiss &
A. Burks (eds.). Cambridge MA: Harvard University Press
Sderbergh Widding, A. 1997. Mentaalinen kuva mietteit elokuvaelmyksest
(The Mental ImageVisions of Cinematic Experience). Lhikuva
23, 714. (in Finnish)
Tarasti, E. 1990. Johdatusta semiotiikkaan. Helsinki: Gaudeamus. (in Finnish)
Tella, S. & Mononen-Aaltonen, M. 1998. Developing Dialogic Communication
Culture in Media Education: Integrating Dialogism and Technology.
Media Education Centre. Department of Teacher Education. University
of Helsinki. Media Education Publications 7.
[http://www.helsinki.fi/~tella/mep7.html]
Tella, S. & Mononen-Aaltonen, M. 2000. Towards Network-Based Education:
A Multidimensional Model for Principles of Planning and Evaluation.
(In this volume)
Tella, S., Mononen-Aaltonen, M. & Kynslahti, H. in collaboration with
Nummi, T., Passi, A., Ristola, R., Sariola, J., Vahtivuori, S. & Wager,
P. 1998. Towards a Communal Curriculum: Strategic Planning and the
Emerging Knowledge of Media Education. In Tella, S. (ed.) Aspects of
Media Education: Strategic Imperatives in the Information Age. Media
Education Centre. Department of Teacher Education. University of
Helsinki. Media Education Publications 8, 184.
[http://www.helsinki.fi/~tella/mep8cc.html]
Veivo, H. & Huttunen, T. 1999. Semiotiikka: merkeist mieleen ja
kulttuuriin. Helsinki: Edita. (in Finnish)
212 Ulla Oksanen
Willemen, P. 1983. Cinematic Discourse: The Problem of Inner Speech. In
Heath, S. & Mellencamp, P. (eds.) Cinema and language. Los Angeles:
The American Film Institute. (Cited in Hietala 1994, 49)
Wertsch, J.V. 1980. The significance of dialogue in Vygotskys account of
social, egocentric, and inner speech. Contemporary Educational Psychology
5, 150162. (Cited in Tella, S. & Mononen-Aaltonen, M.
1998)
Wollen, P. 1977. Merkityksen ongelma elokuvassa. (Signs and meaning in
the Cinema.) Helsinki: Gaudeamus. (in Finnish)
Vygotski, L. 1982. Ajattelu ja kieli. (Thought and Language.) Espoo:
Weilin+Gs. (in Finnish)
Yl-Kotola, M. 1998. Jean-Luc Godard mediafilosofina: rekonstruktio simulaatiokulttuurin
lhtkohdista. Mediatieteen julkaisuja B 1. Lapin yliopisto.
Taiteiden tiedekunta. (in Finnish)
Unprinted Reference
Yl-Kotola, M. 2000. Uusmedian semiotiikka -luennot Helsingin yliopistossa
1314.4.2000. Semiotiikan opintosuunta. (in Finnish)