Multimodal Metaphor

Applications of Cognitive Linguistics 11

Editors Gitte Kristiansen Michel Achard Dirven Rene n Francisco J. Ruiz de Mendoza Iba ez

Mouton de Gruyter Berlin New York

Multimodal Metaphor
Edited by Charles J. Forceville Eduardo Urios-Aparisi

Mouton de Gruyter Berlin New York

Mouton de Gruyter (formerly Mouton, The Hague) is a Division of Walter de Gruyter GmbH & Co. KG, Berlin

which falls within the guidelines of the ANSI to ensure permanence and durability.

Printed on acid-free paper

Library of Congress Cataloging-in-Publication Data Multimodal metaphor / edited by Charles J. Forceville, Eduardo Urios-Aparisi. p. cm. (Applications of cognitive linguistics ; 11) Includes bibliographical references and index. ISBN 978-3-11-020515-2 (hardcover : alk. paper) 1. Metaphor. I. Forceville, Ch. (Charles) II. Urios-Aparisi, Eduardo, 1964 P301.5.M48M85 2009 302.2dc22 2009003856

Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.d-nb.de.

ISBN 978-3-11-020515-2 ISSN 1861-4078

Copyright 2009 by Walter de Gruyter GmbH & Co. KG, D-10785 Berlin All rights reserved, including those of translation into foreign languages. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Printed in Germany

Table of contents

List of contributors ..............................................................................................ix Preface ..............................................................................................................xiii

I. Setting the Scene

Chapter 1 Introduction Charles Forceville and Eduardo Urios-Aparisi...................................................... 3 Chapter 2 Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research Charles Forceville .............................................................................................. 19

II. Multimodal Metaphor in Advertising

Chapter 3 Brand images: Multimodal metaphor in corporate branding messages Veronika Koller ................................................................................................. 45 Chapter 4 Cutting across the senses: Imagery in winespeak and audiovisual promotion Rosario Caballero............................................................................................... 73 Chapter 5 Interaction of multimodal metaphor and metonymy in TV commercials: Four case studies Eduardo Urios-Aparisi ...................................................................................... 95 Chapter 6 Nonverbal and multimodal manifestation of metaphors and metonymies: A case study Ning Yu ........................................................................................................... 119

vi Table of contents III. Multimodal Metaphor in Political Cartoons

Chapter 7 Visual metaphor versus verbal metaphor: A unified account Francisco Yus................................................................................................... 147 Chapter 8 Metaphor in political cartoons: Exploring audience responses Elizabeth El Refaie........................................................................................... 173 Chapter 9 Image alignment in multimodal metaphor Norman Y. Teng .............................................................................................. 197 Chapter 10 Visual metaphoric conceptualization in editorial cartoons Joost Schilperoord and Alfons Maes ................................................................. 213

IV. Metaphors of Emotion in Comics, Manga, and Animation

Chapter 11 Anger in Asterix: The metaphorical representation of anger in comics and animated films Bart Eerden ...................................................................................................... 243 Chapter 12 Pictorial metaphors of emotion in Japanese comics Kazuko Shinohara and Yoshihiro Matsunaka ................................................... 265

V. Metaphor in Spoken Language and Co-Speech Gesture

Chapter 13 Words, gestures, and beyond: Forms of multimodal metaphor in the use of spoken language Cornelia Mller and Alan Cienki ..................................................................... 297 Chapter 14 Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture Irene Mittelberg and Linda R. Waugh ............................................................. 329

Table of contents

vii

VI. Metaphor Involving Music and Sound

Chapter 15 Music, language, and multimodal metaphor Lawrence M. Zbikowski ................................................................................... 359 Chapter 16 The role of non-verbal sound and music in multimodal metaphor Charles Forceville ............................................................................................ 383

VII. Metaphor and Film

Chapter 17 Multimodal metaphor in classical film theory from the 1920s to the 1950s Mats Rohdin..................................................................................................... 403 Chapter 18 Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor in horror films Gunnar Theodr Eggertsson and Charles Forceville ......................................... 429 Subject index.................................................................................................... 451 Author index .................................................................................................... 461 Metaphor and metonymy index ........................................................................ 467

List of contributors
Rosario Caballero Department of Modern Philology University of Castilla-La Mancha Ciudad Real, Spain Alan Cienki Department of Language and Communication Vrije Universiteit Amsterdam Amsterdam, The Netherlands Bart Eerden Research group Visual Rhetoric Avans Academy, AKV|St. Joost Breda, The Netherlands Gunnar Theodr Eggertsson Department of Media Studies University of Amsterdam Amsterdam, The Netherlands Lisa El Refaie Centre for Language and Communication Research Cardiff University Cardiff, United Kingdom Charles Forceville Department of Media Studies University of Amsterdam Amsterdam, The Netherlands Veronika Koller Department of Linguistics and English Language Lancaster University Lancaster, United Kingdom Fons Maes Department of Communication and Information Studies University of Tilburg Tilburg, The Netherlands

List of contributors

Yoshihiro Matsunaka Faculty of Arts Tokyo Polytechnic University Tokyo, Japan Irene Mittelberg Human Technology Centre RWTH Aachen University Aachen, Germany Cornelia Mller Department of Cultural Studies European University Viadrina Frankfurt Oder, Germany Mats Rohdin Department of Cinema Studies Stockholm University Stockholm, Sweden Joost Schilperoord Department of Communication and Information Studies University of Tilburg Tilburg, The Netherlands Kazuko Shinohara Institute of Symbiotic Science and Technology Tokyo University of Agriculture and Technology Tokyo, Japan Norman Y. Teng Institute of European and American Studies Academia Sinica Taipei, Taiwan Eduardo Urios-Aparisi, Department of Modern and Classical Languages University of Connecticut Storrs, CT, USA

List of contributors xi Linda R. Waugh Department of French and Italian University of Arizona Tucson, AZ, USA Ning Yu Department of Modern Languages, Literatures, and Linguistics University of Oklahoma Norman, OK, USA Francisco Yus Department of English Studies University of Alicante Alicante, Spain Lawrence Zbikowski Department of Music University of Chicago Chicago, IL, USA

Preface
The editors would like to thank at least a few among the many people who made this book possible. First of all, although one of us had been toying with the idea for this volume for a long time, it was the chance to organize a conference panel on multimodal discourse at the 9th conference of the International Pragmatics Association (Riva del Garda 2005), that really got things going. In Jef Verschueren and Ann Verhaert we want to thank IPrA for this opportunity. Anke Beck, at Mouton, was enthusiastic about the book project well before it deserved that name: she actually invited us to submit a book proposal when we had scarcely even sent out the call for papers for the conference panel. We thank her for her trust, and Birgit Sievert and Monika Wendland for guiding us through many practicalities at Mouton during later stages. We are also indebted to Gitte Kristiansen, managing editor of the Applications of Cognitive Linguistics series, and to Francisco Ruiz de Mendoza, a staunch promoter of the book from its earliest beginning. Most of all we are grateful, of course, to our authors, who graciously responded to our critical comments on chapter drafts, and our requests for further revisions and fine-tuning. Charles Forceville: I furthermore want to acknowledge how much I benefited from the interactions with the many students who, over the past ten years, followed my course on pictorial and multimodal metaphor in the Media Studies department at the Universiteit van Amsterdam, and from the fine papers they wrote. It is a source of pleasure and pride for me to have two of these former students as authors in this volume. I also want to express my appreciation for the academic associations that, despite their strong orientation toward verbal discourse, provided me with a platform to talk about metaphor in advertising, comics, and film: the International Cognitivist Linguistics Association (ICLA), the Researching and Applying Metaphor (RaAM) association, and the Poetics and Linguistics Association (PALA). It is one of the privileges of being a scholar that one has the opportunity to attend international conferences, where shared professional interests often lead to warm personal contacts. I have good memories of many such conferences. Among the numerous colleagues that have inspired me, I will mention one person by name. Ray Gibbs has always been exceptionally generous to me with his time, encouragement, and expertise. In 2008 I spent six months at the Centre for Advanced Studies of the Royal Flemish Academy of Belgium for Science and the Arts (VLAC), in

xiv

Preface

Brussels, where I had the pleasure of collaborating with Kurt Feyaerts and Tony Veale on the research project The Agile Mind: Creativity in Models and Multimodal Discourse. The former Royal stables at Hertogstraat 1 provided a stimulating environment for carrying out the final editing rounds of this volume. And last but not least, I want to say how lucky I have always been with the sound and commonsense advice of Kuifs agency. Eduardo Urios-Aparisi: I would like to thank my colleagues and the department of Modern and Classical Languages of the University of Connecticut, Storrs. I want to express my deepest gratitude to all those who have helped me through all these years with patience, encouragement and love. Charles Forceville Eduardo Urios-Aparisi Amsterdam, Brussels and Storrs, Connecticut, April 2009.

I Setting the Scene

Chapter 1 Introduction Charles Forceville and Eduardo Urios-Aparisi

All discourse is persuasive in the sense of aiming for some sort of cognitive, emotional or aesthetic effect, or all three together, in its envisaged audience. But purely verbal messages and texts in (mass) communication are nowadays often complemented, or even superseded, by information in other signifying systems. Printed material (advertisements, manuals, instruction books, maps, graphics, cartoons, etc.) usually combine, and establish interactions between, verbal and pictorial information, while most films and TV programs in addition draw on music and non-verbal sound. Internet sites combine text with pictures and sound, and pay attention to graphic lay-out. Spoken language is often accompanied by gestures, while modern product design involves not only what products look like, but also how they sound (e.g., cars motors, their closing doors) or even smell. Such developments reverberate in scholarly research. Classic language and literature faculties in the humanities are on the wane or get transformed and relabeled as media or cultural studies departments. Academic research in the humanities is beginning to shift from a focus on exclusively verbal text to discourses in which language is but one albeit still highly important communicative mode. This inescapable trend toward multimodality, whether applauded or bemoaned, clearly transpires from the rapidly growing number of papers, books, and conference panels with multimodal or one of its cognates in the title. In the current volume this important development in humanities research is studied from the perspective of another, somewhat older paradigm shift: the claim that metaphor is not primarily a matter of language, but structures thought and action. This view was first systematically presented, at least in the English-speaking world, by two book-length studies: Andrew Ortonys (1979) edited volume Metaphor and Thought, which had its second life in a revised and expanded edition in 1993, and George Lakoff and Mark Johnsons monograph Metaphors We Live By (1980; see also Lakoff and Johnson 2003).

Charles Forceville and Eduardo Urios-Aparisi

We believe that the book you have in your hands is pertinent to scholars in both metaphorology and multimodality. Clearly, metaphorists considering themselves adherents of the Conceptual Metaphor Theory (CMT) initiated by Lakoff and Johnson need to take seriously at least one crucial consequence of the tenet that metaphor [is] not a figure of speech, but a mode of thought (Lakoff 1993: 210): that metaphor can occur in other modes than language alone. Indeed they must do so, for if researching non-verbal and not-purely-verbal metaphor does not yield robust findings, this jeopardizes the Lakoff-and-Johnsonian presupposition that we think metaphorically. After all, in that case the supposedly metaphorical nature of human thinking would turn out to be a misconception: what has been presented as the CONCEPTUAL level of metaphor would then simply be verbal metaphor under a different name, disguised in SMALL CAPITALS. Mark Johnson appears to agree, arguing that lurking behind an exclusive focus on language is the prejudice that meaning is only to be found in words. He emphasizes that the processes of embodied meaning in the arts are the very same ones that make linguistic meaning possible (2007: 209). Of course work to correct the onesided emphasis on verbal manifestations has already been done, notably on gesture and pictures, both by authors represented in this book and by others. What is new in this book is that it focuses not so much on non-verbal metaphor per se, but on multimodal metaphor, that is, on metaphors whose target and source are rendered exclusively or predominantly in two different modes/modalities (the terms mode and modality are currently both in use; it is unclear at present which one will catch on) and in many cases the verbal is one of these. The definition of a mode is an extremely thorny one (for more discussion, see Forceville 2006/this volume). For present purposes, the modes to be taken into account are two or more of the following: (1) written language; (2) spoken language; (3) static and moving images; (4) music; (5) non-verbal sound; (6) gestures. Since what can be conveyed in terms of facts, emotions, and aesthetic pleasure differs from one mode to another, the choices for (one) particular mode(s) over (an)other(s) that the producer of a multimodal metaphor has to make is/are bound to affect its overall meaning. One modes potential to render meaning can never be completely translated into that of another mode and sometimes translation is downright impossible. For this reason alone, a healthy theory of (cognitive) metaphor must systematically study non-verbal and multimodal metaphor. It may well be indeed it is very probable that the excessive emphasis on the verbal manifestations of metaphorical thought has blinded researchers to dimensions of the latter that quite simply cannot be cued by the verbal mode.

Introduction

But researchers in the field of multimodal discourse can in their turn benefit from the work done by interdisciplinary-oriented (but often linguistically trained) metaphor scholars. It is true that semiotics, rooted in the structuralism of the 1960s and 1970s, deserves credit for being the first discipline to have conducted sustained research into non-verbal communication, at least if we discount art history, which has necessarily always had a more restricted focus. It is therefore also no coincidence that some of the contributors in this volume propose to marry insights from semiotics to those of cognitivist linguistics and neither is the recent foundation of a journal called Cognitive Semiotics. However, multimodal discourse is a vast territory, comprising a multitude of material carriers (paper, celluloid, videotape, bits and bytes, stone, cloth ), modes (written language, spoken language, visuals, sound, music, gesture, smell, touch), and genres (art, advertising, instruction manual; or at a more detailed level, say, comedy, film noir, Western, science fiction), many of these being further categorizable. It seems at this moment in time impossible, therefore, to provide anything approaching a holistic blueprint of multimodal discourse although attempts have been made (Kress and Van Leeuwen 1996/2006, 2001; Baldry and Thibault 2006; OHalloran 2004; but see Ventola et al. 2004 for more focused approaches). By contrast, systematically tracing the possible manifestations of a specific concept such as metaphor across various material carriers, modes, and genres, will signpost promising scholarly avenues, we trust, for how to analyze yet other aspects of multimodal discourse. One way to date the conception of this book is to say that its seed was planted at The pragmatics of multimodal representations panel that we, the editors, organized at the 9th International Pragmatics Conference (Riva del Garda, Italy, 1015 July 2005). In the call for papers we had emphasized we were particularly interested in multimodal metaphor, and in the end the majority of the submissions focused specifically on this topic. Along with these, other scholars we knew to have the expertise to bridge cognitive linguistics and the budding discipline of multimodal discourse were approached with the request to submit an abstract. They were given detailed guidelines about the books concept, and about how we envisaged each contribution fitting in. In order to ensure internal coherence, it was suggested that all prospective contributors take their cue for the definition of multimodal metaphor from the position paper by Forceville (2006/this volume) or else that they make clear why and how they deviated from it. Moreover, we requested that prospective contributors apply theoretical concepts systematically to one or more real-life case studies, the idea being that this procedure would fruitfully force them to face problems that mere introspective reasoning often circum-

Charles Forceville and Eduardo Urios-Aparisi

vents (cf. Haser 2005: 50). In addition, each chapter is thereby expected to spawn ideas how the proposed procedure can be deployed to analyze other multimodal representations than those examined there. Contributors were also encouraged to present (some of) their conclusions in a form that allows for empirical testing. Most of those we approached responded positively, and of the latter, the majority of the delivered chapters displayed the quality we had in mind. Early drafts of the chapters were extensively commented upon both by the editors and by one other contributing author. The guiding principle running through the chapters is a consideration of which modes play a role in the identification and interpretation of the metaphors studied. Almost invariably, this entails taking into account the genre to which the discourse featuring a multimodal metaphor belongs: advertisements, political cartoons, comics, animation, musical compositions, oral conversations and lectures, feature films. A third recurring dimension is the extent to which a metaphor is not only embodied but also governed by the cultural or professional community in which it functions. We will now briefly introduce each of the chapters in the book. Chapter 2 is a slightly updated version of the position paper on pictorial and multimodal metaphor by Forceville (2006). This paper provides and discusses the definition of multimodal metaphor that contributors to the current volume were asked to use or else explain why they opted for an alternative definition. The first cluster of chapters pertains to multimodal metaphor in advertising. It makes sense to begin with this topic, since advertising has been the subject of a number of studies pertaining to pictorial metaphor the variety of non-verbal metaphor that hitherto has attracted most scholarly attention. This is not surprising, for advertising constitutes a body of texts and practices that is persuasive par excellence. It allows bringing into play the modes of language, visuals, and sound/music. The first contribution in this cluster, Brand images: Verbal and visual metaphor in corporate branding messages, by Veronika Koller (chapter 3), charts how the logos, visuals, and layouts that are used to create companies corporate identities often require or invite the construal of metaphors. Tying in with the pervasive BRANDS ARE LIVING ORGANISMS metaphor, visual elements often subtly encourage the inference of positive corporate qualities that are not necessarily verbalized. Identifying the metaphorical mechanisms deployed to achieve this goal points the way to how the inevitably biased nature of companies selfportraits can be critically examined. Chapter 4 is Rosario Caballeros Cutting across the senses: Imagery in winespeak and audiovisual promotion. The chapter is part of an ongoing

Introduction

research project which is partly based on an impressive corpus of 12,000 wine tasting notes in professional journals, and here takes into account Spanish and French wine advertisements as well. Clearly, since taste and smell wines most important characteristics cannot be directly represented, their verbal and visual descriptions must rely on synaesthesia and metaphor. An important issue in the chapter is the difficulty of the translation of these hardly theorized modes of taste and smell into a shared vocabulary of pictures and words. Another pertinent issue is the role of the cultural background governing both the choice of source domain in purely verbal metaphors describing wines and the choice of visuals in the advertisements. Eduardo Urios-Aparisis Interaction of multimodal metaphor and metonymy in TV commercials: Four case studies (chapter 5) discusses instances of Spanish television commercials. He addresses how Forcevilles (2006/this volume) multimodal metaphor interacts with metonymical mappings, and applies the taxonomy found in Ruiz de Mendoza and Dez Velasco (2002) to multimodal advertising texts, identifying their cognitive value and communicative strategies within this genre. He shows how metaphor and metonymy fulfil different cognitive and discursive roles, serving to identify the target of a metaphor, to limit the correspondences between the domains, or to expand and create new meanings. In Nonverbal and multimodal manifestations of metaphors and metonymies: A case study (chapter 6), Ning Yu provides an in-depth analysis of a single educational message (a non-commercial commercial, if you like) broadcast on Chinese national TV in terms of two conceptual metaphors whose purely verbal varieties have often been discussed: LIFE IS A JOURNEY and LIFE IS A STAGE. He shows how aspects of these metaphors, which in some passages are blended (Fauconnier and Turner 1998, 2002), surface in various modes. In several scenes, moreover, other conceptual metaphors such as UNDERSTANDING IS SEEING and SUCCESSFUL IS UP are shown to play a role, as well as a range of metonymies. The analysis makes clear that whereas thanks to the visuals, the embodied aspects of the metaphors are presumably universally comprehensible, many details can only be fully appreciated by viewers aware of specific Chinese myths and beliefs (cf. Forceville et al. 2006). The second cluster of chapters pertains to a different textual genre: political cartoons. While a crucial presupposition in advertising is that, one way or another, a positive claim is made about the product, service, or idea advertised, political cartoons, by contrast, are characterized by the convention that something critical or negative is conveyed about one or more persons, or a state of affairs, in the world. Chapter 7, Visual metaphor versus verbal

Charles Forceville and Eduardo Urios-Aparisi

metaphor: A unified account, by Francisco Yus, mounts the argument that verbal and visual metaphors are rooted in the same cognitive mechanism. Drawing on Sperber and Wilsons (1995) relevance theory and Fodors (1983) modularity of mind theory, he takes the CMT claim that textual surface manifestations of metaphors can be traced back to conceptual metaphors to imply that there is no substantial difference between how verbal, pictorial, and multimodal metaphors are processed. Analyzing a number of cartoons by the Spanish artist El Roto, Yus demonstrates that the interpretation of each creative metaphor, irrespective of the mode(s) in which it is presented, depends on the formation of ad hoc concepts and on emergent properties (Gineste et al. 2000; Fauconnier and Turner 2002). Elizabeth El Refaies Metaphor in political cartoons: Exploring audience responses (chapter 8) further illuminates the reader about the cartoon genre by investigating two British specimens. As in Yus case study, the source-path-goal schema, with its LIFE IS A JOURNEY manifestation, is emphatically present. Since in both Yus educational commercial and El Refaies cartoons purposiveness as well as temporal development needs to be conveyed, this is hardly unexpected. After providing her own interpretation of the cartoons which turns out to be consonant with their creators intentions El Refaie reports part of a larger research project in which these two cartoons were presented to, mainly non-native, British youngsters. She finds that these adolescents are often seriously deluded about what is happening in the cartoons, with consequences for their interpretations that are as alarming as they are humorous. Norman Tengs Image alignment in multimodal metaphor (chapter 9) addresses the role of patterned visual entities in cartoons. One way of creating similarities between different visual elements is by presenting them as featuring the same orientation, color, size or any other saliently shared aspect of design. Teng discusses how such alignments can play a role in multimodal metaphors. Examining six cartoons by Clay Bennett, he moreover suggests that alignment may be the preferred design choice to convey the abstract concept of similarity between two or more items. Tengs chapter, finally, suggests avenues for research into other multimodal tropes besides metaphor. Joost Schilperoord and Alfons Maes discuss a variety of Dutch cartoons in chapter 10, Visual metaphoric conceptualization in editorial cartoons, arguing that for an appropriate understanding of the metaphors in cartoons image schema-based reasoning needs to be complemented by taxonomic reasoning, since the latter is often the crucial trigger in interpreting the critical stance expressed in editorial cartoons. The authors thus focus not so

Introduction

much on the pragmatic knowledge a viewer brings to a cartoon, but on the text-inherent information that guides metaphor interpretation, which they believe will permit the identification of textual genre-patterns. Examples of three subtypes of pictorial metaphor are examined in detail, and a number of source domains that appear to be particularly popular in cartoon metaphors are identified, such as hospitals, marriage, funerals, and boxing. Based on work by Kvecses (1986, 2000) and Forceville (2005), the next two chapters examine how emotions, specifically the paradigm case of anger, are visualized in comics, and to what extent there is cultural variation in such renderings. This cluster shifts the focus from advertising and political cartoons to comics and animation, retains the cross-cultural dimension, and addresses the notion of structural (in contrast to creative) metaphors. In chapter 11, Anger in Asterix: The metaphorical representation of anger in comics and animated films, Bart Eerden compares Forcevilles findings not only to those surfacing from the analysis of another Asterix album, but also to the data elicited from two animation films based on Asterix albums. After all, since the medium is the message, it is likely that the visual signs communicating an emotion in animated film are not completely identical to those found in comics. Kazuko Shinohara and Yoshihiro Matsunaka pursue the investigations of the EMOTIONS ARE FORCES metaphor in chapter 12, Pictorial metaphors of emotion in Japanese comics, but they provide a novel perspective by analyzing Japanese manga rather than Western comics. As a consequence, they are able to shed light on which visual signs reflect presumably universal aspects of the metaphor, and which are manifestations of knowledge that is tied to a specific culture. Both chapters in this cluster strongly suggest that conceptual metaphors find expression in visual signs in ways that are not always translatable into language, and therefore may be direct manifestations of these conceptual metaphors, unmediated by language. Spoken language and gestures are so closely interdependent that they really should be studied together (McNeill 1992, 2005; Cienki 1998). It is thus to be expected that multimodal metaphor frequently and naturally occurs in face-to-face communication. In the next cluster, two chapters discuss metaphors drawing on the gestural and spoken language modes. In chapter 13, Words, gestures and beyond: Forms of multimodal metaphor in the use of spoken language, Cornelia Mller and Alan Cienki distinguish between various types of monomodal and multimodal metaphor that are possible in spoken language accompanied by gestures, giving examples of each. In addition, they argue that intonation is an under-researched area of conceptual metaphor. Their work supports the central CMT idea that metaphor is a

10 Charles Forceville and Eduardo Urios-Aparisi conceptual phenomenon, but also demonstrates that specific modes each have their own affordances and limitations for conveying dimensions of such conceptual metaphors. Irene Mittelberg and Linda Waugh show in chapter 14, Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture, that gestures may manifest dimensions of conceptual metaphors that are not found in the cooccurring speech and that, moreover, in gesture awareness of metonymy should be considered as an indispensable stage in the process of accessing metaphor. The chapters in the next cluster are specifically devoted to the musical and sonic contributions to multimodal metaphors. Lawrence Zbikowski discusses in Music, language, and multimodal metaphor (chapter 15) how significant aspects of conceptual metaphors in a number of classical and popular music fragments depend exclusively on the musical, as opposed to the verbal, mode. Zbikowski is careful to point out, however, that for these musical elements to be experienced as metaphorical, they need to be considered in conjunction with the theme of the piece. Moreover, not only mappings from language to music are possible, but also vice versa. Zbikowski concludes that to do full justice to the respective contributions of text and music to the various musical pieces scrutinized, in a number of cases a multimodal blending approach (Fauconnier and Turner 2002) provides a better model than a multimodal metaphor construal. In both, he maintains, music appears particularly suitable in supplying sonic analogs to dynamic processes. In chapter 16 in the cluster, The role of non-verbal sound and music in multimodal metaphor, Charles Forceville considers what sonic and musical sources contribute to the identification and interpretation of multimodal metaphors in two genres, commercials and fiction films. Whereas Zbikowski sometimes considers the combinations of text and music best theorizable in terms of blends, Forcevilles cases, drawing on visuals and music often in conjunction with texts appear all to impose a clear directionality for mappings from a source to a target, and hence can typically be considered multimodal metaphors. He ends the chapter with a series of preliminary claims, to be tested in further research in this field. The chapters in the final cluster have been written by scholars with a cognitivist film theory rather than a cognitivist linguistics background. Mats Rohdin, in Multimodal metaphor in classical film theory from the 1920s to the 1950s (chapter 17), reminds us that reflection on non-verbal metaphor has a long tradition in film studies. He examines a series of classic texts that discuss cinematic metaphor, and considers to what extent the various approaches are consonant with the multimodal metaphor model adhered to in

Introduction

11

this volume. Rohdin thus is the only contributor to present a diachronic perspective on the issue of multimodal metaphor. Moreover, he draws attention to the fact that cinematic metaphors may acquire extra meanings because through visual styling they can create intertextual references to other films and phenomena familiar from everyday life. Finally, Rohdin finds that, contrary to expectation, the silent cinema was particularly rich in multimodal metaphors of the verbo-pictorial variety, due to the creative use of intertitles. The final chapter, co-authored by Gunnar Eggertsson and Charles Forceville, is titled Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor in horror films (chapter 18). Its key argument is that human victims in extreme horror films are typically abused as if they were animals. The findings shed light on metaphor theory, the genre of horror films, but they also encourage reflection on the issue of animal rights for, in the spirit of Kvecses (2005) we can adapt a famous dictum and say: show me your metaphors and I will tell you who you are. The division in clusters and chapters chosen loosely on the basis of genres and modes could have been made in different ways, since many other thematic patterns can be detected across the chapters of the book. Without elaborate discussion, we will briefly list some of these patterns, presenting them as something with a status that hovers between hypothesis and research program. Some of the issues have been discussed in relation with verbal metaphors, but often their importance has been underestimated in that realm; others appear to reveal themselves precisely thanks to the multimodal nature of the metaphors that are the specific focus of attention here. Many metaphors are mini-narratives. The paradigmatic NOUN A IS NOUN B formula disguises the dynamic nature of metaphor. Human beings move literally through space and figuratively through time, and it is within these parameters that they need to make sense of their lives. This sensemaking happens through real or imagined metaphor actions; it would perhaps be better to conceive of metaphor as A-ING IS B-ING, since metaphor is always metaphor in action. The A IS B format which maybe became popular also because CMT long discussed only decontextualized metaphors that already came in a ready-made verbal A is B form is no more than a convenient short-hand for what Andreas Musolff calls a metaphor scenario (Musolff 2006). And of course we should not forget that Paul Ricoeur (1977) already strongly emphasized the discursive character of metaphor. Though not always explicitly, all chapters in the volume tie in with this notion of a scenario or a narrative. Target and source in multimodal metaphor may both be concrete entities. Classic CMT has always stressed that human beings can only come to

12 Charles Forceville and Eduardo Urios-Aparisi grips (sic) with the abstract by metaphorically coupling it with the concrete i.e., with that which is perceptible. But the chapters in this volume are reminders that only a target that is concrete is, for instance, depictable, which is important in advertising a product, satirizing a politician in a cartoon, or conveying information about a character in a film. The focus on verbal manifestations of conceptual metaphors, that is, has had as an unfortunate side effect that for instance the stylistic dimensions of metaphors and other tropes have been somewhat ignored by cognitivist scholars (but cf. Semino and Culpeper 2002). Many illuminating (aesthetic as well as persuasive) multimodal metaphors convey something about this specifically styled target in terms of this specifically styled source. Moreover, while the embodied nature of conceptual metaphors is one of the basic tenets of CMT, Caballero (this volume) correctly points out that the embodied domains of smell and taste need rather than provide metaphorical sources. The strong focus on a bottom-up approach (from attested textual manifestations to formulations of the conceptual metaphors which supposedly underlie them rather than the other way round) may also be the reason why in several of the chapters there is some interference of the terminology associated with Max Blacks (1979) interaction theory. Black whose early contributions to cognitive theories of metaphor have insufficiently been acknowledged by most CMT theorists anticipated that metaphor could be a matter of thought rather than language, but discussed specific, creative metaphors in terms of features that were projected or transferred from source to target. CMT favors referring to this process as the partial mapping of entities and knowledge structures from source to target, resulting in a (temporary) understanding of the target in terms of the source but the occasional lapse into Blacks terminology is a healthy reminder that sometimes no more than a single aspect (feature) of the source is mapped. It is impossible to study metaphor without addressing metonymy. Metonymy has over the past decade begun to receive sustained attention from cognitive linguists (Barcelona 2000; Dirven and Prings 2002; Kristiansen et al. 2006). Clearly, each property or feature that is mapped from a source to a target must first have been metonymically related to that source. Of course, a metonym can be an ad hoc one, created by a particular context or shared by a specific community of users (cf. Yus, this volume). In addition, a metonym may have a strong emotional or evaluative relation to its source and it may well be this latter that is the rationale for the metaphor in the first place. Secondly, a given phenomenon may double as the source domain in a metaphor and as metonymically related to the target. If this is the case, the consequence may be that a construal of the relation between two things as

Introduction

13

metaphor is invited rather than forced; after all there may be a realistic, metonymic motivation for the sources presence on the grounds of expected contiguity in the domain of the target. The interaction between metaphor and metonymy is explicitly addressed in the chapters by Urios-Aparisi, Yu, and Mittelberg and Waugh. Non-verbal and multimodal metaphors may make salient certain aspects of conceptual metaphors that are not, or not as clearly, expressible in their verbal manifestations. The role of for instance size and spatial dimensions in source domains (e.g., in POWERFUL IS BIG, HONEST IS STRAIGHT) is more noticeable in visual discourses than in verbal ones. Music, in turn, affords for example scalarity and loudness in ways that can be made productive in source-to-target mappings, and the same holds for a voices timbre or an intonational pattern. Arm-and-hand gestures, both in face-to-face interaction and in the stylized varieties characterizing protagonists behaviors in comics, manga, and animation are embodied actions whose metaphorical exploitation communicates perspectives and emotions not (readily) available in verbal metaphors. A consequence of this is that any translation of these non-verbal and multimodal metaphors into verbal ones necessary for instance to enable scholarly discussion as in this book inevitably is an approximation at best. Metaphor scholars should be acutely aware of this, and reflect on what the choice for one verbalization of a multimodal metaphor over another may entail. The verbal short-hands of multimodal metaphors suggest an explicitness and precision that may well be absent in their originally non-verbal or multimodal, forms. Aspects of this issue are addressed in the chapters by Eerden, Shinohara and Matsunaka, Yu, Yus, El Refaie, Mittelberg and Waugh, Mller and Cienki, Teng, Rohdin, Zbikowski, and Forceville. Personification is a crucial variety of multimodal metaphor no less than of verbal metaphor. Living organisms and animals are attractive choices as source domains both for human target domains and for phenomena such as organizations and cars. This makes sense for a variety of reasons: as humans, we find fellow humans as well as animals provide rich opportunities for the mapping both of idiosyncratic features (snails are typically slow, peacocks proud and beautiful) and for what Black called implicative complexes (Black 1979) and Gentner and Loewenstein (2002) aligned structures. To a considerable extent, the place of humans and animals in the medieval hierarchy of the Great Chain of Being (see Tillyard 1976 [1943], Lakoff and Turner 1989) is still pertinent today, but creatures status can also be strongly influenced by cultural myths (think of the connotations of

14 Charles Forceville and Eduardo Urios-Aparisi the dragon in Western versus Chinese culture). Finally, it is attractive that people and animals move, which allows for numerous ways in which a metaphor producer can focus attention on mappable features particularly in film. Chapters in which this issue of the animal realm, and of living organisms more generally, as source domain, receives attention are those by Koller, Caballero, Urios-Aparisi, Schilperoord and Maes, Forceville, Rohdin, and Eggertsson and Forceville. Under what circumstances can or must a multimodal metaphor be construed? This is a difficult but crucial issue, particularly where a conceptual metaphor is assumed to be present. This can be rephrased as the following question: is the phenomenon under consideration necessarily to be interpreted as a metaphor, i.e., as one thing presented in terms of something that, given the context, belongs to a different category, or are other, non-metaphorical construals of their co-occurrence possible or even likely? This is a critical question for metaphor scholars. If the central tenet of CMT that in essence we think metaphorically is correct, metaphor scholars, working on verbal, non-verbal and multimodal specimens alike, should be able to demonstrate its truth, or at least probability, by showing that the phenomena under consideration can be best explained by postulating that human beings make sense of them by consciously or automatically construing metaphors. But even identifying verbal metaphors as such is no simple affair, although the Pragglejaz Group (2007) has started to develop a procedure for this. To make further progress on this issue it is necessary that alternative hypotheses are specified that might account for the phenomena under discussion (Gibbs and Perlman 2006: 217; for an alternative proposal see Haser 2005: 149 et passim), so that metaphorical and alternative explanations may be coolly juxtaposed and critically debated. This task, no easy matter to start with, is further complicated in the case of metaphors occurring in artistic discourses. Often, in such discourses, coupling two things metaphorically is not necessary to make the segment of discourse in which they occur meaningful, since alternative explanations for their co-occurrence are available. That is, a discourse producer may have reasons not to emphasize that a metaphor is to be construed. Evading censorship, avoiding litigation, or simply wanting to create a polyvalent discourse for aesthetic pleasure can motivate a maker not to produce a strongly signaled metaphor (cf. Forceville 1999: 19196). We are fully aware that many problems still have to be solved in the realm of multimodal metaphor, but we are confident that the present volume will give a substantial boost to its further theorization.

Introduction

15

References
Baldry, Anthony, and Paul J. Thibault 2006 Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Coursebook. London/Oakville: Equinox. Barcelona, Antonio (ed.) 2000 Metaphor and Metonymy at the Crossroads: A Cognitive Perspective. Berlin/New York: Mouton de Gruyter. Black, Max 1979 More about metaphor. In Metaphor and Thought, Andrew Ortony (ed.), 1943. Cambridge: Cambridge University Press. Cienki, Alan 1998 Metaphoric gestures and some of their relations to verbal metaphoric expressions. In Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (ed.), 189204. Stanford, CA: Center for the Study of Language and Information. Dirven, Ren, and Ralf Prings (eds.) 2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New York: Mouton de Gruyter Fauconnier, Gilles, and Mark Turner 1998 Conceptual integration networks. Cognitive Science 22: 133187. 2002 The Way We Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Fodor, Jerry 1983 The Modularity of Mind. Cambridge, Mass.: The MIT Press. Forceville, Charles 1999 The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 17998. 2005 Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. Forceville, Charles, Ed Tan, and Paul Hekkert 2006 The adaptive value of metaphors. In Heuristiken der Literaturwissenschaft. Einladung zu disziplinexternen Perspektiven auf Literatur, Uta Klein, Katja Mellmann, and Steffanie Metzger (eds.), 85109. Paderborn: Mentis.

16 Charles Forceville and Eduardo Urios-Aparisi

Gentner, Dedre, and Jeffrey Loewenstein 2002 Relational language and relational thought. In Language, Literacy, and Cognitive Development, Eric Amsel and James P. Byrnes (eds.), 87120. Mahwah, NJ: Erlbaum. Gibbs, Raymond W., Jr., and Marcus Perlman 2006 The contested impact of cognitive linguistics research on the psycholinguistics of metaphor understanding. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 211228. Berlin/New York: Mouton de Gruyter. Gineste, Marie-Dominique, Bipin Indurkhya, and Vronique Scart 2000 Emergence of features in metaphor comprehension. Metaphor and Symbol 15: 117135. Haser, Verena 2005 Metaphor, Metonymy, and Experientialist Philosophy: Challenging Cognitive Semantics. Berlin/New York: Mouton de Gruyter. Kvecses, Zoltn 1986 Metaphors of Anger, Pride and Love. Amsterdam/Philadelphia: Benjamins. 2000 Metaphor and Emotion: Language, Culture, and Body in Human Feeling. Cambridge: Cambridge University Press. 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Kress, Gunther, and Theo van Leeuwen 1996 Reading Images: The Grammar of Visual Design. 2nd ed. Revised edition published in 2006. London/New York: Routledge. 2001 Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold. Lakoff, George 1993 The contemporary theory of metaphor. In Metaphor and Thought, 2nd ed. Andrew Ortony (ed.), 202251. Cambridge: Cambridge University Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press. 2003 Afterword, 2003. In Metaphors We Live By, 243276. Chicago: University of Chicago Press. Lakoff, George, and Mark Turner 1989 More Than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. McNeill, David 1992 Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press. 2005 Gesture and Thought. Chicago: University of Chicago Press.

Introduction

17

Musolff, Andreas 2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21: 2338. OHalloran, Kay L. (ed.) 2004 Multimodal Discourse Analysis. London/New York: Continuum. Ortony, Andrew (ed.) 1979 Metaphor and Thought. Cambridge: Cambridge University Press. 1993 Metaphor and Thought. 2nd ed. Cambridge: Cambridge University Press. Pragglejaz Group 2007 MIP: A method for identifying metaphorically used words in discourse. Metaphor and Symbol 22: 139. Ricoeur, Paul 1977 The Rule of Metaphor: Multi-disciplinary Studies of the Creation of Meaning in Language. Trans. by R. Czerny, Kathleen McLaughlin, and John Costello. Toronto: University of Toronto Press. Ruiz de Mendoza, Francisco Jos, and Olga Isabel Dez Velasco 2002 Patterns of conceptual interaction. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralph Prings (eds.), 489532. Berlin/New York: Mouton de Gruyter. Semino, Elena, and Jonathan Culpeper (eds.) 2002 Cognitive Stylistics: Language and Cognition in Text Analysis. Amsterdam/Philadelphia: Benjamins. Sperber, Dan, and Deirdre Wilson 1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Blackwell. Tillyard, E.M.W. 1976 [1943] The Elizabethan World Picture. Harmondsworth: Penguin. Ventola, Eija, Cassily Charles, and Martin Kaltenbacher (eds.) 2004 Perspectives on Multimodality. Amsterdam/Philadelphia: Benjamins.

Chapter 2 Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research1 Charles Forceville

Abstract
Conceptual metaphor theory (CMT) has over the past 25 years amply sought to underpin the claim that humans pervasive use of verbal metaphor reflects the fact that they think largely metaphorically. If this tenet of CMT is correct, metaphor should manifest itself not just in language but also via other modes of communication, such as pictures, music, sounds, and gestures. However, non-verbal and multimodal metaphor have been far less extensively studied than their verbal sisters. The present chapter provides a review of work done in this area, focusing on a number of issues that require further research. These issues include the proposal to distinguish between monomodal and multimodal metaphor; reflections on the difference between structural and creative metaphor; the question of how verbalizations of non-verbal or conceptual metaphors may affect their possible interpretation; thoughts as to how similarity between target and source is created; and suggestions about the importance of genre for the construal and interpretation of metaphor. Keywords: Monomodal and multimodal metaphor, pictorial metaphor, structural versus creative metaphor, similarity in metaphor, genre.

1. Introduction Andrew Ortonys edited volume Metaphor and Thought (1979) and Lakoff and Johnsons monograph Metaphors We Live By (1980) were milestone publications in the sense that they marked the switch from research into metaphor as a primarily verbal to a predominantly conceptual phenomenon. The conceptual metaphor theory (CMT), as the Lakoffian-Johnsonian

20 Charles Forceville model is habitually referred to, has been a very productive one (e.g., Gibbs 1994; Johnson 1987, 1993, 2007; Kvecses 1986, 2000, 2002; Lakoff 1987, 1993; Lakoff and Johnson 1999, 2003; Lakoff and Turner 1989; Sweetser 1990; Turner 1996). A key notion in this theory is that the mind is inherently embodied, reason is shaped by the body (Lakoff and Johnson 1999: 5; Chapter 3). Very briefly, what this means is the following. Human beings find phenomena they can see, hear, feel, taste and/or smell easier to understand and categorize than phenomena they cannot. It is perceptibility that makes the former phenomena concrete, and the lack of it that makes the latter abstract. In order to master abstract concepts, humans systematically comprehend them in terms of concrete concepts. Thus abstract concepts such as LIFE, TIME, and EMOTIONS are systematically understood in terms of concrete phenomena. LIFE is understood as A JOURNEY (Hes without direction in his life; Im at a crossroads in my life) but also, for instance, as A STORY (Tell me the story of your life; Lifes a tale told by an idiot ...). TIME is comprehended in terms of SPATIAL MOTION (The time for action has arrived; Time is flying by; He passed the time happily). Emotions are typically represented by drawing on the domain of FORCES. (I was overwhelmed; I was swept off my feet; examples from Kvecses 2000; Lakoff 1993; Lakoff and Johnson 1980). Conceptualizations of many phenomena, CMT proposes, have deeply entrenched metaphorical forms, in which the metaphors target (topic, tenor) is abstract and its source (vehicle, base) is concrete. A metaphors interpretation boils down to the mapping of pertinent features from the source to the target; a mapping that in the case of entrenched metaphors such as the above occurs automatically. Since concreteness is apprehended perceptually, metaphorical source domains are strongly rooted in the functioning of the human body. Metaphorical reasoning is thus governed by the arch metaphor MIND IS BODY (Lakoff and Johnson 1999: 249). A more recent development rooted in CMT is blending theory (Fauconnier and Turner 2002). Rather than postulating a target and a source domain, it presents two (or more) input spaces. The input spaces have both shared and unique characteristics, and it is this combination that allows for the construal of a so-called blended space. Blending theory is a, mainly descriptive, model claiming to be superior to metaphor theory in being able to account for ad hoc linguistic creativity, metaphorical and otherwise. Hitherto it cannot quite convince (for a critical review, see Forceville 2004a), but new work, taking into account pragmatic rhetorical factors, is promising (see Coulson and Pascual 2006; Terkourafi and Petrakis, forthcoming).

Metaphor in a cognitivist framework

21

CMT has inspired conferences (e.g., those organized by the International Cognitive Linguistics Association, and the Researching and Applying Metaphor [RaAM] association), journals (e.g., Metaphor and Symbol, Cognitive Linguistics), as well as empirical research (for references see Gibbs 1994, 2006; Lakoff and Johnson 1999: 58788). Its importance is evident: if CMT is basically correct, it provides crucial insights into what, thanks to embodiment, lays claim to being universal in human cognition, and what is rooted in, and shapes, (sub)cultural differences. However, CMT is restricted in at least the following very important dimension. Even though Lakoff and Johnsons (1980: 5) characterization of metaphors essence as understanding and experiencing one kind of thing in terms of another emphatically avoids the word verbal or linguistic, the validity of CMTs claims about the existence of conceptual metaphors depends almost exclusively on the patterns detectable in verbal metaphors. This entails two dangers: in the first place, there is the risk of a vicious circle: cognitive linguistic research suffers from circular reasoning in that it starts with an analysis of language to infer something about the mind and body which in turn motivates different aspects of linguistic structure and behavior (Gibbs and Colston 1995: 354; see also Cienki 1998). Clearly, to further validate the idea that metaphors are expressed by language, as opposed to the idea that they are necessarily linguistic in nature, it is imperative to demonstrate that, and how, they can occur non-verbally and multimodally as well as purely verbally. Secondly, an exclusive or predominant concentration on verbal manifestations of metaphor runs the risk of blinding researchers to aspects of metaphor that may typically occur in non-verbal and multimodal representations only. This latter awareness, of course, exemplifies a more general principle. Ever since Marshall McLuhans the medium is the message (McLuhan 1964: 24 et passim), it is a truism that as soon as one changes the medium via which a message (including both its factual and emotive aspects) is conveyed, the content of this message is changed as well (see also Bolter and Grusin 1999). Each medium here defined as a material carrier and transmitter of information communicates via one or more signaling systems. The medium of non-illustrated books, for instance, exclusively draws on the mode of written language; radio relies on the modes of spoken language, non-verbal sound, and music; advertising billboards on written language and visuals; and post-silent film on visuals, written language, spoken language, non-verbal sound, and music. If, as is argued here, each of these signaling systems (which will henceforward be called modes) can cue, independently or in combination, metaphorical targets as well as metaphorical sources, a full-blown theory of metaphor

22 Charles Forceville cannot be based on its verbal manifestations alone, since this may result in a biased view of what constitutes metaphor. In this chapter I will sketch how adopting the view that metaphors can assume non-verbal and multimodal appearances can and should guide the research of a new generation of metaphor scholars. I will do so partly by bringing to bear multimodal perspectives on issues already familiar from research by language-oriented metaphor scholars, and partly by discussing issues that have either been neglected by such researchers or are simply not pertinent to purely verbal metaphors. The chapter should be seen as a map of mostly uncharted territory, with only a few details inked in, much of it reporting theory-driven analyses and informed speculation awaiting empirical testing. Multimodal metaphor researchers have a vast amount of work to look forward to.

2. Multimodality versus monomodality In order to distinguish multimodal metaphor from monomodal metaphor, it should first be further clarified what is meant by mode. This is no easy task, because what is labeled a mode here is a complex of various factors. As a first approximation, let us say that a mode is a sign system interpretable because of a specific perception process. Acceptance of this approach would link modes one-on-one to the five senses, so that we would arrive at the following list: (1) the pictorial or visual mode; (2) the aural or sonic mode; (3) the olfactory mode; (4) the gustatory mode; and (5) the tactile mode. However, this is too crude a categorization. For instance, the sonic mode under this description lumps together spoken language, music, and non-verbal sound. Similarly, both written language and gestures would have to be part and parcel of the visual, since one cannot hear, smell, taste, or touch either conventionally written language or gestures (although a blind person feels Braille language and, by touch, can perceive certain gestures for instance those of a statue). If justice is to be done to these distinctions (between images and gestures, between spoken and written language, between spoken language, sounds, and music), other factors need to be taken into account, such as the manner of production (e.g., printed versus Braille letters in relief on paper; signs made with parts of the body versus signs whose use is governed by the grammar and vocabulary rules of a natural language). There are other problematic issues. For instance, what is music and what mere sound may differ from one culture, or period, to another. Similarly, it is impossible to assess objectively where music shades off into

Metaphor in a cognitivist framework

23

sound effect. And is typeface to be considered an element of writing, of visuals, or of both? In short, it is at this stage impossible to give either a satisfactory definition of mode, or compile an exhaustive list of modes. However, this is no obstacle for postulating that there are different modes and that these include, at least, the following: (1) pictorial signs; (2) written signs; (3) spoken signs; (4) gestures; (5) sounds; (6) music (7) smells; (8) tastes; (9) touch. We can now provisionally define monomodal metaphors as metaphors whose target and source are exclusively or predominantly rendered in one mode. The prototypical monomodal metaphor is the verbal specimen that until recently was identical with metaphor tout court, and which has yielded thousands of studies (de Knop et al. 2005; Shibles 1971; Van Noppen et al. 1985; Van Noppen and Hols 1990). A type of monomodal metaphor that has more recently become the subject of sustained research is pictorial or visual metaphor.2 An early discussion of metaphor in pictures is Kennedy (1982). This perception psychologist takes metaphor in the all-encompassing sense of what literary scholars call a trope or a figure of speech (and which Tversky 2001 calls figures of depiction) and identifies some 25 types, including for instance metonymy, hendiadys, and litotes. Kennedys attempts to describe an extensive catalogue of figures of depiction using the tenor/vehicle distinction that Richards (1965) specifically coined for metaphor are sometimes strained. There are other problems. It is a matter for debate whether the names he selects for his examples are always necessarily the best ones, and each trope is illustrated with one or two examples only, making generalizations difficult (the same problem also adheres to Durand 1987). This having been said, Kennedy makes a number of points that are illuminating for a theory of pictorial/visual metaphor-in-the-narrow-sense. In the first place he argues that for a phenomenon to be labeled a visual metaphor it should be understandable as an intended violation of codes of representation, rather than as being due to carelessness or error. Secondly, Kennedy emphasizes that target and source are, in principle, irreversible, which ensures that what he labels metaphor remains commensurate with a generally accepted criterion in theories of verbal metaphor. Thirdly, Kennedy introduces the helpful notion of runes: the kind of non-iconic signs used profusely in comics and cartoons to indicate speed, pain, surprise, happiness, anger and many other phenomena by means of straight or squiggly lines, stars, bubbles etc. surrounding characters or moving objects (see also Smith 1996). In later work, Kennedy elaborates on his theoretical work in various

24 Charles Forceville experiments. Kennedy (1993) reports, among other things, how congenitally blind children metaphorically draw a spinning wheel. Whittock (1990) describes cinematographic metaphor. While his numerous examples are subsumed under ten subtypes, and thus are less wideranging than those by Kennedy, they still go beyond metaphor-in-thenarrow-sense, including for instance metonymy and synecdoche. Whittock is criticized by Carroll (1994) for failing to take into account what the latter considers the most typical variety of visual or cinematographic metaphor, the visual hybrid (see also Carroll 1996). Carroll, unlike Kennedy and Whittock, moreover argues that visual metaphors differ from verbal ones in often allowing for reversal of target and source. In Forceville (2002a), expanding on earlier work (Forceville 1988, 1994, 1995, 1996, 2000), I in turn question Carrolls choice for visual hybrids as core filmic metaphor (Carroll 1996: 218) as well as his proposal for the typical reversibility of target and source in visual metaphors. My argument rests on the claim that Carroll is biased by his exclusive reliance on examples rooted in Surrealist art. My own model, largely developed with respect to advertising representations (but see Forceville 1988) and based on Blacks (1979) interaction theory of metaphor (see also Gineste et al. 2000; Indurkhya 1991, 1992), centers on the answerability of the following three questions: Which are the two terms of the pictorial metaphor?; which is the target and which is the source?; and which is/are the features that is/are mapped from source to target? The last question pertains to the metaphors interpretation: in principle all elements metonymically associated (by a whole community or by a single individual) with the source domain qualify as potential candidates for a mapping. The crucial issue what is actually mapped by a specific addressee in a specific situation is governed by the relevance principle as developed by Sperber and Wilson (1995). (For more discussion of the role of metonymy in metaphor, see various contributions in Dirven and Prings 2002; for the pertinence of Relevance Theory to the interpretation of pictorial metaphors, see Forceville 1996, chapters 5 and 6.) In this model, Carrolls examples would rank as one of three (Forceville 2002b) or four (Forceville 2005a, 2007) subtypes of monomodal metaphor. It is to be noted that the distinction between two of these types metaphor and simile is also made by Kaplan (1990, 1992; see also Rozik 1994; 1998). In contrast to monomodal metaphors, multimodal metaphors are metaphors whose target and source are each represented exclusively or predominantly in different modes. The qualification exclusively or predominantly is necessary because non-verbal metaphors often have targets and/or sources that are cued in more than one mode simultaneously. To give a fictive exam-

Metaphor in a cognitivist framework

25

ple: imagine somebody wants to cue, for whatever reason, the metaphor CAT IS ELEPHANT pictorially in an animation film. She could do this for instance by depicting the cat with a trunk-like snout and large flapping ears; by showing the cat with a canopy on its back in which a typical Indian elephant rider is seated; by juxtaposing cat and elephant in the same salient pose; or by letting the cat behave (for instance: move) in an elephant-like manner. These variants would constitute monomodal metaphors of the pictorial kind, featuring hybrid, contextual, simile, and integrated subtypes respectively (see Forceville 1996, 2002b, 2005a) and of course these subtypes could be combined. Now imagine the producer wishes to cue the same metaphor multimodally. She could for instance have the cat make a trumpeting sound or have another cat shout elephant! to the first one (note that this is not a case of synaesthesia, since there is no conflation of the two domains). In these cases the source domain ELEPHANT would be triggered in two modes (sound and language, respectively) that are different from the target (visuals). By this token, the metaphor would be truly multimodal. But, as in the case of the visual mode alone, the producer would of course not have to choose between any of these modes: she could depict the cat with a trunklike snout and large ears and have it trumpet, and have another cat shout elephant! In this case, the source is cued in three modes simultaneously, only one of these (namely: the visual) exemplifying the same mode as the target. In such a case I also propose to label the metaphor multimodal. Of course the metonymy cueing the source domain in itself is often chosen for its specific connotations. Both tusks and a trunk trigger ELEPHANT, but the former connotes, among other things, aggressiveness, whiteness, costliness and the latter among other things flexibility, sensitivity, and instrument-tospray-water-or-sand-with. For examples, as well as more discussion, of multimodal metaphors involving (moving) images, see Forceville (1999a, 2003, 2004b, 2005b, 2007, 2008). There is also a growing literature on multimodal metaphors involving language and gestures (Cienki 1998; McNeill 1992; Mller 2004), in which the gesture-modality cues the source rather than the target domain (McNeill 2005: 45).

3. Structural versus creative metaphor Lakoff and Turner (1989) have argued that not only metaphors occurring in everyday verbal communication can be traced back to conceptual metaphors, but also those in artistic texts, specifically poetry. Particularly when poems thematize abstract concepts such as life and death, they cannot but draw on

26 Charles Forceville the same conceptual metaphors that permeate non-artistic language. Thus Lakoff and Turner cite, and richly illustrate, many passages featuring LIFE and TIME as metaphorical targets (LIFE IS A JOURNEY, LIFE IS A PLAY, LIFE IS BONDAGE, LIFE IS A BURDEN; TIME IS A THIEF, TIME IS A MOVER, TIME IS A DEVOURER) concluding that although human imagination is strong, empowering us to make and understand even bizarre connections, there are relatively few basic metaphors for life and death that abide as part of our culture (1989: 26). They acknowledge that the art and craft of good poets resides in finding fresh, original verbal formulations for these conceptual metaphors, and that these formulations resonate both with the rest of the poem and with the extra-textual knowledge of the reader. I take it that Lakoff and Turner allow that this, in turn, may result in temporary readjustments of the basic level conceptual metaphors, and thus that they would agree that the linguistic level of the metaphor is not a mere illustration or exemplification of the pre-existing basic conceptual level. But not all verbal metaphors in poetry, as Lakoff and Turner acknowledge, reflect basic conceptual metaphors. While conventional metaphors can be expressed either in common or in idiosyncratic language, modes of thought that are not themselves conventional cannot be expressed in conventional language (1989: 26) and hence require idiosyncratic language. (If I understand her correctly, Renate Bartsch would probably object to the label conceptual metaphor for the metaphorical schema that underlies such novel metaphors. She stipulates that a phenomenon deserves the name of concept only if it has a stable interpretation in a community, and hence must by definition have been linguistically explicated (Bartsch 2002: 50). A stable interpretation requires that the community agrees on the phenomenons characteristic features and these, in turn, reveal themselves in the true predications that can be used for it. Novel metaphors focus attention on non-characteristic features and therefore, in Bartsch reasoning, cannot (yet) have the status of being conceptual.) Often, the border between conventional metaphors and idiosyncratic ones is difficult to draw, not least because conventional metaphors may have idiosyncratic extensions. Lakoff and Turner thus admit that not all poetic metaphors are conventional ones, but the bulk of their examples and discussions pertain to the latter. This is unsurprising, since their aim is to show that poetic metaphors normally tap into conventional ways of thinking (great poets can speak to us because they use the modes of thought we all possess, Lakoff and Turner 1989: xi). Nonetheless Lakoff and Turners account raises some questions. In the first place, it is not clear how representative their chosen examples are of poetic metaphors in general. While they convincingly show that structural metaphors pervade poetry, the relative distribution of metaphors may depend on time and place: older poetry, or non-Western poetry, may feature more, or

Metaphor in a cognitivist framework

27

less, instances of a given metaphor. Gevaert (2001), basing herself on corpus-based data, questions for instance the embodied, timeless status of ANGER IS HEAT claimed for conceptualizations of anger in Lakoff (1987). She demonstrates, among other things, that in the Old English period SWELLING was a much more important source domain in ANGER metaphors than HEAT, and speculates that the latters growing popularity in more recent periods may well be due to the humoral theory that dominated mediaeval times; that is, to cultural no less than embodied knowledge (see also Gevaert 2005). Moreover, a systematic, corpus-based analysis might reveal that many poetic metaphors are not so easily amenable to conventional ones. Numerous poetic metaphors may simply not have abstract concepts such as LIFE, DEATH, TIME, PURPOSE as their target domain. As pointed out by Grady, they may more often be resemblance metaphors than, possibly (near) universal, correlation metaphors (Grady 1999). This would not invalidate Lakoff and Turners impressive findings, but their one-sided emphasis on correlation and generic-level metaphors3 in poetry may inadvertently lead to an uncritical acceptance of the view that most poetic metaphors are of this kind. One important difference between conventional and idiosyncratic metaphors is that the interpretation of the latter is, by definition, far less governed by entrenched, pre-existing correspondences between the schematic structures in target and source. It is only by downplaying this difference that Lakoff and Turner can say that the preservation of generic-level structure is, we believe, at the heart of metaphorical imagination, whether poetic or ordinary (1989: 83; for critical accounts of this view see also Stockwell 1999; Crisp 2003). Secondly, we should not forget that a metaphor can also conceptualize the concrete in terms of the concrete. Lakoff and Turner, to be fair, are aware of this. They discuss at some length the Elizabethan notion of the Great Chain of Being (see e.g., Tillyard 1976 [1943]), which endorsed the idea of natural hierarchies within various types of creatures angels, humans, birds, mammals, etc. and state that the GREAT CHAIN METAPHOR can apply to a target domain at the same level on the Great Chain as the source domain (Lakoff and Turner 1989: 179, emphasis in original). Put differently, metaphors may have targets as well as sources that are directly accessible to the senses. But given that CMT puts great emphasis on metaphors role in conceptualizing the abstract in terms of the concrete, this possibility receives rather scant attention, while CONCRETE IS CONCRETE metaphors are particularly relevant once we leave the realm of the purely verbal. In the case of

28 Charles Forceville monomodal metaphors of the pictorial variety, both target and source are depicted. In advertising, metaphorical targets usually coincide with promoted products and, unsurprisingly, are depicted and hence are necessarily concrete: a beer brand is depicted as a wine; an elegant watch as a butterfly, a close-fitting bathing suit as a dolphins tight and supple skin (examples from Forceville 1996). The same holds for metaphors in feature films (Forceville 2005b; Whittock 1990). In short, to what extent monomodal metaphors of the non-verbal variety and multimodal metaphors are amenable to the correlation metaphors that are the center of attention in CMT is an empirical question. Some of them no doubt do; for instance, the personification of commodities is a very familiar marketing strategy, and ties in with CMT views (Lakoff and Turner 1989: 72). But many pictorial and multimodal metaphors are of the OBJECT A IS OBJECT B type. Traditional CMT has not much to say about these. Even Lakoff and Turners (1989) invocation of the Great Chain metaphor is only of limited use here, since it depends on typological hierarchies that may be subverted, or simply irrelevant, in creative metaphors, many of which function in contexts creating highly specific, ad hoc metaphorical resemblances (see Black 1979). There is a third aspect in which CMT has a one-sided emphasis. As discussed above, the typical source domains concreteness has in CMT been traditionally connected to the notion of embodiment. The embodied nature of source domains emphasizes their physical nature: it is human physical interaction with the world that familiarizes humans with it to such an extent that the resulting knowledge structures can in turn be mapped onto abstract concepts. Knowledge about source domains is not simply a matter of embodiment, however, but also of cultural connotations, as Lakoff and Turner (1989: 66) acknowledge. More recent studies have demonstrated in a variety of ways how the structure of source domains and the salient (and hence: easily mappable) elements in it is influenced by culture (Gibbs and Steen 1999; Kvecses 2005; Shore 1996; Yu 1998). Indeed, the cultural connotations that are metonymically related to a source domain are often more important for potential mappings to a target than its embodied aspects. In a Dutch commercial promoting a Gazelle bicycle in terms of a dressage horse the embodied mapping of riding a horse to riding a bicycle is less important for the interpretation of the metaphor than the mapping of the cultural connotations from the dressage horses owner, champion Anky van Grunsven, to the prospective buyer and user of the bike. Similarly, while advertising a high-tech Senseo coffee machine in terms of a motorbike certainly has

Metaphor in a cognitivist framework

29

embodied aspects, the subcultural connotations associated with motorbikeriding evoked by Steppenwolfs Born to be wild, audible on the commercials soundtrack, are at least as important in the mapping (Forceville 2004b/this volume, 2008; Forceville et al. in preparation). The relevant similarity that is created between target and source pertains to these connotations more than to anything else. The examples bear out Bartsch observation that in metaphor the role of similarity is not restricted to the identity of internal properties of objects and situations, rather similarity also is due to identity of external contiguity relationships between objects, between situations, and it is due to relationships of objects and situations with emotional attitudes, desires, and behavioural dispositions of people (Bartsch 2002: 52). Indeed, it might be ventured that a single, embodied correspondence between target and source is enough to trigger a wide range of further cultural correspondences between target and source, and hence of inferences about the target (Forceville et al. 2006: 107). The old adage that a picture tells more than a thousand words should not blind us to the fact that pictures and other multimodal representations seldom communicate automatically or self-evidently. As in verbal metaphors, it is connotations rather than denotations of source domains that get mapped in metaphors, and these may substantially differ from one (sub)cultural group to another (see e.g., Maalej 2001). Even when non-verbal metaphors verge toward the conventional, as in comics representations of ANGER (Eerden this volume; Forceville 2005c; see Simons 1995 for multimodal instantiations of structural metaphor in preelection TV spots promoting political parties; and Forceville 2006a, Forceville and Jeulink 2007, for discussions of the source-path-goal schema and the LIFE IS A JOURNEY metaphor), it may well be the case that, as in their verbal counterparts (see Kvecses 1986, 2000, 2005), there is cultural variation (see Shinohara and Matsunaka 2003, this volume). The three issues briefly mentioned above (metaphors frequently have concrete rather than abstract targets; many metaphors with idiosyncratic surface manifestations do not reflect correlation metaphors; what gets mapped from source to target domains are often cultural, not embodied, features) are thrown in relief when the study of purely verbal metaphors gives way to that of non-verbal and multimodal metaphors. That is, generalizing observations on metaphor based on the systematic investigation of verbal specimens need to be considered afresh by testing these observations in metaphors occurring in other modes.

30 Charles Forceville 4. The verbalization of non-verbal metaphor and the nature of similarity We have seen that within the CMT paradigm, most surface metaphors should be amenable to a pre-existing conceptual A IS B format. Inevitably, in order to discuss the metaphor, this A and B must be named, i.e., rendered in language. It is by no means a foregone conclusion, of course, that the language of thought is actually a verbal language. The convention to verbalize the image-schematic structures underlying surface metaphors by using SMALL CAPITALS useful inasmuch as this facilitates analyzing them may disguise a number of consequences that seem to me more problematic in the discussion of discourses that are not (exclusively) verbal ones than of purely verbal ones. One of these consequences is that it is the analysts responsibility to find an adequate or acceptable verbal rendering of the metaphors underlying image-schematic level, but such a verbalization, even though used as a convenient shorthand, is never neutral. The design of the Senseo coffee machine suggests the posture of somebody bending over and modestly offering something (i.e., a cup) on a plate. But should this awareness result in the verbalization COFFEE MACHINE IS SERVANT, or is COFFEE MACHINE IS BUTLER more appropriate? Although servant and butler share many features, they also differ: butler is more specific, and may in some people (but not in others) evoke connotations of Britishness and standards of service that servant does not. As a result, the mappings suggested by the two verbalizations may differ. Bartsch might conclude that this very inability to agree on a single verbalization of the source domain that is shared within a community shows that the source has no conceptual status, and reflects a quasi-concept at best (Bartsch 2002: 50). However, to the extent that there is a community that recognizes the source as cueing a serving person, the source admits predicates understood as true in the community (such as is there to serve the user, obeys your requests, and is almost always available). In a visually literate society, a vast number of endlessly repeated and recycled images (such as famous paintings, photographs, film shots, flags, logos, animation characters) evoke specific phenomena and events in a clichd, shorthand manner widely shared within a community, and hence arguably aspire to conceptual status. But this speculation leads us far beyond the concerns of the present chapter and deserves in-depth reflection elsewhere. Another consequence is that verbalization of a non-verbal metaphor is necessarily a conscious action, and a fairly unusual one at that. It is only the scholar writing an academic paper who, to be able to discuss a multimodal

Metaphor in a cognitivist framework

31

metaphor, needs to resort to GAZELLE BICYCLE IS VAN GRUNSVEN DRESSAGE HORSE; the metaphor is not verbalized in this form in the commercial, and it is an open question whether the construal of a non-verbal metaphor requires its verbalization by the audience. The question can be reformulated as follows: does comprehension of a non-verbal or multimodal metaphor imply that recipients mentally verbalize the metaphor? It is an important question, but difficult to test empirically. Since non-verbal modes of communicating by definition do not have the is or is like in order to signal a metaphorical identity relation between two entities belonging, in the given context, to different categories, one issue that deserves attention is by what stylistic means the similarity is triggered. Of course this holds for verbal metaphors that do not have the paradigmatic A IS (LIKE) B format as well (cf. Brooke-Rose 1958; Goatly 1997). But whatever means are chosen in this latter case, the cues are themselves of a verbal nature. In non-verbal and multimodal metaphors, the signals that cue metaphorical similarity between two phenomena are different, and bound to differ depending on the mode(s) in which the metaphorical terms are represented. Here are some possibilities that are deployed in isolation or in combination: Perceptual resemblance. This can only function as a trigger in the case of monomodal metaphors: only a visual representation can perceptually resemble another visual representation; only a sound can perceptually resemble another sound in volume, timbre, or pitch. In the case of visual resemblance, there is a larger range of choices: two things can resemble one another because they have the same size, color, position, posture, texture, materiality, etc. Note that the resemblance need not reside in the things themselves, but may surface in their manner of representation: they may for instance be photographed from the same unusual angle, or filmed with the same unusual camera movement. Filling a schematic slot unexpectedly. Placing a thing in a certain context may strongly, even inescapably, evoke a different kind of thing, namely the thing for which the given context is the natural or conventional place. Put differently, we may encounter deviations from typical gestalts or schemas. For example, when in a musical environment a violin case contains a monkey wrench, this may suggest the metaphor MONKEY WRENCH IS VIOLIN. Simultaneous cueing. If two things are signaled in different modes, metaphorical identification is achieved by saliently representing target and source at the same time. For instance a kiss could be accompanied by the sound of a car crash, of a vacuum cleaner, or of the clunking of chains, to cue metaphorical mappings of, say, disaster, dreary domestic routine, and imprison-

32 Charles Forceville ment, respectively. Alternatively, in a variant on the previous mechanism, two disparate things can be linked because of an unexpected filling of a slot, as when a photograph of a kiss has the caption imprisonment. 5. The influence of genre Human beings in most cases appear to construe a text automatically, very quickly, and probably largely subconsciously as belonging to one genre rather than to another. Anecdotal support for this claim is the experience of channel-surfing: seasoned TV watchers guess in a split second what kind of program they surf into (and decide at once whether they want to spend time with it). Another illustration for the claim is the funny, self-reflexive trailer for the film Comedian (http://nl.youtube.com/watch?v=yXbFuNQwTbs, last consulted 10 July 2008) by Jerry Seinfeld. A man, Jack, sits in a booth in a sound studio to record the voice-over for the trailer, but as soon as he has intoned a few words he is impatiently interrupted by the director:
Jack: Director: Jack: Director: Jack: Director: Jack: Director: In a world where laughter was king No in a world, Jack What do you mean, No in a world? Its not that kind of movie. Oh OK In a land that No in a land either In the time No, I dont think so.

In a rapid exchange Jack makes one abortive attempt after another One man ; When your life is no longer your own ; When everything you know is wrong ; In an outpost ; On the edge of space only to be cut short by the director straightaway. For present purposes the point to be made is that, to an audience with expertise in the area of film, the few words uttered by Jack suffice to cue an entire genre. Finally, Hayward (1994) offers empirically attested support for the claim that people are able to decide very fast to what genre a text belongs. Hayward found that almost 80% of experimental subjects, given randomly selected passages of history or fiction writing, recognized the genre of the work even on the basis of very short passages (5 to 15 words). The genre within which a text (in whatever medium) is presented, or the genre to which it is attributed, determines and constrains its possible interpretations to an extent that is difficult to overestimate (see Altman 1999; Charteris-Black 2004; Forceville 1999b, 2005d; Steen 1994; Zwaan 1993

Metaphor in a cognitivist framework

33

for discussion). For this reason, it is important to study how genre has an impact on the production and interpretation of metaphors (monomodal and multimodal alike). In advertising, for instance, the targets of metaphors often coincide with the product promoted (Forceville 1996). This is to be expected: an advertisement or commercial predicates something about a product, brand, or service, and this neatly and naturally fits the metaphors TARGET IS SOURCE format. Moreover, the features mapped from source to target are positive ones (unless the metaphor is used to disqualify a competitors brand, in which case the mapped features are typically negative). But in feature films, there is no phenomenon that in a similar, natural, way qualifies as a metaphorical target. Metaphors in artistic narratives pertain to phenomena that, for whatever reason, are deemed salient by their producers. These phenomena can be protagonists, but also objects, or even events. The mapped features will often be less clear-cut, and may have a richer aligned structure (Gentner and Loewenstein 2002), than those in advertising. Metaphors in artistic representations may also differ in other respects from those in commercial messages. For instance, while in commercials there will seldom be a question what is target and what is source in a metaphor, an artistic narrative may give rise to two different construals of a metaphor: both A IS B and B IS A are appropriate. (While Carroll 1994, 1996 calls such metaphors reversible, I prefer to say that, in the given context, both the metaphors A IS B and B IS A are pertinent, in order to retain the notion that target and source in a metaphor are, in principle, irreversible.) Commensurate with this, metaphors in artistic contexts presumably allow for greater freedom of interpretation than do metaphors in commercials (cf. also Shen 1995). Another parameter that deserves further research is whether any of the subtypes of pictorial metaphor or of the manifold varieties of multimodal metaphor can be systematically related to certain text genres. For instance, it seems that commercial advertising seldom makes use of the hybrid variety of pictorial metaphors (in Forceville 1996 these were called MP2s). Again, this makes sense: if metaphorical targets typically coincide with products, advertisers would want their product portrayed in their entirety, and not in a manner that might evoke connotations of incompleteness or mutilation. Hybridizing it with a metaphorical source domain would not fit this goal. By contrast, in animation films, or science fiction films, no such problem arises. Finally, it would be worthwhile to investigate whether in the case of multimodal metaphors there are any systematic correlations between textual genres and the modes in which a target and source are represented. In advertisements, the visual mode is typically used for representing the target and

34 Charles Forceville this may well be true for different genres as well. But perhaps alternative patterns in the choice of mode for the source domain are detectable in different types of texts, while this may also change over time within a genre. 6. Concluding remarks Researching multimodal metaphor, in short, is a natural next step in the further development of metaphor studies a development in which theoretical reflection will have to go hand in hand with empirical testing. If creative and conventional metaphor are key factors in human thinking, and if human thinking is reflected in more than verbal manifestations alone, investigating multimodal metaphor is highly worthy of extensive scholarly effort. Given its long disciplinary tradition, the robust insights of metaphor scholarship can in turn fruitfully feed into the budding field of multimodality in general (Kress and Van Leeuwen 1996/2006, 2001; Ventola et al. 2004). Genres to be investigated include advertising (Forceville 1996, 2003, 2007; McQuarrie and Mick 2003; Phillips 2003; Wiggin and Miller 2003; McQuarrie and Phillips 2008), political cartoons (El Refaie 2003), film (Forceville 1999a, 2005b; Rohdin 2003; Whittock 1990), oral speech accompanied by gestures (Cienki 1998; McNeill 1992, 2005; Mller 2004), and design (Cupchik 2003; Van Rompay 2005). And inasmuch as multimodal representations (in the form of advertising, videoclips, games, TV-formats, mainstream films, animation) travel faster and more easily across the world than verbal ones, examining their metaphorical manifestations will help focus on what remains stable and what changes in cross-cultural communication. Furthermore, such work may provide the starting point for how other tropes besides metaphor can assume multimodal appearances (e.g., metonymy, irony, hyperbole, oxymoron, see Gibbs 1993; Kennedy 1982; Teng and Sun 2002). Here the analysis of multimodal metaphor ties in with the study of rhetoric. In a global society in which media are increasingly used, or abused, as mouthpieces for the views of powerful factions (politicians, industry tycoons, religious leaders), the critical analysis of the tools of persuasive discourse in the broadest sense constitutes an excellent interface between research in academia and its possible usefulness in the world beyond its walls. Acknowledgments
I am indebted to Francisco Ruiz de Mendoza and Ren Dirven for comments and suggestions on earlier drafts of this paper.

Metaphor in a cognitivist framework

35

Notes
1. This chapter is virtually identical with the text published as Forceville (2006b). The changes are reference updates and some minor corrections and rephrasings. The topic of metaphor and music also has inspired studies over the past few years, but for lack of expertise in this area I will not dwell on these. For references, see Johnson and Larson 2003; see also Cook 1998; Zbikowski 2002; Thorau 2003; and Spitzer 2004 the last one rather difficult for laymen. Lakoff and Turner describe generic-level metaphors as metaphors which are minimally specific in two senses: they do not have fixed source and target domains, and they do not have fixed lists of entities specified in the mapping (1989: 81). They introduce the term using the example of the EVENTS ARE ACTIONS metaphor which they contrast with LIFE IS A JOURNEY, one of its specific-level instantiations.

2.

3.

References
Altman, Rick 1999 Film/Genre. London: British Film Institute. Bartsch, Renate 2002 Generating polysemy: Metaphor and metonymy. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven, and Ralf Prings (eds.), 4974. Berlin/New York: Mouton de Gruyter. Black, Max 1979 More about metaphor. In Metaphor and Thought, Andrew Ortony (ed.), 1943. Cambridge: Cambridge University Press. Bolter, Jay David, and Richard Grusin 1999 Remediation: Understanding New Media. Cambridge MA: MIT Press. Brooke-Rose, Christine 1958 A Grammar of Metaphor. London: Secker and Warburg. Carroll, Noel 1994 Visual metaphor. In Aspects of Metaphor, Jaakko Hintikka (ed.), 189218. Dordrecht: Kluwer. Carroll, Noel 1996 A note on film metaphor. In Theorizing the Moving Image, 212 223. Cambridge: Cambridge University Press. Charteris-Black, Jonathan 2004 Corpus Approaches to Critical Metaphor Analysis. London: Palgrave-MacMillan.

36 Charles Forceville
Cienki, Alan 1998 Metaphoric gestures and some of their relations to verbal metaphoric expressions. In Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (ed.), 189204. Stanford, CA: Center for the Study of Language and Information. Cook, Nicholas 1998 Analyzing Musical Multimedia. Oxford: Clarendon. Coulson, Seana, and Esther Pascual 2006 For the sake of argument: Mourning the unborn and reviving the dead through conceptual blending. Annual Review of Cognitive Linguistics 4: 153181. Crisp, Peter 2003 Conceptual metaphor and its expression. In Cognitive Poetics in Practice, Joanna Gavins and Gerard Steen (eds.), 99113. London/New York: Routledge. Cupchik, Gerry D. 2003 The interanimation of worlds: Creative metaphors in art and design. The Design Journal 6 (2): 1428. Dirven, Ren, and Ralf Prings (eds.) 2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New York: Mouton de Gruyter. Durand, Jacques 1987 Rhetorical figures in the advertising image. In Marketing and Semiotics: New Directions in the Study of Signs for Sale, Jean UmikerSebeok (ed.), 295318. Berlin/New York: Mouton de Gruyter. Eerden, Bart this vol. Anger in Asterix: The metaphorical representation of anger in comics and animated films. El Refaie, Elisabeth 2003 Understanding visual metaphors: The example of newspaper cartoons. Visual Communication 2 (1): 7595. Fauconnier, Gilles, and Mark Turner 2002 The Way We Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Forceville, Charles 1988 The case for pictorial metaphor: Ren Magritte and other Surrealists. In Vestnik IMS 9, Ale Erjavec (ed.), 150160. Ljubljana: Intitut za Marksistine tudije [Slovenia]. 1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Activity 9: 129. 1995 (A)symmetry in metaphor: The importance of extended context. Poetics Today 16: 677708. 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 1999a The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 17998.

Metaphor in a cognitivist framework 1999b

37

Art or ad? The effect of genre-attribution on the interpretation of images. Siegener Periodicum zur Internationalen Empirische Literaturwissenschaft [SPIEL] 18 (2): 279300. 2000 Compasses, beauty queens and other PCs: Pictorial metaphors in computer advertisements. Hermes, Journal of Linguistics 24: 3155. 2002a The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2002b Further thoughts on delimitating pictorial metaphor. Theoria et Historia Scientiarum 6: 21327 [Torn, Poland]. 2003 Bildliche und multimodale Metaphern in Werbespots [Trans. from English by Dagmar Schmauks] Zeitschrift fr Semiotik 25: 3960. 2004a Review of Fauconnier and Turner (2002). Metaphor and Symbol 19: 8389. 2004b The role of non-verbal sound and music in multimodal metaphor. In Words in their Places: A Festschrift for J. Lachlan Mackenzie, Henk Aertsen, Mike Hannay, and Rod Lyall (eds.), 6578. Amsterdam: Faculty of Arts, VU Amsterdam. 2005a When is something a pictorial metaphor? [Lecture 2 in eight-lecture Course in Pictorial and Multimodal metaphor] http://www. chass.utoronto.ca/epc/srb/cyber/cforcevilleout.html. 2005b Cognitive linguistics and multimodal metaphor. In Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach (ed.), 264284. Cologne: Von Halem. 2005c Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2005d Addressing an audience: time, place, and genre in Peter Van Straatens calendar cartoons. Humor: International Journal of Humor Research 18: 247278. 2006a The source-path-goal schema in the autobiographical journey documentary: McElwee, Van der Keuken, Cole. New Review of Film and Television Studies 4: 241261. 2006b Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven and Francisco Ruiz de Mendoza Ibez (eds.), 379 402. Berlin/New York: Mouton de Gruyter. 2007 Multimodal metaphor in ten Dutch TV commercials. Public Journal of Semiotics 1 (1): 1951. http://semiotics.ca/. 2008 Metaphors in pictures and multimodal representations. In The Cambridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr. (ed.), 462482. Cambridge: Cambridge University Press. Forceville, Charles, Paul Hekkert, and Ed Tan 2006 The adaptive value of metaphors. In Heuristiken der Literaturwissenschaft. Einladung zu disziplinexternen Perspektiven auf Literatur, Uta Klein, Katja Mellmann, and Steffanie Metzger (eds.), 85109. Paderborn: Mentis.

38 Charles Forceville
Forceville, Charles, and Marloes Jeulink 2007 The Source-Path-Goal schema in animation film. Paper presented at 10th International Cognitivist Linguistics conference (ICLC, July) at Jagiellonian University, Krakow (Poland). Forceville, Charles, Michelle Hilscher, and Gerald Cupchik in prep. The interpretation of multimodal metaphors in Dutch commercials by Dutch and Canadian subjects [provisional title]. Gentner, Dedre, and Jeffrey Loewenstein 2002 Relational language and relational thought. In Language, Literacy, and Cognitive Development, Eric Amsel and James P. Byrnes (eds.), 87120. Mahwah, NJ: Erlbaum. Gevaert, Caroline 2001 Anger in Old and Middle English: a hot topic? Belgian Essays on Language and Literature, 89101. 2005 The anger is heat question: Detecting cultural influence on the conceptualisation of anger through diachronic corpus analysis. In Perspectives on Variation, Nicole Delbecque, Johan van der Auwera, and Dirk Geeraerts (eds.), 195208. Berlin/New York: Mouton de Gruyter. Gibbs, Raymond W., Jr. 1993 Process and products in making sense of tropes. In Metaphor and Thought, 2nd ed., Andrew Ortony (ed.), 252276. Cambridge: Cambridge University Press. 1994 The Poetics of the Mind: Figurative Thought, Language, and Understanding. Cambridge: Cambridge University Press. 2006 Embodiment and Cognitive Science. Cambridge: Cambridge University Press. Gibbs, Raymond W., Jr., and Gerard J. Steen (eds.) 1999 Metaphor in Cognitive Linguistics. Amsterdam/Philadelphia: Benjamins. Gibbs, Raymond W., Jr., and Herbert L. Colston 1995 The cognitive psychological reality of image schemas and their transformations. Cognitive Linguistics 6: 347378. Gineste, Marie-Dominique, Bipin Indurkhya, and Vronique Scart 2000 Emergence of features in metaphor comprehension. Metaphor and Symbol 15: 117135. Goatly, Andrew 1997 The Language of Metaphors. London/New York: Routledge. Grady, Joseph 1999 A typology of motivation for conceptual metaphor: Correlation vs. resemblance. In Metaphor in Cognitive Linguistics, Raymond W. Gibbs, Jr., and Gerard J. Steen (eds.), 79100. Amsterdam/ Philadelphia: Benjamins. Hayward, Malcolm 1994 Genre recognition of history and fiction. Poetics 22: 409421.

Metaphor in a cognitivist framework

39

Indurkhya, Bipin 1991 Modes of metaphor. Metaphor and Symbolic Activity 6: 127. 1992 Metaphor and Cognition: An Interactionist Approach. Dordrecht: Kluwer. Johnson, Mark 1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination and Reason. Chicago: University of Chicago Press. 1993 Moral Imagination: Implications of Cognitive Science for Ethics. Chicago: University of Chicago Press. 2007 The Meaning of the Body: Aesthetics of Human Understanding. Chicago and London: University of Chicago Press. Johnson, Mark, and Steve Larson 2003 Something in the way she moves metaphors of musical motion. Metaphor and Symbol 18: 6384. Kaplan, Stuart J. 1990 Visual metaphors in the representation of communication technology. Critical Studies in Mass Communication 7: 3747. 1992 A conceptual analysis of form and content in visual metaphors. Communication 13: 197209. Kennedy, John M. 1982 Metaphor in pictures. Perception 11: 589605. 1993 Drawing and the Blind: Pictures to Touch. New Haven/London: Yale University Press. Knop, Sabine de, Ren Dirven, Ning Yu, and Birgit Smieja (eds.) 2005 Bibliography of Metaphor and Metonymy. Amsterdam/ Philadelphia: John Benjamins. Available electronically only at: http://www. benjamins.com/online/met. Kvecses, Zoltn 1986 Metaphors of Anger, Pride and Love. Amsterdam/Philadelphia: Benjamins. 2000 Metaphor and Emotion: Language, Culture, and Body in Human Feeling. Cambridge: Cambridge University Press. 2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press. 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Kress, Gunther, and Theo van Leeuwen 1996 Reading Images: The Grammar of Visual Design (2nd ed. 2006). London/New York: Routledge. 2001 Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold. Lakoff, George 1987 Women, Fire and Dangerous Things: What Categories Reveal about the Mind. Chicago: University of Chicago Press. 1993 The contemporary theory of metaphor. In Metaphor and Thought, 2nd ed., Andrew Ortony (ed.), 202251. Cambridge: Cambridge University Press.

40 Charles Forceville
Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press. 1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books. 2003 Afterword, 2003. In Metaphors We Live By, 243276. Chicago: University of Chicago Press. Lakoff, George, and Mark Turner 1989 More Than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. Maalej, Zouhair 2001 Processing pictorial metaphor in advertising: A cross-cultural perspective. Academic Research, 1: 1942 [Sfax, Tunisia]. McLuhan, Marshall 1964 The medium is the message. In Understanding Media: The Extensions of Man, 2nd ed., 2335. New York: McGraw-Hill. McNeill, David 1992 Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press. 2005 Gesture and Thought. Chicago: University of Chicago Press. McQuarrie, Edward F., and David Glen Mick 2003 The contribution of semiotic and rhetorical perspectives to the explanation of visual persuasion in advertising. In Persuasive Imagery: A Consumer Response Perspective, Linda M. Scott and Rajeev Batra (eds.), 191221. Mahwah, NJ: Lawrence Erlbaum. McQuarrie, Edward F., and Barbara J. Phillips (eds.) 2008 Go Figure! New Directions in Advertising Rhetoric. Armonk, NY: ME Sharpe. Mller, Cornelia 2004 Metaphors, Dead and Alive, Sleeping and Waking: A Cognitive Approach to Metaphors in Language Use. Habilitationsschrift, Freie Universitt Berlin, Germany. Ortony, Andrew (ed.) 1979 Metaphor and Thought. Cambridge: Cambridge University Press. Phillips, Barbara J. 2003 Understanding visual metaphor. In Persuasive Imagery: A Consumer Response Perspective, Linda M. Scott, and Rajeev Batra (eds.), 297310. Mahwah, NJ: Lawrence Erlbaum. Richards, Ivor Armstrong. 1965 [1936] The Philosophy of Rhetoric. New York: Oxford University Press. Rohdin, Mats 2003 Summary in English. In Vildsvinet I Filmens Trdgrd: Metaforbegreppet inom Filmteorin [The wild boar in the garden of film: The concept of metaphor in film theory], 318329. Ph.D. dissertation, Stockholm: Edita Norstedts Tryckeri AB.

Metaphor in a cognitivist framework

41

Rompay, Thomas van 2005 Expressions: Embodiment in the Experience of Design. Ph.D. diss., Technische Universiteit Delft, The Netherlands. Rozik, Eli 1994 Pictorial metaphor. Kodikas/Code 17: 203218. 1998 Ellipsis and the surface structures of verbal and nonverbal metaphor. Semiotica 119: 77103. Shen, Yeshayahu 1995 Cognitive constraints on directionality in the semantic structure of poetic vs. non-poetic metaphors. Poetics 23: 255274. Shibles, Warren A. 1971 Metaphor: An Annotated Bibliography and History. Whitewater, WI: The Language Press. Shinohara, Kazuko, and Yoshihiro Matsunaka 2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen: Journal of Yokohama Linguistic Circle 4: 118. this vol Pictorial metaphors of emotion in Japanese comics. Shore, Bradd 1996 Culture in Mind: Cognition, Culture, and the Problem of Meaning. New York: Oxford University Press. Simons, Jan 1995 Film, Language, and Conceptual Structures: Thinking Film in the Age of Cognitivism. Ph.D. dissertation, University of Amsterdam, The Netherlands. Smith, Ken 1996 Laughing at the way we see: The role of visual organizing principles in cartoon humor. Humor: International Journal of Humor Research 9: 1938. Sperber, Dan, and Deirdre Wilson 1995 Relevance: Communication and Cognition (2nd ed.) Oxford: Blackwell. Spitzer, Michael 2004 Metaphor and Musical Thought. Chicago/London: University of Chicago Press. Steen, Gerard J. 1994 Understanding Metaphor in Literature: An Empirical Approach. London: Longman. Stockwell, Peter J. 1999 The inflexibility of invariance. Language and Literature 8: 12542. Sweetser, Eve E. 1990 From Etymology to Pragmatics: Metaphorical and Cultural Aspects of Semantic Structure. Cambridge: Cambridge University Press. Teng, Norman Y., and Sewen Sun 2002 Grouping, simile, and oxymoron in pictures: A design-based cognitive approach. Metaphor and Symbol 17: 295316.

42 Charles Forceville
Terkourafi, Marina, and Stefanos Petrakis forthc. A critical look at the desktop metaphor 25 years on. In Proceedings of the 6th International Conference on Researching and Applying Metaphor (RaAM6), University of Leeds, April 2006. Amsterdam/ Philadelphia: Benjamins. Thorau, Christian 2003 Metapher und Variation: Referenztheoretische Grundlagen musikalischer Metaphorizitt. Zeitschrift fr Semiotik 25: 109124. Tillyard, E.M.W. 1976 [1943] The Elizabethan World Picture. Harmondsworth: Penguin. Turner, Mark 1996 The Literary Mind. New York/Oxford: Oxford University Press. Tversky, Barbara 2001 Spatial schemas in depictions. In Spatial Schemas and Abstract Thought, Merideth Gattis (ed.), 79112. Cambridge MA: MIT Press/Bradford book. Van Noppen, Jean-Pierre, Sabine de Knop, and Ren Jongen (eds.) 1985 Metaphor: A Bibliography of Post-1970 Publications. Amsterdam/ Philadelphia: Benjamins. Van Noppen, Jean-Pierre, and Edith Hols (eds.) 1990 Metaphor II: A Classified Bibliography of Publications from 1985 1990. Amsterdam/Philadelphia: Benjamins. Ventola, Eija, Cassily Charles, and Martin Kaltenbacher 2004 Perspectives on Multimodality. Amsterdam/Philadelphia: Benjamins. Whittock, Trevor 1990 Metaphor and Film. Cambridge: Cambridge University Press. Wiggin, Amy A., and Christine M. Miller 2003 Uncle Sam wants you! Exploring verbal-visual juxtapositions in television advertising. In Persuasive Imagery: A Consumer Response Perspective, Linda M. Scott and Rajeev Batra (eds.), 267 295. Mahwah, NJ: Lawrence Erlbaum. Yu, Ning 1998 The Contemporary Theory of Metaphor: A Perspective from Chinese. Amsterdam/Philadelphia: Benjamins. Zbikowski, Lawrence M. 2002 Conceptualizing Music: Cognitive Structure, Theory, and Analysis. Oxford: Oxford University Press. Zwaan, Rolf 1993 Aspects of Literary Comprehension: A Cognitive Approach. Amsterdam/Philadelphia: Benjamins.

II Multimodal Metaphor in Advertising

Chapter 3 Brand images: Multimodal metaphor in corporate branding messages1 Veronika Koller

Abstract
This chapter looks at multimodal metaphors in companies communication of their brand personalities. It is argued that corporate brands are cognitively structured by the metaphor BRANDS ARE LIVING ORGANISMS, often specifically BRANDS ARE PEOPLE. As an intangible abstraction, the brand is made comprehensible by being conceptualized as an ideal person endowed with the traits that are positively evaluated in corporate discourse: growth, flexibility, dynamism, and connectivity. These metaphorical character traits of the brand personality are expressed by the interplay of verbal and visual features in corporate discourse, in particular the illustrations, logos, and layout found in genres such as mission statements and history brochures. The source domain of the BRANDS ARE LIVING ORGANISMS/PEOPLE metaphor is encoded visually, while the target domain tends to be encoded verbally, with occasional additional encoding in the visual mode. It is argued that this dual encoding reinforces the persuasive intent of corporate genres by endowing them with an affective component. Corporate decision-makers use these genres and their multimodal features to communicate the corporate brand to external stakeholders just as much as to address internal ones, i.e., employees, to foster identification with, and loyalty towards, the brand. Keywords: branding, discourse, illustration, layout, logos

1. Introduction This chapter looks at multimodal metaphor in companies communication of their brand personalities. It argues that corporate brands are cognitively structured by a metaphor BRANDS ARE LIVING ORGANISMS, often specifically BRANDS ARE PEOPLE. As an intangible abstraction, the brand is

46 Veronika Koller made comprehensible by being metaphorized as an ideal person endowed with the traits that are positively evaluated in corporate discourse: growth, flexibility, dynamism, and connectivity. These metaphorical character traits of the brand personality are expressed by the interplay of verbal and visual features in corporate discourse, in particular the illustrations, logos, and layout found in genres such as mission statements and history brochures. Corporate decision-makers use these genres and their multimodal features to communicate the corporate brand to external stakeholders such as customers just as much as to address internal ones, i.e., employees (de Chernatony 2002). In both cases, the intention is to foster identification with, and loyalty towards, the brand. However, given that the power dynamics between employees and the company they depend on are very different from those between the corporation and the customers it seeks to win and retain, it may be easier for consumers to disregard or cynically reject the brand image (Holt 2002).

2. A cognitive approach to multimodal metaphor In Cognitive Metaphor Theory (see e.g., Barcelona 2000; Lakoff and Johnson 1999, 2003) metaphor, rather than being a merely decorative literary device, is regarded as an essentially cognitive phenomenon structuring much of human thought. In particular, metaphor is the means by which the human mind conceives of one usually abstract entity in terms of another, usually a more concrete one. The processes by which these metaphorically structured mental models are brought about have been theorized as mappings from a source to a target domain, or blends of two or more input spaces (Fauconnier and Turner 2002). In any case, it is only in a second step that these metaphoric models are realized as metaphoric expressions at the surface level of language, or of any other semiotic mode. Metaphoric expressions have been most exhaustively studied in the verbal mode, i.e., as surface-level linguistic expressions of metaphorically structured mental models. This body of work was followed by research into visual, or pictorial, metaphor (Carroll 1994; Forceville 1994, 1996). However, most of those studies still addressed mono-modal metaphor, in that both source and target domains (or input spaces) were provided in the visual mode and only reinforced, rather than co-constructed, by the verbal co-text. Multimodal metaphor, on the other hand, is constituted by a mapping, or blending, of domains from different modes, e.g., visual and verbal, or visual and acoustic. With a view to the examples presented in this chapter, it seems

Brand images

47

useful to draw on Barthes (1977: 3841) concept of anchoring, according to which the verbal elements of a multimodal text serve to cue and thereby restrict possible interpretations of the visual elements. It will be argued that such anchoring can also be achieved beyond the immediate co-text, by means of intertextuality. For instance, the IBMs mission statements webpage is illustrated with the picture of a DNA helix (figure 2), and while the verbal co-text itself does not include any references to genes, talk about the DNA of a company is shown to be a staple of corporate discourse, thus anchoring an interpretation of the DNA image as the source domain of the metaphor BRANDS ARE LIVING ORGANISMS. Forceville (2006) has rightly pointed out that modes cannot simply be equated with the five senses but that distinctions need to be made between, for instance, different forms of acoustic mode (speech, music, sound; see van Leeuwen 1999). Indeed, if one followed a perceptual taxonomy based on the five senses, it would be impossible to analyze multimodal metaphor as being constituted by visual and verbal elements; after all, verbal elements can be perceived visually as written language, acoustically as spoken language, and haptically as Braille language. Forceville therefore suggests the following categories in the investigation of multimodal metaphor: pictorial signs: These include abstract design elements, as in logos written signs spoken signs gestures: These can be seen as part of the more general category of kinesthetics, i.e. the movement of the body in three-dimensional space2 sounds, to be seen on a continuum with music smells, which are physiologically and hence perceptually related to tastes, and finally touch.

This chapter will focus on the interaction of pictorial and written signs, as they combine into multimodal metaphoric expressions realizing the conceptual metaphor BRANDS ARE LIVING ORGANISMS and specifically, BRANDS ARE PEOPLE. The duality of conceptual metaphors as both operating on the cognitive level and being realized in surface level metaphoric expressions leads to inductive reasoning on the part of the researcher, in that the analysis of metaphoric expressions leads to inferences about underlying metaphoric models. In the case of verbal metaphoric expressions, verbalizations of in-

48 Veronika Koller ferred conceptual metaphors are to some extent motivated by the verbal formulation of the observed expression. For example, the phrase we channeled investments in parallel with our corporate DNA, said by a company executive about their financial strategy (Schwartz 2006), allows for a metaphor COMPANIES ARE LIVING ORGANISMS to be inferred. Obviously, any verbalization reflects the researchers interpretation of the observed expressions, and will in turn steer the analysis of similar phrases. This bias is aggravated in the case of multimodal metaphor, especially when it is the source domain that is expressed in non-verbal form: For example, a companys business principles might feature pictures of a boat race (e.g. ABN Amro bank, http://www.group.abnamro.com/about/business_principles.cfm) with no possible literal interpretation.3 Should the inferred conceptual metaphor be worded as COMPANY X IS A RACING TEAM, or, in more general terms, BUSINESS IS A RACE? In the case of a relay race, the metaphors significance changes again, foregrounding aspects of teamwork rather than competition. The ABN Amro example is further complicated by the verbal co-text realizing the JOURNEY metaphor, referring to the business principles as a compass to guide us on our journey. In view of this complexity, it could be argued that the traditional A IS B type metaphor as such is limiting, and that inferred metaphors should be seen as part of a metaphor scenario (Musolff 2006) with different levels of abstraction as well as cause-effect relations. In our example, this could mean that both COMPANY X IS A RUNNER and BUSINESS IS A RACE are viable. In addition, the metaphoric scenario triggered by the multimodal metaphor could include semi-metaphoric components like company X delivers faster results and outruns the competition. I will return to this question when discussing the examples from corporate discourse below. In a given discourse,4 only particular models will be relevant, i.e., one form of structured knowledge of a particular notion or entity along with its associations, evaluations and affective components will be preferred to others. Subsequently, the metaphoric expressions deriving from metaphoric models and their evaluations become typical of that discourse. Repeated exposure to the same set of metaphoric expressions, extended and elaborated as they may be in various modes, can be assumed to activate the metaphoric mental model underlying them and thereby reinforce it. By the same token, positive evaluation of particular notions would also be reinforced. Discursive reinforcements of mental models, including evaluations, have been referred to as moralized activities, i.e., activities represented by means of abstract terms that distil from them a quality that triggers reference to positive or negative values (van Leeuwen and Wodak 1999: 105). An example would be the metaphoric reference to companies as corporate citizens, which has

Brand images

49

acquired the status of a technical term in corporate discourse and reinforces the positively connoted idea of companies taking responsibility for the wider community they are part of. What models rise to prominence in a discourse is to some extent influenced by the (institutional) power of discourse participants. In the case of corporate branding messages, their senders hold considerable power when addressing employees, while consumers as addressees have the power to withhold their attention, interest and, ultimately, money. This relative power on the part of consumers also grants them easier access to counter-readings that, often cynically, deconstruct brand messages (see for instance the practice of ad busting; Rumbo 2002). Following the sociocognitive tradition in critical discourse analysis (van Dijk 2002, 2005), metaphor can, because of its dual nature, thus be seen as located at the interface between cognition and discourse. Moreover, its selective use in discourse represents an ideologically vested strategy to shape models of social reality, e.g., to present and ingrain particular traits but not others as positive and desirable. By investigating how such a strategy is enacted through multimodal metaphor, the chapter seeks to contribute to the critical study of multimodality (Lassen, Strunck, and Vestergaard 2006), and multimodal metaphor in particular.

3. A cognitive approach to corporate brands It has been noted that inasmuch as pictures are more easily recognized transnationally than languages, pictorial and multimodal metaphors allow for greater cross-cultural access than verbal ones (Forceville 2008). This makes visual elements, including the visual components of multimodal metaphor, particularly attractive as a communicative strategy for multinational corporations. Multimodal metaphor in particular can be used as a tool to meet the persuasive function of most corporate communication, because it requires the texts recipient to construct a meaningful reading by processing verbal and visual elements together. The necessary cognitive effort potentially reinforces a particular conceptualization of the company in the readers mind. Indeed, much research into visual metaphor to date has investigated the phenomenon in contexts such as advertising, often with a view to processes of reception and interpretation (Phillips 2003; Phillips and McQuarrie 2004), or international business magazines (Koller 2005). Supplementing those studies, this chapter looks at corporate branding as the interface between marketing and strategic organizational management. As the outward oriented (i.e., projected) organizational identity (Kapferer

50 Veronika Koller 2002: 185) of a company, a corporate brand is the image a company wishes to convey to its stakeholders. In addition to Kapferers definition of the corporate brand, it is here also understood as inwardly oriented, i.e., communicated to internal stakeholders such as employees.5 A corporate brand is a metonymic condensation of what corporate decision-makers consider most desirable in their organization (Balmer 2006: 39). As such, the corporate brand has to be distinguished from product brands: Although some corporations project both (e.g., Apple as a corporate brand and iPod as one of its product brands), companies may communicate corporate and product brands separately. Examples are McDonalds as a corporate brand without particular product brands and, conversely, Ariel as one of the many product brands marketed by Procter & Gamble, which is not itself a corporate brand. Projecting both seems to be the exception rather than the rule: Apart from historical and strategic reasons for keeping product and corporate brands separate, a duplication of images could ask too much of the cognitive capabilities of a consumer who is already faced with innumerable brands. Ever since the latter half of the 1990s, corporate branding has gradually become more important than product branding, up to the point where tangible products are merely the material extension of a brand (Askegaard 2006: 100) and the battle for positioning takes place in the realm of imagery (Soenen and Moingeon 2002: 30). Reasons for this shift range from increased corporate brand value as an asset in negotiating mergers and acquisitions, to the economies of scale gained by more focused marketing and communication efforts, and the need for employees to identify with their company (Schultz, Hatch, and Ciccolella 2006: 142). In socio-cognitive terms, a corporate brand constitutes the ideal self a company wishes to communicate to others. Ultimately, this ideal self is meant to converge with the epistemic social schema (Kristiansen 2008), i.e., the beliefs a social group, in the present case, stakeholders, has about the nature of another social group or entity, here, the company. The analysis carried out in this study will show how corporate brands are further constructed by drawing on an ought self (Kunda 1999: 472) or deontic social schema (Kristiansen 2008), i.e., the perceived view of stakeholders about what the company should be like. Brands as intangible entities represent the cognitive-affective concepts stakeholders maintain about a particular product, service or, in the case of corporate brands, company. To make these abstract models graspable, companies have since the 1980s sought to conceptualize themselves and their brands as living organisms (Csaba and Bengtsson 2006: 122; Morgan 1986), even endowing them with a quasihuman personality (Christensen and Askegaard 2001; Wee 2004), up to

Brand images

51

the point where brands become metaphorical people interacting with stakeholders.6 In a final twist, the BRANDS ARE PEOPLE metaphor implies that a brand is supposed to incorporate the ideal characteristics the consumer wishes to be seen as having. In an ultimately narcissistic relationship with the BRANDS ARE PEOPLE metaphor, the brands characteristics are theorized to be transferred to the consumer, who thus engages in symbolic consumption (Harquail 2006: 174175), using brands as symbolic resources for the construction and maintenance of identity (Elliott and Wattanasuwan 1998: 132). A central role in the projection of corporate brand image is played by artifacts, or multimodal texts, e.g., logos, architecture, office design, call centre music etc. (Cappetta and Gioia 2006: 212). Such brand artifacts are seen as the vehicles for the transfer of meaning from the brand to the consumer, bringing with them emotional and self-expressive benefits for the consumer (Schultz, Hatch, and Ciccolella 2006: 150). Forceville (2006: 388) notes that the personification of commodities is a very familiar marketing strategy, and it is certainly one that can be extended to (corporate) brand artifacts. Metaphors as a reflection of allegedly shared organizational discourse (Cunliffe and Shotter 2006: 135) are used in a range of modes, most notably visual and verbal, which, often in concert, construct the BRANDS ARE PEOPLE metaphor and endow it with particular characteristics. Such multimodal metaphors tend to take particular forms and to cluster in particular genres.

4. Data selection and analysis Parts of this study are based on a corpus of corporate mission statements. These were chosen as representing a companys ideal self, i.e., the basis for projecting corporate brand images externally and internally. The mission statements in question were taken from the websites of the 2003 Fortune Global 500, a list of the largest companies worldwide that is ascribed great importance by the business community. In particular, the texts comprise the top 50 and the bottom 50 companies, thus building a corpus of 29,925 words. In terms of text type, mission statements come under the headings of Our Vision, Message from the CEO (a feature that is particularly popular with Japanese companies), The essence of our company, Strategy or, indeed, Our Mission. By using these headings interchangeably, companies corroborate Collins and Porrass observation of mission and vision statements as a muddled stew of values, goals, purposes, philosophies, beliefs,

52 Veronika Koller aspirations, norms, strategies, practices, and descriptions (1996: 77, quoted in Czerniawska 1997: 185). Despite this heterogeneity, however, we can identify the mission statement as a genre since the various forms it takes share the communicative purpose of conveying the raison dtre, ambitions and central values of an organization to its internal and external stakeholders (Fox and Fox 2004: 32, 43) and thus foster their loyalty to the institution in question. Defining parameters used to achieve this communicative purpose are hyperbole as realized in superlatives (the best customer experience), absolute quantifiers (each, everywhere), inclusive we and high-affinity deontic modality (it has been necessary to do so) (Fox and Fox 2004: 43, 171; Koller 2008; Swales and Rogers 1995). The key word search function of the Wmatrix corpus analysis interface (see Rayson 2008) was used to ascertain lexical items that were, in statistical terms, significantly overused in the mission statements as compared to the written sampler of the British National Corpus. At 5 per cent probability for log likelihood, the list contains a total of 2,102 types. Filtering out proper names and technical terms, the top 15 items to possibly denote central concepts in corporate discourse are: innovation, respect, excellence, integrity, performance, trust, teamwork, responsibility, growth, committed / commitments, creativity, competitive, transparency, professionalism and fairness. In the persuasive genre represented by the texts in the corpus, even the less obvious of these lexical items are found to be evaluated positively when linked back to their co-text, for instance: (1) (2)
We will invest in those businesses that promise the greatest value growth. (Continental AG) The exceptional quality of our workforce is a valuable competitive edge. (Mobil) [My emphases, VK]

We can therefore posit that these lexical items represent the predominant values in corporate discourse, and hence the ideal type traits of a companys brand personality. How then do verbal and visual modes in corporate discourse interact to convey the ideal corporate brand personality? The data collected to answer this question are taken from a range of genres promoting the corporate brand, such as brochures and the corpus of mission statements mentioned above. The texts therefore represent instances of the secondary corporate communication of marketing (Balmer 2006: 37). In general, the components and characteristics of the BRANDS ARE PEOPLE metaphor are seen as conveyed in illustrations, logos and layout, together with their verbal co-text: The graphic designers brief that corporate design reflect the true

Brand images

53

reality of the company means that ideally, if you analyse a companys design you can also trace its identity [and] personality (Johansson and Holm 2006: 140). Although not discussed in this chapter, illustrations, logos and layout, and other low-sensuality artifacts (Pratt and Rafaeli 2006: 281) should be understood as acting in concert with other corporate artifacts in communicating brand concepts. Further, while the present analysis focuses on multimodal metaphors in which verbal components anchor a metaphoric interpretation of visual elements, discursive construction and cognitive conceptualization of brands can equally be realized in different modes, e.g., sounds and music, and the study can therefore be taken as a starting point for empirically testing its claims against further data involving different modalities.

5.

Illustration, logo, layout

In the data, multimodal metaphors with visual and verbal components feature in three ways: Firstly, illustrations on corporate websites support conventional metaphors such as BRANDS ARE LIVING ORGANISMS. Secondly, logos and logo elements tend to visualize particular characteristics ascribed to the BRANDS ARE PEOPLE metaphor as a special case of the LIVING ORGANISMS metaphor. Such characteristics are often expressed using spatial concepts as source domains (e.g., illustrating dynamism, balance, or openness). Thirdly, layout can also visualize source domains of metaphoric expressions that are anchored by verbal elements, e.g., when connecting design elements in the corporate color are used in the brochure of a company that claims to have a networking personality. 5.1 Illustration

As mentioned above, the About sections on corporate websites typically include a companys values, vision, mission, philosophy and strategy, with these terms often being used interchangeably. Ultimately elaborating on the corporate identity and projecting it as corporate brand image, these subsections are a good starting point to ascertain the multimodal metaphoric construction of BRANDS ARE LIVING ORGANISMS and its specific form BRANDS ARE PEOPLE. It is noteworthy how many websites in the corpus contain images of human beings, mostly children, and of nature.

54 Veronika Koller

Figure 1. Hitachi corporate philosophy (http://www.htachi.com/about/vision/index.html, accessed 20 August 2007). Copyright in these images is owned by Hitachi, Ltd. Reproduced with permission.

In the triptych structure of the above illustration (figure 1), a verbal representation of the brand name, Hitachi, against the background of what looks like the inside of a corporate building, appears in the left-hand picture, followed by the low-angle shot of a skyscraper a staple of corporate iconography and the picture of a tree. The latter is over-determined, in that it serves, most obviously, as a literal illustration of Hitachis key strategic theme Sustainable Environmental Solutions. Further, the text is typical of the mission statement corpus in including metaphoric expressions of growth ([Were] aiming for high growth in the global market), and the tree can be interpreted as a visual representation of the source domain in the metaphor FINANCIAL SUCCESS IS NATURAL GROWTH. Finally, the dual visual connection of the brand with culture, as symbolized by the building, on the one hand and nature, as represented by the tree, on the other finds a reflection in the verbal co-text:

Brand images (3)

55

Since its founding in 1910, Hitachi has acted from a corporate philosophy of contributing to society through technology. In the intervening years, the world and society have changed greatly, but we have never lost our pioneering spirit, based on the principles of harmony and sincerity (my emphasis, VK).

By setting up a dichotomy of technology and harmony, the latter is conceptually linked to what one would normally expect as the opposite of technology, namely nature. Not only does the size of the tree then suggest an age equaling that of the company, but it may also be used to visualize the notion of harmony, as linked to nature. In mission statements there is a clear, literal link between harmony and nature, as evidenced by the following examples:
(4) We will shoulder the responsibility to contribute to a sustainable society in harmony with nature. (Asahi) Stay in harmony with nature; blend in with local societies; and put our hearts into creating a more vibrant, richer culture. (NYK Line) To conduct fair and open business operations while acknowledging our social responsibilities and aspiring for harmony with our global environment. (Toppan)

(5)

(6)

If the picture of the tree is indeed meant to metaphorically represent natural growth, with the verbal elements further cueing a link between growth and harmony, then we are dealing with a typical case of partial mappings in promotional genres, in which all negatively connoted knowledge about the source such as the disarray and imbalance brought about by growth are muted (Ungerer 2000). Understood as such, we would be confronted with a multimodal metaphor in which the source domain (NATURAL GROWTH) is represented visually, while the verbal co-text relates another concept (HARMONY) to the source domain, linking both to the target domain of the brand. The relation between visuals and language here makes it possible, if not necessary, to construe a metaphor FINANCIAL SUCCESS IS NATURAL GROWTH. In that sense, multimodal metaphors that are based on the interplay between verbal and visual elements are not unlike pictorial similes (Forceville 1996). This unforced metaphorical interpretation may well serve to make instances like these powerful precisely because the metaphor is neither explicit nor strongly signaled (Forceville 1999).

56 Veronika Koller A similar multimodal metaphor BRANDS ARE LIVING ORGANISMS can be found in the relevant section of IBMs website, with the visual element again acting as a representation of the source domain (figure 2).

Figure 2. IBM values (available at http://www-03.ibm.com/ibm/history/exhibits/ valueone/valueone_intro.html , accessed 20 August 2007). Reprint courtesy of International Business Machines Corporation, copyright International Business Machines Corporation.

The stylized picture of a DNA helix next to the company name here instantiates the BRANDS ARE LIVING ORGANISMS metaphor and represents a companys values as an inherent and inalterable quality. The verbal text includes the source domain only once (to bring IBMs values to life) but explicitly mentions the target domain (what IBM has always been about and always will be about). The juxtaposition of present perfect and future tense points toward the inalterable nature of what is understood to be the brands core and thus foregrounds this particular semantic component of the metaphor. While the text does not include the word DNA as such, the visual representation of the term intertextually draws on an established metaphoric expression in corporate discourse. Witness the following examples:
(7) To be successful at a house, a designer has to have an almost visceral sense of what is now called the brands DNA: the abstract spirit of the woman the clothes are made for, and the design vocabulary to communicate it. (Friedman 2004) We all know that these are highly competitive markets, but I firmly believe that the stellar attributes which are in the DNA of the brands and

(8)

Brand images

57

operating culture will enable the groups businesses to successfully differentiate themselves and to compete effectively through a variety of complementary retail channels. (Simon Fox, chief executive officer at HMV, quoted in Killgren and Callan 2005) (9) [The corporate culture] really has gotten into the DNA, says Ms Considine, who will stay on as executive chairman. I dont say that lightly I used to be a biochemist. (Beales 2006)

In the last example, the metaphoric expression is given particular weight and credibility by linking it to its source domain via the speakers professional background. The genetic metaphor has not escaped the attention of brand theorists, who have raised the question whether such an essentialist notion is still viable given the fragmented consumer identities found in postmodern markets (Csaba and Bengtsson 2006: 124125). On the other hand, essentialist brand identity may be reinforced in corporate discourse to provide orientation and the idea of reliability another key word in the mission statement corpus for consumers. Illustrations on corporate websites suggest that brand values are essentialized as stable and inherent in a brand that is metaphorically conceptualized as a living organism. Corporate logos specify the metaphor to BRANDS ARE PEOPLE, revealing what allegedly inalterable traits are ascribed to such a metaphorical human being. 5.2 Logo

Logos are central elements of corporate brand communication. Mostly multimodal in themselves, they combine written and pictorial signs, sometimes in addition to music in so-called sound logos, i.e., short jingles associated with a brand. Intended to convey a highly condensed form of the brand image, a logo meets cognitive and affective functions, serving as a visual cue for the recall of information previously received In addition to the cognitive responses of identifying the brand and remembering information, the logo can also elicit affective responses, such as recall of pleasurable advertising and positive brand experiences (Pimentel and Heckler 2003: 106). Companies and the designers employed by them are remarkably verbose when it comes to explaining the meaning of their logos to the public. Clearly, the function of logos as instances of condensed brand communication, as well as the ever increasing importance of brands, have led to a previously unseen vanity with respect to the signifiers that organizations themselves

58 Veronika Koller choose to manufacture (Christensen and Cheney 2000: 251). Energy company Totals visual identity portal is a case in point (figure 3):

Figure 3. Total logo. Reproduced with permission.

The companys logo shows the corporate name underneath an open ball made up of interlaced tapes in red, yellow, light blue, and navy. The accompanying text offers the following interpretation: (10)
The logos spherical shape suggests both the Earth and the international scope of our business. Constructed from a set of colored, curving lines, it symbolizes the complexity of an ever-changing world. The logos colors symbolize multiple energies. The colors chosen naturally evoke the idea of heat. The intersecting curving lines convey an impression of movement. The combination suggests concentrated, controlled energies serving heat, light and movement. The logo is airy and light, signifying that energy is more than just a natural resource, but the product of humankinds ingenuity and exchanges.7

This may provide interesting reading for graphic designers but is unlikely to rivet the attention of stakeholders. Indeed, it is questionable whether especially consumers can, in their typically superficial encounters with logos, really appreciate such symbolic meanings, or if they even wish to do so (see Pimentel and Heckler 2003: 126). Grandiose explanations are also offered by car manufacturer Mazda. The Mazda logo features the name underneath a silver circle surrounding a gently curved element that resembles a stylized bird. The company offers the following interpretation on their website:8
(11) The brand symbol expresses Mazdas dedication to continuous growth and improvement. It is a symbolic development of the Mazda M, and shows the company stretching its wings as it soars into the future.

The text goes on to trace the meaning of the brand name to the West Asian deity Ahura Mazda and elaborates the positive values it stands for, before concluding, rather flatly: It also derives from the name of our founder, Jujiro Matsuda.

Brand images

59

Common denominators in the two logo interpretations above seem to be openness, lightness, and movement. While the first two of those values can be literally visualized by open and non-solid circles, static images rely on metaphor for representing movement as well as its associated qualities of energy, activity, and dynamism. A metaphorical element used to this end in many logos is a bow or curve (see figures 4a and 4b).

Figure 4a and 4b. Curved elements in logos. The HVB logo is reproduced with permission. The Vanteon logo is owned by Vanteon and is used with permission from Vanteon.

Be they for banks (HVB), sportswear companies (Nike),9 software consultants (Vanteon) or even not-for-profit organizations such as the British Home Office,10 curved logo elements (swooshes) seem ubiquitous, up to the point of clich (Lindsay 2000). Perceived as conveying global reach [and] impact full spectrums (Brannon Cashion, senior vice president at Addison Whitney, quoted in Lindsay 2000: 204), curved logo elements visually represent one of the key words in the mission statement corpus, global. While this design feature is not in itself metaphoric representing as it does a stylized version of the globe other possible interpretation such as drive and impetus are better candidates. Given that dynamic, proactive, and agile are again key words in the mission statements corpus, curved logo elements may be interpreted as contributing to a multimodal metaphor of dynamic speed. This reading hinges on whether the elements are understood as indexical or symbolic. Swooshes could be seen as varieties of what Kennedy (1982) has called runes, i.e., the action lines used in comics to indicate, among other things, movement. In a later paper (Kennedy, Green, and Vervaeke 1993), such action lines are compared to a trail, as if left by lights on the moving body, like time-lapse photography (p. 247). Understood as such, action lines would be indexical in that they point to a moving object. In composite logos, however, curved elements include the brand name instead of the moving object, making the whole logo a multimodal metaphor that includes the target domain in the verbal and the source domain in the visual mode.11

60 Veronika Koller It has been noted that the logo [has] a significant role in triggering the cognitive frame that a company wishes to be perceived in (Baruch 2006: 182). In corporate discourse, particular frames will be prioritized over others. Thus, characteristics such as dynamic and active are overrepresented at the expense of contemplative and receptive, both verbally as lexical items and visually as logo elements. Given the ubiquity not only of certain logos but also of pervasively used logo elements such as the swoosh, such endlessly repeated and recycled images arguably aspire to conceptual status (Forceville 2006: 390). The mental models of activity and continuous movement have achieved a currency that makes these models and their linguistic and visual expressions almost defining notions of the discourses and other practices of corporations. Due to the prestige and influence in late capitalism of all things corporate, such models become positively connoted in other areas as well and are likely to be desired as identity traits by many consumers. Hence, metaphorically constructing brands as people is intended to appeal to stakeholders and incite them to interact with the brand and co-produce its identity (Csaba and Bengtsson 2006: 124), in the hope of acquiring some of its characteristics by doing so. In how far this strategy can be successful in the face of widespread consumer indifference and even cynicism is an open question. 5.3 Layout

We have seen how the Hitachi website (figure 1) uses visual elements to represent the source domain of the BRANDS ARE LIVING ORGANISMS metaphor. Moreover, the brand name was accompanied by visual elements signifying the brands main characteristics. This device points to a third area of use of multimodal metaphor in corporate branding discourse, namely layout. Not just nameable objects, such as buildings and trees, but also layout can assist in construing a metaphor, even though this strategy may require a visually literate audience in order to be successful. To take but one example, in various samples of the brand communication of banking group HSBC (brochures, advertisements, posters, websites; see Koller 2007) red, one of its corporate colors, acts as a structuring device: Red sub-headings, red frames and red bullet points all serve layout functions in arranging pictorial elements. In one brochure, red lines connect paragraphs about the various places where the company does business. The lines also frame details of easily recognizable, even clichd local symbols such as a Chinese dragon or the Eiffel Tower. As such, the layout of the brochure is

Brand images

61

an example of pictorial grouping, understood as symmetric image alignment, i.e., the spatial organization of pictorial elements in terms of size, orientation or distance (Teng and Sun 2002; see also Teng, this volume). If, as in the HSBC example, the aligned elements depict things of the same kind, their grouping produce[s] the cognitive effect of inviting the viewers to see the depicted entities as belonging to the same category (Teng and Sun 2002: 300). Furthermore, the corporate color red is metonymic in that it stands for the whole HSBC group. At the same time it functions to illustrate the idea that HSBC connects places and people, by dint of the red lines literally connecting the symbols of different localities the company is present in. The metonymy is again cued by its verbal component, as the layout just described can be found in HSBCs Business Connections brochure. Similarly, in their history brochure, the company makes references to its network 24 times and to connection(s) four times in a total of 11,563 words. Curiously, both these central terms feature in nominal form only, yet it could be argued that the vectors12 and lines of the brochures layout are equivalent to action verbs (Kress and van Leeuwen 2006: 46). Another statistically overused word in mission statements is connecting, as in the following instance:
(12) Our brands are connecting with customers in ways that will last a lifetime. (Whirlpool)

Thus, the layout of corporate branding messages is another indication of how all elements metonymically associated with the source domain qualify as potential candidates for a mapping (Forceville 2006: 384/this volume). In integrated communications, the source domain of BRANDS ARE LIVING ORGANISMS as such is provided by illustrations, while logos and layout specify the conceptualization of corporate brands to BRANDS ARE PEOPLE and provide visualizations of that metaphorical persons character traits. Given the persuasive genres involved, such traits will be ideal rather than real, both for the identity of the corporate brand and for that of the stakeholders interacting with it.

6. Discussion and conclusion This leaves us with the question why corporate brands should be personified in the first place. What are the specific affordances of BRANDS ARE PEOPLE that make this metaphor appropriate for the persuasive genres of corporate branding communication? As Lakoff and Turner (1989: 72) note with regard

62 Veronika Koller to poetic metaphor, [p]ersonification permits us to use our knowledge about ourselves to maximal effect, to use insights about ourselves to help us comprehend abstract concepts. Branding messages use personification to aid cognitive processing of the brand concept, compressing the complex system that is a corporation into an abstract bundle of characteristics and making this abstraction graspable by linking it to human personality as the source domain. As far as those personality traits are supposed to be desirable, personification also serves a persuasive purpose. The metaphoric personality traits, or brand values, are communicated by having verbal elements such as key words anchor interpretation of visual elements. Indeed, the pattern to emerge from the above examples is that the target of the BRANDS ARE LIVING ORGANISMS/PEOPLE metaphor is encoded verbally, with the company name in figure 1 being integrated into a visual element. The source domain in all cases is encoded visually, and occasionally in additional verbal form, e.g., in the Hitachi mission statement (figure 1) elaborating on growth, the IBM values statement (figure 2) talking about bringing values to life or in the HSBC Business Connections brochure (my emphasis, VK). Dual encoding of the source domain seems to be less common in logos, but this and other results will have to be tested on a much broader scale. Multimodal encoding has the potential for further persuasion by reinforcing the desirable characteristics in two modes. The aim of persuasive genres is therefore less likely to be met if the visuals were omitted, especially since the positive values that are visualized carry affective components that are intended to reinforce the brand message. Moreover, brand characteristics can reflect readers perceived personality, as shown in this description of the following two (product) brands:
(13) Hachez [chocolate] is the brand of the banker, the businessman in a dark suit and bowler hat. If it was a person, it would be a distinguished elderly gentleman with a distinct feel for quality and indulgence. (Hasso Nauck, chief executive officer at Hachez, quoted in Willenbrock 2005: 103, trans. VK) A volkswagen is like a polite, good-looking person. A nice, friendly attitude, rather unassuming, definitely not a show-off it is a goodnatured car, a bit moody every now and then, just like a human being. A loyal, reliable life partner. (Volkswagen customers quoted in Laudenbach 2005: 87, trans. VK)

(14)

The second of the above quotations is elaborated by the writer of the article, who remarks that his interview partners seem to be describing themselves

Brand images

63

when listing the attributes of their cars. This narcissistic relationship between the consumer and the brand is made possible by first ascertaining social and psychological aspects of the target audience through market research and subsequently constructing the brand personality as mirroring that of the ideal consumer. This conceptualization is effected through marketing, particularly branding messages and their typical features, such as multimodal metaphor. Unsurprisingly, given the persuasive thrust of marketing, the practice is highly manipulative in that it addresses consumers selfimage and exploits potential lack of self-esteem. On a more positive note, sociological research into brand communities (Muniz and OGuinn 2001) has made a case for brands fostering social cohesion among individuals identifying with a brand, rather than fragmenting societies into narcissistic individuals engaging in compensatory brand consumption. Given the nature of logos as condensed branding messages, such symbols obviously play a central role as textual resources for such brand communities (Muniz and OGuinn 2001: 423). The double-edged nature of branding as a corporate discourse practice is captured in its description as a (global) ideoscape [that] provides the ideological basis for the establishment of new meaning systems, new practices and new identity forms for the members of the consumer culture (Askegaard 2006: 98). Although vocal criticism of branding (Klein 2000) has ironically become part of branding itself, companies still have a vital interest in not having their brands demystified. One strategy to mystify brands is naturalization: Verbally and visually, texts promoting corporate brands draw on LIVING ORGANISM metaphors to recast social practices as natural kinds and thus make them less vulnerable to criticism. Rothbart and Taylors (1992) observation that social categories tend to be viewed in an essentialist fashion as homogeneous and inalterable natural kinds is crucial in this context. The question has been raised whether essentialist notions of brands are suitable for postmodern consumer identities (Csaba and Bengtsson 2006: 130131). However, given the persuasive thrust of branding communication, it can equally be argued that brands illusions of stability and inalterability not only respond to public mistrust but actually attract stakeholders who lack orientation in highly fragmented societies, for better or worse. Yet, with promotional texts, genre is key: The designing institution in each case is a company whose ulterior motive in producing and distributing texts is promoting itself as a brand and, ultimately, maximize profits. Audiences can be expected to be aware of, and even cynical towards, the profit

64 Veronika Koller motive (Messaris 1997: 159). Encountering brand images in recognized persuasive genres will therefore constrain interpretation and make uncritical acceptance of the communicated brand image unlikely. In this context, it is crucial to remember that often the viewer is invited rather than forced to understand the visuals as encoding the BRANDS ARE LIVING ORGANISMS/PEOPLE metaphor. Outside the context of the specific instance of brand communication, there would be little to suggest that illustrations, logos or layout should be taken metaphorically; it is only in the given genre that they suggest metaphoric construal, at least to some viewers. Also, the instances discussed above leave some leeway for alternative interpretations of visual elements, e.g., the rooflike protectiveness offered by the Home Office logo (see n. 10). Finally, as indicated above, at least consumers may simply not be interested enough to engage with corporate branding messages on more than superficial terms. To be sure, this may be different for other stakeholders: Branding messages are also directed inwardly, i.e., they communicate the corporate brand identity to employees with the intention to foster employee loyalty and identification (Schultz, Hatch, and Ciccolella 2006: 143). Given the institutional power of the sender vis--vis this particular stakeholder group, employees may be less able to afford the cynicism and/or indifference with which consumers may react to branding messages. Even when employees feel cynical towards their employers brand identity, voicing such cynicism on record is rarely advisable. Given the reports of senior managements attempts to control ethnographic research into corporate discourse (personal communication with Jo Angouri, 13 December 2005), it is therefore difficult to find more than anecdotal evidence for negative or indifferent employee attitudes toward corporate brands. We can therefore expect corporate branding efforts to increasingly be directed inwardly, leading to further corporate self-absorption (Christensen and Cheney 2000) on the part of companies with their metaphorical brand personalities and their multimodal expressions thereof. Notes
1. 2. I would like to thank the two editors as well as Rosario Caballero for their helpful comments on a draft version of this chapter. The embodiment approach to metaphor (Lakoff and Johnson 1999) sees kinesthetics as the foundation of basic, primary metaphoric models that are

Brand images

65

acquired early in life, e.g. GOOD IS UP (Grady 1997). However, note the difference between kinesthetically founded conceptual metaphor as a cognitive phenomenon and gestural metaphor as a surface-level expression of such a metaphoric model. 3. The image of the race could be literal as an illustration of, say, a marathon sponsored by the company in question. A case could be made for sponsoring practices literalizing aspects of the brand personality the company constructs for itself, e.g., that of fastness and speed. 4. Discourse is here defined as the total of texts produced, distributed and received between members of a particular social field, e.g., the business community. 5. In branding and organizational theory, an internally communicated corporate brand image is sometimes referred to as brand identity (Hatch and Schultz 2000: 22). 6. Indeed, the metaphor BRANDS ARE PEOPLE has become so entrenched in corporate discourse that it has been used as a methodological tool. Thus, Davies and Chun (2002) developed a Corporate Personality Scale to measure the difference between internal and external images of organizations. 7. http://www.total.com/identite/portail/en/index.htm, accessed 12 February 2009. 8. http://www.mazda.com/profile/vision/, accessed 12 February 2009. 9. http://www.nike.com/nikeos/p/nike/en_GB/, accessed 12 February 2009. 10. http://www.homeoffice.gov.uk/, accessed 20 August 2007. In its combination of traditional (coat of arms) and modern (curve) elements, the Home Office logo is a good example of the transition from time-honored heraldic symbols to contemporary abstract logos (see Baruch 2006, Frick 1996, Mautner forthcoming, Pimentel and Heckler 2003). The curved element can also be read as reinforcing the word home over which it arches, with both drawing on notions of safety and protection. 11. The metaphoric nature of such logos becomes even more obvious when the curve is interpreted as a movement heavenwards. 12. In Kress and van Leeuwens theory of visual grammar, vectors are formed by depicted elements that form an oblique line, typically a diagonal one that indicates an actor and the goal acted upon. In the HSBC example, connecting lines without an indicator of directionality mean something like is connected to, is conjoined to, is related to (Kress and van Leeuwen 2006: 59).

References
Askegaard, Sren 2006 Brands as a global ideoscape. In Brand Culture, Jonathan E. Schroeder and Miriam Salzer-Mrling (eds.), 91102. London/New York: Routledge.

66 Veronika Koller
Balmer, John T.M. 2006 Corporate brand cultures and communities. In Brand Culture, Jonathan E. Schroeder and Miriam Salzer-Mrling (eds.), 3449. London/New York: Routledge. Barcelona, Antonio (ed.) 2000 Metaphor and Metonymy at the Crossroads: A Cognitive Perspective. Berlin/New York: Mouton de Gruyter. Barthes, Roland 1977 Image, Music, Text. Transl. by Stephen Heath. London: Fontana. Baruch, Yehuda 2006 On logos, business cards: The case of UK universities. In Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael G. Pratt (eds.), 181198. Mahwah, NJ: Lawrence Erlbaum. Beales, Richard 2006 Teaching a gorilla to tap-dance. Financial Times 5 July. Available from http://search.ft.com/searchArticle?page=5&queryText=DNA& y=0&javascriptEnabled=true&id=060705000146&x=5. Accessed 19 September 2006. Cappetta, Rossella, and Dennis A. Gioia 2006 Fine fashion: Using symbolic artifacts, sensemaking, and sensegiving to construct identity and image. In Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael G. Pratt (eds.), 199219. Mahwah, NJ: Lawrence Erlbaum. Carroll, Noel 1994 Visual metaphor. In Aspects of Metaphor, Jaakko Hintikka (ed.), 189218. Dordrecht: Kluwer. de Chernatony, Leslie 2002 Would a brand smell any sweeter by a corporate name? Corporate Reputation Review 5: 114132. Christensen, Lars Thger, and Sren Askegaard 2001 Corporate identity and corporate image revisited: A semiotic perspective. European Journal of Marketing 35: 292315. Christensen, Lars Thger, and George Cheney 2000 Self-absorption and self-seduction in the corporate identity game. In The Expressive Organization: Linking Identity, Reputation, and the Corporate Brand, Majken Schultz, Mary Jo Hatch, and Mogens H. Larsen (eds.), 246270. Oxford/New York: Oxford University Press. Collins, James C., and Jerry I. Porras 1996 Building your companys vision. Harvard Business Review 74: 65 77. Csaba, Fabian F., and Anders Bengtsson 2006 Rethinking identity in brand management. In Brand Culture, Jonathan E. Schroeder and Miriam Salzer-Mrling (eds.), 118135. London/New York: Routledge.

Brand images

67

Cunliffe, Ann L., and John Shotter 2006 Linguistic artifacts in organizing and managing. In Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael G. Pratt (eds.), 119137. Mahwah, NJ: Lawrence Erlbaum. Czerniawska, Fiona 1997 Corporate Speak: The Use of Language in Business. London: Macmillan. Davies, Gray, and Rosa Chun 2002 Gaps between the internal and external perceptions of the corporate brand. Corporate Reputation Review 5: 144158. Elliott, Richard, and Kritsadarat Wattanasuwan 1998 Brands as symbolic resources for the construction of identity. International Journal of Advertising 17: 131144. Fauconnier, Gilles, and Mark Turner 2002 The Way We Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Forceville, Charles 1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Activity 9: 129. 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 1999 The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 179198. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379 402. Berlin/New York: Mouton de Gruyter. 2008 Metaphors in pictures and multimodal representations. In The Cambridge Handbook of Metaphor, Raymond W. Gibbs, Jr. (ed.). 462482. Cambridge: Cambridge University Press. Fox, Renata, and John Fox 2004 Organizational Discourse: A Language-Ideology-Power Perspective. Westport, CT/London: Praeger. Frick, Wolfgang 1996 Corporate Design als Identittsstrategie: ber die Relevanz der Deckungsgleichheit zwischen Selbst- und Fremdbild. [Corporate design as identity strategy: On the relevance of converging self and other images] Ph.D. dissertation, University Salzburg. Friedman, Vanessa 2004 New blood reinvents a venerable DNA. Financial Times 12 October. Available from http://search.ft.com/searchArticle?page=3&query

68 Veronika Koller
Text=DNA&y=0&javascriptEnabled=true&id=041012000957&x=5 Accessed 19 September 2006. Grady, Joseph 1997 Foundations of Meaning: Primary Metaphors and Primary Scenes. Ph.D. dissertation. University of California Berkeley. Harquail, Celia V. 2006 Employees as animate artifacts: Employee branding by wearing the brand. In Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael G. Pratt (eds.), 161180. Mahwah, NJ: Lawrence Erlbaum. Hatch, Mary Jo, and Majken Schultz 2000 Scaling the Tower of Babel: Relational differences between identity, image, and culture in organizations. In The Expressive Organization: Linking Identity, Reputation, and the Corporate Brand, Majken Schultz, Mary Jo Hatch, and Mogens H. Larsen (eds.), 1135. Oxford: Oxford University Press. Holt, Douglas B. 2002 Why do brands cause trouble? A dialectical theory of consumer culture and branding. Journal of Consumer Research 29: 7090. Johansson, Ulla, and Lisbeth S. Holm 2006 Brand management and design management. In Brand Culture, Jonathan E. Schroeder and Miriam Salzer-Mrling (eds.), 136152. London/New York: Routledge. Kapferer, Jean-Nol 2002 Corporate brand and organizational identity. In Corporate and Organizational Identities: Integrating Strategy, Marketing, Communication and Organizational Perspectives, Bertrand Moingeon and Guillaume Soenen (eds.), 175193. London/New York: Routledge. Kennedy, John M. 1982 Metaphor in pictures. Perception 11: 589605. Kennedy, John M., Christopher D. Green, and John Vervaeke 1993 Metaphoric thought and devices in pictures. Metaphor and Symbolic Activity 8: 243255. Killgren, Lucy, and Eoin Callan 2004 HMV names new chief. Financial Times 18 July. Available from http://search.ft.com/searchArticle?page=4&queryText=DNA&y=0& javascriptEnabled=true&id=06071Mautner8003693&x=5. Accessed 19 September 2006. Klein, Naomi 2000 No Logo: Taking Aim at the Brand Bullies. New York: Picador.

Brand images

69

Koller, Veronika 2005 Designing cognition: Visual metaphor as a design feature in business magazines. Information Design Journal and Document Design 13: 136150. 2007 The worlds local bank: Glocalization as a strategy in corporate branding discourse. Social Semiotics 17: 111130. 2008 Our customers embrace us as an essential partner: Corporate brands as socio-cognitive representations. In Cognitive Sociolinguistics: Language Variation, Cultural Models, Social Systems, Gitte Kristiansen and Ren Dirven (eds.), 389418. Berlin/ New York: Mouton de Gruyter. Kress, Gunther, and Theo van Leeuwen 2006 Reading Images: The Grammar of Visual Design. 2nd ed. London/New York: Routledge. Kristiansen, Gitte 2008 Idealized cultural models: The group as a variable in the development of cognitive schemata. In Body, Language and Mind. Volume 2: Sociocultural Situatedness (Cognitive Linguistics Research 35.2), Roslyn M. Frank, Ren Dirven, Tom Ziemke, and Enrique Bernrdez (eds.), 409432. Berlin/New York: Mouton de Gruyter. Kunda, Ziva 1999 Social Cognition: Making Sense of People. Cambridge, MA: MIT Press. Lakoff, George, and Mark Johnson 1999 Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books. 2003 Metaphors We Live By. 2nd ed. Chicago: University of Chicago Press. Lakoff, George, and Mark Turner 1989 More Than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. Lassen, Inger, Strunck, Jeanne, and Torben Vestergaard (eds.) 2006 Mediating Ideology in Text and Image. Amsterdam/Philadelphia: Benjamins. Laudenbach, Peter 2005 Wagen ohne Volk. [Volkswagen: a car without people] brand eins 8: 8490. Lindsay, Greg 2000 I want a unique logo just like theirs. Fortune 24 July: 204205. Mautner, Gerlinde forthc. The spread of corporate discourse to other social domains. In Language and the Market, Helen Kelly-Holmes and Gerlinde Mautner (eds.). Basingstoke: Palgrave Macmillan.

70 Veronika Koller
Messaris, Paul 1997 Visual Persuasion: The Role of Images in Advertising. London/ Thousand Oaks, CA: Sage. Morgan, Gareth 1986 Images of Organization. London/Thousand Oaks, CA: Sage. Muniz, Albert M., and Thomas C. OGuinn 2001 Brand community. Journal of Consumer Research 27: 412432. Musolff, Andreas 2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21: 2338. Phillips, Barbara J. 2003 Understanding visual metaphor in advertising. In Persuasive Imagery: A Consumer Response Perspective, Linda M. Scott and Rajeev Batra (eds.), 297310. Mahwah, NJ: Lawrence Erlbaum. Phillips, Barbara J., and Edward F. McQuarrie 2004 Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing Theory 4: 113136. Pimentel, Ronald W., and Susan E. Heckler 2003 Changes in logo design: Chasing the elusive butterfly curve. In Persuasive Imagery: A Consumer Response Perspective, Linda M. Scott and Rajeev Batra (eds.), 105127. Mahwah, NJ: Lawrence Erlbaum. Pratt, Michael G., and Anat Rafaeli 2006 Artifacts and organizations: Understanding our objective reality. In Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli and Michael G. Pratt (eds.), 279288. Mahwah, NJ: Lawrence Erlbaum. Rayson, Paul 2008 Wmatrix: A web-based corpus processing environment. Computing Department, Lancaster University. Available from http://ucrel.lancs. ac.uk/wmatrix/. Rothbart, Myron, and Marjorie Taylor 1992 Category labels and social reality: Do we view social categories as natural kinds? In Language, Interaction and Social Cognition, Gn R. Semin and Klaus Fiedler (eds.), 1136. London/Thousand Oaks, CA: Sage. Rumbo, Joseph 2002 Consumer resistance in a world of advertising clutter: The case of ad busters. Psychology & Marketing 19 (2): 127148. Schultz, Majken, Hatch, Mary Jo, and Francesco Ciccolella 2006 Brand life in symbols and artifacts: The LEGO company. In Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli

Brand images

71

and Michael G. Pratt (eds.), 141160. Mahwah, NJ: Lawrence Erlbaum. Schwartz, Jonathan 2006 The five founding principles that drive innovation. Financial Times September 13. Available from http://www.ft.com/cms/s/17cd2eae42c4-11db-8dc3-0000779e2340.html. Last accessed 11 February 2009. Soenen, Guillaume, and Bertrand Moingeon 2002 The five facets of collective identities: Integrating corporate and organizational identity. In Corporate and Organizational Identities: Integrating Strategy, Marketing, Communication and Organizational Perspectives, Bertrand Moingeon and Guillaume Soenen (eds.), 1334. London/New York: Routledge. Swales, John M., and Priscilla Rogers 1995 Discourse and the projection of corporate culture: The mission statement. Discourse and Society 4: 223242. Teng, Norman Y., and Sewen Sun 2002 Grouping, simile, and oxymoron in pictures: A design-based cognitive approach. Metaphor and Symbol 17: 295316. Ungerer, Friedrich 2000 Muted metaphors and the activation of metonymies in advertising. In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 321340. Berlin/New York: Mouton de Gruyter. van Dijk, Teun A. 2002 The discourse-knowledge interface. In Critical Discourse Analysis: Theory and Interdisciplinarity, Gilbert Weiss and Ruth Wodak (eds.), 85109. Basingstoke/New York: Palgrave. 2005 Contextual knowledge management in discourse production: A CDA perspective. In A New Agenda in (Critical) Discourse Analysis: Theory, Methodology and Interdisciplinarity, Ruth Wodak and Paul Chilton (eds.), 71100. Amsterdam/Philadelphia: Benjamins. van Leeuwen, Theo 1999 Speech, Music, Sound. London: Macmillan. van Leeuwen, Theo, and Ruth Wodak 1999 Legitimizing immigration control: A discourse-historical analysis. Discourse Studies 1: 83118. Wee, Thomas Tan Tsu 2004 Extending human personality to brands: The stability factor. Journal of Brand Management 11: 317331. Willenbrock, Harald 2005 Der Unterschied. [The difference] brand eins 2: 98103.

Chapter 4 Cutting across the senses: Imagery in winespeak and audiovisual promotion Rosario Caballero

Abstract
The present chapter surveys some of the metaphors recurrently used in wine promotion. The starting point is the tasting note, a genre that basically deals with translating a sensorial and highly subjective experience into comprehensible and, above all, shareable terms. This endeavor implies the abundant use of figurative language. If we take into account that tasting notes are the verbal translations of organoleptic experiences (i.e., perceived by a sense organ), the issue here is whether the metaphors used are monomodal or multimodal according to the definition of multimodal metaphor in the present volume. Some of the metaphorical frames in winespeak are also exploited to promote wine in commercials and print advertisements particularly in the latter. Both genres show that imagery may encompass modes other than language but, at the same time, here the difficulties of communicating smell and taste are particularly conspicuous. Indeed, should we have to choose one mode of expression for this particular endeavor, wine advertising points to language as the best albeit also limited tool at our disposal. In this regard, my second aim in this chapter is to discuss the problems derived from attempting to translate verbal metaphors into images in wine advertising the metaphors in this case being unquestionably multimodal and the weight of culture in using and, above all, interpreting figurative phenomena rendered via images. Keywords: Winespeak, sensory perception, culture, tasting note, advertising genres, cross-modal metaphor

1. Introduction Wine is becoming a cultural icon in an emerging hedonistic sub-culture accessible to an ever larger number of consumers. Among the genres articulating the discourse of wine, the tasting note (henceforth, TN) has played a

74

Rosario Caballero

critical role in introducing people to wine-tasting procedures and language and, at the same time, has critically contributed to promoting wine. And, yet, irrespective of the growing popularity of this beverage, wine jargon (winespeak) still retains some of the mystique traditionally associated with the topic a mystique that partly rests on the use of figurative language. One of the reasons underlying the figurative quality of winespeak is the shortage of terms available to describe two crucial sensory experiences in wine tasting, namely, smell and taste. Thus, their verbalization usually involves using metonymies (ripe flavors), similes (it smells like a barnyard), and synesthetic metaphors (it smells crisp). In other words, since there is no single lexicon with the expressive potential to cover aroma and flavor nuances, their communication is inextricably linked to metaphor (in the broad sense of the term). However, hitherto there are few studies devoted to exploring metaphor in wine discourse (although see Lehrer 1975, 1983, 1992, 2006; Peynaud 1987; Bruce 2000; Amoraritei 2002; Gluck 2003). In this regard, one of my aims in this chapter is to survey the figurative language used by wine critics in TNs. Particular attention will be paid to the crosssensory/modal quality of such language in the genre and, hence, the metaphors motivating it. For multimodal metaphors I have adopted Forcevilles definition as those whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006: 384/this volume), yet have broadened its scope to some extent. For if we take into account that TNs are the verbal translations of organoleptic experiences (i.e., those perceived by a sense organ), the metaphors involved may be equally seen as cutting across various senses and/or modes of expression i.e., as being multimodal in the broadest sense of the term. Some of the metaphorical frames in winespeak are also exploited to advertise wine in commercials and print advertisements particularly in the latter. Both genres show that imagery may encompass modes other than language (as discussed throughout this book) yet, at the same time, here the difficulties of communicating smell and taste are particularly conspicuous. In other words, it is one thing to acknowledge the heuristic role of metaphor in wine discourse, and it is another to ignore the limitations of the modes in which metaphor is instantiated for sharing sensory experiences with a given audience. Indeed, should we have to choose one mode of expression for this particular endeavor, wine advertising points to language as the best albeit also limited tool at our disposal. In this regard, my second aim in this chapter is to discuss the problems derived from attempting to translate verbal metaphors into images in wine advertising the metaphors in this case being unquestionably multimodal.

Cutting across the senses

75

In the present discussion, major insights have been drawn from an ongoing research project devoted to exploring metaphor in a corpus of 12,000 TNs and 100 print advertisements retrieved from American and British wine publications. However, for this chapter I have also used a small corpus comprising texts and adverts in Spanish and French.1 The main reason for including these lies in the fact that adverts need not necessarily be translated into the language of the specialized magazine inserting them (French being a case in point given its former status as the language every oenologist and wine lover should master). Moreover, although the metaphors and metonymies underlying winespeak largely cut across languages and cultures (Nedilko 2006), this is not always the case when images are concerned, or when images and words combine in wine adverts. In this regard, comparing how wines from different countries are advertised may bring to the fore the weight of culture in using and, above all, interpreting figurative phenomena rendered via images. A clarification concerning my use of the term culture in this chapter is in order at this point. Thus, culture covers both the shared beliefs, knowledge and world view(s) characterizing national groups as well as the more specific beliefs, knowledge and view(s) of the community articulated around wine in the latter case, culture cutting across national or regional differences and foregrounding the importance of topic and shared interests in building up discourse communities or cultures within cultures. The chapter is organized as follows. It starts with a survey of the figurative language used by wine critics in TNs. This is followed by a discussion of the cross-modal or multimodal quality of such language and its possible relationships with other figurative phenomena. Then I discuss the use of imagery in advertising genres, pointing to their heavy dependence on the verbal mode and the difficulties arising from this fact.

2. Figurative language in tasting notes The tasting note is one of the most representative and popular genres in wine discourse, as well as a key instrument in the process of wine acculturation. TNs are short texts (from 20 to 200 words) devoted to describing and evaluating wine, and are prototypically organized in three distinct sections that capture the three canonical steps in any wine tasting procedure, namely, the assessment of wines (a) color, (b) smell metonymically referred to as the wines nose/nariz/nez in English, Spanish, and French respectively, and (c) mouth-feel which subsumes smell, taste, and touch, and is metonymically referred to as the wines palate/boca/bouche.

76

Rosario Caballero

Another conspicuous characteristic of TNs is their heavy reliance on figurative language, which, as happens with their rhetorical structure, is shared by texts written in the three languages under survey. The figurative repertoire illustrated in the genre draws upon diverse metaphorical frames, among which the most salient portray wines as if they were LIVING ORGANISMS, TEXTILES, or THREE-DIMENSIONAL ARTIFACTS. (As discussed later, this does not mean that wine critics actually equate wines with such entities, but the labels may be helpful for surveying the most salient domains involved in wine commentary and will, therefore, be used throughout the chapter.) Textile renderings of wines in English are conveyed through reference to wine elements as their cloak, glove, frock, or mantle, that is, as PIECES OF CLOTHING which may well suggest a personified view of wines. Nevertheless, textile terms are mostly used either to describe the structural properties of wines (e.g., these have a fabric or weft, may burst at their seams, and can be variously described as a tapestry, open-knit, well meshed, or tightly wound) or to evaluate their feel in the tasters mouth (e.g. silky, velvety, satiny, pillowy). In contrast, Spanish TNs yield fewer and less varied textile expressions, although we also find nouns like capa (robe) used to refer to the intensity of a wines color, adjectives such as aterciopelado (velvety) or sedoso (silky) qualifying wines mouth-feel, and descriptions of tannins as cloth that tapiza(n) la boca (literally, carpet the mouth). Finally, French critics also refer to a wines color as its robe (the term being, in fact, borrowed by Spanish and English), and use terms like appret (starch), soyeux (silky), and velout (velvety) to assess the mouth-feel of wines, and toff to qualify a full-bodied wine.2 Textile metaphors are illustrated in the following examples:
(1) (2) (3) (4) Blackberry jam and pepper were well integrated into the fabric of this wine A monster in a beautiful frock. loads of velvety tannins. Los taninos [del vino] son puro terciopelo. [The tannins are pure velvet] Trs belle robe rouge pourpre quilibre d aux tannins fins et soyeux. [Very beautiful red purple color/robe balance due to fine and silky tannins] En bouche, il se dploie comme un tapis de velours sur lequel dansent les armes. [In the mouth, it unfurls like a tapestry on which aromas dance]

(5)

Cutting across the senses

77

The metaphorical frame WINES ARE THREE-DIMENSIONAL ARTIFACTS may be further decomposed in two related frames. On the one hand, we find wines described as GEOMETRICAL ENTITIES which, accordingly, can have edges (Sp. puntas), layers, contours, backs, and fronts, and are often evaluated as being square (Fr. carr), angular (Sp. anguloso, Fr. anguleux), wide (Fr. ample), long (Sp. largo, Fr. long), pointed (Sp. afilado), deep (Sp. profundo, Fr. profond) or round (Sp. redondo, Fr. rond). Indeed, according to the French wine expert Peynaud, the ideal wine should be round or spherical (Peynaud 1987). On the other hand, some terms suggest a view of wines as BUILDINGS. In English, this metaphorical frame is evoked by reference to wines as edifices or monuments which are constructed, built, buttressed, backed up or fortified by all or some of their constituents, or by evaluative terms such as monumental, massive, monolithic, foursquare or skyscraper-like. Although this architectural frame appears to be less productive in Spanish and French, terms like armado (reinforced), construido (built), and ensamblado (assembled) in Spanish, and French charpent (well structured/built)3 suggest that critics also draw upon the domain of architecture to discuss the structural properties of wines see also Lehrer (2006) in this respect. By way of illustration, consider the following passages:
(6) (7) (8) (9) Solidly built yet balanced [wine]. The fruit [in this wine] barely peeks through the wall of chewy tannins on the finish. Bien armado, con cuerpo y taninos firmes [Well assembled, with body and firm tannins] La bouche a des tanins matures, bien fondus mais qui donnent une architecture bien dfinie. [The mouth has ripe, well integrated tannins which provide a well defined architecture]

All in all, however, the most salient metaphorical frame in the three languages at issue is WINES ARE LIVING ORGANISMS. This is far from surprising since wine is a mutable entity resulting from an organic process: the juice extracted from grapes changes considerably along its life inside both oak casks and bottles a process referred to as breeding or ageing (Sp. crianza, Fr. levage) as well as during the very act of drinking it. Thus, among the many terms used to describe wines evolutionary state we find forceful (Sp. vigoroso, Fr. vigoreux), weak (Sp. dbil, Fr. faible), youthful (Sp. joven or juvenil, Fr. jeune), tired (Sp. agotado, Fr.

78

Rosario Caballero

fatigu), or old (Sp. viejo, Fr. vieux or senile). Nevertheless, this organic frame is not only concerned with wines physiological dimension, but is also used to talk about its structural, behavioral, and social properties. Wines structural components are frequently described by means of anatomically biased lexis. For instance, the trickles on the inside of a glass indicating the alcohol concentration of a wine are conventionally referred to as its legs or tears (Sp. lgrimas, Fr. jambes or larmes), and the general effect or weight of a wine in the tasters mouth is referred to as its body (Sp. cuerpo, Fr. corps). Accordingly, wines can be described as light-bodied (Sp. con cuerpo ligero, Fr. lger) and full-bodied (Sp. con cuerpo, Fr. cors) or, more conspicuously, as brawny (Sp. musculoso, Fr. muscl), fat (Sp. graso, Fr. gras), flabby (Fr. toff), fleshy (Sp. carnoso, Fr. charnu), lean or slim (Fr. mince), or thin (Sp. con poco cuerpo, Fr. maigre). Two other common terms in wine description are masculine (Sp. masculino or viril, Fr. viril) and feminine (Sp. femenino, Fr. fminin).4 The behavioral properties of wines are usually assessed by lexis typical of the human realm. This personifying view of wines is cued by adjectives such as aggressive (Sp. agresivo, Fr. agressif), pretty-handsome (Fr. joli), in-your-face, upfront, or honest (Sp. franco, Fr. franc), sexy, demure or shy (Sp. hurao, Fr. rserv), or expressive (Sp. expresivo), suave (Sp. amable, Fr. aimable), or civilized (Sp. civilizado). Finally, English terms like blue blood, clone, pedigree, sister, mate, sibling or peer, and Spanish primognito (firstborn) or hijo (son) foreground the social and kinship dimensions of wines. The examples below illustrate several instantiations of this anthropomorphic frame:
(10) The big, bold, and sultry 2001 Chevalier-Montrachet is a highly expressive, fleshy, supple wine This white chocolate, truffle, cream, and spice-scented beauty is medium-bodied, concentrated, and sexy. [This wine] has a nicely buried backbone of acidity and tannin. ultimate impression of both muscle and flesh. Like its older sibling, [this wine] will be delicious in its first 34 years of life. Un gran tinto de la Ribera, con casta y finura. [A great red from Ribera, with breed and finesse]

(11) (12) (13)

Cutting across the senses (14)

79

[Este vino] es una bestia encerrada de gran complejidad. Nariz poderosa muy compleja. En boca es musculoso, tiene cuerpo, estructura, potencia carnosidad todava es muy joven Posee raza y fuerza. [a caged beast of great complexity powerful and very complex nose. In mouth it is brawny, with body, structure, power, flesh it is still very young has breed and strength] Nez expressif, dj bien ouvert Joli nez, assez complexe. Attaque franche et volumineuse. Joli potentiel de vieillissement. [Expressive nose, already well open Beautiful nose, quite complex. Straightforward and voluminous Beautiful potential for aging] Jambes colores. Trs beau nez Trs belle saveur fruite. ... Charnu. Du corps. ... Trs beau vin. [Colored legs. Very beautiful nose Very beautiful fruity flavor Fleshy. Full-bodied. Very beautiful wine] Rserv, complexe, ferme, charnu, compact et pre, dune longueur ternelle, frache et race: un grand vin encore dans les limbes qui ne dvoilera sa vraie nature que dans dix ans. [Reserved, complex, firm, fleshy, compact and rough, incredibly long, fresh and thoroughbred: a big wine still in a dream world which will reveal its true character in ten years]

(15)

(16)

(17)

As pointed out earlier, one of my aims in this chapter is to show that, together with showing the heuristic role of figurative language in conveying disparate sensory experiences, the passages seen so far also point to the multimodal quality of wine critics commentary. This is the topic of the next section.

2. Metaphors in winespeak: Monomodal or multimodal? A core assumption in cognitive approaches to metaphor is that abstract thinking is heavily determined by the functioning of the human body (Lakoff and Johnson 1999). Simply put, our concrete experiences in and with the world provide the basic data for understanding abstract, non-concrete concepts. Nevertheless, although helping our understanding of the most abstract via the most concrete is one of the most salient properties of metaphor, this does not rule out the concreteness of both the source and target in certain metaphors. A case in point is image metaphor, which involves the mapping of concrete, topological information across disparate domains (for a detailed discussion on this type of metaphor see Caballero 2006). Another conspicuous albeit often neglected trait of metaphor is that it can be rendered in

80

Rosario Caballero

modes of expression other than language (e.g. pictures, sounds, or gestures) as well as involve various modalities or senses (e.g. vision, hearing, touch, or taste). An extreme example of the latter is synesthetic metaphor, which maps information across sensory domains or modes (Day 1996; Ramachandran and Hubbard 2001; Yu 2003). Both the concreteness and multimodal quality of some metaphors are important for discussing the figurative language used by critics in TNs. Thus, the raison dtre of the genre is to put olfactory and gustatory experiences into words. Yet, although smell and taste are two of the most basic and firsthand i.e., concrete human experiences, the verbalization of the subtle differences among the vast array of aromas and flavors at our disposal is no easy matter. Figurative language compensates for the poverty of the lexicon in this respect: in general, the expressions used in wine commentary denote entities and qualities concrete enough to be of use in this difficult context. Most expressions, however, defy Lakoff and Johnsons claims that human beings find phenomena they can see, hear, feel, taste and/or smell easier to understand and categorize than phenomena they cannot. It is perceptibility that makes the former phenomena concrete, and the lack of it that makes the latter abstract (1999: 249). In fact, very often the situation in winespeak seems to be the reverse: wine critics need to draw upon experiences other than smell and taste to categorize and verbalize these. In turn, smell, taste, and language (itself expressible in various ways) represent three different modalities or modes, which leads to a view of the metaphors underlying winespeak as intrinsically multimodal rather than conceptual in the sense the latter term is used in mainstream cognitive linguistics. Take, for instance, terms like nose, body or mute, all of which have been discussed as rendering a human view of wines personification being a canonical example of conceptual metaphor in the literature. However, wine critics use them to describe sensory experience: as has been noted, nose refers to a wines aroma(s), body refers to the weight of a wine as sensed in the palate, and mute or expressive have nothing to do with the articulatory properties of the wine at issue, but mean that the elements making up a wines bouquet and/or flavor are difficult or easy to perceive. Of course, the description of wine as a living organism or, more specifically, as a human being in wine discourse is fully congruent with the physiological properties of that beverage. Nevertheless, many other terms apparently drawing from the human domain as well as expressions borrowed from the textile or architectural domains are, in fact, used to express the sensations provoked by wine in the drinkers nose and mouth i.e., involve perception through the senses rather than abstract concepts or quali-

Cutting across the senses

81

ties. This does not mean that some of the metaphors cannot through repeated use in textual interaction be further extended and, thus, conform to the metaphorical frames discussed earlier. Yet, regarding such frames as conceptually rather than cross-sensually concerned would betray the nature of the experience metaphorically portrayed and communicated. Indeed, the complexity and interest of the figurative expressions discussed so far arises from the fact that their underlying mappings seem to involve different modes. Thus, whereas many of the targets of the metaphors are smell(s), taste(s) and mouth-feel (i.e., physical, perceptual experiences via sense organs), the sources concern both concrete (three-dimensional artefacts or the human body) and abstract (kinship) entities and domains an idiosyncrasy worth investigating, yet which falls outside the general aims of the present volume. Nevertheless, since both the source and target in the figurative expressions are rendered verbally, hence in the same communicative mode, these belong to the monomodal type of metaphor in Forcevilles (2006) sense. Having said this, let us have a look at wine advertisements, and assess whether these contain truly multimodal metaphors, that is, metaphors whose target and source are (primarily) rendered in different modes (e.g., via images and language).

3. Imagery in wine advertising Wine advertising is peculiar in a number of ways. In the first place, it is fairly conservative in general, particularly when compared with commercials and adverts promoting other consumer goods. For one thing, most wine adverts show the product in a very straightforward (literal) manner so that audiences cannot be mistaken as to the brand at issue the underlying assumption being that the brand alone sells. Moreover, when a given winery enjoys a high status among wine lovers, adverts usually consist of a close-up of wine labels which suggests that their purpose is to remind readers of the wines existence rather than persuading them to buy the product (i.e., raise brand awareness and loyalty). A final conspicuous characteristic of wine advertising is its heavy reliance on language: when elaborate or innovative images are used, they appear to be but illustrations of what is expressed verbally. Metaphorwise the word-image combination in wine adverts is challenging for several reasons. In many cases, images may be seen as figurative only when their accompanying slogans are also taken into account. Pictorial metaphors may also seriously contradict what is being communicated via the

82

Rosario Caballero

verbal metaphors in the accompanying body text or, what is worse, show the literal referent of the entity denoted by a given figurative expression or term irrespective of its relevance and/or meaning in the wine realm (e.g., adverts exploiting anthropomorphic metaphors, as discussed in section 3.1.). This, in turn, suggests that understanding the pictorial metaphors in wine adverts is no easy matter, and nearly always requires acquaintance with wine discourse in general and, of course, with the figurative language used by the community of wine lovers. As already pointed out, the approach to multimodal metaphor followed in this chapter is the one provided by Forceville (2005, 2006, 2007). Two important steps in Forcevilles scheme are (a) to establish the domains involved in the metaphors which, in turn, implies determining whether the metaphors favor certain source domains, and (b) to see which types of multimodal metaphor are illustrated in the data under analysis. These questions are addressed in the following sections. 3.1. Image-language combination Given the fierce competition in the sector and the overuse of certain procedures to enlarge production, most wine adverts are particularly concerned with explaining the winemaking process followed by the winery so that their products achieve the sought-after sense of terroir (i.e., how the wine exhibits the characteristics of the vineyard site). Unsurprisingly, the adverts thus oriented favor the anthropomorphic frame outlined earlier, playing up the kinship and physiological aspects of the wine at issue in order to highlight its pedigree or lineage. When the metaphor is used for this purpose, it is mostly instantiated through language. A nice example of this is a Spanish commercial designed for Navarra wines (one of the Spanish guarantees of origin and quality of wines), where the audience is shown a middle-aged man who appears to be talking to his departing son in the following terms:
Youre leaving tomorrow. The world awaits you and Im sure that when they know you theyll love you. Were proud of you at home. We know that youll become one among the great. Some wont appreciate your worth: pay no attention. Mmm Others will see a winner in you: dont boast. Dont be afraid of time: it will make you mature and, when you grow old, do it with dignity. Always remember the land where you were born. Son, Ive given you everything I know. Ill miss you. OK, enough. You have a long trip tomorrow. NAVARRA REDS, SONS OF OUR LAND. [Translated from Spanish, RC] Bassat. Reproduced with permission.

Cutting across the senses

83

Only after the last sentence we are shown the son thus addressed: a bottle of wine resting on a table. The advert is true to its purpose, namely, advertising a wine-producing region rather than a particularly winery. Accordingly, the bottle bears no label and the camera is solely concerned with the father, who metaphorically stands for the region. Of course, the language used is unmistakably figurative, yet we are aware of this only after we are shown the bottle-son at the very end of the commercial this delay being a common advertising strategy that seeks to surprise audiences. The commercial is a true example of multimodal metaphor in that the target (Rioja wines) is rendered visually and the source (son) is rendered verbally.
Being the latest generation in a line of truly great wines means that you have evolved from very solid roots. You have also grown up sheltered by the knowledge of one family. You are loyal to one name only Muga, whose tradition and character allow you to grow even further and develop new forms of art, hence Torre Muga. Torre Muga comes from only the best vintages and is pampered throughout its traditional winemaking process. [] [translation from Spanish by RC].

Figure 1. Print ad for the Spanish Torre Muga winery. WINES ARE PEOPLE

Personification is also exploited in the following three examples, all of which promote specific brands and attempt to render a human view of wines via language and images. Figure 1 shows several products of a well-known Spanish winery, yet is particularly concerned with promoting the latest product as explicitly acknowledged throughout the text namely, Torre Muga. Following the aforementioned prototypical technique, the excellence

84

Rosario Caballero

of this wine as well as that of its predecessors is warranted by describing the procedures followed in their making. The situation here is somewhat different from that of the advertisement discussed above: although personification is mainly cued by language, we could contend that the wines shown in the advert pose for the audience as in a family portrait. Moreover, since the advert is particularly concerned with promoting Torre Muga, this is foregrounded in the picture (i.e., it has been moved to the front and appears larger than the other two).6 However, the question remains as to whether the image would be regarded as metaphorical on its own i.e., without the accompanying text. The role of language is also paramount in the following two adverts, both exploiting personification as well, yet in subtly different ways.
With wines it is the other way round. Only if you have grown up/been raised in a basement/cellar within an oak cask, you are of noble birth [translation from Spanish by RC].

Figure 2. Advertisement for the Spanish Montecillo wine: WINE IS BABY.

The Spanish wine in figure 2 is portrayed as a human being by language focusing on its birth, raising and noble origin. The image does, indeed, reinforce the verbal metaphor: both the wicker recipient and the white cloth resemble a cradle, which helps see the bottle as a newborn baby.5 However, should we ignore the text, the image would not be seen as metaphorical but, rather, as literally showing the way wines are usually presented at the table in any restaurant. Likewise, the following French advert plays with the idea of wines being born. The slogan used here is more complex and playful. Thus, although the

Cutting across the senses

85

literal translation is We show you our latest nose, the term nez may also be understood as born should we read the slogan aloud (nez and n being homophones in French).

Figure 3. Advertisement for Chatau Loville Poyferr

The image in figure 3 may be read in at least one of two ways. On the one hand, it may be seen as a very theatrical way of introducing the wine to an audience due to the low angle perspective of the picture: this shows a veil going up just as curtains do in theatres the genres conventions allowing for this dramatic introduction since nobody would do this with a wine. On the other hand, if we use our imagination the veils wrapping the wine may well be seen as resembling the delicate cloth that prototypically covers newborn babies which would be commensurate with the playful use of nez/n. However, although using the term nose to refer to the wine as a whole may reinforce this human portrayal, the physiological, infantile traits suggested by the second meaning get lost. Moreover, nothing in the image suggests that it is the wines aromas that appear to be highlighted in the advert (the nose of wines being, indeed, one of their most distinctive and praiseworthy traits) even if this is precisely what the term nose means in winespeak. Going back to the questions posed at the beginning of this section, the adverts seen so far illustrate the penchant of advertisers for anthropomorphic metaphors. They also show the usefulness of this frame to foreground traits

86

Rosario Caballero

of both product and producer, particularly the aforementioned questions of lineage, nobility, ageing etc. Finally, should we need to determine the type of metaphor instantiated in the adverts following Forcevilles (1996) scheme they would be close to what he calls verbo-pictorial metaphor, that is, a metaphor whose source is visually represented and the target is verbally represented or vice versa. Nevertheless, the exploitation of anthropomorphism in wine advertising is not always so successful despite its weight in wine commentary. This is particularly conspicuous when adverts focus on a concrete sensory trait of the wine at issue such as its flavor, bouquet, texture, length (i.e., aftertaste), etc., and combine both anatomically-biased expressions with images of human body parts to do so. Adverts playing on the term body are very revealing in this respect. This is the case in a Spanish campaign which advertises the same wine by showing either a naked male body or a female one under a huge label reading Body.7 This image-word combination is infelicitous for two reasons. In the first place, the only trait articulated by body in winespeak is the weight of wines as perceived in the mouth all other properties of human bodies being totally irrelevant in this particular context. In other words, the tactile experience encapsulated in the jargon term body (however unrelated to touch the term may seem to the layperson) gets lost in the image, which can be interpreted in a number of ways. Moreover, the indiscriminate use of male and female bodies to promote the same product deviates from the way the adjectives masculine and feminine are used in wine discourse since both are used to describe dramatically different wines. Among the few exceptional cases where advertisers do try to convey the sensory properties of wines via images I found the following one:
() They say that then it started swirling, slowly first releasing berry notes then faster its pink freshness turning violet. And some say its eyes were those of a woman () [translation from Spanish by RC].

Figure 4. Advertisement for Gran Feudo Chivite wine: SWIRLING WINE IS SWIRLING BALLET DANCER.

Cutting across the senses

87

In Figure 4 we also find language and image side by side; yet, in this particular case, the image does not need the help of language to be understood. Thus, the advertisers have used well-known ballet dancer Tamara Rojo to visually convey the swirling of a wine within a glass i.e., the necessary first step to release its aromas and assess its nose. Moreover, swirling also helps tasters grasp the first impression of wines texture (remember the aforementioned use of legs or tears in this regard) before this is finally assessed inside their mouths. In other words, the dancers swirling is used to represent both the wines swirling and its texture. Given the characteristics of the dancers dress (which, as ross, is pink), this texture appears to be assessed as fresh, light, smooth and soft or supple. All such traits are characteristic of ros wines like the one here advertised, draw upon tactile experiences typically felt outside rather than inside the mouth (i.e., through skin contact) and, therefore, may be seen as cases of synesthesia particular to the wine domain, and usually co-occur with one of the most conventional figurative terms used in texture assessment: silky. Of course, the image may be interpreted as also illustrating the metaphor A WINE IS A BALLERINA, yet all in all it is the releasing of aromas and the wines texture that are at stake in the advert. As it is, equating a wine with a ballerina may, indeed, reinforce the aforementioned traits of the wine at issue. In sum, the advert not only is congruent with some of the figurative terms used by expert tasters in TNs, but, according to an informal discussion held with some wine connoisseurs, also appears to successfully convey them in visual form. Together with illustrating some of the problems derived from attempting to represent the metaphors in winespeak in visual form, the adverts shown so far may be used to discuss the pertinence of culture in metaphor interpretation whether metaphor is represented verbally, visually, or both.

3.2. Culture and wine advertising As Nedilko points out, the terminology of viticulture and winemaking is marked by globalization and the interpenetration of different cultures, which leads to a large number of borrowings and neologisms (2006: 138). This is also the case with the figurative frames motivating a large number of the lexical repository in the wine realm. Yet, however useful this uniformity may be for discussing wine in some genres (e.g., the TN), this does not mean that cultural idiosyncrasies do not also slip in verbal commentary or that they cannot be exploited for particular purposes in other genres. Indeed,

88

Rosario Caballero

images are good exponents of the weight of culture in both producing and interpreting messages whether this is done through metaphors or otherwise. Thus, the same image may mean totally different things to people immersed in Western culture, a national culture within this broader frame, or a more specific community built upon shared professional or ludic interests and, more often than not, cutting across national differences. This is also postulated by Forceville, who acknowledges that the old adage that a picture tells more than a thousand words should not blind us to the fact that pictures and other multimodal representations seldom communicate automatically or self-evidently. As in verbal metaphors, it is connotations rather than denotations of source domains that get mapped in metaphors, and these may substantially differ from one (sub)cultural group to another (2006: 389). Although I fully agree with Forcevilles views, I think that knowledge of the target domain and, above all, of the conventions, beliefs, needs, etc. of the community whose metaphorization of the world is scrutinized should not be underestimated either. Before taking this point further, let us consider two other wine adverts.

Figure 5. WINES ARE JEANS

Figure 6. Osborne arrives at Malpica.

The first advert (figure 5) belongs to a campaign promoting the wines of Ironstone Vineyards (California, US). Here we have a comparison between wines and jeans, a simile rendered in pictorial form, yet also cued through language (just like our wine). Concerning the former, we have both source (jeans) and target (wines) visually represented in the advert. This textile portrayal is anchored by language: on the one hand, the adjectives soft,

Cutting across the senses

89

easy, and comfortable are used to qualify both jeans and wines; on the other, the verb try on reinforces the wines-jeans equation. As to the properties involved in the comparison, soft and comfortable point to texture (i.e., tactile sensations), whereas easy suggests the casual, everyday, uncomplicated quality of both jeans and the wines thus qualified which encompasses more things than just sensory experience like, for instance, that both can be worn/drunk with nearly everything and at any occasion. Indeed, part of the success of this advert lies in the fact that the aforementioned adjectives are frequently used in wine commentary focused on wines texture. However uncontroversial the advert may seem at first sight, for people in their middle age it may seem odd to qualify jeans as soft unless they are really old and worn out. Of course, soft albeit new jeans are the result of modern cloth manufacturing procedures such as stonewashing (which may well be seen as a pun on the winery's name, Ironstone) as well as current jeans fashion. However, for people in their forties, the connotations of jeans whether visually or verbally cued cover anything but softness. Age and fashion considerations apart, the advert could be interpreted as highlighting the casual, young, uncomplicated, everyday qualities of the wines at issue a view that fits the North American lifestyle also promoted by other consumer goods such as cosmetics or clothes. Moreover, although softness is usually a result of ageing, this advert tells us explicitly that Ironstone wines are ready to be consumed i.e., do not need to be kept inside the bottle for a while to acquire such a property. However desirable and sought-after such qualities may be in modern winemaking and in certain wine markets, the question remains as to whether wine lovers from other countries would find the wines thus advertised worth buying. Indeed, one of the assumptions in countries such as France or Spain as well as the main feature of some famous vintages is, precisely, that great wines stand out among others for their sense of terroir (which rules out the connotations derived from adjectives such as easy and comfortable) or the winemaking and ageing process used (which usually takes time). Moreover, softness may be a desirable quality in a wine (meaning that it is round, fruity and lacks aggressive tannins) or, rather, be regarded as negative in the sense that it indicates that the wine at issue provides little impact on the palate i.e., is somewhat watery. The advert of a Spanish wine in figure 6 offers a dramatically different picture, the slogan reading Osborne arrives at Malpica. At first sight, there is nothing figurative here: what we have is a close up of a grapevine and a slogan that tells us that a well-known Spanish brand (Osborne) has been recently established in Malpica (a location in Spain). Yet, the advert

90

Rosario Caballero

also includes the brands trademark, a bull, which was originally created to advertise one of the most popular Spanish brandies (Veterano) in billboards, and has become a distinctive landmark of the Spanish landscape as well as a national identity symbol if we heed the numerous reproductions of Osbornes bull in the T-shirts, tea mugs, etc. sold to tourists. Finally, the surname Osborne in Spain is loaded with connotations that bring to mind wines as well as bull breeding which may be missed by those people not familiar with Spanish culture. In my view, the advert nicely illustrates a pictorial metonymy, that is, an image that provides access to the entity bull via one of its most prototypical traits: its head and, more specifically, its horns. Moreover, since I am acquainted with Spanish TNs, I could not help but interpret the metonymy as cueing one of the best compliments a wine may be paid, namely, that of being thoroughbred. For whereas this English term is associated with horses or dogs, its Spanish equivalent casta is prototypically used to qualify bulls. This characteristic shows both in the animals behavior and in its appearance, and horns are largely responsible for the latter. My specific claim here is that any Spaniard could easily interpret the advert as equating wines with bulls regardless of the presence of the trademark or the connotations verbally cued by the term Osborne, yet would be seen as merely showing a grapevine by many audiences outside Spain. A different issue is whether the trait metonymically mapped (lineage, breed) would be understood by Spanish people outside the wine realm. Several related questions are relevant concerning this last point, and are applicable to any advert. The first one is: would non-expert audiences understand what is communicated in wine adverts at all? Of course, any reader acquainted with advertising discourse will know that something positive is being promoted via language, images, or both. S/he will presumably make use of her/his background knowledge to interpret the adverts knowledge which may include different kinds of information and, most importantly, will be culturally biased. In turn, bringing in cultural knowledge in advert and/or metaphor understanding also prompts two more questions: What do we mean when we use the term culture? How do the different ways in which the term may be applied affect the understanding of metaphors in adverts? For if culture is understood as the shared beliefs, knowledge and world view(s) characterizing national groups, the adverts may of course be understood, yet the interpretation of the images and, above all, of the metaphors thus rendered may very well be more open given the heterogeneous nature of the audience (due to their diverse backgrounds, and hence diverse concerns and expectations). In

Cutting across the senses

91

contrast, a particular discourse community like the one articulated around wine (be it composed of professionals or, simply, wine lovers) represents a subculture within a much broader cultural frame, and is characterized by specific knowledge schemas, needs and interests. It seems reasonable to expect, then, that the ways in which the adverts will be understood will differ radically across various communities. For one thing, wine lovers and aficionados bring in knowledge of the domain being metaphorized in the adverts, as well as of the jargon used in other promotional genres like the tasting note. They may also be acquainted with numerous brands from different wine producing countries, and be aware of the connotations in, for instance, a name such as Osborne. A different issue is whether the two adverts in figures 5 and 6 represent successful selling strategies given the possible prejudices towards wines being compared to jeans or to bulls according to national cultural with a capital C differences. The issue, however, asks for research procedures that go beyond the purposes of this chapter (namely, a user-centered approach), but is open for further research into how the production and interpretation of the imagery used in the audiovisual genres are affected by the specificity of the audience at which they are aimed.

4. Concluding remarks

The discussion in this chapter has been mainly concerned with the experiential dimension of metaphor, that is, with how it is used by wine critics and advertisers to articulate the sensory experience(s) afforded by wines in different media and communicative modes. Thus, I have provided a brief overview of the metaphors found in two promotional genres within wine discourse: TNs and wine adverts. Concerning the former genre, I have explained wine critics abundant use of figurative language as responding to their need to overcome the difficulties inherent in communicating the various organoleptic experiences conflating in wine tasting. Concerning the latter genre, the adverts here shown illustrate a penchant for anthropomorphic metaphors, and largely fall into Forcevilles verbo-pictorial variety of multimodal metaphor. Nevertheless, the foregoing discussion has also revealed the problems derived from representing anthropomorphic metaphors in both verbal and visual form, particularly those concerned with the sensory properties of wines (the exploitation of the term body being a case in point of the contradictory, even wrong views often rendered by the image-language combination). In this respect, however attractive the image-language combination may be to sell the product, it also poses many problems for the meta-

92

Rosario Caballero

phor researcher since it not always captures the previously discussed crosssensory or multimodal dimension of most metaphors in winespeak. A point also worth noting is the fact that many of the metaphors rendered in pictorial form are actually the translation of metaphors acquired or learnt consciously or unconsciously via such a cultural manifestation as language rather than being embodied in the cognitive sense of the term. A final aim of the chapter has been to draw attention to the role of culture in understanding the metaphors used to advertise wine: both the broad national culture of the audience at which the adverts are aimed as well as the specific culture articulated around wine. For, indeed, as Forceville (2006: 389) acknowledges, the connotations from the source domains in the metaphors are important in their interpretation, yet these also leave more room for interpreting them. In my view, this endeavor also requires knowledge of the target domain (in the present case, wine) as well as of the schemas underlying the particular worldview of the wine community.
Acknowledgments
I am grateful to Veronika Koller, Eduardo Urios-Aparisi and Charles Forceville for their comments on earlier drafts, as well as to Ernesto Suarez-Toste (Universidad de Castilla-La Mancha) for his insightful remarks on some of the issues addressed in this chapter as well as for providing the wine magazines used to build the French corpus.

Notes
1. The project is currently being undertaken by Ernesto Suarez-Toste and Rosario Caballero. The English examples belong to the corpus used in the research and have been retrieved from the following sources: Decanter, Wine Enthusiast, Wine Spectator, The Wine Advocate, Wine News, The Wine Pages, and The Wine Anorak. The Spanish data have been extracted from Sobremesa, and Gua Proensa de los Mejores Vinos de Espaa. French tasting notes belong to La Revue du Vin de France, and Vinum. This term is derived from toffe, meaning cloth, fabric. The term charpent derives from charpent, which means skeleton, frame, and framework and is used in French both to describe built artifacts and people. In this regard, the term is a nice example of the frequent use of anatomical lexis in architecture and vice versa. This is also the case with Spanish esqueleto (which can be translated as skeleton, framework

2. 3.

Cutting across the senses

93

4.

5.

6.

7.

or, simply, structure). For a detailed account of the relationship between both domains, see Caballero (2006). Although the terms are used for descriptive purposes, and wine lovers defend their neutral quality, the sexist connotations of the adjectives are often pointed out by female scholars as I have personally experienced at various conferences (see also Amoraritei 2002). Indeed, both terms are perfect examples of the intrusion of culture (and, in many cases, controversy) in the interpretation of figurative expressions. Thus, in spite of whatever expectations the readers cultural background may arouse, there is no evaluation inherent in the masculine/feminine distinction: there is nothing per se good or bad in being one or the other; if anything, it is undeniable that the current market trends favor power over elegance (i.e., masculine over feminine), but this is essentially circumstantial. The terms simply state that the drinkers natural expectations from a specific wine will not be met because the wine is unusual for its group in terms of mouth-feel and structure (for a detailed discussion on how both terms are used by wine experts, see Caballero and Suarez-Toste forthcoming). Notice also that the winespeak term referring to the wine basket is cradle, although, in this case, the underlying metaphor if any would involve physical resemblance, in which case it would illustrate image metaphor rather than any other type. I am indebted to Raquel Segovia (who collaborated in the aforementioned research project on wine metaphors) and Eduardo Urios-Aparisi for this insight. The adverts here discussed are not shown for copyright reasons.

References
Amoraritei, Loredana 2002 La mtaphore en Oenologie. Metaphorik.de 3: 112. Available from www.metaphorik.de/03/amoraritei.htm (last accessed 23 March 2003). Bruce, Nigel 2000 Classification and hierarchy in the discourse of wine: Emile Peynauds The Taste of Wine. English for Special Purposes Journal 2326: 149164. Caballero, Rosario 2006 Re-Viewing Space: Figurative Language in Architects Assessment of Built Space. Berlin/New York: Mouton de Gruyter. Caballero, Rosario, and Ernesto Suarez-Toste forthc. A genre approach to imagery in winespeak: Issues and prospects. In Researching and Applying Metaphor in the Real World, Alice

94

Rosario Caballero Deignan, Lynne Cameron, Graham D. Low, and Zazie Todd, (eds.). Amsterdam/Philadelphia: John Benjamins.

Day, Sean 1996

Synaesthesia and synaesthetic metaphors. Psyche 2 (32). Available from www.users.muohio.edu/daysa/types.htm. Last acccessed 7 October 2005. Forceville, Charles 2005 Cognitive linguistics and multimodal metaphor. In Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach (ed.), 264284. Cologne: Von Halem. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. 2008 Pictorial and multimodal metaphor in commercials. In Go Figure! New Directions in Advertising Rhetoric, Edward F. McQuarrie and Barbara J. Phillips (eds), 178204. Armonk, NY: ME Sharpe. Gluck, Malcolm 2003 Wine language. Useful idiom of idiot-speak? In New Media Language, Jean Aitchison and Diana Lewis (eds.), 107115. London/New York: Routledge. Lakoff, George, and Mark Johnson 1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books. Lehrer, Adrienne 1975 Talking about wine. Language 51: 901923. 1983 Wine and Conversation. Bloomington: Indiana University Press. 1992 Wine Vocabulary and Wine Description. Verbatim: The Language Quarterly 18: 1315. 2006 Wine and conversation: A new look. In Empirical, Cognitive-based Studies in the Semantics-Pragmatics Interface, Jzsef Andor and Pter Pelyvs (eds.). Oxford: CRISPI, Elsevier Science. Nedilko, Anatoly 2006 Viticulture and winemaking terminology and terminography. Terminology 12: 137164. Peynaud, mile 1987 The Taste of Wine. Trans. Michael Schuster. London: Macdonald Orbis. Ramachandran, Vilayanur, and Edward Hubbard 2001 Synaesthesia A window into perception, thought and language. Journal of Consciousness Studies 8: 334. Yu, Ning 2001 Synesthetic metaphor: A cognitive perspective. Journal of Literary Semantics 32: 1934.

Chapter 5 Interaction of multimodal metaphor and metonymy in TV commercials: Four case studies Eduardo Urios-Aparisi

Abstract
This chapter analyzes four prototypical cases of interaction of multimodal metaphor and metonymy in television commercials. Three questions are addressed: how multimodal metaphor and metonymy interact; how this interaction can contribute to meaning creation in the commercial; and how multimodal metaphor and metonymy interact in the cognitive and persuasive aspects of a multimodal genre. A holistic analysis of these texts needs to bear in mind the diversity of modes and submodes which participate in this genre as well as adopt a dynamic perspective on the cognitive processes and their taxonymy, following Ruiz de Mendoza and Dez Velasco (2002). The analysis shows how metaphor and metonymy interaction hinges on cognitive as well as communicative roles and motivations. Keywords: metaphor, metonymy, multimodal metaphor-metonymy interaction, expansion, reduction, highlighting, advertising discourse, media, TV commercials

1. Introduction: television commercials in the advertising world The presence and importance of multimodal metaphors for the theory of cognitive metaphor and figurative language has been well attested by Forceville (2006/this volume, 2007, 2008) and by the chapters in this volume. In a multimodal context, words, images and sounds can represent different domains and establish mappings which result in metaphor-producing relationships. As Radden (2002: 413) points out, those domain mappings can also create metonymy-producing relationships. Only recently research on metonymy has shown the important role it plays in cognition (Panther and Radden 1999, Barcelona 2000, Dirven and Prings 2002, Panther and Thornburg 2003). Nonetheless, except for Mittelberg and Waugh (this volume), the references to metonymy in relation to multimodal metaphor have generally

96

Eduardo Urios-Aparisi

been made in passing, although multimodal metonymy has been deemed worthy of consistent study (Forceville 2008: 298). In the context of advertising, metonymy is an important cognitive process and it not only reveal[s] rhetorical strategies (Forceville 2008: 298) but also has an important role in motivating metaphor, highlighting its mappings, and consequently metonymy can define and represent reality and how the product should be perceived by the audience. In this chapter, I address the following questions: how multimodal metaphor and metonymy interact, how this interaction can contribute to meaning creation in the commercial, and how the relation between cognition and persuasion as multimodal metaphor and metonymy are embedded in the genre of advertising. Except for Forceville (2007, 2008) and Yu (this volume), the study of conceptual metaphor in advertising has focused mainly on printed advertising. Printed advertising has been approached from a cognitive and pragmatic point of view (e.g., Tanaka 1994, Forceville 1996, Teng and Sun 2006), from a rhetorical point of view (e.g., Scott and Batra 2003, Phillips and McQuarrie 2002), and from a discourse analysis point of view (e.g., Cook 2001). The four cases analyzed in this paper are prototypical cases of metaphor-metonymy interaction within the multimodal context of television commercials. In a commercial, metaphor is an integrated experience of words, images, sounds and meanings. As Kress and Van Leeuwen (1996: 17) suggest, of the visual and verbal modes, each has its own possibilities and limitations of meaning. A holistic study of meaning in a multimodal context requires that particular modes of communication should be seen in their environment, in the environment of all the other modes of communication which surround them, and of their functions (Kress and van Leeuwen 1996: 33, italics in original). It is, therefore, necessary to integrate the visual, verbal and auditory modes and to theorize how they contribute to the overall meaning of the commercial and the intention of the advertiser to represent the advertised product in a positive light, so the spectator feels compelled to buy or use it (cf. Forceville 1996). In a commercial a metaphor is a primarily unidirectional act meant to define the product and its benefits for the consumer. In most cases, the claims made about the product need to be seen as motivated and natural. The creative team uses cognitive resources in order to fulfill the claim about the product, motivate the meaning by associating it to the real experience of the spectator and to his conceptions of the world. In this sense, advertising works very much like poetry, as Barthes (1988 [1964]) already argued. The creative team uses the same resources as, for instance, a poet, in order to

Interaction of multimodal metaphor and metonymy

97

ground the metaphors in common experience and in the knowledge of the consumer. In the next section, I will discuss the theoretical framework of this chapter. This discussion will lead to the microanalysis of four commercials that were awarded prizes at the Festival Publicitario de San Sebastin, nowadays called Festival El Sol.1 As mentioned before, television commercials are particularly good examples of interaction between visual, verbal and sound modes. The analysis of these multimodal commercials will involve the identification of target and source, a discussion of how they draw on more than one mode, and an analysis of the various types of metaphor and metonymy interaction. These types follow and expand on the kinds of conceptual interaction found in Ruiz de Mendoza (1999) and Ruiz de Mendoza and Dez Velasco (2002). The analyses of these commercials show how all of them aim to integrate the advertised product within what Sperber and Wilson (1995) call the cognitive environment of the audience, using the modal resources of the cinema and television so that its central message is always somehow in praise of the product. 2. Multimodal metaphor and metonymy A multimodal metaphor is a cognitive process in which two domains are represented in two different modes. In the study of multimodal metaphor, the surface representation should be taken into account in order to further the understanding of the metaphorical mappings. As Forceville suggests, clearly, which channel(s) of information (language, visuals, sound, and gestures, among others) are chosen to convey a metaphor is a central factor in how a metaphor is construed and interpreted (2007: 15). We may be allowed to see either the target or the source, but either of these can be merely suggested by any of a great variety of devices. For instance, the target (often: the advertised product) can be conveyed by one of its parts or by its logo or jingle, and the source can be explicitly represented or implicitly inferred. In fact, research on advertising has found that making claims about a product by means of indirect representations can create positive inferences and a more receptive attitude toward the brand by the audience (McQuarrie and Phillips 2005). Jakobson (1971 [1956]) situated metonymy and metaphor as two poles of cognitive processing: the metonymic pole accounts for contiguity relations between linguistic elements, while the metaphoric pole is the result of similarity relations between two domains (cf. Mittelberg and Waugh this volume

98

Eduardo Urios-Aparisi

on gestures). The interaction of metonymy and metaphor shows that they are not two opposite poles, but two parts of a continuum from literalness to metaphor, as Radden (2002: 409) suggests. Metonymy is understood here as an internal mapping of a subdomain within the same experiential domain (Lakoff and Johnson 1980, Radden 2002; for the topic of multimodal metonymy per se, see Yu this volume). While a metaphorical mapping bridges the distance between entities that are experienced as belonging to two different domains, in metonymy a mapping is connected to the mental highlighting or activation of one (sub)domain over another (Barcelona 2002, Croft 1993). The target and source domains in a metaphor establish symmetrical correspondences between different concepts in a way that does not happen in metonymy. For instance, in the metaphor of LOVE IS A JOURNEY the concepts in the target (lovers, love relationship, etc) correspond to the concepts in the source (travelers, vehicle, etc) as Lakoff (1993: 208) has shown. As Barcelona (2002) says, in metonymy this correspondence is asymmetrical: the metonymic source projects its conceptual structure onto that of a target, not by means of a systematic matching of counterparts, but by conceptually foregrounding the source and by backgrounding the target (cf. Barcelona 2002: 226, italics in original). For instance, in case 4, the front of the car stands for the whole car or even the whole car company. The foregrounding of the source (the car front: bumper, spoiler with company logo, head and turning lights) highlights features of the car which the advertiser intends to underscore. In contrast with metaphor, which can be either referential or predicative (e.g., Warren 2006), metonymy has been considered to have mostly a referential function (Lakoff and Turner 1989: 103). Other functions can be meaning extension (cf. Taylor 2002: 325) or pragmatic inferencing (cf. Panther and Thornburg 2003). The distinctions between metaphor and metonymy are fuzzy. An instance of how metonymy and metaphorical mappings can overlap is found in a discussion in Forceville (1996). In a printed advert a beer bottle is pictured in a wine cooler, thus expressing the metaphor BEER IS CHAMPAGNE. This metaphor is developed from the metonymy which connects both target and source to a single domain: [alcoholic] drinks.2 Research on metonymymetaphor interaction has led to different typologies. Goossens (1990) was the first to analyze their interaction in linguistic action expressions and created the term metaphtonymy. This term included four types of combination in cases of meaning extension: metaphor from metonymy, metonymy within metaphor, demetonymisation inside a metaphor and metaphor within metonymy.

Interaction of multimodal metaphor and metonymy

99

These combinatory entities may be useful for the explanation of meaning extension in linguistic expressions, although these categories do not seem adequate for the kind of combinations found in multimodal texts. In this context, the categories proposed by Ruiz de Mendoza (1999) and Ruiz de Mendoza and Dez Velasco (2002: 512) in terms of target and source seem to be more suitable. They suggest a different view of metonymy which considers that of the three general types of metonymies (PARS PRO TOTO, TOTUM PRO PARTE and PARS PRO PARTE, see above and Kvecses and Radden 1998 for a detailed analysis of the different metonymies) only the first two are in fact metonymies. The names of the two types are TARGET-IN-SOURCE as in She is taking the pill (pill for contraceptive pill) or SOURCE-IN-TARGET as in All hands on deck (hands for sailors) (Mendoza and Dez Velasco 2002: 497). Two metonymies can also interact as, for instance, in the example Wall Street is in panic: the location of the institution (Wall Street) for the institution (New York Stock Exchange) (PLACE FOR INSTITUTION) and the metonymy of the institution for the people who work in it (Ruiz de Mendoza and Dez Velasco 2002: 512). These authors also distinguish two processes: expansion and reduction. These processes are present in five types of conceptual interaction. Metonymic expansion of a metaphoric source can be exemplified in the case To beat ones breast, where the basis of the source of the metaphor is a metonymy. In this metonymy the source is a person beats his breast and the target a person beats his breasts in order to show his sorrow about a situation. This is the source for the metaphor a certain person makes an open show which may be a pretence in order to express his sorrow about a certain situation (Ruiz de Mendoza and Dez Velasco 2002: 5201). In metonymic expansion of a metaphoric target, as in to knit ones brows, knitting is the source of a metaphor with a metonymy in the target: person who frowns for person frowning because of anger. Metonymic reduction of one of the correspondences of the target domain of a metaphor can be found in to win someones heart where LOVE is the target domain of the metaphor LOVE IS A PRIZE, and HEART FOR LOVE is the metonymic reduction within the target domain. Metonymic expansion of one of the correspondences of the target domain of a metaphor can be exemplified by to catch someones ear. In the source: person catches an object, the object of the source corresponds to ear in the target and this is the source for metaphorical mapping with attention. Finally, a metonymic expansion of one of the correspondences of the source domain of a metaphor is found in to bite the hand that feeds you, where hand is the source domain of the metaphor HELPING IS FEEDING and stands for feeder (2002: 522527).

100

Eduardo Urios-Aparisi

Meaning in the context of a commercial is not static but progresses either by creating a short fiction or by layering diverse meanings over the advertised product. In this regard, the interaction of metaphor and metonymy and the processes of expansion and reduction between domains are similar to those in the verbal mode. Visual perceptions and these perceptions may be decoded by the same specialized mental module according to Yus (this volume). But, as Stckl (2004: 14) says, submodes constitute a mode in that they provide the building blocks of a modes grammar. The submode discussed in this chapter is color, but other submodes could include the line and shape, tone, color, movement and rhythm of the moving image (about these components of the moving image see Block 2008). The creative team of a commercial is generally aware of each mode and its submodes. In the short length of a commercial, every single entitiy of the message is considered in order to elicit our attention and emotions by simulating various significant features of our real-world visual experiences (Messaris 1997: 266). 3. Case studies 3.1 Metonymic reduction of metaphorical correspondences between domains: New model of car (1986) In this commercial, the metaphor is explicitly represented in the images: CAR IS A LONG-JUMP ATHLETE, a variant of the metaphor CAR (MACHINE) IS A PERSON. As we can see in figure 1, the visual montage shows the long-jump athlete and the car in cross-cut shots. This metaphor is a clear case of personification. Within this metaphorical conceptualization of cars, the metaphor is represented clearly with an ingenuous use of the montage. This explicit representation of the metaphor is clearly different from the implicit meaning of case 4, which I will argue is also a case of personification. Nonetheless, this case is a particularly good instance of the use of metonymy to highlight the features that are being mapped between the two domains. As we can see from the selected images of this commercial, the source is shown first at a distance, while the target is only seen in parts until both are on the racing track ready to do the long-jump. The cross-cutting of images enhances the identification of the target with its source including mappings of the athletes running shoes and legs, which highlight other features of the car such as its sporty properties. The property that is metonymically cued in

Interaction of multimodal metaphor and metonymy

101

Athlete

Car

Words

7 Ahora ms versiones Now more versions

9 Renault 11

10

11

Figure 1. Fasa Renault 1986 01 Salto Renault 11 Salto Jump.

102

Eduardo Urios-Aparisi

both domains is the power of the cars engine (and metonymically the whole car), and the power of the athlete who can jump very far. Just before the jump, the car appears completely and the male voice-over says: Now more versions The verbal mode explicates the target of the commercial: the new properties and more powerful motor of a car model. The attributes by which both domains are cued constrain the metaphorical mappings by highlighting those features which are relevant for the target domain and the intention of the advertiser. Although both source and target of the metaphor are identified by the montage of images of the athlete and the car, the voice-over further anchors the commercials message by clarifying the target domain of the metaphor. The montage of the images of the athlete and the car identify both target and source of the metaphor, and verbal anchorage further clarifies the target of the metaphor. The potential mappings in the metaphor CAR IS PERSON are limited to the ones the advertiser is interested in activating by creating metonymical visual correspondences between the domains and by the verbal modality naming those conceptual features. 3.2 Metonymic reduction in target and motivation of metaphor: Tea drink (1987)

Figure 2. Hipnosis (hypnosis) (1987). Product by CPC Espaa. Advertising company: J. Walter and Thompson.

This commercial creates the metaphor a TEA TAG IS A HYPNOTIZERS WATCH metaphor in order to transfer the powers of hypnosis to the relaxing impact of a brand of tea. In this metaphor the target takes on the role of the hypnotizers watch and the movement of the tea tag is one of the cues to the source domain of the metaphor. The representation of the tea follows two metonymies: EMBLEM FOR PRODUCT (Tea Tag for Tea Bag), and EFFECT FOR CAUSE (Relaxation for Drinking Tea), summarized in Table 1. The first metonymy highlights the products tag as it can resemble a hypnotizers watch and allows for the metaphor TEA TAG IS A HYPNOTIZERS WATCH to be enacted. The second metonymy EFFECT FOR CAUSE (Relaxa-

Interaction of multimodal metaphor and metonymy

103

tion for Drinking Tea) motivates this metaphor. As hypnosis is associated with falling into a trance-like state, the commercial enacts the quintessential feature of tea by identifying and making the tea tag act as a hypnotizers watch.
Table 1. Metonymies to represent TEA. Metonymy type
EMBLEM FOR PRODUCT

Metonymy in commercial Tea Tag for Tea Relaxation for Drinking Tea

Explanation Visual representation of the product Highlighted consequence of drinking tea

EFFECT FOR CAUSE

Table 2. Modes in metonymy and metaphor for case 2. Figure Metonymy Components
TARGET SOURCE

Visual Tea drink Tea tag hanging from tea cup Tea Tag Hypnotizers watch Submode 1: movement of the watch Submode 2: Soft colors and low modality.
SOFT COLOR IS WARMTH

Words

Sound

The logo of the product

Metaphor

TARGET SOURCE

Words a hypnotizer says to a patient

Hushed voice of the hypnotizer The tic-toc of the watch

The visual mode could be divided into several submodes such as color or movement. As mentioned above, the submodes are building blocks of each mode (cf. Stckl 2004: 14). The movement of the tea tag acts out the way a

104

Eduardo Urios-Aparisi

hypnotizers watch moves. The soft colors in a gold and brown hue and the slightly out of focus image represent the view of the person who is falling under the spell of the watch. The hue and colors are stereotypically warm and thus represent the feeling of wellness that the product is supposed to give and associate to the metaphor SOFT COLOR IS WARMTH and, thus, to the metaphor AFFECTION IS WARMTH which can be found in the context of emotional relationships (cf. Kvecses 2000: 93) and, if that is the case, the color submode would integrate experiences of friendship and emotion with the product. The visual mode also activates the sound mode with the speakers hushed tonality which is similar to the color and hues of the image. The modes and their association with different components in the metonymy and metaphor in this commercial are listed in table 2. In conclusion, the metaphor is elaborated through a complex interaction of modal techniques. Each mode and submode associates the product with the main metaphor TEA TAG IS A HYPNOTIZERS WATCH and relates it to sensations of warmth, relaxation and wellness. 3.3 Metonymy in source and expansion by implicit metaphorical mappings This commercials point is not to advertise any new model, but it is a public announcement and praise for the safety fixtures and reliability of all the cars of that brand. Therefore, it is a kind of corporate advertising with a public service tone, most likely aired around a time of high traffic and traveling, as the voice-over clearly implies: En estos das mucha gente saldr a la carretera (These days a lot of people will go on the highway). The commercials design is very simple: the camera focuses on the front grill of a Volvo car from the 240 series which was developed in the 1970s.3

Figure 3. Respuesta (answer) (1987). Product advertised: Volvo car. Advertising company: CID.

It can be considered an example of metonymy in advertising since all the mapped features are metonymically motivated, as we will see. The square and bulky shape of the front with its metallic look, together with the bumpers and large headlights, is a visual metonymy: CAR FRONT FOR CAR. The front

Interaction of multimodal metaphor and metonymy

105

of the car is a complex image. In it we can see the frame of the car with its bulky design, the big bumpers and lights, the license plate with the word Volvo in blue capitals with a white background (the official typeface of the brand; Egyptian according to http://www.volvoclub.org.uk /history/volvo_ logo.shtml) and the logo which is located on top of the radiator. These two elements are symbolic metonymies of the product and they have metonymical (LOGO FOR PRODUCT) as well as other meanings (on logos as multimodal metaphors, see Koller this volume).4 The image represents the identity of the corporation in three aspects: the product, the corporate signs and symbols, and the main features they want to associate themselves with: reliability, strength and safety.
Table 3. Metonymies and metaphor in Case 3 Figure Metonymy 1
PART FOR WHOLE

Components
TARGET SOURCE

Visual Car Front of the car Front of the car with logo and official typeface Cars reliability and safety Front of the car Changing weather conditions

Words

Sound

Metonymy 2
PART FOR WHOLE

TARGET SOURCE

Changing weather conditions and sounds associated to moving car Person: Advice to a driver by the voice-over

Metaphor

TARGET SOURCE

Car

Metonymy 3
PART FOR WHOLE

TARGET SOURCE

Company Intelligent car, Logo and official typeface

106

Eduardo Urios-Aparisi

After the first image of the car, the camera pulls back and the front of the car moves through all kinds of weather and driving conditions smoothly and surely, and this driving is accompanied by metonymical sounds and images which can be associated with these driving conditions and to passengers and other persons: the sound of children getting in the car, the door closing, the noise of the car engine, the rain, thunder and snow (also visible in the images), the turn signal noise and image, the noise and the image of a ball suddenly bouncing in front of the car, the car horn and brakes. On the one hand, the images are metonymies of the car as a whole, perhaps referentially highlighting the cars strength, sturdiness and immutability towards the changes in the road conditions. Also, as mentioned above, the color submodality would also contribute to these meanings. On the other hand, the language used does not refer to the product at all. The male voiceover addresses the consumer directly by giving advice about how to drive when there is high traffic during vacation time (the images suggest that it is winter). The words insist that the driver be sensitive, drive safely and prudently, and focus on the importance of the family (referenced metonymically as carga or load) and on arriving safely (which implies that it is better to arrive safely than fast). As we can observe, this commercial features good examples of metonymic references in various modes. Metonymy, in this case, is mostly referential with respect to the car, its passengers, and the weather and driving conditions, but this metonymy can create further implicit meanings by the audiences knowledge of car-safety and brands. Besides these components, there is one obvious absence in the visual representations: that of the driver who is directly addressed in the words. Whereas the commercial features various metonymies, the voice-overs address predicates of the car some of the properties which generally are associated with a person, the driver of the car: sensitivity and prudence. Therefore, as part of the creation of the corporate image, the car seems to be personified (CAR IS PERSON). A personification is a kind of ontological metaphor in which the target is understood in human terms (Lakoff and Johnson 1980). It can be explained along the lines of Lakoff and Turners (1989: 195) discussion of metaphors like PERSONS ARE ANIMALS (Achilles is a lion) within the cultural model of the GREAT CHAIN METAPHOR. According to this metaphor, attributes and behaviors are associated with animate creatures within a hierarchical scale: the Great Chain is a scale of forms of being human, animal, plant, inanimate object and consequently a scale of the properties that characterize forms of being reason, instinctual behavior, biological function, physical attributes, and so on (Lakoff and Turner 1989: 167). In this commercial, we understand the

Interaction of multimodal metaphor and metonymy

107

cars behavior as it goes through different driving conditions in terms of some features that can only be associated with a human beings behavior (prudence, sensitivity and steadfastness). This mapping is complex since it is not only between the properties of the source and the target, but also between the relationships of those properties to their domains. Therefore, human sensitivity and prudence is intelligent and high-order and so car behavior is also intelligent and high-order.4 The metaphor of an intelligent car (CAR IS PERSON) is an extension of a complex metonymy: the front part of the car by juxtaposition to the voiceover which addressees the audience, while at the same time the front of the car itself is shown and probably credited with the behaviors described by the words. The commercial as a whole is metonymically representing the brand, bestowing on it the features associated with the car. 3.4 Metaphorical expansion of metonymy in the source domain Commercial 4 advises women to do family planning instead of resorting to abortion. This commercial is another instance of extension of a metonymy but in this case in the source domain. The first visual metaphor of this commercial is MAKING A DECISION IS PULLING A THREAD. The camera focuses on a yellow bootie situated over a soft white light against a diffused dark background. Shortly after the male voice-over starts speaking, we see two fingers starting to pull a thread of the bootie and the words explain how women sometimes have to make a decision, and how this decision is hard and sometimes traumatic (see the complete text in figure 4). The first image is easily identifiable with a baby by the metonymy BABY BOOTIE STANDS FOR BABY, a variant of the metonymy GARMENT FOR PERSON. The bootie is the source of the metonymy and also part of the source of the metaphor that is developed as the hand pulls the thread. Another metaphor is more specifically identified by the words and the image: ABORTING IS UKNITTING BABY BOOTIE. Undoing the baby bootie is at the same time the act of making a decision, and since this object can be identified metonymically with the baby, its unraveling is the result of a decision which is considered difficult or traumatic. During the commercial, abortion is never mentioned, but it is metonymically implied by the decision to abort and the experience of going through the abortion (both PART FOR WHOLE metonymies). These words are diagrammatically represented in the images. As difcil decisin or siempre dura and traumtica (respectively, shots 3, 5 and 7) are uttered, the

108

Eduardo Urios-Aparisi

perspective on the bootie changes and the camera shows a close-up of the thread of the bootie slowing down. The source has been identified by the visual mode while the target is in the verbal mode throughout the commercial until the final sentence: no vivas pendiente de un hilo (dont live hanging by a thread) which is made literal in the image by showing the last thread of the bootie. The final expression is a conventional metaphor in Spanish and returns to some extent to the first metaphor: MAKING A DECISION IS PULLING A THREAD. The commercial leaves the opportunity to reach other conclusions to the audience.

1. Muchas mujeres Many women

2. se han visto obligadas a tomar have been obliged to make

3. una difcil decisin a difficult decision

4.

5. Una decisin siempre dura A decision, always hard

6. y en ocasiones and sometimes

7. traumtica. traumatic

8. Evita esta experiencia Avoid that experience

9. No vivas pendiente de un hilo Dont hang by a thread

10. Planifcate Make plans Figure 4. Patuco (baby bootie) (1988) Ministerio de Sanidad (Health Department, Spanish Government). Advertising company: Vitruvio.

Interaction of multimodal metaphor and metonymy

109

In table 4, I have summarized the metonymy and the metaphors of this commercial. It is an emotionally charged commercial. The bootie, out of all the possible pieces of baby clothing, is readily associated with a baby (older children do not wear booties); also, it shows with more immediacy the shape of the body, and, finally, it has immediate connections to life and movement: crawling and walking. The metonymy motivates the metaphor, but also maps other meanings to the target of the metonymy (the baby). This process together with the music (the so called Brahms Lullaby) creates another emotional layer to the commercial by reliving the decision making process in the enactment of undoing a bootie.
Table 4. Metonymy and metaphor in case 4. Figure Metonymy
GARMENT FOR PERSON

Components
TARGET SOURCE: TARGET SOURCE

Visual
BABY BOOTIE

Word

Metaphor 1

MAKING A DECISION UNDOING A BOOTIE ABORTING:

Metaphor 2

TARGET

metonymically implied by the words (PART FOR WHOLE)

SOURCE

UNKITTING BABY-BOOTIE

Metaphor 3 No vivas pendiente de un hilo Dont live hanging by a thread (extension of Metaphor 1)

TARGET

TO LIVE OR TO BE IN DANGER IS

SOURCE

Last thread of the baby bootie [visual representation]

TO HANG BY A THREAD

4. Conclusions In this analysis I have identified the following metaphor-metonymy interactions:

110

Eduardo Urios-Aparisi

a) Metonymic reduction limits the possible correspondences between domains by highlighting the features of both domains that are to be matched metaphorically (case 1). In this case, the metaphor CAR IS PERSON is determined by the visual identification of the correspondences and thus the commercial takes advantage of the persuasive potentials of a variant of this metaphor: CAR IS ATHLETE. b) Metonymic reduction in metaphorical target visualizes the product and highlights the feature that will motivate the metaphor (case 2). c) Source-in-target metonymy represents the product and the company and it is also the target of the verbally expressed source domain (in case 3). The metonymy highlights the relevant part and lends it to further mappings because of its visual presence throughout the commercial and because of background knowledge about this particular car and brand. d) Source-in-target visual metonymy (GARMENT FOR PERSON) is expanded with a series of metaphors. Each metaphor builds on the previous one and relates to the first metonymy. Obviously, this is not a comprehensive list and other interactions are possible. Perhaps other cases can identify the same kinds of interactions in the source domain as Ruiz de Mendoza and Dez Velasco (2002) found in their linguistic corpus. In these cases, metonymy has mostly a limiting role, as it identifies the target of metaphor or the correspondences between both domains, while metaphor expands this identification with other mappings. In case 3, the metonymy taps into the background knowledge to suggest further interpretations. To some extent, it is not limiting or exclusively referential but creates further correspondences. In the dynamics of a television commercial, metonymical mappings, on the one hand, do not only substitute for or represent the product, but they can link the product to domains which can be relevant for the products promotion: expanding or constraining the interpretation of metaphorical mappings (see El Refaie this volume for other uses of a metonymy as anchoring). Metaphor, on the other hand, can also make the metonymy progress towards mappings that go beyond the presence of the product and try to convey additional meanings such as emotional representation and poetic effects in the cognitive environment of the audience, as mentioned earlier. The different modes and submodes can contribute to the creation of a pausible representation of reality and can also be associated to other metaphors. For instance, as mentioned in case 2, the hue and colors are warm and soft. Those colors can represent the metaphor SOFT COLOR IS WARMTH and further, it can connect to the metaphor AFFECTION IS WARMTH. This is clearly contrasted in case 3. In this car commercial, mainly in blue and grey

Interaction of multimodal metaphor and metonymy

111

the colors can be associated not only with the difficult driving conditions but also with the strength and endurance of iron or steel. While in case 4, the choice of yellow in the color of the baby bootie intentionally avoids colors like blue or pink, which are gender-specific. The diffused light of the background highlights the baby bootie and its dramatic undoing. In the case of metaphor, they include meanings that aim at supporting the visual consistency and the identification with those features which are stereotypical of the domain. In brief, I would conclude: 1. Television commercials are dynamic texts in which all modes can contribute to multimodal metaphors either in the source domain or the target domain. 2. In order to understand how metaphor creates meaning, metaphor needs to be studied within its embeddedness in the context of the commercial and the persuasive functions of advertising. 3. The interactions of metaphor and metonymy show that layering of rhetorical figures is not random, but follows clear cognitive patterns which restrict and define their design and persuasion. As seen in case 2, the metonymy has a double function: represent the target for the metaphor in a way that can be realistic for the metaphorical representation, and motivate the message of the commercial. Once metonymical correspondences are mapped, the commercial can create additional metaphorical mappings. Also, metonymy can identify those entities which are to be transferred from the two domains, as in case 1. 4. As shown, a metaphor expands the meaning by associating new domains with the original metaphor or metonymy. It creates further imagery that can trigger more emotional or intellectual associations with the product. 5. The grounding of the meaning of a commercial in the viewers knowledge and experience can be accomplished by various means. One of them is metonymy, and this is consonant with general views on this figure. Metonymy is considered closer to literalness in the literalness-metaphor continuum (Radden 2002 and Dirven 2002), and is frequently used in realist art (Jakobson 1971 [1956]). In narrative it can highlight conventional belief, structure episode development and, thus, help interpretation (cf. Pankhurst 1997). In the commercials, the metonymy activates or highlights an aspect of the reality of the product. This feature can be recognized by the audience most easily or can be productive to provoke implicit positive meanings. As shown, it also can constrain the amount of possible correspondences in a metaphorical mapping.

112

Eduardo Urios-Aparisi

6. The modes in a commercial are structured around a clearly defined target, and the need to persuade or represent a product in a positive light. The kind of products which are advertised, and even some elements not addressed here such as the length of a commercial, all contribute to the need for interaction of metaphors and metonymies. A comprehensive analysis of multimodal metaphors needs to identify the genre which shapes them and steers their interpretation (Forceville this volume). The importance of connecting discourse and conceptual metaphor has been shown by Cameron (2003) in education discourse, Urios-Aparisi (2004) in interaction, Caballero (2006) in architecture discourse, and Caballero (this volume) in winespeak. Further research is needed to account for the roles of submodes and how they contribute to the domain construction mainly in visual modes. As we have seen, their contribution to the source domains in 2 and 3 is relevant to the metaphors they form, and in 4 it contributes to the dramatic staging of the commercial, but to what extent can they create other metaphorical mappings? As we have seen, three of the four cases are personifications. This is probably no coincidence. Objects, animals and other experiences are best understood as representations or extensions of human beings. The experience of objects and vehicles as animated human beings reflect the natural application of the cognitive model GREAT CHAIN METAPHOR and also the anthropocentric dimension of this cognitive model (cf. Dirven, Polzenhagen, and Wolf 2007: 1228 for this metaphor and the language of oppression). Underlying cognitive structures are present in the use of the camera, montage, color and other (sub)modalities which can be construed by the audiences cognitive resources. The inextricable relation between metaphorical and metonymical mappings determines how meaning is created. The analysis of these commercials shows how metonymy engages the products representation to the cognitive environment of the audience as it is conceived by the advertiser. Such criteria underlie the choices made by the creators of these commercials regarding the modes and the entities which are used to represent the product.

Acknowledgments
I would like to thank Charles Forceville, Brian Patrick, John Bardem and Cristin Siebert for their insightful comments on, and thorough revisions of, earlier versions of this chapter.

Interaction of multimodal metaphor and metonymy

113

Notes
1. 2. The commercials can be viewed online at the Instituto Cervantes in the Museo Virtual de Arte Publicitario (Muvap): http://cvc.cervantes.es/actcult/ muvap/in sala V: Creatividad publicitaria audiovisual. A similar commercial but with a juice drink is analyzed by Phillips and McQuarrie (2004). Both printed ads highlight how beer or juice drinks differ from wine in their production and suggest that both beer and juice drinks acquire the status of wine when they are produced in a cellar. According to http://www.swedecar.com/volvo_history.htm in 1972: The US traffic safety administration (NHTSA) purchased a number of Volvo 240s, which were used to set the safety standards against which all new cars on the US market were tested. The importance of the safety image is still paramount in Volvos marketing. In their webpage section Experience Volvo, the section on safety is first and within this section they have another special on how Volvo saved my life, (see http://www.volvocars.us/experience/ safety.htm). The logo is an adaptation of the symbol for iron: a circle with an arrow pointing upwards towards the right (cf. http://www.volvoclub.org.uk/history/ volvo_logo. shtml). Further analysis would pertain to the slogan of this commercial Respuesta segura (sure answer). This slogan is a conventional metaphorical meaning of the word respuesta as the effect of an action is the response to a question: the effect of an action is a verbal response. As part of the personification of the car, the slogan maps the features of talking and reacting with intelligence associated with humans.

3.

4. 5.

References
Barcelona, Antonio 2000 Introduction: The cognitive theory of metaphor and metonymy. In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 128. Berlin/New York: Mouton de Gruyter. 2002 Clarifying and applying the notions of metaphor and metonymywithin cognitive linguistics: An update. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.), 202277. Berlin/New York: Mouton de Gruyter. Barthes, Roland 1988 [1964] The Semiotic Challenge. New York: Hill and Wang. Block, Bruce 2008 The Visual Story. 2nd edition. Amsterdam, etc.: Focal Press.

114

Eduardo Urios-Aparisi

Caballero, Rosario 2006 Re-viewing Space: Figurative Language in Architects Assessment of Built Space. Berlin/New York: Mouton de Gruyter. this vol. Cutting across the senses: Imagery in winespeak and audiovisual promotion. Cameron, Lynne 2003 Metaphors in Educational Discourse. London/New York: Continuum. Cook, Guy 2001 The Discourse of Advertising. London/New York: Routledge. Croft, William 1993 The role of domains in the interpretation of metaphors and metonymies. Cognitive Linguistics 4: 335370. Dirven, Ren 2002 Metonymy and metaphor: Different mental strategies of conceptualization. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralph Prings (eds), 75112 Berlin/New York: Mouton de Gruyter. Dirven, Ren, and Ralph Prings (eds.) 2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New York: Mouton de Gruyter. Dirven, Ren, Frank Polzenhagen, and Hans-Georg Wolf 2007 Cognitive Linguistics, Ideology and Critical Discourse Analysis. In The Oxford Handbook of Cognitive Linguistics, Dirk Geeraerts and Hubert Cuyckens (eds.), 12221240. Oxford: Oxford University Press. El Refaie, Elisabeth this vol. Metaphor in political cartoons: Exploring audience responses. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379402. Berlin/New York: Mouton de Gruyter. 2007 Multimodal metaphor in ten Dutch TV commercials. Public Journal of Semiotics 1: 1951, available at http://semiotics.ca/. 2008 Pictorial and multimodal metaphor in commercials. In Go Figure! New Directions in Advertising Rhetoric, Edward F. McQuarrie and Barbara J. Phillips (eds.), 272310. Armonk, NY: ME Sharpe. this vol. The role of non-verbal sound and music in multimodal metaphor.

Interaction of multimodal metaphor and metonymy

115

Goossens, Louis 1990 Metaphtonymy: The interaction of metaphor and metonymy in figurative expressions for linguistic action. In By Word of Mouth: Metaphor, Metonymy and Linguistic Action in a Cognitive Perspective, Louis Goosens, Paul Pauwels, Brygida Rudzka-Ostyn, Anne-Marie Simon-Vandenbergen, and Johan Vanparys (eds.), 159174. Amsterdam/Philadelphia: Benjamins. Jakobson, Roman 1971 [1956] Two aspects of language and two types of aphasic disturbances, in Selected Writings II: Word and Language. The Hague: Mouton. Koller, Veronika this vol. Brand images: Multimodal metaphor in corporate branding messages. Kvecses, Zoltan 2000 Metaphor and Emotion: Language, Culture and Body in Human Feeling. Cambridge: Cambridge University Press. 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Kvecses, Zoltan, and Gnter Radden 1998 Metonymy: Developing a cognitive linguistic view. Cognitive Linguistics 9: 3777. Kress, Gunther, and Theo van Leeuwen 1996 Reading Images: The Grammar of Visual Design. London/New York: Routledge. Lakoff, George 1987 Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago: University of Chicago Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press. Lakoff, George, and Mark Turner 1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. Messaris, Paul 1997 Visual Persuasion: The Role of Images in Advertising. Thousand Oaks, CA/London/New Delhi: SAGE. McQuarrie, Edward F., and Barbara J. Phillips 2005 Indirect persuasion in advertising: How consumers process metaphors in pictures and words. Journal of Advertising 34: 721. Mittelberg, Irene, and Linda Waugh this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture

116

Eduardo Urios-Aparisi

Pankhurst, Anne 1997 Interpreting unknown worlds: Functions of metonymic conceptualization in William Goldings The Sea Trilogy. Language and Literature 6: 121131. Panther, Klaus-Uwe, and Gnther Radden 1999 Metonymy in Language and Thought. Amsterdam/Philadelphia: Benjamins. Panther, Klaus-Uwe, and Linda L. Thornburg, (eds.) 2003 Metonymy and Pragmatic Inferencing. Amsterdam/Philadelphia: Benjamins. Phillips, Barbara J., and Edward F. McQuarrie 2002 The development, change, and transformation of rhetorical style in magazine advertisements 19541999. Journal of Advertising 31 (4): 113. 2004 Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing Theory 4 (12): 113136. Radden, Gnther 2000 How metonymic are metaphors? In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 93 108. Berlin/New York: Mouton de Gruyter. Ruiz de Mendoza, Francisco Jos 1999 Introduccin a la teora cognitiva de la metonimia. Granada: Mtodo ediciones. 2000 The role of mappings and domains in understanding metonymy. In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 109132. Berlin/New York: Mouton de Gruyter. Ruiz de Mendoza, Francisco Jos, and Olga Isabel Dez Velasco 2002 Patterns of conceptual interaction. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralph Prings (eds), 489532. Berlin/New York: Mouton de Gruyter. Sperber, Dan, and Deirdre Wilson 1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Blackwell. Scott, Linda, and Rajeev Batra (eds.) 2003 Persuasive Imagery: A Consumer Response Perspective Mahwah, NJ/London: Lawrence Erlbaum. Stckl, Hartmut 2004 In between modes. In Perspectives on Multimodality, Elija Ventola, Cassily Charles, and Martin Kaltenbacher (eds.), 930. Amsterdam/ Philadelphia: Benjamins.

Interaction of multimodal metaphor and metonymy

117

Tanaka, Keiko 1994 Advertising Language: A Pragmatic Approach to Advertisements in Britain and Japan. London/New York: Routledge. Taylor, John 2002 Category extension by metonymy and metaphor. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralph Prings (eds.), 323347. Berlin/New York: Mouton de Gruyter. Teng, Norman Y., and Sewen Sun 2002 Grouping, simile, and oxymoron in pictures: A design-based cognitive approach. Metaphor and Symbol 17: 295316. Urios-Aparisi, Eduardo 2004 Quarrelling about metaphors on love: A pragmatic approach. In Current Trends in the Pragmatics of Spanish, Maria Elena Placencia and Rosina Mrquez Reiter (eds.), 283310. Amsterdam/ Philadelphia: Benjamins. Warren, Beatrice 2006 Referential Metonymy. Lund: Almqvist and Wiksell. Yu, Ning this vol. Nonverbal and multimodal manifestations of metaphors and metonymies: A case study. Yus, Francisco this vol. Visual metaphor versus verbal metaphor: A unified account.

Chapter 6 Nonverbal and multimodal manifestations of metaphors and metonymies: A case study Ning Yu

Abstract
This paper intends to analyze, within the cognitive linguistic paradigm, the nonverbal and multimodal manifestations of metaphors and metonymies in an educational commercial screened on China Central Television (CCTV). Specifically, it shows how two major conceptual metaphors, LIFE IS A JOURNEY and LIFE IS A STAGE, are manifested in dynamic visual and aural, as well as verbal, discourse. The various visual, aural and verbal elements are interactive with and dependent upon each other when they combine into a conceptual blend with input spaces in visual, aural and verbal modes. This blend contains conspicuous juxtapositions of various kinds, simultaneous or sequential, which cast in relief the unity and contrast between the Chinese and the Western, between thought and action, between primitivity and modernity, and between tradition and innovation. They all contribute to the central theme of the commercial that China, thanks to a motivation for change that originates in her heart, has been undergoing the process of modernization and globalization while retaining her cultural identity. Keywords: conceptual metaphor, conceptual metonymy, conceptual blend, primary metaphor, complex metaphor, nonverbal and multimodal manifestations

1. Introduction This chapter intends to demonstrate the nonverbal and multimodal manifestations of metaphors and metonymies within the cognitive linguistic paradigm, focusing on the example of an educational advertisement screened on China Central Television (CCTV). In contrast with commercial advertisements, on CCTV, educational advertisements are designed to influence peoples way of thinking and understanding for the purpose of promoting public welfare. They are, therefore, usually called advertisements for public good

120

Ning Yu

(). While educational advertisements on CCTV are distinct from ordinary TV commercials, which have business promotion as their purpose, I will still use the term commercial or TV commercial, in its broader sense, to refer to the one under discussion in this chapter. In this chapter the term metaphor is used in a broad and a narrow sense. The broad sense includes both metaphor and metonymy in the narrow sense of the terms. Actually, the distinction between metaphor and metonymy is scalar, rather than discrete: they seem to be points on a continuum of mapping processes (Barcelona 2000b: 16). According to cognitive linguistics, metonymy is a more fundamental cognitive phenomenon than metaphor, and metaphor is very often motivated by metonymy (see Barcelona 2000a; Dirven and Prings 2002; Panther and Radden 1999). This cognitive linguistics view of metaphor and metonymy will gain further support in the analysis that follows. The TV commercial to be analyzed is about two minutes long. It converges on the linguistic presentation of a short verbal message like a motto:
In everyones heart there is a big stage; however big ones heart is, that is how big the stage is

While this verbal line is itself metaphorical in nature, it serves, I will argue, as the core of the educational TV commercial that is constructed as a blend of cultural beliefs and conceptual metaphors presented visually and aurally as well as verbally. I will show that there are two major conceptual metaphors underlying this commercial: LIFE IS A JOURNEY and LIFE IS A STAGE. Of these two the former is manifested visually, i.e., through visual images, whereas the latter is realized as a multimodal metaphor, i.e., a metaphor whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006a: 384/this volume). I will suggest that although the TV commercial is centered on a Chinese country girl who dances all the way from a rural village to a modern metropolis, it can be interpreted allegorically as containing a personification of China, which has been undergoing a process of modernization and globalization. In what follows, I first provide a brief review of the cognitive linguistic paradigm of metaphor studies. Then, following a synopsis of the TV commercial, I analyze the nonverbal and multimodal manifestations of metaphors and metonymies involved to reveal how a multi-faceted metaphorical complex is achieved via contributions from and constructions in visual and aural, as well as verbal, modes.

Nonverbal and multimodal metaphors and metonymies

121

2. Cognitive linguistic theory of conceptual metaphor During the past two decades, cognitive science has seriously challenged the fundamental assumption that most of our thinking about the world is literal, directly corresponding to external reality (see e.g., Gibbs 2006; Lakoff and Johnson 1999). The results of cognitive linguistic studies show that human minds are embodied in the cultural world, and thinking and reasoning are largely metaphorical and imaginative, shaped by embodied and acculturated experiences (e.g., Gibbs 1994, 2006; Johnson 1987; Lakoff 1987; Lakoff and Johnson 1980, 1999). It is argued that all cognition is embodied in cultural situations (Gibbs 1999: 156). According to the conceptual metaphor theory of cognitive linguistics, metaphor is not merely a figure of speech, but also a figure of thought, giving rise to understanding one conceptual domain in terms of another conceptual domain.1 Conceptual metaphors in peoples conceptual systems influence to a considerable extent how they think, understand, reason, and imagine in everyday life, and many concepts, especially abstract ones, are structured and mentally represented in terms of metaphor (Gibbs 1999: 145). It is worth stressing that the experiential basis of conceptual metaphors is both bodily and cultural. Cognitive linguistics maintains that our minds are embodied in such a way that our conceptual systems draw largely upon the peculiarities of our bodies and the specifics of our physical and cultural environments (e.g., Gibbs 1994, 2006; Lakoff and Johnson 1980, 1999). According to the cognitive linguistic view, conceptual metaphors emerge from the interaction between body and culture: they are grounded in bodily experience, but shaped by cultural understanding. In order to answer the question why some metaphors are widespread or even universal and others are culture-specific, the newer version of Conceptual Metaphor Theory puts forth a decomposition account based on the distinction between two kinds of conceptual metaphors: primary metaphors and complex metaphors (see Grady 1997a, 1997b, 2005; Kvecses 2002, 2005; Lakoff and Johnson 1999, 2003). In short, primary metaphors are derived directly from experiential correlations, or conflations in everyday experience that pair subjective experience and judgment with sensorimotor experience (Lakoff and Johnson 1999: 49), whereas complex metaphors are combinations of primary metaphors and cultural beliefs and assumptions and, for that reason, tend to be culture-specific. For instance, Lakoff and Johnson (1999: 6061) suggest that the complex metaphor A PURPOSEFUL

122

Ning Yu

LIFE IS A JOURNEY is composed of the following cultural belief (reformulated here as two propositions) and two primary metaphors:
PEOPLE SHOULD HAVE PURPOSES IN LIFE PEOPLE SHOULD ACT SO AS TO ACHIEVE THEIR PURPOSES PURPOSES ARE DESTINATIONS ACTIONS ARE MOTIONS

Whereas the two primary metaphors (PURPOSES ARE DESTINATIONS and ACTIONS ARE MOTIONS), based on common bodily experience, are likely to be universal, the complex metaphor (A PURPOSEFUL LIFE IS A JOURNEY) is not or less likely to be so. This is because its validity in a particular culture depends on this cultures holding the combination of the two propositions (PEOPLE SHOULD HAVE PURPOSES IN LIFE and PEOPLE SHOULD ACT SO AS TO ACHIEVE THEIR PURPOSES) and the two primary metaphors, as listed above. A more recent development that is related to Conceptual Metaphor Theory is Blending or Conceptual Integration Theory (e.g., Fauconnier and Turner 1998, 2002; Turner and Fauconnier 1995). Different from conceptual metaphor theorys conceptual mapping from a source to a target domain, this theory postulates conceptual integration of two or more input spaces, which share a generic space of common characteristics, into a new blended space. This multiple-space model can account for various metaphorical and non-metaphorical aspects of conceptual phenomena (see also Grady, Oakley, and Coulson 1999; Kvecses 2002; Forceville 2004 for detailed discussions). A major difference between conceptual metaphor theory and blending theory is that the former is typically concerned with entrenched conceptual relationships and the ways in which they may be elaborated whereas the latter often focuses on novel conceptualizations, which may be short-lived (Grady, Oakley, and Coulson 1999). In an effort to bridge the theories of conceptual metaphor and conceptual integration, it is proposed that primary metaphors, as entrenched metaphoric correspondences between concepts, often function as inputs to metaphoric blends in the process of conceptual integration (Grady 2005). The rise of the cognitive linguistic paradigm of metaphor studies has created opportunities for the study of nonverbal and multimodal metaphors. However, as Forceville (2006a) points out, Conceptual Metaphor Theory has so far been restricted in an important dimension. While it characterizes metaphors as primarily conceptual in nature and only secondarily manifested in language, the validity of its claims about the existence of conceptual

Nonverbal and multimodal metaphors and metonymies

123

metaphors depends almost exclusively on linguistic evidence in the form of verbal metaphors. If, as it claims, metaphor fundamentally characterizes thinking, and is thus not an exclusive attribute of language, it should be able to produce nonverbal manifestations as well as the purely verbal ones that have so far been the central concern of conceptual metaphor studies. If metaphor does not necessarily appear in verbal form, conceptual metaphor theory can hardly afford to ignore the nonverbal realm. For pioneering work on nonverbal and multimodal metaphors within the cognitive linguistic paradigm, readers are referred to Forcevilles empirical and theoretical studies (e.g., 1994, 1996, 1999, 2002, 2005, 2006a, 2006b). In the light of Forcevilles (2006a) argument, this chapter is part of the attempt to demonstrate, within the cognitive linguistic paradigm, that conceptual metaphors can be manifested nonverbally and multimodally as well as verbally.

3. Synopsis of the commercial Here is a synopsis of the TV commercial under discussion. At the beginning, a close-up shot from a low camera angle focuses on a Chinese country girl, wearing peasant-style attire and posed for Western ballroom dancing (figure 1). With the playing of the slow-tempo music of the Chinese folk song Lan Huahua,2 the girl starts dancing elegantly but repetitively, turning around and around, all alone, in a snow-covered countryside. She keeps turning and turning, along a country path, and through a village with small country houses (figure 2). As she dances past, it can be seen that she has a gracious smile on her face, apparently absorbed in the joy of dancing the Westernstyle ballroom dance, despite the fact that she does not even have a dancing partner. Then, in an urban setting, she dances past a traditional-looking tall dark-red wall (figure 3) and then a Western-looking sculpture (figure 4), and finally up to the top of a skyscraper, against the metropolitan background bathing in the sun (figure 5). At this point, the line In everyones heart there is a big stage appears on the screen, getting closer and bigger, as the backdrop turns into a darkened screen (figure 6). All of a sudden, the audio shifts from the slow-tempo Chinese folk music to fast-tempo Western ballroom dance music. Now, the country girl is dancing with a male partner in a swallow-tailed tuxedo. Together they make a great variety of beautiful moves and poses (figure 7). Again, they dance past the Western-looking sculpture (figure 8) and the traditional-looking dark-red wall (figure 9), and then back up to the round

124

Ning Yu

top of another skyscraper, this time with 24 other pairs of similar-looking dancers following them. While taking the leading role, the first pair dances around the top of the skyscraper, followed by the other 24 pairs (figure 10). Then, as the leading couple dances in the foreground, the remaining 24 pairs change into a matrix of four by six dancing in the background (figure 11). At this time, the line However big ones heart is, that is how big the stage is draws nearer and larger while the background fades into a black screen (figure 12). After this, the country girl is alone again, turning around slowly to a stop (figure 13). Finally, as the audio shifts back to the Chinese folk music of Lan Huahua, the girl stands still, with her back toward the audience, looking far at the skyline of the modern metropolis (figure 14). The final scene provides a global view - produced by a long shot from a high camera angle - of the big city with its many tall buildings.

Figure 1. Posed for dancing

Figure 2. Dancing past a village

Figure 3. Dancing past a wall

Figure 4. Dancing past a sculpture

Figure 5. Dancing on a skyscraper

Figure 6. Stage in the heart

Nonverbal and multimodal metaphors and metonymies

125

Figure 7. Dancing with a partner

Figure 8. Dancing past the sculpture

Figure 9. Dancing past the wall

Figure 10. Leading the way

Figure 11. Leading and supporting

Figure 12. Size of heart and stage

Figure 13. Dancing alone again

Figure 14. Gazing afar standing

4. Analysis In this section, I analyze the TV commercial to show that its didactic and aesthetic effects are achieved through, among other things, nonverbal and multimodal manifestations of two common conceptual metaphors: LIFE IS A JOURNEY and LIFE IS A STAGE. While the former is realized almost exclu-

126

Ning Yu

sively through moving images, accompanied by musical sounds, the latter one is manifested visually and aurally, as well as verbally through the linguistic message: In everyones heart there is a big stage; however big ones heart is, that is how big the stage is. Other than these two conceptual metaphors, the TV commercial also contains several conceptual metonymies, which both motivate and constitute the metaphors, and help set in relief the unity and contrast of cultural identity and globalization that characterize contemporary China. Before I proceed to analyze the conceptual metaphors and metonymies that structure the TV commercial, I first briefly comment on the Chinese cultural conceptualization of the HEART and the verbal message that serves as the core of the advertisement under discussion. In the tradition of Chinese culture, the heart (xin) is regarded as the organ for thinking and understanding as well as feeling, and more generally as the central faculty of cognition (Yu 2003, 2005, 2007a, 2007b). This cultural conceptualization of the heart contrasts with the Western dualism that maintains the heart-mind dichotomy, i.e., the heart is the seat of emotions whereas the mind, associated with the brain, is the center of thoughts. In light of the above comment, I would like to point out that the verbal message in the TV commercial is a manifestation of the Chinese conceptualization of the heart as the central faculty of cognition, as well as an instantiation of the popular conceptual metaphor LIFE IS A STAGE. On the stage of life, people play various roles, some being more important and successful than others. Ones degree of success in external life (the size of the stage) is attributed and related to the mental capacity of ones internal world, the heart (the size of the heart), in a metaphorical fashion. That is, only when one can think big (i.e., with a big heart) can one act big on the big stage of life. So interpreted, the verbal message of the TV commercial reveals the following combination of propositions and metaphors:
(1) a. b. c. d.
HEART IS THE THINKING ORGAN THAT DESIGNS ACTIVITY OF LIFE SUCCESS IN LIFE ORIGINATES IN HEART DEGREE OF MOTIVATION FOR SUCCESS IS SIZE OF HEART MORE MOTIVATED FOR SUCCESS IS BIGGER OF HEART

In this group, (1a) and (1b) are two propositions that reflect the Chinese cultural conceptualization of the heart whereas (1c) and (1d) are metaphors that are rooted in the cultural beliefs of the heart as the central fac-

Nonverbal and multimodal metaphors and metonymies

127

ulty of cognition. The metaphorical nature of the culturally constructed understanding of the heart is quite obvious. This understanding can be summarized by a more general complex metaphor: ONES MENTAL CAPACITY IS SIZE OF ONES HEART. 4.1. LIFE IS A JOURNEY I first analyze the conceptual metaphor LIFE IS A JOURNEY, which is fundamentally manifested through moving images. In the TV commercial under discussion, the girl undertakes a journey, going or, more exactly, dancing all the way from the field of the snow-covered countryside to the top of a skyscraper in a large metropolitan area. This journey, however, is metaphorically designed and constructed to manifest, visually, the common conceptual metaphor LIFE IS A JOURNEY. In other words, it is not a physical journey taking place through space, but one that is a metaphor for subjective experience and abstract advancement in life. The cueing of the dancing as a JOURNEY metaphor for abstract life is achieved mainly through the shifts of scenes that show spatial, as well as temporal, leaps and bounds across various physical settings. While the country girl repeats her monotonous but graceful act of dancing alone, her path extending through conspicuous differences in the environment, free from any restrictions, makes a literal reading of her dancing impossible. The LIFE IS A JOURNEY metaphor, which is based on the SOURCE-PATHGOAL schema,3 creates mappings from the source domain of journey to the target domain of life and establishes correspondences between various items within these two conceptual domains, such as those shown in (2). The arrows indicate the direction of the metaphorical mappings from the source to the target domain.
(2) LIFE IS A JOURNEY
SOURCE TARGET

a. b. c. d. e. f.

JOURNEY TRAVELER STARTING POINT TRAVEL ON JOURNEY PATH OF JOURNEY DESTINATION

LIFE PERSON INITIAL STATE EXPERIENCE IN LIFE WAY OF LIFE GOAL

The experiential motivation for this metaphor is that people do undertake many journeys as part of their life. In this sense, the metaphor LIFE IS A

128

Ning Yu

has a metonymic basis when JOURNEY is mapped onto LIFE, characterized by a conceptual metonymy PART FOR WHOLE. In the TV commercial under discussion, the traveler is the country girl. For her, traveling is dancing a Western ballroom dance that she really enjoys, even though she does it initially without a partner. For her, the path of the journey runs from the cold of the snow-covered countryside to the warmth of the sun-bathed modern metropolis, and from the country field to the top of a skyscraper in a big city. More abstractly, this is a path of going upward in spatial conceptualization of success in life. The correspondences listed in (3) and (4) are some of the entailments of the mappings given in (2) (see also Lakoff 1993):
JOURNEY

(3) EXPERIENCE IN LIFE IS TRAVEL ON JOURNEY a. DIFFICULT TRAVEL BAD EXPERIENCE b. EASY TRAVEL GOOD EXPERIENCE c. FAST MOTION FAST PROGRESS d. SLOW MOTION SLOW PROGRESS (4) WAY OF LIFE IS PATH OF JOURNEY a. PHYSICAL CONDITIONS b. BUMPY PATH c. SMOOTH PATH
ABSTRACT STATES DIFFICULT WAY EASY WAY

This is how conceptual metaphors systematically transfer inferences or entailments, based on our bodily experience, from the source to the target domain. It is noteworthy that LIFE IS A JOURNEY is a complex metaphor that represents the combination of some primary metaphors and a number of cultural beliefs, as shown in (5):
(5) LIFE IS A JOURNEY a. PEOPLE SHOULD HAVE GOALS IN THEIR LIFE b. PEOPLE SHOULD ACT SO AS TO ACHIEVE THEIR GOALS c. STATES ARE LOCATIONS d. CHANGES ARE MOVEMENTS (FROM ONE TO ANOTHER LOCATION) e. CAUSES ARE FORCES f. ACTIONS ARE SELF-PROPELLED MOTIONS g. PURPOSES ARE DESTINATIONS

Here, (5a) and (5b) present two propositions as the cultural beliefs or assumptions upheld by people who subscribe to the complex metaphor LIFE IS A JOURNEY; (5cg) are primary metaphors of the so-called Event Structure

Nonverbal and multimodal metaphors and metonymies

129

Metaphor, which is a metaphor system responsible for the conceptualization of various abstract events (see Lakoff 1993; Yu 1998, Ch. 5). In the commercial, the country girl has her goal in life and she acts, persistently, to attain this goal. During the process, she has changed the state of her life. More specifically, she has made changes by moving (in this case, dancing a Western-style ballroom dance) from one location (the field of the backward countryside) to another location (the top of a skyscraper of a modern city). The stark contrast between these two locations represents, metaphorically, the marked difference between two states in her life that the change has brought to her. The cause for this change is the motivation or ambition in her heart, which is the force that pushes her to make repetitive but beautiful moves (keeping turning around in her ballroom dance) in a consistent and persistent manner, until she reaches her destination, when she is joined by a male dancing partner, and then followed by 24 other pairs of dancers. Her destination (her goal) is the top of the skyscraper, which looks like a highly elevated stage (the STAGE metaphor), from which she can enjoy a full view of the world around her. Physically, a vantage point on a higher location enables people to see farther, and that is what happens when the country girl gazes afar on the top of the skyscraper at the very end of the commercial. Metaphorically, the physical perception of seeing is mapped onto the mental function of understanding in the primary conceptual metaphor UNDERSTANDING IS SEEING (see Lakoff and Johnson 1999; Sweetser 1990; Yu 2004). Thus, we have a two-step mapping: HIGHER LOCATION SEEING FARTHER UNDERSTANDING BETTER. Of these two mappings, the first is metonymic whereas the second is metaphorical. Both mappings are manifested in the TV commercial visually. At this point, I want to underscore another aspect of the significance of the country girl ending up on the top of a skyscraper after dancing all the way from the country field. This, I believe, is the visual manifestation of a primary conceptual metaphor SUCCESSFUL IS UP, i.e., A MORE SUCCESSFUL STATE IS A HIGHER LOCATION, which is combined with the LIFE IS A JOURNEY metaphor to form another complex metaphor, as in (6):
(6) SUCCESSFUL CAREER IN LIFE IS UPWARD MOVEMENT ON JOURNEY a. LIFE IS A JOURNEY b. SUCCESSFUL IS UP

The list in (7) below gives some of the conceptual mappings entailed by (6b) from the concrete domain of space to the abstract domain of success.

130

Ning Yu

(7) SUCCESSFUL IS UP a. UP b. HEIGHT OF LOCATION c. HIGH LOCATION d. HIGHER e. LOWER

SUCCESSFUL DEGREE OF SUCCESS IN LIFE SUCCESSFUL STATUS IN LIFE MORE SUCCESSFUL LESS SUCCESSFUL

That is, DEGREE OF SUCCESS IN LIFE IS HEIGHT OF LOCATION. HIGHER is mapped onto MORE SUCCESSFUL whereas LOWER is mapped onto LESS SUCCESSFUL. A higher location represents a more successful status in life, and a lower location represents a less successful status in life.4 In short, the journey that the country girl has undertaken is a journey from a small village to a large metropolis, and from backwardness to modernity. At the end of the journey, she can enjoy, at a very high vantage point, a global view of her world that she could not have had if she had not danced all the way from the field of the countryside to the top of a skyscraper in a modern city. 4.2. LIFE IS A STAGE Now I turn to analyzing the multimodal manifestation of the conceptual metaphor LIFE IS A STAGE. This conceptual metaphor is, again, a complex metaphor that represents the combination of a number of components at different levels. First consider (8) below:
(8) LIFE IS A STAGE a. PEOPLE ACT TO ACHIEVE SUCCESS IN LIFE b. PEOPLES ACTION IN LIFE IS EVALUATED BY OTHERS c. ACTION IN LIFE IS ACTION ON STAGE d. STATES ARE LOCATIONS e. ACTIONS ARE SELF-PROPELLED MOTIONS

Here (8c) is the key metaphorical component. Since ACTION IN LIFE IS ACTION ON STAGE, after the identical item, ACTION, on both sides of the equation is eliminated, we have LIFE IS A STAGE. But what is the experiential motivation for (8c)? I argue that this metaphor is motivated by a more fundamental figurative relationship, a metonymy, ACTING ON STAGE STANDS FOR ACTING IN LIFE, which is a specific instantiation of the more general conceptual metonymy PART STANDS FOR WHOLE. That is, acting on the stage is only part of the whole, acting in life, and here we use a part to stand for the whole. In the above list, (8d) and (8e) represent two primary meta-

Nonverbal and multimodal metaphors and metonymies

131

phors in the Event Structure Metaphor System on which the complex metaphor LIFE IS A STAGE is based. Life is a series of states whereas a stage is a special kind of location. The actions that people take in life, whether concrete or abstract, are generally understood as self-propelled motions through space. In this particular case, actions taken in life are metaphorically conceptualized as artistic moves of ballroom dancing. Besides, I assume that the cultures that subscribe to the LIFE IS A STAGE metaphor also hold the propositions in (8a) and (8b), in combination with (8c). Thus, the conceptual parallel is perceived as follows. People act to achieve success in life, just as performers act to achieve success on the stage; their actions in life are evaluated by others, just as actors and actresses performances are evaluated by their audience. The LIFE IS A STAGE metaphor establishes, for instance, the correspondences between the following elements in the two conceptual domains.
(9) LIFE IS A STAGE
SOURCE TARGET

a. b. c.

STAGE PERFORMANCE ON STAGE ROLES ON STAGE

LIFE ACTIVITY IN LIFE PEOPLE IN LIFE

In the TV commercial, the country girl dances all the way from a small village to a big city, so the stage of her life is indeed very big. Her performance can be divided into four phases. In the first phase, she is alone and starts dancing ballroom dance. Her moves, though graceful, are repetitive, metaphorically representing, I suggest, her persistence and perseverance in pursuit of her goal. The fact that the Chinese country girl dances a Western ballroom dance, rather than a Chinese folk dance, is unexpected and for that matter really significant. In China, the Western ballroom dance is often referred to as the international standard dance. During the past 15 years or so, a goal that China has been trying to achieve to implement the reform and open-door policy is to be connected with the international track/rail (), i.e., to meet the international standard. Thus, the country girls persistence in dancing the international standard dance can be seen as a visual metaphor of Chinas effort to meet the international standard. In the second phase, the country girl is joined by a male dancing partner wearing the standard ballroom dance apparel (i.e., a black swallow-tailed tuxedo and black leather shoes), as in sharp contrast with her Chinese peasant-style clothing. Together, they make all kinds of beautiful moves and poses, accompanied by Western ballroom dance music. Their fast-tempo

132

Ning Yu

movements, accompanied by the fast-tempo music, are metaphorical of their speedy advancement in life. Such a conspicuous juxtaposition of the Chinese and the Western in the pair dance seems to be a visual cue for Chinas cooperation with the outside, especially the Western world. This cooperation is essential for Chinas process of modernization and globalization. In the third phase, the country girl, with her dancing partner, plays a leading role in dancing when followed by 24 other couples, and is metaphorically a leader in life. The leading role in the group dance seems to cue the increased influence of China in the international arena or on the international stage. In the fourth and last phase, the girl is alone again, stops dancing, and gazes afar while standing still. This is when she achieves some deep understanding of life when gaining a far vision, i.e., UNDERSTANDING IS SEEING (see Lakoff and Johnson 1999; Yu 2004): In everyones heart there is a big stage; however big ones heart is, that is how big the stage is. While enjoying the global vision at the high vantage point, the girl stands with her back toward the viewers, looking ahead into the future (since according to the Chinese version of the TIME IS SPACE metaphor the future is ahead; see, e.g., Lakoff 1993; Yu 1998, Ch. 4). This shot design seems to put the viewers into a following position, i.e., they are invited to follow suit looking ahead into Chinas bright future while sharing the girls understanding of life as specified by the verbal message of the advertisement. The metaphorical mappings in (9b) and (9c), for instance, have the following entailments, transferring further knowledge from the source to the target domain:
(10) ACTIVITY IN LIFE IS PERFORMANCE ON STAGE a. SUCCESS ON STAGE b. FAILURE ON STAGE c. VARIETY IN PERFORMANCE (11) PEOPLE IN LIFE ARE CHARACTERS ON STAGE a. LEADING CHARACTERS ON STAGE b.
SUPPORTING CHARACTERS ON STAGE SUCCESS IN LIFE FAILURE IN LIFE DIVERSITY IN ACTIVITY

MORE SUCCESSFUL PEOPLE IN LIFE LESS SUCCESSFUL PEOPLE IN LIFE

As shown in (10) and (11), the entailments all contribute to the systematic mappings from the source to the target domain, activated by the LIFE IS A STAGE metaphor. In the TV commercial, the country girl is indeed very successful in life since, later in the show, she is playing a leading role in the foreground of the stage.

Nonverbal and multimodal metaphors and metonymies

133

However, LIFE IS A STAGE is still not sufficient to capture the figurative meaning of the verbal message in particular and the TV commercial in general. It still needs to combine with another primary metaphor, SUCCESSFUL IS BIG, so as to form another complex metaphor, A SUCCESSFUL LIFE IS A BIG STAGE. This further combination is given in (12).
(12) A SUCCESSFUL LIFE IS A BIG STAGE a. LIFE IS A STAGE b. SUCCESSFUL IS BIG

Given in (13) below are some of the conceptual mappings entailed by (12b) from the concrete domain of spatial dimensions to the abstract domain of success.
(13) SUCCESSFUL IS BIG a. BIG b. SIZE OF OBJECT c. BIG OBJECT d. BIGGER e. SMALLER
SUCCESSFUL DEGREE OF SUCCESS IN LIFE SUCCESSFUL STATUS IN LIFE MORE SUCCESSFUL LESS SUCCESSFUL

That is, the physical size of an object is mapped on to the degree of success.5 In (12) the OBJECT is specified as a stage. Thus, (14) lists some of the conceptual mappings entailed by (12):
(14) A SUCCESSFUL LIFE IS A BIG STAGE a. BIG b. SIZE OF STAGE c. BIG STAGE
SUCCESSFUL DEGREE OF SUCCESS IN LIFE SUCCESSFUL STATUS IN LIFE

That is to say, the size of ones stage is metaphorically correlated with the degree of success in ones life: DEGREE OF SUCCESS IN LIFE IS SIZE OF STAGE. The bigger ones stage is (of course metaphorically), the more successful one is in life. It is worth noting that the top of the skyscraper, where the girl, her dancing partner, and 24 other pairs are dancing, looks very much like a big stage. Thus, the metaphor A SUCCESSFUL LIFE IS A BIG STAGE is manifested visually as well as verbally. In summary, the LIFE IS A STAGE metaphor is manifested visually through moving images, accompanied by musical sounds, as well as linguistically through the verbal message appearing on the TV screen. The fact that the country girl has made remarkable progress in life is revealed via visual metaphors. First, the change in physical location of her dancing from the

134

Ning Yu

backward countryside to the modern metropolis metaphorically suggests her abstract advancement in life. Second, the change in the manner of her dancing, from solo dancing, to dancing as a couple, and to dancing as the leading pair of a group, also metaphorically suggests her general progress in life. It is worth mentioning that the metaphor A SUCCESSFUL LIFE IS A BIG STAGE also has its metonymic motivation, which can be expressed by the metonymy STAGE STANDS FOR PERFORMANCE ON STAGE, or more generally LOCATION OF ACTIVITY STANDS FOR ACTIVITY. The network of metonymic and metaphoric relations involved is shown below:
STAGE FOR PERFORMANCE + LIFE IS A STAGE + SUCCESSFUL IS BIG STAGE SIZE OF STAGE BIG STAGE

PERFORMANCE ON STAGE DEGREE OF SUCCESS ON STAGE SUCCESSFUL PERFORMANCE ON STAGE

ACTIVITY IN LIFE DEGREE OF SUCCESS IN LIFE SUCCESSFUL ACTIVITY IN LIFE

That is, as the first step, STAGE is mapped metonymically onto the PERFORMANCE ON STAGE and, as the second step, PERFORMANCE ON STAGE is mapped metaphorically onto ACTIVITY IN LIFE. 4.3. Multimodal metonymies Apart from the two major conceptual metaphors discussed above, the TV commercial has also deployed a number of metonymies to achieve its didactic purpose and artistic effect, as has already been touched upon above. For instance, in the verbal message In everyones heart there is a big stage; however big ones heart is, that is how big the stage is, we can say that, initially, the reference to the stage is a metonymy for the performance on the stage, i.e., STAGE FOR PERFORMANCE ON STAGE, or more generally LOCATION OF ACTIVITY FOR ACTIVITY. In this case, the figurative mapping takes place from one thing to another within the same conceptual domain. It is through further mapping across the domains that the metaphor ACTIVITY IN LIFE IS PERFORMANCE ON STAGE is constructed. In this section, I discuss several other metonymies in the visual and aural modes. In effect, these metonymies under analysis are all integrated into the complex of conceptual metaphors. The first visual metonymy is STYLE OF CLOTHING STANDS FOR CULTURE. The girl wears typical peasant-style clothing, which is metonymically associated with traditionally agricultural Chinese culture. It fits well into the rural setting at the beginning of the commer-

Nonverbal and multimodal metaphors and metonymies

135

cial. However, this style of clothing is conspicuously at odds with Western ballroom dance. The most conspicuous contrast appears when the country girl in the Chinese peasant-style clothes is dancing the Western ballroom dance with a male partner in a black swallow-tailed tuxedo, which is the standard Western-style ballroom dance apparel. It is a conspicuous visual blend of contrasting Chinese and Western styles. Subsequently, 24 other pairs of dancers join them, wearing exactly the same attire as they do. I would suggest that the country girls peasant-style clothing is metonymic for the cultural identity of the Chinese in general, and it is part of the visual metaphor for the retention of cultural identity in the process of modernization and globalization. Although her surroundings have changed drastically over time, her Chinese peasant-looking appearance has remained the same. Another metonymy I want to mention is STYLE OF DANCE STANDS FOR CULTURE. Ballroom dance is associated metonymically with Western cultures in the developed countries that embody modernity and superiority in various areas in the world today. In the TV commercial, the country girl could have danced a Chinese folk dance, which would have been very appropriate for her identity represented metonymically by her Chinese peasantlooking appearance. Instead, what we see is a conspicuously inappropriate blend of the Chinese peasant-looking appearance and the Western elegance of ballroom dance. As is masterfully designed, indeed, the Chinese country girl dancing Western ballroom dance is a powerful visual metaphor for the process of modernization and globalization that China has been undergoing in the past twenty years or so. The metonymy STYLE OF DANCE STANDS FOR CULTURE, realized visually, is an important component of that complex metaphor. In the process of mapping, we can trace the following steps of metonymic mapping governed by the principle of contiguity: BALLROOM DANCE WESTERN CULTURE DEVELOPED COUNTRIES MODERNIZATION AND GLOBALIZATION. However, if we omit and ignore the two intermediate steps, we have a cross-domain mapping that is metaphorical: BALLROOM DANCE IS PROCESS OF MODERNIZATION AND GLOBALIZATION. The third metonymy to be discussed is STYLE OF PHYSICAL SETTING FOR CULTURE. This metonymy is visualized mainly by the juxtaposition of two conspicuous contrasts representing different cultures. The first contrast consists of the countryside versus the metropolis. In the countryside, which is the physical setting of the first portion of the commercial, we see an open field covered by white snow, and a small village with small wood houses (in the Northeast of China). In stark contrast, what we see in the second half of the commercial is a large metropolitan area with numerous skyscrapers (in Shanghai). This represents the contrast between the underdeveloped and the

136

Ning Yu

developed, and between primitivity and modernity. The second contrast is that between the tall dark-red wall and the Western-looking sculpture, which are both shown twice. The first time, the country girl dances past them alone, and the second time, the girl and her partner dance past them together. The tall dark-red wall looks like those enclosing the Forbidden City (in Beijing), the royal palace that was off limits to ordinary people in the last feudal dynasties of China. It is, therefore, metonymically associated with a traditional culture of isolation characteristic of China before it was opened up to the outside world some twenty years ago. The Western-looking sculpture, by contrast, is a visual metonymy of the influence of Western culture present in contemporary China following the implementation of its open-door policy. It is worth pointing out that the visual contrasts brought out by the metonymy STYLE OF PHYSICAL SETTING FOR CULTURE play an important part in the visual manifestation of the two conceptual metaphors, LIFE IS A JOURNEY and LIFE IS A STAGE. Finally, I turn to the metonymy STYLE OF MUSIC STANDS FOR CULTURE. Two kinds of music are played through the commercial. At the beginning, as the country girl starts dancing her ballroom dance, the accompanying music is not Western ballroom dance music, but the music of the Chinese folk song Lan Huahua. Playing Chinese folk music instead of Western ballroom dance music adds to the cultural context and cultural identity created by the visual images of the country girls Chinese peasant-style attire and the physical setting of the Chinese countryside. That is, music is used as one of the tools to create cultural context and cultural identity. After the pair and group dancing, the country girl is alone again, standing motionless on top of a skyscraper, gazing at the panorama of the modern metropolis. The audio track, at this point, shifts back from the Western ballroom dance music to Lan Huahua, the Chinese folk song music, for the final seconds of the TV commercial. This shift in musical style is designed, I argue, to suggest, metonymically, the retention of cultural identity despite the fact that the physical setting has changed from the countryside to the metropolis, and from primitivity to modernity. The country girl has not lost her cultural identity, her appearance remaining the same, even though her state of life has drasticallychanged, as metaphorically and metonymically represented by the change of locations and physical settings. She is now embedded in a modernized and globalized environment, as visually represented by the global view of a modern metropolis, but her cultural identity is retained, as represented visually by her Chinese peasant-style attire, and aurally by the Chinese folk music played for the last few seconds to complete the commercial.

Nonverbal and multimodal metaphors and metonymies

137

In sum, the metonymies discussed in this section all fall into one general pattern, where PROTOTYPICAL ITEMS OF A CULTURE STAND FOR THAT CULTURE or SALIENT FEATURES OF A THING STAND FOR THAT THING. As we have seen, this conceptual metonymy can be manifested visually and aurally as well as verbally. 5. Conclusion One of the crucial insights of cognitive linguistic theory of metaphor is that verbal metaphors systematically manifest underlying conceptual metaphors. There is already ample and still growing linguistic evidence, in support of this claim, discovered by empirical studies of a broad spectrum of world languages and from a cross-cultural perspective. If, as cognitive linguists have argued, metaphor is primarily conceptual in nature as a cognitive mechanism characterizing the mode of thought or the way of thinking, it follows that conceptual metaphors should emerge in nonverbal manifestations as well as verbal ones. So far, there are not many studies focused on nonverbal and multimodal manifestations of conceptual metaphors despite the fact that such studies are theoretically essential to consolidate the validity of conceptual metaphor theory (see Forceville 2006a). This overwhelming preference to the study of verbal over nonverbal manifestations of conceptual metaphors needs correcting for the sound development of conceptual metaphor theory. The present study represents part of the attempt toward that end. In this chapter, I have analyzed the nonverbal and multimodal manifestations of two conceptual metaphors through dynamic visual and aural, as well as verbal, discourse. I have shown that these conceptual metaphors are complex ones composed of cultural beliefs and assumptions, other complex and primary metaphors, and metonymies. The various visual, aural and verbal elements are interactive with, and dependent upon, each other when they combine into a conceptual blend with input spaces in visual, aural, and verbal modes. This blend contains conspicuous juxtapositions, simultaneous or sequential, of contrasting visual, aural or verbal images that are metonymic and metaphoric in nature. These juxtapositions bring to the fore the unity and contrast between the Chinese and the Western, between thought and action, between primitivity and modernity, and between tradition and innovation. They all contribute to the central theme of the TV commercial that China, thanks to a motivation and ambition for change that originates in

138

Ning Yu

her heart, has been undergoing the process of modernization and globalization while retaining her Chinese characteristics.
Table 1. Some possible multimodal mappings of the metaphors LIFE IS A JOURNEY and LIFE IS A STAGE. Metaphors Source:
JOURNEY

Visual (1) the country girl dancing from the field of the countryside to the top of a skyscraper in the metropolis; (2) enjoying the global view at height and looking afar and ahead (a) the person/China undergoing the process of modernization and globalization; (2) gaining a deep understanding upon success and looking into the future the girl in peasant clothing (1) dancing Western ballroom dance alone; (2) dancing in pair with a male partner in Western tuxedo; (3) dancing as the leading couple; (4) still wearing the same peasant clothing in a changed environment The person/China, with her cultural identity, (1) making self-propelled effort toward modernization and globalization; (2) working in cooperation with the West; (3) advancing as a leader in development; (4) retaining cultural identity in modernization

Aural (1) starting with slow-tempo music; (2) switching to fast-tempo music; (3) winding down with slow-tempo music again (1) starting at a self-propelled pace; (2) switching to a fast speed; (3) slowing down to a stop

Verbal

Target:
LIFE

Source:
STAGE

(1) starting with Chinese music; (2) switching to Western music; (3) ending with Chinese music again

In everyones heart there is a big stage; however big ones heart is; that is how big the stage is.

Target:
LIFE

(1) Chinese culture; (2) internationalization; (3) retention of cultural identity

Success in life originates internally in ones mind; only when one thinks big can one act big and achieve big success in life.

Nonverbal and multimodal metaphors and metonymies

139

Table 1 summarizes some possible multimodal mappings of the two major conceptual metaphors, LIFE IS A JOURNEY and LIFE IS A STAGE, studied in this chapter. As can be seen, the first metaphor, LIFE IS A JOURNEY, is basically realized in the visual mode, whereas the aural mode performs a metonymic function in support of the visual manifestation of the metaphor in question, with the tempo of the music indexical of the tempo of the movement on the journey in the visual mode. The role of the verbal mode is absent in this case. With the second metaphor, LIFE IS A STAGE, the verbal message, which is only displayed on the screen, is crucial in its manifestation. This message, however, is very schematic, with all the details furnished in the visual mode by concrete visual images. In the aural mode, the contrast of Chinese and Western music is again metonymic for the respective cultures. In playing this metonymic function, however, the music, switched from Chinese to Western and back to Chinese again, reinforces the visual images that are metaphorical of the process of modernization and globalization and the retention of cultural identity. Thus, it is in interaction that the elements in different modes give rise to nonverbal and multimodal manifestation of metaphors and metonymies in the TV commercial. Acknowledgments
An earlier version of this paper, titled Cultural identity and globalization: Multimodal metaphors in a Chinese educational advertisement, was presented at the 12th Annual Conference of the International Association of Intercultural Communication Studies held in San Antonio in August 2006, and appeared in China Media Research 3(2): 2532, 2007. I am greatly appreciative of Charles Forceville and Mats Rohdin for their insightful comments on the earlier versions of this chapter, but I am solely responsible for any errors that may remain.

Notes
1. For cognitive linguistic studies of metaphor, metonymy, and figurative language in general, see, e.g., Barcelona (2000a), Dirven and Prings (2002), Gibbs (1994), Gibbs and Steen (1999), Johnson (1987), Kvecses (2002, 2005), Lakoff (1987), Lakoff and Johnson (1980, 1999), Lakoff and Turner (1989), Panther and Radden (1999), Sweetser (1990), Turner (1991, 1996), and Yu (1998). More exactly, Lan Huahua is a folk song from the northern Shaanxi Province, which belongs to the part of China considered as the place of origin of

2.

140

Ning Yu Chinese civilization. In China, songs of this kind are known as northern Shaanxi folk songs. See Forceville (2006b) for a detailed discussion of the SOURCE-PATH-GOAL schema in multimodal manifestations in three documentary films. The schema, which structures the concept of JOURNEY (starting point, travel, and destination) literally, is analyzed as shaping the understanding of such more abstract concepts as PURPOSEFUL LIFE (ambition, action, and achievement) and STORY (beginning, middle, and end) by metaphorical extension. The verbal manifestation of the metaphor SUCCESSFUL IS UP or A MORE SUCCESSFUL STATE IS A HIGHER LOCATION is extremely rich in Chinese. One lexical example should suffice here: the compound word gao-di (, literally high and low) means relative superiority or inferiority (e.g., in a contest or competition). Thus, zheng ge gao-di (, literally vie for the high and the low) means vie with each other to see who is better, and nan fen gao-di (, literally hard to distinguish between the high and the low) means hard to tell which is better. A nonverbal example, which is internationally accepted as a standard practice in sports competition, is that the first, second, and third place winners of an event are distinguished by the height of the platforms on which they receive their medals or trophies: i.e., the champions platform is the highest, and the runner-ups is higher than the third-place winners (and in addition, the champions platform is usually in the middle, i.e., IMPORTANT IS CENTRAL). Parallel to this distinction in height (and centrality) is the difference in height (and centrality) of the three national flags raised to honor the corresponding three athletic winners. In Chinese, again, the primary metaphor SUCCESSFUL IS BIG, parallel to IMPORTANT IS BIG, is manifested linguistically in an extremely rich fashion. For instance, a high-ranking official (gao-guan) is also called a big official (da-guan); a big wrist (da-wan) is a big-brand shining star (da-pai ming-xing) in acting or singing who has a big name (ming-qi da), a big shot (da-heng) in business is called a big sum (of money) (da-kuan), a VIP in general is a big person or big character (da-renwu), and so on and so forth. Various instances of nonverbal manifestation of SUCCESSFUL IS BIG can be found in daily life, too. We can go back to the sports example of the platform in note 4, i.e., the champions platform should be the biggest if there is a difference in size. Besides, trophies, if there are any, also vary in size with the places of their winners, with the first-place winner given the biggest one. And, if financial prizes are awarded, they should be related directly to the places of the winners too, with the champion getting the biggest or largest amount of money on a fattest check. Associated with big stars or big persons in general are big things, ranging from big cars (limousines) to big houses (mansions).

3.

4.

5.

Nonverbal and multimodal metaphors and metonymies

141

References
Barcelona, Antonio (ed.) 2000a Metaphor and Metonymy at the Crossroads: A Cognitive Perspective. Berlin/New York: Mouton de Gruyter. Barcelona, Antonio 2000b Introduction: The cognitive theory of metaphor and metonymy. In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 128. Berlin/New York: Mouton de Gruyter. Dirven, Ren, and Ralf Prings (eds.) 2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New York: Mouton de Gruyter. Fauconnier, Gilles, and Mark Turner 1998 Conceptual integration networks. Cognitive Science 22: 133187. 2002 The Way We Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Forceville, Charles 1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Activity 9: 129. 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 1999 The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 179198. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2004 Review of Fauconnier and Turner (2002). Metaphor and Symbol 19: 8389. 2005 Visual representations of the idealized cognitive model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2006a/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. 2006b The SOURCE-PATH-GOAL schema in the autobiographical journey documentary: McElwee, van der Keuken, Cole. New Review of Film and Television Studies 4: 241261. Gibbs, Raymond W., Jr. 1994 The Poetics of Mind: Figurative Thought, Language, and Understanding. Cambridge: Cambridge University Press. 1999 Taking metaphor out of our heads and putting it into the cultural world. In Metaphor in Cognitive Linguistics, Raymond W. Gibbs,

142

Ning Yu

Jr. and Gerard J. Steen (eds.), 145166. Amsterdam/Philadelphia: Benjamins. 2006 Embodiment and Cognitive Science. Cambridge: Cambridge University Press. Gibbs, Raymond W., Jr., and Gerard J. Steen (eds.) 1999 Metaphor in Cognitive Linguistics. Amsterdam/Philadelphia: Benjamins. Grady, Joseph 1997a Foundation of meaning: Primary metaphors and primary scenes. Ph.D. dissertation, Department of Linguistics, University of California at Berkeley. 1997b THEORIES ARE BUILDINGS revisited. Cognitive Linguistics 8: 267 290. 2005 Primary metaphors as inputs to conceptual integration. Journal of Pragmatics 37: 15951614. Grady, Joseph, Todd Oakley, and Seana Coulson. 1999 Blending and metaphor. In Metaphor in Cognitive Linguistics, Raymond W. Gibbs, Jr. and Gerard Steen (eds.), 101124. Amsterdam/Philadelphia: Benjamins. Johnson, Mark 1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University of Chicago Press. Kvecses, Zoltn 2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press. 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Lakoff, George 1987 Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago: University of Chicago Press. 1993 The contemporary theory of metaphor. In Metaphor and Thought, 2nd ed., Andrew Ortony (ed.), 202251. Cambridge: Cambridge University Press. Lakoff, George, and Mark Johnson 1980 Metaphor We Live By. Chicago: University of Chicago Press. 1999 Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books. 2003 Afterword, 2003. In Metaphors We Live By, 243276. Chicago: University of Chicago Press. Lakoff, George, and Mark Turner 1989 More Than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press.

Nonverbal and multimodal metaphors and metonymies

143

Panther, Klaus-Uwe, and Gnter Radden (eds.) 1999 Metonymy in Language and Thought. Amsterdam/Philadelphia: Benjamins. Sweetser, Eve E. 1990 From Etymology to Pragmatics: Metaphorical and Cultural Aspects of Semantic Structure. Cambridge: Cambridge University Press. Turner, Mark 1991 Reading Minds: The Study of English in the Age of Cognitive Science. Princeton, NJ: Princeton University Press. 1996 The Literary Mind. Oxford: Oxford University Press. Turner, Mark, and Gilles Fauconnier 1995 Conceptual integration and formal expression. Metaphor and Symbolic Activity 10: 183203. Yu, Ning 1998 The Contemporary Theory of Metaphor: A Perspective from Chinese. Amsterdam/Philadelphia: Benjamins. 2003 Chinese metaphors of thinking. Cognitive Linguistics 14: 141165. 2004 The eyes for sight and mind. Journal of Pragmatics 36: 663686. 2005 The Chinese heart as the central faculty of cognition. Paper presented at the 9th International Cognitive Linguistics Conference, Seoul, Korea. July 2005. To appear in Culture, Body, and Language: Conceptualizations of Internal Body Organs across Cultures and Languages, Farzad Sharifian, Ren Dirven, Ning Yu and Susanne Niemeier (eds.). Berlin/New York: Mouton de Gruyter. 2007a Heart and cognition in ancient Chinese philosophy. Journal of Cognition and Culture 7: 2747. 2007b The Chinese conceptualization of the heart and its cultural context: Implications for second language learning. In Applied Cultural Linguistics: Implications for Second Language Learning and Intercultural Communication, Farzad Sharifian and Gary B. Palmer (eds.), 6585. Amsterdam/Philadelphia: Benjamins.

III Multimodal Metaphor in Political Cartoons

Chapter 7 Visual metaphor versus verbal metaphor: A unified account Francisco Yus

Abstract
Multimodal metaphors are those whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006: 384/this volume), mainly with a verbal-visual interface of source and/or target. When multimodality is analyzed in metaphors, the verbal and visual inputs are wrongly treated as different phenomena demanding different interpretive strategies when searching for a metaphoric interpretation. In this chapter, on the contrary, it is claimed that the comprehension of verbal, visual and multimodal metaphors involves similar mental procedures. Although the perception of images differs from linguistic decoding, reaching an interpretation of metaphors entails similar adjustments of conceptual information of texts and images and multimodal combinations, regardless of the modal quality of the input. Keywords: Relevance theory, conceptual upload, ad hoc concepts, ad hoc pointers, visual-conceptual interface

1. Introduction In this chapter, visual metaphor comprehension is compared to verbal metaphor comprehension and analyzed mainly from a cognitive pragmatics point of view (specifically within relevance theory, as proposed by Sperber and Wilson 1995 [1986]), but also with reference to cognitive linguistics where appropriate. The main claim underlying this chapter is that the comprehension of verbal and visual metaphors involves similar mental procedures. While the perception of images is obviously different from linguistic decoding, reaching an interpretation of visual metaphors also entails an adjustment

148 Francisco Yus of conceptual information a stage during comprehension that will be called conceptual upload in the same way as verbal metaphors. Therefore, although it is not denied that the combination of visual and verbal inputs in multimodal metaphors can indeed generate interesting interpretive outcomes, as the chapters in this book demonstrate, in many studies of multimodality visual and verbal metaphors are wrongly treated as different phenomena. By contrast, in this chapter it will be argued that both types of metaphor (and also multimodal metaphors with combinations of text and image) are decoded by specialized mental modules, which deliver schematic information that has to be enriched inferentially in order to obtain the intended interpretation. Besides, as will be illustrated below with several Spanish political cartoons by El Roto, visual metaphors can be arranged on a scale depending on the gap existing between the prototypical referent of the image and the cartoonists intended referent, which has to be adjusted inferentially in the same way as in verbal metaphors, whose interpretation involves the hearers inferential adjustment of the concept that the speaker encodes in order to obtain the speakers intended interpretation. In this sense, the cartoons analyzed in this chapter contain metaphors of a pictorial/visual nature, whereas multimodal metaphors, as defined by Forceville, are metaphors whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006: 384). However, it follows from my central thesis in this chapter that the model presented here is capable of accommodating multimodal metaphors in the same way as metaphors with only verbal or only visual inputs.

2. Relevance theory and ad hoc concept formation Sperber and Wilsons (1995 [1986]) relevance theory (henceforth RT) predicts that human comprehension follows two stages:
i. Following a path of least effort, test interpretive hypotheses (disambiguations, reference resolutions, enrichments, implicatures) in order of accessibility. Stop when the interpretation satisfies the current expectation of relevance.

ii.

For instance, an advertisement by London Transport quoted in Tanaka (1994) only contained the text Less bread, no jam. The reader of this ad will follow a path of least effort and conclude, initially, that bread and

Visual metaphor versus verbal metaphor

149

jam have to do with food, but faced with the incongruity between food and the advertised transport company, the reader will continue testing interpretive hypotheses concluding, at a second stage, that bread is a colloquial word for money and jam refers to traffic jams. The reader will now be satisfied at this interpretation and stop processing here. Specifically, for RT, comprehension does not normally start in a communicative vacuum, but takes place against a context of previous utterances whose interpretation (stored in the short-term memory) works as a background against which new information is processed. Cognitive linguistics has also drawn attention to the role of context for the right comprehension of metaphors. For example, for Conceptual Metaphor Theory, metaphors may be activated as part of the hearers understanding of context, and this will make the interpretation of metaphors easier at subsequent stages in discourse. Similarly, conceptual blending theory stresses the role of context in metaphor comprehension: because cognitive activity mediates the relationship between words and the world, the study of meaning is the study of how words arise in the context of human activity, and how they are used to evoke mental representations (Coulson, quoted in Tendahl and Gibbs 2008: 1843). This cumulative background context of previous utterances in the conversation is normally absent in the processing of visual metaphors, which are inserted in media discourses such as newspapers, billboards or magazines, and hence the viewers1 have to interpret them from scratch, without this readily available short-term memory store of information. This does not mean that visual metaphors do not require a great deal of background knowledge for their satisfactory interpretation. The metaphors used in cartoons, such as the ones analyzed in this chapter, are often related to recent news-worthy events whose knowledge is essential to get the right extent of the metaphoric mappings (cf. Peamarn 1996; El Refaie this volume; Schilperoord and Maes this volume; Forceville 2005). RT predicts two clear-cut phases during interpretation: one of decoding and one of inference. The first one is in charge of the language module of the mind (Fodor 1983), which apprehends a linguistic sequence and yields a de-contextualized but grammatical logical form which has to be enriched in order to be meaningful. By contrast, cognitive linguistics disregards modularity in favor of what is called the embodied-mind hypothesis, according to which the same neural mechanisms used in perception and bodily movement play a role in all forms of conceptualization, including the creation of lexical fields and abstract reasoning (Ruiz de Mendoza 2005: 36).

150 Francisco Yus For RT, language does not encode thoughts, but only clues that help the hearer access the speakers thoughts, which are often more complex than the literal meaning encoded by the utterances. For example, the (b) versions of the following utterances are closer to the thoughts that the speaker intended to communicate with them than the schematic (and communicatively useless) (a) versions, the ones actually uttered:
(1) (2) a. Its too wide. b. The table that I bought yesterday is too wide to go through the door. a. The cinema is some distance from here. b. The cinema is too far to go walking.

Secondly, and following a relevance-seeking criterion, the hearers mind undertakes an inferential process of mutual parallel adjustment of explicit content, implicatures and context (including the information from preceding utterances) until a satisfactory interpretation is achieved, at which point processing stops. In my opinion, despite the apparent differences (see El Refaie 2003: 8590), this model of utterance interpretation is applicable to visual and multimodal metaphor comprehension in the same way as to verbal metaphor comprehension. As pointed out above, there is always a greater or lesser informational gap between what the speaker says (what is encoded) and what the speaker intends to communicate with the utterance. This task often involves an adjustment of the conceptual information encoded, that is, interpretation involves the creation of ad hoc concepts during interpretation (see Carston 2002; Pilkington 2000). Since we store many more concepts in our mind than words to encode them, inevitably there is a greater or lesser amount of adjustment of encoded concepts needed in order to grasp the speakers intended interpretation. This is applicable to almost every concept, not just the adjustment of concepts regarding metaphor comprehension. Consider, for instance, the examples provided in (3) below (Vega-Moreno 2004: 317):
(3) a. The sofa is soft. b. Baby skin is soft. c. The cat is soft.

The hearer of (1a-c) is expected to adjust the encoded concept soft into a more appropriate and contextualized type of softness that specifically applies to sofas, skins and cats respectively, that is, adjust into more relevant ad hoc concepts SOFT*, SOFT**, and SOFT***.2

Visual metaphor versus verbal metaphor

151

Ad hoc concepts can be formed either by a process of narrowing of the encoded concept (what the hearer interprets is more specific than the encoded concept) or a process of broadening of the encoded concept (the hearer interprets more general or broader information than that encoded), all of them resulting from a relevance-seeking procedure.3 These processes are illustrated in (4) and (5) respectively:
(4) a. The fish attacked some people near here (FISH* = dangerous fish, e.g., sharks). b. The boy has a temperature (TEMPERATURE* = higher temperature than normal). c. It will take some time to fix the car (SOME TIME* = longer than it usually takes). a. The steak is raw (RAW* = undercooked). b. She is a genius! (GENIUS* = not literally a genius, but having some of his/her qualities). c. It was quiet in the street last night (QUIET* = with very little noise).

(5)

Within RT, verbal metaphor comprehension also involves a pragmatic adjustment (broadening, narrowing, or both simultaneously), and RT suggests a different approach to metaphor analysis than does cognitive linguistics. Briefly, the former is more interested in the role of metaphor for communication and hence in the role of context favoring a certain adjustment of concepts, while the latter mainly focuses on the cognitive motivation for certain metaphors, their conceptual organization and the inference patterns involved in their creation. But, as such, the accounts are not mutually exclusive (see Ruiz de Mendoza and Prez Hernndez 2003; Ruiz de Mendoza 2005; Tendahl and Gibbs 2008). In Vega-Moreno (2004: 208), three main types of ad hoc concepts are proposed for verbal metaphors: (a) Ad hoc concepts which contain qualities which are applicable to all the prototypical referents of the encoded concepts and also to a range of other referents, as in (6) below:
(6) A. Why does your boyfriend want you to go with him everywhere? B. Because he is a baby. (BABY* denotes a person who cannot be independent, cannot look after himself, cant do things alone, etc. These are qualities applicable to all babies (as prototypical referents) and also to some adults such as the speakers boyfriend).

152 Francisco Yus (b) Ad hoc concepts which contain qualities which are applicable to some of the prototypical referents of the encoded concepts and also to a range of other referents, as in (7) below:
(7) Being the only boy, Dave has always been the prince of the house. (PRINCE* denotes a subset of princes who are spoilt and do as they please, as well as a set of young boys who are not princes but are spoilt and do as they please).

(c) Ad hoc concepts which contain qualities which are applicable to none of the prototypical referents of the encoded concepts but are applied to other referents, as in the utterance quoted in (8):
(8) I tried to persuade him to change the essay topic but there was no way. He is an iron bar. (IRON BAR* denotes people who are difficult to convince, persuade, etc., qualities which are not found in iron bars as prototypical referents).

In my opinion, these three cases are not only inherent to verbal metaphor comprehension, but are also found in the processing of visual metaphors. In this sense, case (c) is interesting because it gives rise to the so-called emergent features or emergent properties which apparently do not belong to the target domain of the metaphor but seem to emerge during comprehension (Gineste, Indurkhya, and Scart 2000; Wilson and Carston 2006). These emergent properties might appear to be found only in the interpretation of verbal metaphors but, as will be argued below, they are also frequent in visual metaphor comprehension (cf. Yus 2003a) and in any multimodal combination of text and image. Many explanations have been suggested for the creation of these emergent properties.4 I will follow an interesting proposal by Vega-Moreno (2004) within a relevance-theoretic point of view, and I will argue that this proposal is applicable, in a similar way and with the necessary adjustments, to the processing of emergent properties in visual or multimodal metaphors. More generally, I will show to what extent conceptual assessment is involved in visual metaphor comprehension, basically through what will be called stable versus innovative conceptual upload. The analysis will be divided into several steps that the reader is expected to go through during the interpretation of a visual metaphor. Comparisons with verbal metaphor comprehension will be made where necessary, and there is an inherent claim in this proposal: that combinations of text and image in multimodal metaphors de-

Visual metaphor versus verbal metaphor

153

mand similar interpretive procedures involving conceptual adjustment of encoded information. Variations are only found in the way schematic information is obtained by mental modules (in a more holistic way in pictures, in a more linear way in texts).

3. Stages in visual metaphor comprehension: A proposal 3.1. Perception: Visual versus verbal My claim is that processing visual metaphors does not differ substantially from processing verbal metaphors, the main difference being the way in which the input is transferred to the central inferential processor. RT predicts, following Fodors (1983) theory of the modularity of mind, a contextfree decoding of a linguistic string by the language module, which sends a de-contextualized string of linguistic information to the central processor in order to be enriched inferentially into a fully contextualized (and optimally relevant) interpretation that supposedly matches the speakers intended one. Visual information, on the other hand, is decoded by another module: the perceptual module. The language module and the perceptual module share similar properties: (a) they are fast and automatic (i.e., they are capable of a high-speed transference of information, and they are automatically activated by the appropriate type of input: linguistic and visual respectively); (b) they are domain-specific (both modules are only activated by a specific type of input); (c) they are part of our genetic endowment (i.e., they are not learned and possess an evolutionary quality); and (d) they have a uniform path of development (unfolding) across individuals and cultures. In short, these mental modules get activated automatically when the appropriate type of input reaches them, and both yield de-contextualized pieces of evidence of the senders intention to communicate some information. This context-free information is then enriched inferentially in order to obtain a fully satisfactory (i.e., relevant) interpretation of the verbal or visual input. However, these qualities of modules do not entail that no choices are ever made during this phase of verbal or visual decoding. For instance, the language module often has to choose between two possible logical forms for the same linguistic string.5 In the same way, it has been demonstrated that although the perceptual module seems to engage in a one-to-one matching between object and referent, it also has to make choices as to what visual information it is actually processing. Specifically, when readers interpret a visual metaphor, they start by perceiving the image, that is, by identifying

154 Francisco Yus the visual input. This is done through a subconscious or sub-attentive comparison with previously stored information on the visual attributes of the object or objects depicted (see McMahon 2003; Kriegel 2004). When the image is supposed to be intentionally communicated to the readers beyond a simple perceptual recognition, processing moves one step beyond into a more conscious stage of interpretation, loaded with inferential activity (see below).6 This mental storage of prototypical referents that we possess is made up of two basic types of information which undergo a constant process of updating and stabilization through subsequent visual perceptions: (a) Prototypical visual referent: encyclopedic entry containing visual elements and attributes that an item depicted in an image is typically made of. For instance, the prototypical referent of an image of a cat would contain visual attributes that are stored as typical of cats (type of hair, colors, ears, whiskers, paws) and which allows for an easy visual identification. (b) Prototypical visual syntax: other items typically associated with another object depicted in an image. In general, processing is faster if the visual arrangement of objects in the image fits our storage of prototypical visual syntax for these objects, a sort of visual schema that precedes and influences actual perception (cf. McMahon 2003: 266). In the above cat example, we will expect to find visual representations of this animal in specific scenarios with objects forming a prototypical visual syntax regarding its representation (e.g., cat on a branch, on a mat, playing with wool). In general, as the number of visual features of the image which belong to the prototypical visual referent increases and its prototypical visual syntax fits stored schemas, the effort involved in its processing will decrease accordingly.7 Highly iconic images are normally filled with features fitting the prototypical visual referent of the image that the reader possesses, but there can be other images containing less prototypical features, generating socalled scales of iconicity. Besides, visual perception shares a bottom-up and a top-down quality. It is bottom-up because the reader constructs and integrates the prototypical visual referents from the available visual elements (as claimed by Gestalt theory). But at the same time it is top-down because readers test the visual input against their mental storage of prototypical visual referents, anticipating, as it were, and even influencing the recognition of the object depicted in the image.8 The perceptual module draws on a conceptual repertoire that contains a range of visual referents and is subject to constant revision and updating through subsequent visual perceptions of similar images. Perception is never isolated, and past exposure to objects constrains future perceptions (Villafae and Mnguez 1996: 100). Hence, each perception of the

Visual metaphor versus verbal metaphor

155

physical object or picture to which the referent is associated helps the reader to update the prototypical referent that was created when the reader first perceived it.

3.2. Ad hoc pointers The previous section on visual perception is important to understand why certain images are interpreted metaphorically. In my opinion, the key to a shift from a purely denotative interpretation of the image, often subattentive, to a connotative metaphoric interpretation, loaded with inferential processing, lies in the detection of an incongruity that turns up between the activation of the stored prototypical visual referents during perception and the actual visual configuration of the image or images making up the visual metaphor (Forceville 1996: 115). This kind of incongruity has been labeled ad hoc pointer (Yus 2005), in the sense that an ad hoc visual arrangement or configuration created by the author with specific communicative purposes points towards a connotative interpretation, alerts the reader towards a connotative interpretation, often a metaphorical one. This idea entails an increased mental effort in moving beyond a sub-attentive visual perception into an effort-demanding inferential activity in search of the right metaphoric mapping from what we can label the source image to what we can call the target image. Of course, as we enter this inferential phase, the reader takes the responsibility of grasping the intended metaphoric interpretation (or his/her own personal interpretation) and the author of the image can only hope that the reader will be able to select the appropriate encyclopedic features associated with the visual referents of the images and infer which are the ones involved in the metaphoric interpretation (El Refaie 2003: 81; this volume). Incidentally, there may be visual ambiguity, in which a metaphoric interpretation of the image is intended but a purely denotative interpretation is also valid, that is, occasions on which there is no apparent ad hoc pointer and hence the metaphoric interpretation may not be accessed. In these cases, it is the readers search for an optimally relevant interpretation that will guide them beyond a purely denotative interpretation. In general, visual metaphors are integrated in other discourses (images in advertisements, cartoons in the press) and the readers know that these images are intended to communicate specific, non-denotative information, and hence they will not be cognitively satisfied at a purely denotative level.

156 Francisco Yus 3.3. Visual-conceptual interface Upon detecting the ad hoc pointer, the reader of the image(s) enters another stage in interpretation, which I will call visual-conceptual interface, inbetween a sub-attentive perception of the images and a fully inferential extraction of a relevant connotative (i.e., metaphoric) interpretation of the image. At this stage, the reader aiming at an optimally relevant interpretation has to raise a number of preliminary hypotheses concerning the intended relationship that holds between the depicted images and the encyclopedic (conceptual) information stored about the referents of these images, mostly of a stereotypical quality. In short, the readers would ask themselves questions such as the following: 1. Which are the two images related metaphorically? Are both present in the picture? Visual metaphor involves a mapping of information transferred from one image to another, which we have called source image and target image respectively. Often both images are present in the picture (either fused together or separated) but sometimes one of them normally the source image is absent. Therefore, there are different degrees of mental effort involved in processing visual metaphors depending on whether both the source image and the target image are depicted in a metaphoric visual configuration, or one of the images is absent and is only accessible through an inferential operation regarding the encyclopedic information on its prototypical referent. At the same time, some mental effort has to be devoted to identifying the source and target images in the first place, which are not always clearly distinguishable, even when both images are present. 2. What kind of visual arrangement is there between the images? The reader is also expected to infer what relationship holds between the previously identified source image and target image. Is the target image supposed to be like the source image? Is it opposed to the source image? (cf. Phillips and McQuarrie 2004.) 3. Are the prototypical encyclopedic referents of the images themselves the ones that are going to undergo inferential adjustment in order to obtain a metaphoric interpretation or do the images stand for a different encyclopedic referent? I believe that visual metaphor comprehension, in a similar way to verbal metaphor comprehension, also involves an access to and adjustment of conceptual information stored in or attached to the encyclopedic prototypical referents of the image or images depicted. This implies that it is of utmost importance to determine whether the author intends the most accessible referents of the images to undergo metaphoric processing or whether the intended sources of metaphor have to be found elsewhere. This is the case of

Visual metaphor versus verbal metaphor

157

images that stand metonymically for other referents. For instance, in the visual metaphor described in (9), it is the prototypical referent of the image buildings that works as target domain for metaphoric mapping, whereas in (10), the visual referent of book page is not expected as target domain, since it stands for culture in general, which is the intended target domain.
(9) (10) Images depicted: Lorry throwing buildings into a garbage dump. Metaphor: BUILDINGS ARE RUBBISH (El Roto, El Pas, 296-2003). Images depicted: A page of a book as a paper serviette inside a dispenser. Metaphor: CULTURE IS A PAPER SERVIETTE (photograph by Chema Madoz).

Figure 1. Cartoon by El Roto, El Pas, 6 June 2003.

Similarly, in a cartoon by El Roto (figure 1), a syringe is depicted with a television tower instead of the needle. An incongruity in the visual syntax of the image works as an ad hoc pointer alerting the readers to a metaphoric interpretation.9 The readers search for relevance will lead them to dismiss the conceptual features of the prototypical referents of syringe and television tower as the ones undergoing metaphoric assessment, and they will probably infer, instead, that syringe stands metonymically for drugs in general, and that television tower stands metonymically for television in general as a mass medium, and the conceptual features of drugs and television are the intended source and target of this visual metaphor. The cartoon also includes the text la gran droga (the big drug), which works as an anchorage (in Barthes 1977 sense) of the image facilitating the metonymic relationship between syringes and drugs in general. Using the cognitive linguistics terminology that distinguishes source-in-target metonymies and

158 Francisco Yus target-in-source metonymies (Ruiz de Mendoza and Dez Velasco 2002), in this case we would, instead, encounter examples of source-in-source metonymy and/or target-in-target metonymy between prototypical referents depiced and the intended referents. Specifically in figure 1, both the source image (tower) and the target image (syringe) stand metonymically for the actual source and target referents undergoing metaphoric interpretation (television/drugs).

3.4. Conceptual upload The ad hoc pointer and the preparatory phase of visual-conceptual interface lead to a fully inferential stage in the processing of the image or images, which will be called conceptual upload. Since this stage is centered upon the inferential assessment and adjustment of conceptual information attached to the prototypical encyclopedic referents of the images (or the referents intended through metonymy), the distinction between visual and verbal input to metaphoric interpretation no longer matters (we are now at a cognitive, fully inferential phase of interpretation). The reader has now entered a fully inferential stage that takes either the information from the verbal utterance or the identified visual images as blueprints or clues for an optimal metaphoric interpretation. At this stage of conceptual upload, and following a relevance-oriented path of accessibility, the reader will compute conceptual features stored in the encyclopedic information of the intended referents of the images (either the prototypical referents of these images themselves, or the referents to which these images point metonymically, as commented upon above) and will try to find the ones that can be applied to the other image, a mental procedure which can be called ad hoc choice of image-associated conceptual features. This is a similar inferential activity to the one intended to obtain metaphoric mappings in verbal and multimodal metaphors. Sometimes this assessment of possible ways in which the images can be related does not result in any metaphorical outcome despite the visual incongruity, because the reader is unable or unwilling to find any metaphoric connections between these images. On other occasions, though, the readers search for relevance will lead to a metaphoric interpretation and to a selection of features which can be mapped from the source image to the target image. In this sense, two possible types of conceptual upload can be identified:

Visual metaphor versus verbal metaphor

159

(1) Stable conceptual upload of image-associated conceptual features takes place when interpreting the visual metaphor involves an adjustment of one or several features belonging to the prototypical encyclopedic referent of the image depicted. The maker of a visual metaphor fitting this type will expect all the readers to have a similar store of conceptual information filling up the prototypical referent of the image or images depicted. But, crucially, in this type of conceptual upload the encyclopedic feature of the referent associated with the source image maintains its conceptual stability, and the reader only has to broaden, as it were, its denotation to include the referent associated with the target image. The types 1 and 2 of ad hoc concept formation that were introduced in section 1 above for verbal metaphors would belong to this type of conceptual upload: Type 1. When one or several conceptual features of the prototypical encyclopedic referent associated with the source image that the author intends the reader to apply to the target image can be found in all the prototypical referents represented in the source image. In this case, by means of a process of conceptual broadening, an ad hoc CONCEPT* is created and applied to the target image. For instance, in the aforementioned cartoon by El Roto depicting a lorry throwing buildings into a huge garbage dump, one or several of the features of the prototypical encyclopedic referent rubbish (for example being useless, having no quality, etc.) are mapped onto the encyclopedic referent buildings associated with the target image. These conceptual features form an ad hoc concept RUBBISH*, which is the result of broadening the prototypical concept rubbish in order to cover the unusual referent buildings of the target image. Type 2. One or several conceptual features of the prototypical encyclopedic referent of the source image that the author intends the reader to apply to the target image can be found in some but not all of the prototypical encyclopedic referents represented in the source image (or referred to by this image). Again, a process of conceptual broadening is required so that an ad hoc CONCEPT* is created. For example, in another cartoon by El Roto (figure 2) the reader can see a man on a surfboard sliding on a huge wave, but the wave is made of buildings, instead of water, and this unusual wave looks as if it is about to break on the surfer. The author probably intends to communicate metaphorically that the urge to build houses in Spain (the construction wave or construction bubble) will eventually break (or burst) and harm us in the same way as the huge wave is about to break on the surfer and probably harm him as well. The harming quality of breaking waves can be found in some (but not all) of

160 Francisco Yus the prototypical encyclopedic referents of the image depicted, specifically only those waves which are big enough to break onto surfers and harm them, and a new ad hoc concept WAVE* is created as a result of an adjustment (broadening) in order to fit the new encyclopedic referent construction bubble of the target image.

Figure 2. Cartoon by El Roto, El Pas, 19 October 2003.

(2) Innovative conceptual upload of image-associated conceptual features. No conceptual features that the author intends the reader to apply to the referent associated with the target image seem to be found in the prototypical encyclopedic referent associated with the source image. These conceptual features are not stabilized in the prototypical referents but arise, in the same way as do emergent properties in verbal metaphors, as part of the readers relevance-seeking interpretation procedure. This kind of conceptual upload fits the third type of ad hoc concept construction introduced in section 1 above for verbal metaphors. The reader will consider encyclopedic features not directly applicable to the referent of the target image. Inevitably, this mental operation entails the adjustment of one or several of these features and, as a consequence, they will be deprived of their conceptual stability when attributed to the referent associated with the target image. Unlike cases 1 and 2 above, in which the features were minimally adjusted in order to include the new referent but maintained their conceptual quality, in this third case a substantial adjustment of the features is required and only by losing their stability can they be applied to the new referent. In fact, the resulting emergent properties are not directly applicable to the target, and hence a deep process of adjustment is

Visual metaphor versus verbal metaphor

161

required, in the same way as are emergent properties that arise in case 3 of conceptual adjustment during verbal metaphor comprehension. For example, another cartoon by El Roto (El Pas, 9 June 2002) depicts a goal keeper clearing a book (instead of the expected ball) with his fists. In the metaphor THE BOOK IS A BALL, none of the features of the prototypical referent ball seems to be applicable to the referent book of the target image. Whatever metaphoric interpretation the reader chooses (e.g., football makes people reject books or football stops people from getting real culture if book stands metonymically for culture) will emerge during the processing of the referents of the images, and will involve more inferential effort than a simple broadening of one or several features of the prototypical encyclopedic referent of the source image in order to obtain an ad hoc concept BALL*. More inferential activity will have to be devoted to adjusting some feature(s) that belong to the prototypical encyclopedic referent. Vega-Moreno (2004: 318f) exemplifies this possible explanation of emergent properties with the verbal metaphor communicated in (11) below:
(11) Jane: I know I have to speak to my boss but I am afraid of him. He is such a bulldozer!

In this metaphor, our knowledge of bulldozers does not include information about them being stubborn or disrespectful, and hence this metaphoric quality of the boss emerges during interpretation. In a nutshell, the hearer can select, as a starting point, the assumptions that bulldozers are machines and are used to remove obstacles in their way. These are not directly applicable to the boss (as it would be in stability-preserving cases 1 and 2 of ad hoc concept formation). Cognitive linguistics would explain this metaphor by claiming that there is an underlying conceptual metaphor THE MIND IS A MACHINE.10 For RT, though, the hearer creates an ad hoc concept BULLDOZER* which involves a radical adjustment of its denotation so that it also includes a kind of removal, a type of obstacle, and a range of situations that warrant the derivation of a relevant metaphoric interpretation. In other words, an ad hoc concept REMOVE OBSTACLES IN THE WAY* is created with an adjustment which not only applies to machines, but also includes the act of despising, rejecting, undermining peoples feelings and peoples thoughts. As will be seen in section 4.2 below, there is a similar conceptual adjustment in the case of type-three visual metaphors and, in principle, in any multimodal combination of text and image.

162 Francisco Yus 4. Visual metaphor comprehension: Some examples 4.1 Examples of visual metaphor comprehension involving stable conceptual upload

Figure 3. Cartoon by El Roto, El Pas, 23 June 2002.

(a) THE EARTH IS A SAUCEPAN (figure 3) 1. The reader finds figure 3 in a newspaper and infers that its author intends to communicate some information by means of this wordless cartoon. 2. He perceives the iconic signs of the cartoon by a bottom-up and topdown matching with previously stored prototypical visual referents of the item(s) depicted. A saucepan is identified. The continents of the earth are also identified. These are superimposed on the saucepan. 3. An incongruity arises during the perception of the elements of the image. The earth and a saucepan cannot be fused in one image. There is an anomalous visual arrangement regarding the mental storage prototypical combinations of objects depicted together (what above was labeled visual syntax of the image) that works as an ad hoc pointer that alerts the reader to an intended metaphoric interpretation beyond the simple depiction and perception of the drawing in the cartoon. 4. The reader enters a visual-conceptual interface, in which a number of hypotheses are made and the so-called prototypical visual referents of the images (already dealt with by the perceptual module) are contrasted with the parallel prototypical encyclopedic referents of these images. The readers hypotheses at this stage will prepare the ground for a fully inferential stage of visual metaphor comprehension, and should include the following conclusions: (a) the saucepan is the source image; (b) the earth is the target image (a likely conclusion obtained by the reader in a relevance-

Visual metaphor versus verbal metaphor

163

seeking procedure); (c) the encyclopedic referent EARTH has qualities of the encyclopedic referent SAUCEPAN; and (d) these are the prototypical referents intended by the author; the images of the earth and the saucepan do not stand metonymically for other referents. 5. The reader starts computing assumptions in order of accessibility following a relevance-guided procedure, beginning with the ones which belong to the prototypical encyclopedic referent depicted in the source image: SAUCEPAN. 6. One of the encyclopedic features of the prototypical encyclopedic referent SAUCEPAN seems to be directly applicable to the earth: heats up gradually, since the earth, due to global warming and the so-called greenhouse effect is also heating up gradually. This involves the creation of an ad hoc concept SAUCEPAN* whose quality heats up gradually remains relatively stable in the metaphoric process (i.e., undergoes a minimal adjustment via broadening). This new concept is applicable to all saucepans and also metaphorically to the new encyclopedic referent EARTH. In this sense, this could perhaps be a case of ontological metaphor in Lakoff and Johnsons (1980) terminology, since in this case an inherent quality of saucepans is attributed to the target image. It would also fit Ruiz de Mendozas (1998) one-correspondence metaphor, since in this case only one correspondence between the source and the target is exploited. This visual metaphor would fit case 1 of ad hoc concept formation, in which the intended conceptual feature is found in all the prototypical referents depicted by the image plus a number of other entities included through broadening. 7. The presence of SAUCEPAN* in what can be called the explicit content of the referents attached to the items depicted in the cartoon warrants the derivation of a number of possible implicated conclusions. Cartoons are a good example of a medium in which current news-worthy events play a part in the generation of implicated conclusions. In this case, if the reader knows about the fact that, at the time the cartoon was published, there was a debate on the Kyoto protocol and whether Japan and Australia would sign it (i.e., if this information is manifest to him/her, in RT terms), this information will influence both the accessibility to the visual metaphor and the mental effort devoted to its processing. (b) THE BALLOT BOX IS A DICE (figure 4) Steps 1. and 2. as above. 3. The reader enters a visual-conceptual interface, in which a number of hypotheses are made concerning the encyclopedic referents of the images, once the prototypical visual referents have been perceived, again preparing

164 Francisco Yus the ground for a fully inferential stage. Some conclusions should be derived: (a) the dice is the source image; (b) the dice stands metonymically for gambling with dice and more generally for all types of gambling (and hence the encyclopedic referent intended by the author to undergo metaphoric transference is not DICE, but GAMBLING); (c) the ballot box is the target image (facilitated by the readers background knowledge about the fact that the cartoon was published in a time of political elections); (d) the ballot box is in a metonymic relationship to political elections (and hence the referent intended by the author to undergo metaphoric transference is not BALLOT BOX, but POLITICAL ELECTIONS in general or more specifically the ones taking place in the near future); (e) the political elections have qualities of gambling.

Figure 4. Cartoon by El Roto, El Pas, 19 February 2005.

4. The reader starts computing assumptions in order of accessibility following a relevance-guided procedure, beginning with the ones which belong to the prototypical encyclopedic metonymic referent of the source image: GAMBLING. 5. One of the encyclopedic features of the prototypical referent gambling seems to be directly applicable to political elections: involves an unpredictable outcome. Strictly speaking, this cartoon should be included in type 1, if we consider that all dice involve unpredictability. However, we may also hypothesize that since there is also a kind of gambling that generates a wholly predictable outcome (for instance gambling involving the use of loaded dice which always produce the desired result) in this case not all the referents for image of a dice would contain this quality. In this more unlikely case, the reader would be expected to create an ad hoc concept GAMBLING* whose feature would involve an unpredictable outcome applicable to most (but not all) kinds of gambling and also to other unpredictabil-

Visual metaphor versus verbal metaphor

165

ity-loaded events such as political elections. Consequently, this visual metaphor would now fit case 2 of ad hoc concept formation during visual metaphor comprehension. 6. The presence of GAMBLING* in what can be called the explicit content of the image depicted in the cartoon warrants the derivation of a number of possible implicated conclusions, some strongly implicated (e.g., the unpredictability of Spanish elections) and other weakly implicated (e.g., the more emotion-connoted implication that no Spanish party is trustworthy).

4.2. Example of visual metaphor comprehension involving innovative conceptual upload (a) THE TENNIS RACKET IS A SHARKS JAW Phillips and McQuarrie (2004: 123) reproduce an advertisement for a tennis racket in which a sharks jaw has been superimposed on a racket frame. How would the audience process this ad? Basically steps 1 to 3, as proposed for the previous examples, would also apply here. The unusual fusion of images and the anomalous visual syntax (i.e., bringing together visual elements racket and jaw which are not stored as part of the prototypical arrangement of objects such image can normally be made of or surrounded by) work as an ad hoc pointer to a non-denotative interpretation of the images. 4. The reader then enters the so-called visual-conceptual interface, in which a number of questions are asked regarding the relationship between the images depicted (already obtained via perception) and the encyclopedic information attached to them. Among others, some conclusions that the reader would be expected to derive are these: (a) the jaw is the source image; (b) the racket is the target image (a conclusion facilitated by the context of the advertisement, in which it is made clear that the author of the ad intends to characterize the racket in some way, so that it is eventually purchased); (c) the jaw stands metonymically for SHARKS in general; (d) the racket stands metonymically for THE PLAYERS SKILL; (e) the players skill acquires qualities of a shark (aggressiveness, fearful attack). 5. The reader then starts computing assumptions in order of accessibility following a relevance-guided procedure, starting with the ones listed in the prototypical encyclopedic referent made accessible by the source image: SHARK. 6. Again, although intuitively the reader can feel that the intended interpretation has to do with some form of aggressiveness, there is nothing in the

166 Francisco Yus behavior of sharks that can be directly applicable to a tennis players skill (although, as a stereotypical feature of sharks, it would also be labeled as ontological metaphor, in Lakoff and Johnsons 1980 terminology), and therefore all the eventual transference of information will inevitably involve emergent properties arising in the relevance-guided comprehension procedure. Whatever conceptual information ends up being applied to the encyclopedic referent associated with the target image will involve a loss in the stability of the storage of this information. 7. The readers then assess qualities of the encyclopedic referent associated with the source image and will adjust their denotation drastically in order to fit not only the aggressiveness of an animal, but also the aggressiveness of a tennis player. These qualities include the information that sharks are aggressive animals. These are possible constituents of the ad hoc concept SHARK* but not directly applicable to playing skill, so the reader has to adjust (broaden) the quality AGGRESSIVE* that belongs to SHARK* so that it also covers the tennis players skill. Other features such as the way sharks chase and attack their preys may also be adjusted (depending on the readers willingness to devote additional mental resources to this task), leading to the ad hoc concepts CHASE* and ATTACK*. 8. These adjustments are necessary in order to draw the implicated conclusion that the user of this racket will play with an unusual degree of aggressiveness. As above, the adjusted concepts warrant the derivation of a number of possible implicated conclusions.

5. Verbal before visual or vice versa? From the examples analyzed in the previous section, we can conclude that interpreting visual metaphors also involves a great deal of conceptual upload and adjustment following a criterion guided by a search for relevance. In this sense, it is worth commenting that many visual metaphors are original in the way they create a metaphoric link through an anomalous visual arrangement, while others seem to include an anchorage of previously used verbal metaphors which are simply transferred to a visual medium and were probably stored previously as conventionalized metaphors. In these cases, the metaphor-seeking conceptual assessment can indeed be speeded up by the fact that a particular feature of the prototypical encyclopedic referent of the source image has been made prominent by previous use through verbal means, or even facilitated by the fact that the visual metaphor only exists because there is an underlying verbal one. In a way, this is the counterpart of

Visual metaphor versus verbal metaphor

167

the well-studied fact that verbal metaphors also draw on a conceptual repertoire of visual sensory schemas which aid in the metaphoric attribution, to the extent that these images often end up becoming conventionalized in the language and deprived of their sensory metaphoric power.11 Therefore, it comes as no surprise that so many metaphors involve the mediation of visual sensory information contained in image schemas (see Lakoff 1987), or involve what can be called re-visualization of conventionalized verbal metaphors, which is extensively used by cartoonists. Several steps are involved in the comprehension of the cartoons that fit this quality: (a) initially, an image is much more effective (i.e., vivid) than the range of coded options available to communicate a thought. (b) A metaphor is created that contains a schema as a referent. This schema contains visual sensory information.12 (c) Repeated use of the metaphor makes it lose its sensory vividness and it ends up becoming conventionalized and hence people stop regarding it as a metaphor. (d) The cartoonist takes this conventionalized metaphor and re-visualizes it, as it were, forcing the reader to re-incorporate into its processing all the sensory vividness that the metaphor had already lost. An example is a cartoon drawn by El Roto (El Pas, 15 April 1996) depicting the earth split into two parts, and with a big gap between the North and the South hemispheres. There are people trying to jump from the Southern hemisphere onto the Northern one but they inevitably fall into the huge gap. This metaphor, which can be described as there is an abyss between the North and the South, reproduces the aforementioned steps: (a) the sensory information of an abyss is more vivid than other coded options to communicate depth and distance between A and B; (b) a metaphor THE DIFFERENCE BETWEEN NORTH AND SOUTH IS AN ABYSS is created containing visual sensory information; (c) the metaphor ends up becoming conventionalized and loses its sensory power (people stop seeing an abyss when uttering the metaphor); (d) the cartoonist re-visualizes the information contained in the metaphor, forcing the reader to see the sensory qualities of the image schema that had been lost due to conventionalization.

6. Concluding remarks Interpreting visual metaphors does not differ substantially from verbal metaphor comprehension. Both kinds of metaphor are decoded by a specialized mental module (Fodor 1983) which delivers schematic information that has to be enriched inferentially in order to obtain the intended interpretation (an

168 Francisco Yus optimally relevant one). The de-contextualized perception of images is not relevant enough (some incongruity in the image or images works as an ad hoc pointer directing the viewer towards a metaphorical interpretation) and the reader has to engage in subsequent interpretive steps involving the access to encyclopedic information either directly related to the referent depicted in the image, or made prominent by metonymic relationship to other encyclopedic referents. At this stage, the reader will adjust the conceptual information in his or her search for relevance and will generate appropriate ad hoc CONCEPTS*. This is the same kind of conceptual adjustment that takes place in the interpretation of verbal metaphors. In short, conceptual information has to be accessed and adjusted in any type of metaphor. It is only the type of decoded input feeding the inferential processor that makes a difference. Indeed, the mode in which the reader is presented with the coded information that has to be adjusted inferentially plays a major role in the quantity and quality of metaphoric conclusions derived. Normally, pictures have a more powerful impact on the reader due to their holistic gestalt-like processing and are good for visualizing conventionalized concepts such as abyss in the example above. Utterances, on the other hand, are linear, and readers make interpretive hypotheses as text is processed in a word-by-word integration into phrases and sentences, which entails differences in the way literal and implicated meanings are generated.

Notes
1. Since all the visual metaphors analyzed or referred to in this article are taken from newspapers, from now on the viewer of the visual metaphor will be referred to as reader. I will use the convention of adding asterisks to stress the fact that the accompanying word is an ad hoc concept and not an encoded concept. In fact, context plays an important role aiding the addressee in determining the kind of CONCEPT* that the speaker or the author intends. Normally, in the course of a conversation, previous utterances and background knowledge about the speaker work as an important short-term-memory storage of information against which new utterances are interpreted. Vega-Moreno (2004: 317) explains this with the metaphor my boss is a shark. If it is clear from previous turns of the conversation or from general encyclopedic information about the speaker that he is happy with his boss, the concept AGGRESSIVE may be adjusted to denote a kind of (positive) aggressiveness that involves energy and assertiveness (represented as AGGRESSIVE*). However, processing the metaphor on the assumption that the speaker is afraid of his bosss tactics

2. 3.

Visual metaphor versus verbal metaphor

169

and techniques, the concept AGGRESSIVE would be adjusted to denote a kind and level of (negative) aggressiveness (AGGRESSIVE**). 4. One famous explanation is provided by the so-called conceptual blending theory (Fauconnier and Turner 1998). It suggests that a subset of the attributes and relational structure from the source and target domains are imported into a blended space where they can be combined and supplemented with information from encyclopedic knowledge. These hybrid models, or blends, are useful in explaining emergent properties. 5. The existence of a choice of two possible logical forms for the same linguistic input is often exploited by humorists in some of their jokes. In Yus (2003b: 1304), it is claimed that the source of humor in some jokes lies in the fact that the language module of the addressee has to choose between two possible de-contextualized logical forms extracted from the humorists utterance, as in the following example: Postmaster: Heres your five-cent stamp. Shopper (with arms full of bundles): Do I have to stick it on myself? Postmaster: Nope. On the envelope. 6. It is utterly important to identify the image as intentionally communicated (ostensive in RT terms) and not simply as visual information unintentionally exuded, as it were, from the environment. The former carries a presumption of eventual relevance which the latter lacks. This stage of intention ascription is important because it constrains all the subsequent inferential activity devoted to the processing of the visual stimulus and the amount of effort that the reader will be willing to devote to this inference beyond a purely subattentive identification of the image. 7. In fact, there is experimental evidence that the visual system uses principles of coherence to detect whether the visual information corresponds to a unique object or belongs to separate, interrelated objects, and different specialized brain cells are devoted to these tasks (see Humphreys and Heinke 1998). 8. The fact that we normally perceive only one interpretation very rapidly indicates that we see far more than the immediate information falling on our retina. The highly accurate guesses and inferences that we make rapidly and unconsciously are based on a wealth of knowledge of the world and our expectations for the particular scene we are seeing (Cavanagh 1998). 9. The change in mode from drawing to photographic style in this example may indicate that the visual syntax not only points towards the metaphor but also the humor of the vignette by the unexpected association between the building and the syringe. Perhaps the decoding of the photographic mode versus the drawing modes connected in this picture provoke the search for extra implicit meanings besides the anomalous visual arrangement. I would like to thank E. Urios-Aparisi for pointing this out to me. 10. Specifically, what guarantees the interpretation is the existence of an underlying conceptual mapping from bulldozer to human being whereby we understand [the bosss] behavior in terms of the figurative behavior (i.e. the

170 Francisco Yus

way the machine functions) that we attribute to a bulldozer (Ruiz de Mendoza and Prez Hernndez 2003: 29). 11. Tendahl and Gibbs (2008: 1834) assert that the motivation for metaphorical language is found in recurring sensorimotor patterns of experience that are continually enacted as neural processes in the moment of thinking, speaking, and understanding. Such recurring sensorimotor patterns at least motivate the existence and continued use of many conventional metaphors and some novel extensions or elaborations of these in creative metaphorical language. 12. As correctly argued within cognitive linguistics, this is an example of the typical mental operation that helps people conceptualize vague or abstract domains of knowledge in terms of more specific and familiar knowledge such as the one provided by sensory input.

References
Barthes, Roland 1977 Image, Music, Text. London: Fontana. Carston, Robyn 2002 Thoughts and Utterances: The Pragmatics of Explicit Communication. Oxford: Blackwell. Cavanagh, Patrick 1998 Top-down processing in vision. In Encyclopedia of Cognitive Science (MITECS), 839840. Cambridge, MA: MIT Press. El Refaie, Elisabeth 2003 Understanding visual metaphor: The example of newspaper cartoons. Visual Communication 2 (1): 7595. this vol. Metaphor in political cartoons: Exploring audience responses. Fauconnier, Gilles, and Mark Turner 1998 Conceptual integration networks. Cognitive Science 22: 133187. Fodor, Jerry 1983 The Modularity of Mind. Cambridge, MA: The MIT Press. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2005 Addressing an audience: Time, place, and genre in Peter van Straatens calendar cartoons. Humor 18: 247278. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379402. Berlin/New York: Mouton de Gruyter.

Visual metaphor versus verbal metaphor

171

Gineste, Marie-Dominique, Bipin Indurkhya, and Vronique Scart 2000 Emergence of features in metaphor comprehension. Metaphor and Symbol 15: 117135. Humphreys, Glyn, and Dietmar Heinke 1998 Spatial representation and selection in the brain: Neuropsychological and computational constraints. Visual Cognition 5: 147. Kriegel, Uriah 2004 Perceptual experience, conscious content, and non-conceptual content. Essays in Philosophy 5 (1): 114. Lakoff, George 1987 Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago: University of Chicago Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: The University of Chicago Press. McMahon, Jennifer A. 2003 Perceptual constraints and perceptual schemata: The possibility of perceptual style. Journal of Aesthetics and Art Criticism 61 (3): 259272. Peamarn, Cristina 1996 El humor grfico y la metfora polmica. La Balsa de la Medusa 3839: 107132. Phillips, Barbara J., and Edward F. McQuarrie 2004 Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing Theory 4 (12): 113136. Pilkington, Adrian 2000 Poetic Effects: A Relevance Theory Perspective. Amsterdam/ Philadelphia: Benjamins. Ruiz de Mendoza, Francisco 1998 On the nature of blending as a cognitive phenomenon. Journal of Pragmatics 30: 259274. Ruiz de Mendoza, Francisco 2005 Linguistic interpretation and cognition. In Cultural Matrix Reloaded. Romanian Society for English and American Studies. Seventh International Conference, Elena Croitoru, Daniela Tuchel and Michaela Praisler (eds.), 3664. Bucarest: Didactica Si Pedagogica. Ruiz de Mendoza, Francisco, and Olga I. Dez Velasco 2002 Patterns of conceptual interaction. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralph Prings (eds.), 489532. Berlin/New York: Mouton de Gruyter. Ruiz de Mendoza, Francisco, and Lorena Prez Hernndez 2003 Cognitive operations and pragmatic implication. In Metonymy and Pragmatic Inferencing, Klaus-Uwe Panther (ed.), 2349. Amsterdam/Philadelphia: Benjamins.

172 Francisco Yus

Schilperoord, Joost, and Alfons Maes this vol. Visual metaphoric conceptualization in editorial cartoons. Sperber, Dan, and Deirdre Wilson 1995 [1986] Relevance: Communication and Cognition. Oxford: Blackwell. Tanaka, Keiko 1994 Advertising Language. A Pragmatic Approach to Advertisements in Britain and Japan. London/New York: Routledge. Tendahl, Markus, and Raymond W. Gibbs 2008 Complementary perspectives on metaphor: Cognitive linguistics and relevance theory. Journal of Pragmatics 40: 18231864. Vega-Moreno, Rosa 2004 Metaphor interpretation and emergence. UCL Working Papers in Linguistics 16: 297322. Villafae, Justo, and Norberto Mnguez 1996 Principios de Teora General de la Imagen. Madrid: Pirmide. Wilson, Deirdre, and Robyn Carston 2006 Metaphor, relevance and the emergent property issue. Mind and Language 21 (3): 404433. Yus, Francisco 2003a Conceptos ad hoc en el procesamiento de la metfora. El caso de las vietas humorsticas en la prensa, paper delivered at Jornadas sobre Texto/Imagen, Ciudad Real (Spain): University of Castilla-La Mancha, December. 2003b Humor and the search for relevance. Journal of Pragmatics 35: 12951331. 2005 Ad hoc concepts and visual metaphor? Towards relevant ad hoc pointers, paper delivered at the 9th International Pragmatics Conference, Riva del Garda (Italy), July.

Chapter 8 Metaphor in political cartoons: Exploring audience responses Elisabeth El Refaie

Abstract
Using data from a study of young peoples responses to British newspaper cartoons, this chapter considers the ways in which readers interpret multimodal metaphors of the verbo-visual variety. One of the central tenets of Conceptual Metaphor Theory is that many metaphors derive from our bodily experience and are thus likely to be understood in similar ways by all human beings. But in fact there is increasing evidence that the interpretation of metaphors is partly dependent upon peoples socio-cultural background, as well as on the contexts in which the metaphors are used. The results of our study suggest that some metaphorical mappings in cartoons, such as those between size and power/status, and between movement through space and the passing of time, might be understood more generally and at a more intuitive level than more elaborate structural metaphors, which tend to be interpreted in different ways by different individuals. Keywords: audience research, embodiment, newspaper cartoons, verbo-visual metaphor

1. Introduction This chapter analyzes the use of multimodal metaphors of the verbo-visual variety in political cartoons and explores the ways in which such metaphors are understood by viewers from different backgrounds. The discussion draws on data from a study of young peoples responses to British newspaper cartoons about the 2004 US presidential elections.1 Political cartoons offer a good opportunity to explore multimodal metaphor, because metaphor is a very common device used by cartoonists (Edwards 1997; Philippe 1982; Morrison 1992; Templin 1999) and most car-

174

Elisabeth El Refaie

toons combine visual and verbal codes. As I argue in section 2, using two cartoons from our study as examples, newspaper cartoons typically act as a bridge between fact and fiction, combining actual current events with an imaginary, make-believe world created by the cartoonist (Edwards 1997: 8). While this metaphorical process of transferring meaning from the makebelieve to the real world tends to be conveyed predominantly in the visual mode, most cartoon metaphors also rely to some extent on verbal cues. Sometimes either the target or the source is represented exclusively through language, but more frequently verbal labels in cartoons are used as a means of specifying important aspects of a primarily visual metaphor. In the case of political cartoons, Forcevilles (2006: 384/this volume) definition of multimodal metaphors as metaphors whose target and source are each represented exclusively or predominantly in different modes thus needs to be interpreted in a way that also embraces such asymmetrical verbal-visual relationships. Section 3 explores the issue of universality versus individuality in the understanding of metaphors in cartoons. Conceptual Metaphor Theory is based on the proposition that metaphor derives from our bodily experience and is thus an essential part of our everyday patterns of thinking. This suggests that most instances of metaphor will be understood in similar ways by all members of a language community. But in fact it is becoming increasingly clear that the choice and interpretation of metaphors is partly dependent upon the participants social and cultural background (Kvecses 2005; Proctor, Proctor, and Papasolomou 2005), as well as on the specific contexts in which the metaphors are encountered (Ritchie 2004). In section 4, I describe the data and methods used in our study of young peoples readings of political cartoons, including the measures we took to ensure that we did not pre-empt their responses. I also explain our decision to consult the makers of the two cartoons used in the study, Nicholas Garland and Peter Schrank, about their intentions.2 Although we do not consider the artists to be the ultimate arbiters of the meaning of their work, it proved revealing to compare the intended meanings with the analysis of the cartoons by the author of this chapter and with the interpretations generated by the young people participating in the study. The fifth section of this chapter discusses some of the results of our study. The main focus is on the way readers recognize and interpret multimodal metaphors, but this cannot always be separated from more general considerations of how people read visual meaning and how their world knowledge influences the interpretation process. The discussion of the data therefore sometimes goes beyond the issue of multimodal metaphor in the

Metaphor in political cartoons

175

narrow sense and includes some reflections on wider issues of cartoon interpretation. In the final section I reconsider the results and propose ideas for further research. 2. Bridging the gap from fiction to fact If, as most researchers now accept, metaphors operate at the level of thought rather than being merely linguistic, then any form of communication can be seen as an instance of metaphor if it is able to induce a metaphoric thought or concept (Kennedy, Green, and Vervaeke 1993: 244). Since the early 90s, researchers have been discovering manifestations of metaphor in various non-verbal modes, thereby providing additional evidence for the existence of metaphorical thought patterns (Seitz 1998). It has also been shown that metaphors can be cued in more than one mode simultaneously (Forceville 2004; 2006). However, the search for commonalities must not distract from potential variations in meaning arising from the genre in which a metaphor occurs. As Sol Worth ([1974] 1981: 161) pointed out in an early discussion of visual metaphor, [i]t is the fact that we learn the agreed-upon rules for the intentional creation of meaning within specific contexts that makes metaphor possible. Therefore, genre is likely to have an important influence on the choice of metaphors by producers, the form these metaphors take, and the ways in which they are recognized and interpreted by audiences. Most of the research on visual and multimodal metaphor has so far focused upon its use in advertising (Forceville 1996; Kaplan 1992; Messaris 1997; Phillips 2003; Scott 1994), where the communicative purpose is obvious: to attract the attention of potential customers and create (implicit) cognitive links between the product and some desirable abstract quality. Because of this, visual metaphors in advertising are often highly creative and unusual. Researchers have tended to focus on this level of explicit metaphorical meaning and to disregard the issue of whether adverts also contain more basic orientational and ontological metaphors, which are thought to structure human perception and experience at a very fundamental level (Lakoff and Johnson 1980; Kvecses 2002). Clearly, in the case of political cartoons we are dealing with a completely different genre, with its own distinctive styles, conventions, and communicative purposes. A political cartoon is an illustration, usually in a single panel, published on the editorial or comments pages of a newspaper. Generally, the

176

Elisabeth El Refaie

purpose of a political cartoon is to represent an aspect of social, cultural, or political life in a way that condenses reality and transforms it in a striking, original, and/or humorous way. The field of politics is often complex and bewildering, and cartoons offer a way of explaining the significance of real life events and characters through the means of an imaginary scenario. As Edwards puts it, in cartoons people and events are depicted as something that they are not in order to arrive at a new definition of what they are (Edwards 1997: 128). The frame around a cartoon functions as an implicit metacomment, signaling to the newspaper reader that it is to be viewed as part of the dramatic cartoon world, as opposed to the real world of serious news reports, commentaries and newspaper photographs (Baldry and Thibault 2006: 17). Although cartoons often depict clearly ludicrous situations, they draw on readers real-life experiences and rely on their wider interpretive competences (Dines-Levy and Smith 1988: 244). A generic convention of cartooning, in contrast to advertising, is that the goal is generally to expose something bad or shameful rather than to highlight the positive. Not surprisingly, cartoonists will thus often fall back on stereotypes and systematic metaphorical concepts that represent the complexity of the world in simpler and often very negative terms. Orientational metaphors, which link spatial orientation with more abstract meanings, seem to be particularly common in political cartoons. According to Lakoff and Johnson (1980: 14ff), our physical and cultural experience of the up-down orientation, for instance, motivates a whole range of common metaphors, including HIGH STATUS IS UP/LOW STATUS IS DOWN and HAVING CONTROL OR FORCE IS UP/BEING SUBJECT TO CONTROL IS DOWN. In cartoons, size is commonly used to indicate the relative salience or importance of the various elements (El Refaie 2003: 85), and, since a difference in size is closely linked to up-down orientation, it can also imply power differentials. In her analysis of US newspaper cartoons about the 88 Primaries, for instance, Edwards (1995) found that the Democratic candidates were frequently represented as the Seven Dwarfs, which very effectively conveyed the message that they were all equally irrelevant, weak and powerless. One of the cartoons we used in our study (figure 1) is another good example of how size can be imbued with a metaphorical meaning. As cartoonist Peter Schrank explained (personal communication, June 2005), he intended the giant boot to express the utter indifference of the US administration toward the UK; Tony Blair had been used by them, but hes powerless. I like the way the cowboy boot as a simple object represents the whole administrations and current US cultures attitude. In this case the

Metaphor in political cartoons

177

intention was thus to use exaggerated size to signify the abuse of power and force, rather than simply to increase the salience of this visual element.

Figure 1. Peter Schrank, Independent on Sunday, 15.10.2004

Some authors believe that there are significant differences regarding what the visual and the verbal mode are able to convey effectively (Messaris 1994; Kress 2000). For instance, the visual mode differs from language in that it is simply not possible to represent abstract meaning visually without recourse to symbols, metonyms, or metaphors (El Refaie 2003; Forceville 2005). So, in our example the concept of an unequal relationship between the British Prime Minister and the US President could not have been expressed literally in pictorial form at all and had to be translated into a metaphorical image. Conversely, because images always represent a particular instance of someone or something, they are more specific than words, capturing nuances of meaning that would be hard to convey through language. Moreover, images often evoke profound emotional responses which are hard to explain and of which the viewer may not always be entirely conscious (Zakia 2002: 233255). The precise look of the cowboy boot and Tony Blairs facial expression in figure 1, for instance, speak volumes about the cartoonists opinion of both politicians. The cartoon in figure 2, which was published on the day of the 2004 US Presidential elections, uses size and foregrounding to indicate salience: even at a glance, we can recognize that the larger-than-life matchbox is clearly essential to the argument the cartoonist is trying to make. The central metaphor in this cartoon is based on the representation of George Bush as a small child, which could be verbalized as BUSH IS TODDLER. In formal terms, this can be described as a monomodal metaphor of the pictorial variety, or, more specifically, as a hybrid (Forceville 1996: 163) or fusion (Carroll 1996) metaphor, where the target and the source are visually amalgamated into one

178

Elisabeth El Refaie

spatially bounded object. Such fusion is very common in political cartoons, since caricatures often overlay the features of a famous personality onto any imaginable being or object (Bell 2004). Provided the caricatured personality is recognized, this type of visual metaphor should work perfectly well without the support of a verbal label. Here the face of George Bush, the target of the metaphor, is visually amalgamated with the body of a toddler, which represents the source. Once both target and source have been identified, the reader is invited to map properties of a prototypical toddler (playful, irresponsible, stubborn, impetuous) onto the American president.

Figure 2. Nicholas Garland, Daily Telegraph, 2.11.2004

However, this example demonstrates that newspaper cartoons are typically about more than just the characteristics of a person. Rather, they tend to represent a particular situation, event or action in terms of something else, which in this case was paraphrased by the artist Nicholas Garland (personal correspondence, June 2005) in the following way: It is dangerous to give a small child a box of matches to play with President Bush is not to be trusted to be sensible and wise in my opinion. This concept is clearly more complex, and it requires the cartoon to be read as a narrative, representing not just participants, but also events and temporal sequence. Although still images are not particularly suited to the task of expressing action and events (Kress 2000), even single-panel images such as political cartoons are able to convey some narrative meaning through the depiction of movement that is frozen in the instance of representation (Schirato and Webb 2004: 87). This sense of activation can be increased by the use of vectors, strong, often diagonal lines formed obliquely by depicted objects or people, which indicate the direction of an action (Kress and van Leeuwen 1996: 4378) and, in some cases, through conventionalized motion lines (Horn 1998: 136) leading to or from a moving element. The cartoonist thus

Metaphor in political cartoons

179

relies on every readers ability to complete in his or her head what is suggested by an image, including the actions that precede and follow the depicted moment (Edwards 1997: 53). Since static images are also unable to express chronology directly, cartoonists are bound to use spatial relations to indicate the passing of time. According to Lakoff and Johnson (1980: 4144), we are in any case used to thinking about time in terms of space, with the future in front of us, the present right by us, and the past behind. We imagine the passing of time in two different ways: in the TIME IS A MOVING OBJECT metaphor, we are facing toward the future, which creeps up on us and which we have to meet head on. However, time can also be conceptualized as a stationary landscape through which we move in the direction of the future (were approaching the end of the year). Although these two metaphors are not consistent, they are nevertheless coherent by virtue of being special cases of the same underlying metaphor TIME PASSING IS MOTION (Lakoff 1993: 217). In the two cartoons shown above, action is implied through frozen movement and strong diagonal vectors, while chronology is suggested through the relative position of the main active participants, who, in accordance with the MOVING OBSERVER metaphor, are shown traveling through time from the past to the future. In figure 1, the past is in the foreground and the future in the background, so that the viewer is facing in the same direction as the main participants and, like them, is confronted with the prospect of an uncertain future. The image of someone walking along and, reaching a fork in the road, having to decide which way to go, is an instantiation of the LIFE IS A JOURNEY metaphor, which has been described as one of the most conventional ways of conceptualizing the link between space and time in Western cultures (Lakoff 1993). Perhaps partly because of the conventionality of this metaphor, which is firmly rooted in the embodied source-pathgoal schema (Johnson 1987), visual information alone may suffice to cue the idea of two destinations representing future decisions. However, a more specific reading of this cartoon, which could perhaps be verbalized as FUTURE US FOREIGN POLICY IS BOOT CRUSHING OTHER COUNTRIES, requires more detailed information about the various slots in this schema; this additional information is at least partially provided by verbal tags. The slot of moving observer, for instance, is further specified through a doubly coded metonym: Since the American President is frequently referred to as W in order to distinguish him from his father, the use of this letter in the cartoon is likely to support the reading of the cowboy boot as a visual metonym for Bush, a reading principally triggered by his Texan background and his famous predilection for Western-style casual

180

Elisabeth El Refaie

dress. The detailed drawings of the wall and nuclear power station, metonyms for Irans controversial nuclear program and the long-running territorial conflict between Israelis and Palestinians, provide additional information about the precise goals of the boot, as do the visual symbols of the vulture and the dove, representing war and peace respectively. The verbal elements on the signs pointing to Israel + Palestine and Iran + North Korea provide important support in the identification of these visual elements in the background, which may otherwise be difficult to recognize. This example clearly demonstrates that reading political cartoons often requires readers to draw on several different kinds of interpretive strategies, including the ability to recognize visual symbols and the targets of metonyms. Similarly, in figure 2 the basic interpretation of a conceptual link between space and time involves only the visual mode and is not dependent upon verbal information. Again, the image of the toddler crawling towards something cues the MOVING OBSERVER metaphor, although here the orientation is reversed, so that the background of the image represents the past and the foreground the future. This places the viewer in a knowing position where, in contrast to Bush-the-toddler, he or she is already able to see only too clearly what the next few years are likely to bring. This general orientational schema provides the basis for a more complex structural metaphor, which we might verbalize as ACHIEVING RE-ELECTION IS GETTING HOLD OF A BOX OF MATCHES. In this case, only the source is visually present, while the target must be gleaned from the pictorial context, any verbal clues, or more generally the world knowledge of the viewers (Forceville 1996: 109). The inscription on the box of matches is instrumental in pointing the viewer towards a more exact interpretation of the general SPACE-IN-FRONT IS FUTURE metaphor, since the precise concept of SPACEIN-FRONT IS FOUR MORE YEARS IN OFFICE could not have been rendered exclusively in the visual mode and needed to be supplied partially as a verbal message.3 In this context, the box of matches thus denotes Bushs bid to be re-elected and the opportunity this would offer him to cause even more havoc, while the flames are clearly meant to represent some of the Presidents foreign policies, such as the invasion of Iraq and the launch of his socalled war on terror, which by the time of the cartoons publication were already being seen by many as disastrous. In the case of the complex structural metaphors FUTURE US FOREIGN POLICY IS BOOT CRUSHING OTHER COUNTRIES and ACHIEVING RE-ELECTION IS GETTING HOLD OF A BOX OF MATCHES I believe we are dealing with examples of true multimodal metaphors, even though, strictly speaking, they are not covered by Forcevilles (2006: 384) definition, which stipulates that

Metaphor in political cartoons

181

in multimodal metaphors target and source must be represented exclusively or predominantly in different modes. In our examples, the verbal tags offer additional but nevertheless essential information, which helps the reader identify the precise source or target of a complex metaphor. In order to embrace such typical instances of multimodal metaphor in political cartoons, Forcevilles definition would thus have to be extended to include cases where target and source are partially represented in different modes.

3. Universality versus individuality According to the standard view of metaphor in a cognitivist framework, metaphors are based on embodied human experience (Lakoff and Johnson 1999; Turner 1996), which is seen as universal. For example, since we all feel hot as a result of physical exertion or excitement, metaphors based on the concept of INTENSITY IS HEAT seem entirely natural to us (Kvecsec 2005: 18). Similarly, we all experience a connection between our movement through space and the passing of time. This would suggest that everybody understands such basic ontological and orientational metaphors intuitively and, very often, at the level of unconscious or barely conscious thought processes. While primary metaphors are thought to be universal, they can be combined to form more complex mappings between different domains of experience, and at least some of these structural metaphors are assumed to be language or culture specific. For instance, there is evidence that in many languages anger is conceptualized as a fluid or gas under pressure in a container, but in Zulu it is apparently also sometimes understood as OBJECTS IN THE HEART (Kvecses 2005: 69). However, such a neat split between universal primary metaphors and more culturally influenced structural metaphors may be deceptive. It is perfectly possible that some universal experiences do not lead to universal metaphors and that bodily experience may in some cases be overridden by both culture and cognition (Kvecses 2005: 4). Moreover, some metaphors may not be based on bodily experience at all, but rather on purely cultural considerations and cognitive processes (Talebinejad and Dastjerdi 2005). Even more importantly, perhaps, the choice of metaphors by a communicator and the recognition and interpretation of these metaphors by audiences are likely to be strongly dependent upon the social context in which they are used. As Ritchie (2004: 277279) points out, the same metaphorical statement, my wife is an anchor, may be used to mean completely different things

182

Elisabeth El Refaie

in different conversational contexts, implying, for instance, either feelings of contentment and stability or a sense of boredom and frustration. Building on the concept of working memory, Ritchie (2004) suggests that people interpret metaphors by drawing on readily accessible elements from their long-term memory, as well as all the ideas that were recently activated through the communicative interaction, including the perceived relationship between participants and the degree to which they expect their perceptions and ideas to be shared (Ritchie 2004: 272). If peoples cognitive representations of the common ground are radically different, then they may well interpret the same metaphor in very different ways. Whether the processing of a metaphor stops as soon as an initial interpretation is reached or whether it continues to generate various metaphorical entailments will depend on the nature of the conversation, the relationship between participants, the available time, distractions and perceived importance. Metaphor elaboration may also be encouraged by the intrinsic pleasure involved in thinking and solving puzzles. In the case of political cartoons, it is perfectly possible that some basic ontological and orientational metaphors are understood in a similar way, perhaps even intuitively, by all members of a culture or language community, irrespective of their background and prior knowledge. However, it seems more likely that the interpretation of all metaphors in cartoons is at least to some extent dependent upon the context in which they are used and the working memory of the individual reader. Since political cartoons typically refer to topical events, it is clear that they will be read differently if taken out of their original context and viewed at a later date. Apart from lacking knowledge about a particular event or political issue, readers may also be unfamiliar with aspects of the fictional world on which the cartoon is based. In Western cultures, at least, it is common for cartoonists to use metaphors drawn from everyday life, sport and popular culture in order to ensure that they are in fundamental harmony with the cultural literacy of their public (Fischer 1996: 122; Mumford 2001: xi). But cartoonists clearly sometimes misjudge their public, and the background and experiences of some individual readers may lead them to interpret a cartoon in very different ways to the one intended. Metaphors are by their very nature open to more than one interpretation, which makes their use more risky for communicators, but also potentially more interesting for interpreters. In fact, if cartoons were too easy to understand, they would probably not provide the same sense of pride and achievement to those viewers who manage to solve these intriguing mental puzzles.

Metaphor in political cartoons

183

4. Data and Methods This chapter is based on data collected in phase one of our study into the ways in which young people understand cartoon images, including the multimodal metaphors they often contain. For this purpose, we conducted oneto-one interviews with 25 young people between the ages of sixteen and nineteen in Bradford, a city with a large British Asian population. Of the 13 men and 12 women, 13 were Muslims from a Pakistani or Bangladeshi background, while two each were Hindus and Sikhs from Indian families. The family of one young man had come to Britain from Afghanistan, and there were also one British-Indian, one black, and six white British-born youngsters who described themselves either as Christian or Atheist. Most of our respondents were doing A-levels, whereas six of the participants were taking vocational courses. The majority of the interviews were conducted in July 2005 and a small number in November of the same year. We used five cartoons on the US Presidential elections published several months previously, in October or November 2004. Since the participants had volunteered to take part in the study, we expected them to be on average a little more interested in political issues and more likely to follow the news than others in their peer group. Although more than half of our interviewees initially described themselves as not very or not at all interested in politics, during the interviews the Muslim students in particular actually turned out to hold quite passionate views about those geopolitical issues that they felt concerned them directly, such as the invasion of Iraq and the conflict between Israel and Palestine (Hrschelmann 2008). If they read any newspapers at all, the respondents tended to read the local paper or, more rarely, a tabloid or the occasional broadsheet newspaper. In the UK, local newspapers and tabloids tend not to carry many cartoons of an explicitly political nature.4 Not surprisingly, many of the young participants were thus not very familiar with the political cartoon genre. The semi-structured one-to-one interviews, which lasted between 30 and 45 minutes each, took place at the Technology College and local Further Education College where the young people were studying. After explaining that the cartoons were all about the recent Presidential elections in the USA, the cartoons were discussed one by one, using the entire newspaper page in order to ensure that at least some of the context was preserved. We encouraged respondents to read any of the headlines that they would normally read in conjunction with the cartoon before describing the drawing and then attempting an interpretation. In fact, while the articles below the cartoons did generally comment on the US Presidential elections, none of them related

184

Elisabeth El Refaie

directly to the content of the cartoons. This meant that, in the few cases that the respondents did avail themselves of the chance to read the headlines, this did not necessarily assist them in their interpretation of the cartoons. In order to ensure that the interview data were comparable, we used the same basic interview schedule, consisting of a small number of open and non-leading questions such as: Can you recognize any of the depicted characters?; How would you describe their mood/feelings?; What do you think the cartoonist wanted to say with this cartoon?; What are your own thoughts and feelings when you look at this cartoon? While taking care not to prejudice the participants in their responses, we occasionally used additional prompts in order to elicit more details or to clarify their answers. For instance, if respondents seemed to be struggling to describe the mood of the figures in the drawing, we might ask them to describe the relationship between the depicted characters and to imagine what they might be feeling. If they overlooked certain important details, such as the inscription four more years on the matchbox in figure 2, we would generally draw their attention to these and ask what they might be referring to. In order to preserve the anonymity of the interviewees, responses were coded with a letter from A to Y. ER indicates that I conducted the interview, and KH stands for my collaborator, Kathrin Hrschelmann. The symbol (.) in the transcriptions indicates a hesitation or pause in the respondents delivery. We also sent short questionnaires to the creators of the cartoons used in the study, asking them to describe the intended meanings of their drawings. In addition to completing the questionnaire, most of the artists agreed to a telephone interview, during which they talked about their general working conditions and practices, and explained in more detail what had given them the inspiration for the cartoons used in the study, why they had chosen particular symbols and metaphors, and what they were trying to express through their work. Clearly, the meanings intended by the cartoonist are not the only ones that a particular cartoon or, indeed, a particular multimodal metaphor is able to generate, nor are his or her intentions to be equated with its correct or most valid interpretation. By its very nature, the cartoon genre is ambiguous and open to multiple readings, and the process of making sense of a cartoon is likely to be strongly dependent upon the individual interpreters background and experiences. In fact, our own detailed analysis of the cartoons corresponded very closely with the artists intended meanings. This may be linked to the fact that we probably represent the sort of politically minded regular newspaper readers that the cartoonists apparently had in

Metaphor in political cartoons

185

mind when creating their work. In contrast to this, the young peoples readings of the cartoons and the multimodal metaphors they contained reflected their very different interests and preoccupations, as well as perhaps a degree of unfamiliarity with cartoon conventions.

5. Results: Interpreting cartoon metaphors As I showed in section 2, two of the cartoons used in our study contain several layers of metaphorical meaning, from the basic connection between movement through space and the passing of time to the more complex associations between a make-believe mini narrative and events in the real world. In the case of the cartoon by Peter Schrank, published in the Independent on Sunday (figure 1), another basic metaphor that seemed to evoke a common response was that of the darkened sky, which for the four participants who mentioned this aspect indicated a sense of danger and impending disaster. The orientational TIME IS MOTION THROUGH SPACE metaphor also seemed to be understood in a similar way by all the respondents, since nearly everybody equated the concept of the boot walking towards Iran and North Korea with future actions, with the two signposts pointing in opposite directions representing alternative decisions. One exception to this general rule was respondent I, who described the boot as coming from Palestine and other Muslim countries, where it has caused a lot of destruction. For this young man, the figure stuck to the sole of the boot embodied the innocent people in Afghanistan who have suffered George Bushs wrath or so-called wrath, people that suffered in Iraq. Similarly, respondent J, a young Muslim woman who was clearly very concerned with the plight of Palestinian civilians, thought the small figure represented a Palestinian:
J I think he is trying to show all (.) all this war going on (.) on in Palestine (.) thats been going on for the past year or so (.) its trying to (.) show that (.) its George Bushs fault and you know he is just crushing people for nothing without realizing it and people are yelling at him look what you are doing to our country but hes not taking no notice and he is just crushing people under his feet (.) and now hes been to Palestine then hes going to Iran and so hes going across the world doing it to all the countries

In fact, eleven respondents did not recognize Tony Blair, seeing the drawing instead as symbolizing an ineffective politician or, more generally, the weak

186

Elisabeth El Refaie

and powerless: So there is a big authority leader going towards the way he wants and there are smaller people getting stuck (F). Many of the participants said it was the W on the boot that allowed them to make the connection to George Bush, while a few seemed to rely more on the general context or the Texan connotations of the cowboy boot to draw this conclusion. Even among the eight respondents who did not make the metonymic connection between the boot and George Bush there was still unanimous agreement about the fact that the person symbolized by the boot was powerful and ruthless. Participants talked about him having the ability to go where he likes or do as he likes and his actions were described as standing, stepping, crushing, or trampling on someone less powerful: sort of dont care who he steps on (M). Thus, the metaphorical link between size and power, status, or force was understood in a similar way by everybody, although the degree to which the boot was seen as ferocious and sadistic seemed to depend on whether or not Blair and Bush were recognized and the degree to which the respondent identified with the figure squashed under the boot. Clearly, only the eleven respondents who recognized the references to both politicians were able to read the cartoon in the way it was intended by the cartoonist, namely as a comment on the relationship between the two men and their respective countries:
R Tony Blair is basically just (.) hes following George Bush round like a lost sheep (.) even though that Tony Blair does have his own (.) he wants to do his own things (.) hes like scared that (.) he wont let go cause hes scared of losing America (.) and thats why I think that Tony Blair he just isnt a suitable leader

This excerpt also demonstrates that, when discussing the meaning of a particular metaphor, respondents often introduced new metaphors or similes to express their thoughts. The description of Tony Blair following Bush around like a lost sheep evokes a very different image from the one represented in the cartoon and shifts the focus from the actions of the US President to those of the British Prime Minister. The young people struggled to recognize other aspects of the cartoon image as well. For instance, most of our interviewees read out the place names on the signposts but were unable to make sense of the little drawings in the background. The wall behind the signpost pointing to the left was described by six respondents as a wall or fence, while four students realized that the drawing of a power station on the other side had something to do with nuclear energy or nuclear weapons. The latter drawing reminded two interviewees of the oil industry, while others described it as a crane, buildings,

Metaphor in political cartoons

187

satellites and communication (B), a building with a big ball on top of it (N) and a door of a mosque and buildings (L). Only two of the respondents (K and U) were sufficiently well informed about the related geopolitical issues to be able to refer to these in their interpretation of the cartoon. In Ritchies (2004) terms, most of the young people did not have these facts stored in their long-term memory, or at least they were not able to access them when cued by the cartoon image. Unsurprisingly, this influenced their readings of the two metonyms WALL IS ISRAEL-PALESTINE CONFLICT and POWER STATION IS NUCLEAR ISSUE. 21 of the participants were also not able to name either the vulture or the dove (or both), referring to them instead as a big bird (T), a turkey (J), an eagle (K), and as a normal (smaller) bird (J, T) or pigeon (P) respectively. With some respondents, it was simply that they could not think of the correct term, but in many other instances the students were apparently not aware of the conventional meanings associated with these symbols. However, for these respondents the size and other physical features of the birds nevertheless seemed to act as powerful metaphors for particular attitudes and characteristics. Respondent S, for instance, described the bird on the left as homeless and the one on the right as selfish, greedy. As pointed out above, this lack of political background knowledge and familiarity with common cultural symbols did not prevent the vast majority of our respondents from understanding that the fork in the road represented a choice of future actions according to the source-path-goal schema. This would seem to support a tentative conclusion that basic orientational metaphors are generally quite widely and easily understood, particularly if they are represented through a very conventional visual image, like the one in figure 1. By contrast, the interpretation of metaphors of a more elaborate structural nature (e.g. FUTURE US FOREIGN POLICY IS BOOT CRUSHING OTHER COUNTRIES) requires readers to use any verbal tags, pictorial symbols and/or metonyms to assemble more precise information about target and source and to apply the general schema to a specific social or political scenario. The cartoon by Nicholas Garland (figure 2) was again recognized by almost all the participants to be a narrative image, and spatial relations were used to comprehend the sequence of events. However, in this case an understanding of the nature of future action was more tightly connected to grasping the implications of the inscription (four more years) on the matchbox. 23 out of 25 respondents understood that the matches had something to do with the future, but this interpretation was generally not forthcoming until after the inscription had been noted and read out in the course of the inter-

188

Elisabeth El Refaie

views. Only respondent X seemed not to understand that the four more years referred to the future: the matchbox (.) four years (.) is that how long he has been president? Another young person was unsure about how to read the sequence of events: it looks like he is going for the matches but theres already a fire lit (.) no idea what thats supposed to mean (H). Five of the participants explicitly made the connection between this inscription and the length of a Presidential term in office in the US, with one young man drawing attention to the number of matches lying on the ground: there are four matches out representing four more years (U). The great majority (18) seemed to grasp from the context that the inscription must refer to George Bushs re-election, even though they did not explicitly refer to the length of the US Presidential term and may not have been aware of this fact. As I discussed in section 1, the more elaborate structural metaphor BUSH IS TODDLER is a monomodal metaphor of the visual variety, which is completely independent of the verbal mode. However, it does require the reader to identify the caricature of the American President. Six people did not recognize George Bush as the target of this metaphor, in which case they tended to read the cartoon in a rather literal way, as a depiction of the threatening nature of fire. One young woman, for instance, was clearly worried by the picture of a baby in extreme danger and speculated that the cartoon was trying to convey a message do not leave matches out for little babies (E). In this context, the interviewee was puzzled by the inscription on the box, speculating that it might refer to the age of the child or to the longevity of a particular brand of quality matches:
KH have a look at the box of matches (.) what do you think four more years means? E never actually noticed that before (.) four more years four more years (.) it could have a double meaning four more years (.) unless a child is four years old or could live for few more years rather than putting the child in danger (.) or it could be branding matches could last a long time

In addition to the respondents who did not recognize the US President, 12 of the participants did not seem to see the figure in the drawing as childlike at all, which meant that the source of this metaphor was also not perceived in the same way by everyone. In fact, one participant described the figure as an old woman (F) and another as a gentleman:
B okay theres an image of a gentleman crawling towards a box of matches which says four more years on them (.) so Im guessing its to do with the election (.) theres smoke and fire in the background

Metaphor in political cartoons

189

which means perhaps the gentleman is going to strike a match and add more fire to the background make it even worse (.) thats what it shows ER hmm (.) and can you identify who the character (.) do you think its anyone specific? B no (.) no ER and can you describe the the = B = Ive met him I cant remember his name Ive met him (.) no Im an ex member of youth parliament and so Ive met all of these top guys (.) but I cant remember his name

As this example shows, by no means can it be taken for granted that all newspaper readers will recognize the caricatures of even the most wellknown politicians. One young woman taking part in our study, for example, did not appear to be at all sure what the US President looked like, although she thought she had seen him on TV once in a blue moon (O). Respondent B, by contrast, clearly had a keen interest in politics and had even met some politicians face-to-face. He nonetheless seemed unable to see any similarity between this drawing and the US President and was instead trying to name a British politician who might fit the bill; in our subsequent conversation it turned out that he was probably thinking of John Prescott, the British Labour Party politician and Deputy Prime Minister. Cartoonists employ a particular shorthand style when drawing famous politicians and it is therefore hardly surprising that people who are not familiar with this genre or with a particular artists work will struggle to recognize some caricatures. But even among those respondents who identified both the source and the target of the metaphor BUSH IS TODDLER, the similarities they saw between the two domains often diverged from or went beyond the meanings intended by Garland. For example, one woman (Y) initially explained this metaphor by pointing out that people actually see Bush as a small child and not very intelligent, but she later went on to generate more entailments, suggesting that, like a baby, he seems to need taking care of: whenever you see him he is always surrounded by so many I dont know bodyguards and everything (.) he likes the attention if you like. Respondent R focused on the idea that the more something is perceived to be naughty, the more a child wants to do it, while another participant suggested that the cartoonist had drawn Bush as a grinning baby in order to show his absolute confidence in his victory: especially at a young age (.) like possibly a toddler crawling towards four more years in power (.) he could be really confident (.) and because of that hes showing that he can do it without all his American citizens (I).

190

Elisabeth El Refaie

While most people saw the fire as a metaphor for general problems, conflict, war and destruction, or more specifically the invasion of Iraq or 9/11, respondent A interpreted the fire in the background as a reference to the difficulties George Bush had to face in order to be re-elected: he has been through hell to get four more years. Participant S, by contrast, thought the figure crawling towards the match box looked like John Kerry and saw the matches as representing his being under stress: he looks so pressurized cause hes got a flame up the back and hes trying to reach to the matches cause that shows that hes got tension (.) and hes got a lot of stress due to the voting four more years to work (.) he has to organize it present it everything so it all goes well. Although most of the interviewees did make a connection between the matches, fire and danger of some sort, only four explicitly mentioned the idea that children should not be allowed to play with matches and that if you play with matches you get burnt (U).

6. Conclusion and suggestions for further research Multimodal metaphors, particularly those involving pictures and language, offer the opportunity to explore different peoples understanding of the same material, by asking them to verbalize their interpretation process. In our study we were able to compare and contrast our own analysis of the various levels of metaphor with the ways in which the same metaphors were read by a group of young people. We were also at least to some extent able to relate the various responses to the young peoples background, prior knowledge, and experiences. This allowed us to explore the different roles of the verbal and the visual mode in multimodal metaphor and to begin to address the thorny issue of universality versus individuality in metaphor understanding. However, in view of the relatively small number of participants in this study, the following are just tentative conclusions, which would require further investigation. The results of our study indicate that some metaphorical mappings in political cartoons, such as those between size and power/status and between movement through space and the passing of time, might be understood more generally and at a more intuitive level than more elaborate structural metaphors. These orientational metaphors are also generally monomodal, in the sense that visual information alone is sufficient to cue the conceptual link between a more concrete and an abstract meaning. However, reading a cartoon usually requires the various slots of basic schemata to be filled in with more specific and detailed information. The

Metaphor in political cartoons

191

interpretation of the complex structural metaphors, ACHIEVING RE-ELECTION IS GETTING HOLD OF A BOX OF MATCHES and FUTURE US FOREIGN POLICY IS BOOT CRUSHING OTHER COUNTRIES, for instance, requires readers to draw on verbal clues (four more years, place names on the sign posts) as well. This, I propose, justifies their inclusion in the category of multimodal metaphors, in which case Forcevilles (2006: 384) definition would need to be slightly extended to comprise all instances of metaphor where the target and source are represented exclusively, predominantly, or partially in different modes. The data also reveal that the more complex structural metaphors could only be read in the way intended by the cartoonist if respondents had a certain level of general knowledge and a familiarity with current affairs and particular political figures. Clearly, the identification of metaphors is only part of the process of cartoon interpretation as a whole, which means that the study of how people understand multimodal metaphors must also include a consideration of more general processes of meaning-making, which in turn are closely linked to an individuals education, background, and experience of the genre in which the metaphor occurs. For instance, since most of our respondents were not accustomed to the cartoon genre, it is hardly surprising that many of them struggled to identify the caricatures of even very wellknown politicians. Even if the target and the source of a metaphor were identified in the way they were intended by the cartoonist, the exact interpretation of it and the entailments it generated varied a great deal from one interviewee to another and reflected their current interests and preoccupations. This is consistent with the view that people interpret metaphors by drawing on elements from their working memory (Ritchie 2004: 272). The Muslim respondents, for instance, appeared to be particularly conscious of the suffering of the civilian population in the Middle East, which was reflected in the way many of them tended to map properties of extreme cruelty and ruthlessness from the cowboy boot onto the American President. Another good example of the important role of personal experience in the interpretation process is the way in which respondents applied completely different features of toddlers to George Bush in the context of his bid for re-election. Unfortunately, asking people to verbalize visual meaning is also not without its theoretical and methodological problems. If, as many authors believe, the verbal and the visual mode differ with respect to what they are able to communicate, then any translation from one mode to the other will involve some loss of meaning. Thus, verbal responses may not provide entirely reliable data for judging what people are actually thinking when they

192

Elisabeth El Refaie

look at a visual metaphor, particularly if the respondents are unaccustomed to such essentially artificial verbalization tasks. Perhaps future researchers will be able to devise methods of exploring multimodal metaphor that are not wholly dependant upon respondents verbal dexterity. Another difficulty concerns Ritchies (1994) observation that the same metaphor can assume entirely different meanings when encountered in different contexts. Although we tried to replicate the ordinary reading situation as closely as possible, for instance by providing the whole newspaper page, the interview situation was in fact far from ordinary. Most of the young people were not particularly avid newspaper readers, and they tended not to be very familiar with the political cartoon genre. Under normal circumstances they would probably never have encountered these cartoon images, or, if they had, they may not have paid them much attention. Apart from a couple of interviewees who were unable or unwilling to continue to discuss the two cartoons beyond a very superficial description, our respondents were nevertheless very cooperative and tried hard to glean some meaning from the images. Prompted by our questioning, they were often prepared to generate further metaphorical entailments, which they would otherwise perhaps not have considered at all. Although we tried to keep our interventions to a minimum and not to ask any leading questions, it is probably inevitable that our own interests to some extent influenced their readings of the cartoons and the metaphors within them. In spite of these limitations, the study provides striking new evidence for the unpredictability of metaphor interpretation in real life situations. Our results suggest that analysts of all types of metaphor, be they verbal, visual or multimodal, must guard against taking their own interpretations for granted and must pay more attention to the pragmatics of metaphor use in particular contexts.

Notes
1. 2. 3. Elisabeth El Refaie and Kathrin Hrschelmann, Editorial cartoons and geopolitical perceptions, April 2005April 2007. We would like to thank the British Academy for supporting this project (Grant No. SG-39469). We are very grateful to Peter Schrank and Nicholas Garland for their kind permission to reprint their cartoons and for agreeing to be interviewed. For some readers, the verbal mode might play another, less explicit, role in this cartoon; for them, the drawing might evoke the verbal expression playing with fire. In this case, it could be said to represent a striking instance of a cartoon metaphor in which the cartoonist merely secures what language

Metaphor in political cartoons

193

4.

has prepared (Gombrich 1971: 128; cf. El Refaie 2009). None of the participants in our study referred to this expression. The other three cartoons used in the study were taken from the Guardian, the Independent and the Daily Mail respectively.

References
Baldry, Anthony, and Paul J. Thibault 2006 Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Coursebook. London/Oakville: Equinox. Bell, Steve 2004 Drawing Tony. The Guardian 17 July: 11. Carroll, Noel 1996 A note on film metaphor. Journal of Pragmatics 26: 809822. Dines-Levy, Gail, and Gregory W. H. Smith 1988 Representations of women and men in Playboy sex cartoons. In Humour in Society: Resistance and Control, Chris Powell and Gregory E. C. Paton (eds.), 234235. Basingstoke/London: MacMillan. Edwards, Janis L. 1995 Wee George and the Seven Dwarfs: Caricature and metaphor in campaign 88 cartoons. Inks 4 (May): 2634. 1997 Political Cartoons in the 1988 Presidential Campaign: Image, Metaphor, and Narrative. New York/London: Garland. El Refaie, Elisabeth 2003 Understanding visual metaphor: The example of newspaper cartoons. Visual Communication 2 (1): 7596. 2009 Multiliteracies: How readers interpret political cartoons. Visual Communication 8 (2): 181205. Fischer, Roger A. 1996 Them Damned Pictures: Explorations in American Political Cartoon Art. North Haven CT: Archon Books. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2004/this vol. The role of non-verbal sound and music in multimodal metaphor. In Words in their Places: A Festschrift for J. Lachlan Mackenzie, Henk Aertsen, Mike Hannay, and Rod Lyall (eds.), 65 78. Amsterdam: Faculty of Arts, VU Amsterdam. 2005 Visual representations of the idealized cognitive model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Ap-

194

Elisabeth El Refaie

plications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. Gombrich, Ernst H. 1971 The cartoonists armoury. In Meditations on a Hobby Horse and Other Essays on The Theory of Art. 2d ed., 127142. London/New York: Phaidon. Hrschelmann, Kathrin 2008 Youth and the geopolitics of risk after 11 September 2001. In Fear: Critical Geopolitics and Everyday Life, Rachel Pain and Susan J. Smith (eds.), 139152. Aldershot: Ashgate. Horn, Robert E. 1998 Visual Language: Global Communication for the 21st Century. Washington: MacroVU. Johnson, Mark 1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination and Reason. Chicago: University of Chicago Press. Kaplan, Stuart Jay 1992 A conceptual analysis of form and content in visual metaphors. Communication 13: 197209. Kennedy, John M., Christopher D. Green, and John Vervaeke 1993 Metaphoric thought and devices in pictures. Metaphor and Symbolic Activity 8: 243255. Kvecses, Zoltn 2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press. 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Kress, Gunther 2000 Text as the punctuation of semiosis: Pulling at some of the threads. In Intertextuality and the Media: From Genre to Everyday Life, Ulrike H. Meinhof and Jonathan Smith (eds.), 132154. Manchester: Manchester University Press. Kress, Gunther, and Theo van Leeuwen 1996 Reading Images: The Grammar of Visual Design. 2nd ed. 2006. London/New York: Routledge. 2001 Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold. Lakoff, George 1993 The contemporary theory of metaphor. In Metaphor and Thought, 2d ed., Andrew Ortony (ed.), 202225. Cambridge: Cambridge University Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago/London: The University of Chicago Press.

Metaphor in political cartoons 1999

195

Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books. Messaris, Paul 1994 Visual Literacy: Image, Mind, and Reality. Boulder, CO/Oxford: Westview. 1997 Visual Persuasion: The Role of Images in Advertising. Thousand Oaks, CA: Sage. Morrison, Susan S. 1992 The feminization of the German Democratic Republic in political cartoons 198990. The Journal of Popular Culture 25 (4): 3552. Mumford, Alan 2001 Stabbed in the Front: Post-War General Elections Through Political Cartoons. Canterbury: Centre for the Study of Cartoons and Caricature. Philippe, Robert 1982 Political Graphics: Art as Weapon. Oxford: Phaidon Press. Phillips, Barbara J. 2003 Understanding visual metaphor in advertising. In Persuasive Imagery: A Consumer Response Perspective, Linda M. Scott and Rajeev Batra (eds.), 297310. Mahwah, NJ/London: Lawrence Erlbaum. Proctor, Tony, Stella Proctor, and Ioanna Papasolomou 2005 Visualizing the metaphor. Journal of Marketing Communications 11 (1): 5572. Ritchie, David 2004 Metaphors in conversational context: Toward a connectivity theory of metaphor interpretation. Metaphor and Symbol 19: 265287. Schirato, Tony, and Jen Webb 2004 Understanding the Visual. London/Thousand Oaks/New Delhi: Sage. Scott, Linda M. 1994 Images in advertising: The need for a theory of visual rhetoric. Journal of Consumer Research 21 (2): 252274. Seitz, Jay A. 1998 Nonverbal metaphor: A review of theories and evidence. Genetic, Social and General Psychology Monographs 124 (1): 95119. Talebinejad, M. Reza, and H. Vahid Dastjerdi 2005 A cross-cultural study of animal metaphors: When owls are not wise! Metaphor and Symbol 20: 133150. Templin, Charlotte 1999 Hillary Clinton as threat to gender norms: Cartoon images of the First Lady. Journal of Communication Inquiry 23 (1): 2036.

196

Elisabeth El Refaie

Turner, Mark 1996 The Literary Mind. Oxford: Oxford University Press. Worth, Sol 1981 Studying Visual Communication. Philadelphia: University of Pennsylvania Press. Zakia, Richard D. 2002 Perception and Imaging. Boston/Oxford: Focal Press.

Chapter 9 Image alignment in multimodal metaphor Norman Y. Teng

Abstract
This chapter focuses on how image alignment as a design strategy figures in the construction of multimodal metaphors. Six editorial cartoons from The Christian Science Monitor are used as illustrative examples. Image alignment can take many forms. It can be linear, curvilinear, or exhibit a two-dimensional pattern. It works by making some constituent components of the alignment salient, surprising, evocative, or otherwise noticeable, or by making the shape of the overall alignment conspicuous and unexpected. Sometimes it is only implicitly involved in a design choice. How non-pictorial elements in a multimodal metaphor interact with the aligned pictorial components is explained by concrete examples. As to the conceptual basis for image alignment as a design strategy, a tentative thesis is put forward for future research: image alignment renders the abstract concept SIMILARITY visible on the basis of the experiential correlation that motivates the primary metaphor SIMILARITY IS ALIGNMENT. Keywords: cartoon, design, image alignment, primary metaphor, multimodal metaphor

1. Introduction This chapter focuses on image alignment and how it figures in multimodal metaphor. Image alignment as a design strategy was explored in Teng and Sun (2002), which elaborated and extended Forcevilles (1996) account of pictorial simile. The core idea of this strategy is this: when pictorial components are approximately aligned with one another with respect to size, orientation, and distance, the alignment thus formed is apt for expressing an idea that connects these pictorial components. For example, when the pictorial components depict things of different kinds, the alignment is apt for expressing pictorial simile. As another example, if the components depict things that

198

Norman Y. Teng

can be seen as incompatible, the alignment can express an oxymoron in pictorial terms. A quick look at the following cartoon (figure 1) may give us a good sense of how image alignment figures in pictorial representation. This cartoon features a pattern of image alignment apt for expressing pictorial simile. A newspaper is placed in alignment with books on a shelf. The newspaper is positioned in the middle of this alignment, and the books that flank it on both sides are all well-known horror novels. The front-page headline of the newspaper indicates that the news is about the United States. News and horror stories are of different narrative styles and presumably belong in different genres; however, the alignment suggests some similarity between the components. This cartoon, then, suggests that news about the US is similar to a horror story. The word HORROR engraved on the front of the upper shelf further supports this reading. Following the standard A IS LIKE B format of simile, this pictorial simile can be labeled AMERICAN NEWS IS LIKE HORROR NOVEL (see Teng 2006: 7374 for further discussion of this example). The expressiveness of image alignment as shown in figure 1 is not confined to pictorial representation, but also plays an important role in the design of multimodal metaphor.

Figure 1. The Horror Show, by Clay Bennett, The Christian Science Monitor, May 13, 2004, page 8. 2004 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

Multimodal metaphor is a newly formed research topic proposed and explored by Forceville (2006/this volume, 2008). This newly defined research topic incorporates Lakoff and Johnsons (1980a, b) insight that the occurrence of metaphors is not restricted to language. It directs attention to the

Image alignment in multimodal metaphor

199

phenomenon that metaphorical thoughts can be present in different modes of representation, and, more importantly, in a diverse range of combined versions of different modes of representation. Briefly, a metaphor is multimodal when its target and source are each represented exclusively or predominantly in different modes of representation, or when its targets and/or source are cued in more than one mode simultaneously (Forceville 2006: 384). I shall restrict my discussion to a subtype of multimodal metaphor, previously called verbo-pictorial metaphor (Forceville 1996: 148162), and examine how image alignment as a design strategy may figure in it. Six examples are discussed in the next section. All of them are taken from The Christian Science Monitor, for which they were created by Clay Bennett, the papers editorial cartoonist. (The reader may survey Bennetts cartoons, which are full of verbo-pictorial metaphors, by visiting his personal website at http://www.claybennett.com.) The purpose of the following discussion is to give a robust sense of how image alignment participates in the construction of multimodal metaphors. In the final section, a tentative thesis concerning the conceptual basis for image alignment as a design strategy is put forward for future research.

2. Image alignment as a design strategy Let us begin with a simple form of image alignment juxtaposition. Consider figure 2, which juxtaposes two images. The one in the left panel is a blurred image of a garbage can brimming with a mixture of rubbish and surrounded by scattered emptied tins and wasted food. The one in the right panel is essentially the same as that in the left panel, but it has such high resolution that people can see more clearly what it is about. The alignment suggests that the two images are connected via some idea. The word Television printed at the bottom of the left panel and the phrase HighDefinition Television printed at the bottom of the right panel further suggest what that idea is: TV programs are piles of garbage, and watching programs on a high-definition television will not change that. This alignment and the printed words jointly form a metaphor which, following the standard A IS B format, can be labeled TV PROGRAMS ARE GARBAGE. This metaphor is multimodal in that the target TV PROGRAMS is chiefly represented through the verbal representations Television and High-Definition Television, with the aid of the pictorial element that marks out the difference in image resolution, and the source GARBAGE is exclusively represented in the pictorial mode of representation.

200

Norman Y. Teng

Figure 2. High-Definition Television, by Clay Bennett, The Christian Science Monitor, June 15, 1998, page 8. 1998 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

Consider next figure 3. Again there is a pattern of alignment. The image in the left panel depicts a rhino, and the words Southern Africa above it indicate where the rhino is located. The image in the middle panel depicts a panda, and the words Western China above it indicate where the panda is located. The image in the right panel depicts a dove with an olive branch in her beak, and the words Middle East above it indicate where the dove is located. The words endangered species are printed on a rectangular box, which is superimposed on the upper parts of the panels. The label endangered species, positioned this way, suggests that the alignment puts the three depicted creatures in the same category, and conveys the message that rhinos in Southern Africa, pandas in Western China, and doves in the Middle East are all endangered species. It is assumed that people know that rhinos in Southern Africa are endangered, as are pandas in Western China. However, doves are not really endangered species, and they are not indigenous to the Middle East. Joining the doves to the endangered species suggests a metaphorical reading, and the readers are expected to take notice of the fact that doves and olive branches are often used as a symbol of peace in Western cultures. Taken together, this design suggests the metaphorical reading that peace in the Middle East, like rhinos in Southern Africa and pandas in Western China, is an endangered species. This metaphor is multimodal in that the target PEACE IN THE MIDDLE EAST is chiefly represented by the image of a dove holding an olive branch in her beak, and the source ENDANGERED SPECIES is chiefly represented by the category label endangered species, aided by the depiction of a rhino and a panda that serve as illustrative examples of endangered species. Following the standard A IS B

Image alignment in multimodal metaphor

201

format, this metaphor can be labeled PEACE IN THE MIDDLE EAST IS AN ENDANGERED SPECIES.

Figure 3. Endangered Peace, by Clay Bennett, The Christian Science Monitor, May 24, 2002, page 10. 2002 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

Alignment in figures 1, 2, and 3 is linear along the horizontal axis, but it can also exhibit other patterns. Consider figure 4. This cartoon depicts scores of worshippers aligning themselves in a two-dimensional pattern, which is underlined by the layout of the carpets. The white gowns that the worshippers wear and the decorative patterns on the carpets are further cultural elements that suggest to the readers that the worshippers are Muslims. The worshippers are on all fours, and most of them point their bodies in the same direction. At the center of this alignment, however, one worshipper, bearing a rifle and a pistol, holds his body in the opposite direction. The overall alignment suggests the interpretation that all the worshippers belong in the same group; they all believe in Islam and live according to its rules. However, the depiction of the opposite body-orientation at the center of this alignment suggests that the rifle-bearing worshipper is an exception. This cartoon is an editorial cartoon from The Christian Science Monitor, and the readers surely understand this important detail. While admiring the cartoons gentle sense of humor and the empathy it expresses with Muslims, the readers may also take in its political stance and read the message as follows: unlike the rest of Muslims, the rifle-bearing man is not really a true believer, or his religion is not true. The word terrorism printed on his belt and the bewildered look of another worshipper beside him reinforces this interpretation. The metaphor embedded in this overall interpretation can be labeled MISGUIDED BELIEF IS AN OPPOSITE BODY ORIENTATION. Many readers would undoubtedly arrive at the same interpretation without noticing the word on

202

Norman Y. Teng

the belt of the rifle-bearing man, because of the prototypical depiction of a terrorist. (I thank Francisco Yus for pointing out this to me.) This implies that the word terrorism does not play any essential role in their interpretation, and, as a result, the metaphor in this cartoon is hardly multimodal. It is worth emphasizing, however, that terrorism plays an important role from a design point of view. It guides peoples interpretations, and for the people who do not take the depiction of the rifle-bearing man as a prototypical depiction of a terrorist, it frames their interpretation. (For further discussion on the guiding and framing of image interpretation, which Barthes examined under the concept of anchoring, see Barthes 1985 [1964]: 2830, and Forceville 1996: 7174.) From this viewpoint, MISGUIDED BELIEF IS AN OPPOSITE BODY ORIENTATION can be considered a multimodal metaphor, in that it becomes multimodal when the target MISGUIDED BELIEF is chiefly represented through the word terrorism, aided by the depiction of a rifle-bearing man, and the source OPPOSITE BODY-ORIENTATION is exclusively represented by a pictorial component in the image alignment. Whether a design is multimodal depends on how it achieves its effect, and it is important to specify in what way a particular metaphor is, or becomes, a multimodal metaphor.

Figure 4. Call to Prayer, by Clay Bennett, The Christian Science Monitor, September 24, 2001, page 8. 2002 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

Figure 5 exhibits alignment in a curvilinear form. In figure 5, two army officers are pondering possible scenarios and what action the military should take. The American flag and the US badge on their upper arms indicate that they are US army officers. The words war on terrorism on the door window depicted in the upper left corner of the picture indicate that it is military strategies for war on terrorism that they are pondering. They seem to come

Image alignment in multimodal metaphor

203

to terms with a difficult situation, and reach a decision: an endless deployment of armed forces and non-stop military operations throughout a vast geographical area. The metaphor embedded in this interpretation is chiefly represented by the model tanks on the map, which line up into a curving shape reminiscent of the mathematical symbol of infinity . The model tanks metonymically represent the deployment of armed forces, the map represents a vast geographical area, and the curving alignment reminiscent of the mathematical symbol of infinity metaphorically represents what the strategic deployment will lead to. This is a multimodal metaphor, which can be labeled THE DEPLOYMENT OF ARMED FORCES AGAINST TERRORISM IS AN INFINITY DEPLOYMENT IN A VAST GEOGRAPHICAL AREA.

Figure 5. Terrorism Strategy, by Clay Bennett, The Christian Science Monitor, September 13, 2004, page 8. 2004 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

As shown by the above examples, image alignment can take many forms. It can be linear along the horizontal axis (figures 1, 2, and 3), a twodimensional pattern (figure 4), or a curvilinear form (figure 5). Despite the diverse forms it can take, alignment of any of these forms can be strategically deployed in the construction of multimodal metaphors through its central use for connecting a set of pictorial components in a way that is apt for expressing a certain idea. Figures 25 show how this strategic deployment may work. In figure 2, the two pictures of garbage cans are essentially the same except the contrast in image resolution between them. This contrast is a design choice made within the framework defined by the overall alignment. In figure 3, the dove, unlike the rhino and the panda, cannot properly be labeled endangered species; it is much smaller in size than the other two animals; and, more important, it is symbolic of peace. This contrast, again, is a design choice in the framework defined by the overall alignment. It is

204

Norman Y. Teng

worth noting here that alignment is defined in terms of iconic features such as size, orientation, and distance, and, as a result, the symbolic representation in figure 3 is anchored to the alignment via the iconic function of the pictorial components. In figure 4, the worshipper at the center of the alignment is in stark contrast to the rest of worshippers with respect to their body-orientation. This is a design choice, too. All these choices make some constituent components of the alignment salient, surprising, evocative, or otherwise noticeable. In figure 5, the model tanks line up into a shape reminiscent of the mathematical symbol of infinity. This is a design choice that makes the shape of the overall alignment conspicuous, and, probably, unexpected in that context. In contrast to figure 3, the symbolic representation in figure 5 is achieved via the overall shape of the alignment, rather than a particular aligned component. Design choices of the sort just described point to the directions in which the readers should take their interpretations, but they do not determine their interpretations. The above discussion of figures 25 shows how one may proceed to work out pertinent and well-balanced interpretations.

Figure 6. Its an Escher-type Economy, by Clay Bennett, The Christian Science Monitor, November 5, 2003, page 8. 2003 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

Figures 6 and 7 show that image alignment may be only implicitly involved in a design choice. In figure 6, a business person and a laborer each stand in an erect position, watching a zigzag arrow on a large sheet, which covers most of the wall. The word economy printed on the arrow indicates that it is a graphic report on the economy. The Y shaped lines, which represent the edges and the corner, define an unconventional three-dimensional spatial frame. The business person and the laborer are in positions orthogonal to

Image alignment in multimodal metaphor

205

each other within this spatial frame. The business persons position defines a viewpoint from which he sees a growing trend in the economy. The laborers position defines another viewpoint from which he sees a downward trend in the economy. Notice that the business person and the laborer in figure 6 would stand next to each other if they were aligned in a normal, spatial frame. It is against this implicit understanding of the normal alignment that the unconventional spatial frame and the consequent opposing viewpoints are made possible and salient. This design gives a succinct, metaphorical account of the economic situation. The target is the economic trend; it is represented by the arrow and the word economy. The source is the direction that the arrow is supposed to point in. It is represented by the arrowhead and its spatial relations to the positions of the business person and the laborer. Connecting the target to the source yields a metaphor that cannot but be formulated in somewhat laborious terms: THE DIRECTION OF THE ECONOMIC
TREND IS THE DIRECTION OF THE ARROW AS IT IS VIEWED FROM EITHER THE BUSINESS PERSONS POSITION OR THE LABORERS POSITION IN THE UNCONVENTIONAL SPATIAL FRAME. This metaphor expresses a critical stance on

the economy by reminding people of the existent alternative perspectives, and using the unconventional spatial frame, instead of a normal spatial frame, metaphorically suggests that the conventional assumption that a growing economy will eventually benefit all people is not, or no longer, valid. Alignment implies an orderly arrangement of pictorial components, and the examples we have encountered up to this point are effective in exposing how an orderly arrangement of pictorial components may figure in the construction of multimodal metaphors. Nonetheless, a disorderly array of pictorial components may also participate in the construction of multimodal metaphors, especially if such an array is intended for a particular effect against a backdrop of some understood, orderly alignment. Figure 7 is a case in point. In this cartoon, five sheets of paper are arranged in a rather untidy way. Moreover, the four sheets in the background have been damaged some letters were cut out from them. The sheet in the foreground is intact, and the letters cut out from the background sheets have been pasted on it. The scissors, the glue, and the scattered shreds give further evidence of the clipping and pasting. The symbol SeCuRitY thus created on the foreground sheet is a jumble of both lower case and upper case letters, and the letters are not lined up in an orderly way. One can tell from the context that the symbols printed on the background sheets were Liberty, Justice, Equality, and Freedom, but, because of the clipping and pasting, they are now in bad shape. This design suggests the following interrelated interpretations: (a) the

206

Norman Y. Teng

cuttings metaphorically mean that liberty, justice, equality, and freedom have been severely and dangerously compromised; (b) the clipping and pasting metaphorically mean that liberty, justice, equality, and freedom have been curtailed in the service of security; and (c) the jumbled form of SeCuRitY metaphorically mean that security measures have been badly managed. On top of all this, the fact that the five sheets of paper are not well arranged suggests the metaphorical reading that issues concerning liberty, justice, freedom, equality and security have not been handled carefully. It is worth emphasizing that disorder need not be a poorly thought-out design choice. It can be carefully crafted so as to suggest that an action has been performed. The disarray in figure 7 testifies to such a design choice. It is the implied actions, rather than the things portrayed, that figure importantly in this construction of multimodal metaphor.

Figure 7. Homeland Security, by Clay Bennett, The Christian Science Monitor, October 11, 2001, page 8. 2001 The Christian Science Monitor (www.csmonitor.com). All rights reserved. Reprinted with permission.

3. Conclusions and suggestions for further research In this chapter, I have focused on image alignment as a design strategy in the construction of multimodal metaphors. Various uses of this strategy and the design choices that point out the directions the interpretations should take are described. How verbal and other non-pictorial elements in a multimodal metaphor interact with the aligned pictorial components is explained by concrete examples. A few further suggestions that are more amenable to empirical evaluation may be considered. In figure 2, Television and HighDefinition Television are essential to the interpretation of the cartoon. Most

Image alignment in multimodal metaphor

207

people probably will have no clue how to read the garbage images if the verbal representations are removed from the picture. It is also likely that many people will have no idea what figure 3 is intended to mean if endangered species, Southern Africa, Western China, and Middle East are deleted from it. By contrast, if the target audiences are already familiar with the political cartoon genre, terrorism in figure 4 is probably dispensable because of the prototypical depiction of a terrorist. Economy in figure 6 is also likely dispensable because of the typical portrayal of a boss and a laborer. As to the mathematical symbol of infinity in figure 5, if the model tanks were lined up into a shape other than the mathematical symbol for infinity, the metaphorical meaning would be lost. Finally, the lower and upper case letters in figure 7 are definitely verbal elements, but they are also pictorial components by design. In addition to the texts guiding and framing readers interpretation, the garbage in figure 2, the dove-with-olive branch in figure 3, the terrorist in figure 4, the American flag in figure 5, and the boss and the laborer in figures 6 are all pictorial components that carry symbolic or cultural meanings familiar in the Western world. When the target audiences are conversant with the symbolic and cultural meanings, those pictorial components can be good choices for communicative purposes. (For an empirical study of audience responses to political cartoons, see El Refaie, this volume) It is worth noting that the cartoons metaphors are mainly pictorial in their mode of representation, and the verbalizations of the metaphorical meanings can sometimes be a laborious task. Figure 6, for example, is a case in point (see Forceville 2006: 390392 about the implications of this phenomenon for metaphor research). From a design perspective this should not be a surprise, since words and pictures belong to different modes of representation and are suitable for different communicative purposes. The idea of image alignment as a design strategy discussed above offers a perspective from which different modes of representation (here pictures and language) can be deftly combined and coherently understood. It is also worth noting that the patterns of image alignment discussed above should not be taken to be instances governed by the invariance principle, one version of which runs as follows: Metaphorical mappings preserve the cognitive topology (that is, the image-schema structure) of the source domain, in a way consistent with the inherent structure of the target domain (Lakoff 1993: 215; for a slightly different version, see Turner 1991: 172 182; for a critical discussion, see Brugman 1990, Lakoff 1990, Turner 1990; for a discussion of how this principle fares against blending theory, see Turner 1996: 108109; for a discussion of how this principle should be

208

Norman Y. Teng

further revised, or abandoned, see Lakoff and Johnson 2003: 253254). The patterns of image alignment, despite their image-schematic and topological nature, are not something to be preserved or overridden in the metaphorical mappings. Or to put it differently, the idea of target domain overrides is just not appropriate to frame the issues concerning the role of image alignment in the construction of multimodal metaphors. Instead, it is more suitable to consider them as constructional schemas, and the design choices described in the previous section as elaborations of the schemas. (For a recent discussion of constructional schemas in cognitive grammar, see Langacker 2008: 167 174; for a discussion of elaboration, see Langacker 2008: 198205.) This line of thinking gives us a way to apply cognitive grammar and its usagebased approach to design research. (I thank Francisco Ruiz de Mendoza and an anonymous reviewer for prompting this clarification.) It also suggests the hypothesis that primary metaphors may well serve as a common conceptual basis for multimodal constructions. A metaphor is primary in the sense that the association of the target with the source is directly based on an experiential correlation between them (Grady 1997a: 4748; see also Grady 1997b, 1999, 2005, Grady and Johnson 2000, Johnson 2007: 178179, Lakoff and Johnson 1999: 4573). Moreover, the source and target concepts refer to basic dimensions of experience, the shared structure of which coincides largely with parameters relevant to the characterization of basic grammatical categories in cognitive grammar, and the metaphorical mappings appear not to be governed by the invariance principle. (Grady 2005: 16061607). One entry in Gradys (1997a: 281299) list of primary metaphors is SIMILARITY IS ALIGNMENT. The linguistic examples and the experiential motivation for this metaphor are given below: Motivation:
Objects may be oriented in the same way because they serve similar functions, are involved in similar processes or acted on by similar forces. And/or, orientation is a basic parameter for perceptual/cognitive classification.

Examples:
Her new dress is very much in line with those worn by her co-workers. There are stunning parallels between these two novels (Grady 1997a: 283).

Let us suppose that Grady is correct about this metaphor (and I think he is). One may ask what is the relationship between this metaphor and the image alignment discussed in this chapter. My guess is that this metaphor provides

Image alignment in multimodal metaphor

209

the requisite conceptual underpinning for image alignment (I thank Charles Forceville for suggesting this point to me). It seems then that image alignment renders the abstract concept SIMILARITY visible on the basis of the experiential correlation that motivates the primary metaphor SIMILARITY IS ALIGNMENT. In the last resort, it is this primary metaphor that sustains the framework for the design choices described above. One may further hypothesize that each design strategy that enables people to render an abstract concept visible on the basis of the relevant experiential correlation in fact is based on a primary metaphor. More data from a diverse range of designs have to be examined before one can substantiate this conjecture about primary metaphors and design strategies.

Acknowledgments
This study is supported by a grant from Taiwans National Science Council. I have greatly benefited from Charles Forcevilles extensive comments and suggestions on earlier versions of this chapter. I thank Kathleen Ahrens, Sewen Sun, Francisco Yus, Francisco Ruiz de Mendoza, and an anonymous reviewer for their helpful critical remarks, and Judd Kinzely and Jeffrey Cuvilier for their hints on how to improve the writing. I was kindly given an opportunity to present an earlier version of this chapter at the International Symposium on Language, Culture and Cognition, Taiwan. I thank William Croft and I-wen Su for their comments and suggestions on that occasion. Another version was given as a lecture talk at the Institute of European and American Studies, Academia Sinica. I thank LeeJoy Cheng, Wan-Chuan Fang, Der-Chin Horng, Richard Hwang, Chyong-Fang Ko, Te-Hsing Shan and Ruey-Ling Tzeng for their probing questions and suggestions on that occasion.

References
Barthes, Roland 1985 [1964] Rhetoric of the image. [Trans. by Richard Howard] In The Responsibility of Forms: Critical Essays on Music, Art, and Representation, 2140. New York: Hill and Wang. Brugman, Claudia 1990 What is the invariance hypothesis? Cognitive Linguistics 1: 257 266.

210

Norman Y. Teng

Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. 2008 Metaphor in pictures and multimodal representations. In The Cambridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr. (ed.), 462482. Cambridge: Cambridge University Press. Grady, Joseph E. 1997a Foundations of meaning: Primary metaphors and primary scenes. Ph.D. diss., University of California, Berkeley. 1997b THEORIES ARE BUILDINGS revisited. Cognitive Linguistics 8: 267 290. 1999 A typology of motivation for conceptual metaphor: Correlation vs. resemblance. In Metaphor in Cognitive Linguistics, Gerard J. Steen and Raymond W. Gibbs, Jr. (eds.), 79100. Amsterdam/ Philadelphia: Benjamins. 2005 Primary metaphors as inputs to conceptual integration. Journal of Pragmatics 37: 15951614. Grady, Joseph E., and Christopher Johnson 2000 Converging evidence for the notions of subscene and primary scene. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.), 533554. Berlin/New York: Mouton de Gruyter. Johnson, Mark 2007 The Meaning of the Body: Aesthetics of Human Understanding. Chicago: University of Chicago Press. Lakoff, George 1990 The invariance hypothesis: Is abstract reason based on imageschemas? Cognitive Linguistics 1: 3974. 1993 The contemporary theory of metaphor. In Metaphor and Thought 2nd ed., Andrew Ortony (ed.), 202251. Cambridge: Cambridge University Press. Lakoff, George, and Mark Johnson 1980a Metaphors We Live By. Chicago: University of Chicago Press. 1980b Conceptual metaphor in everyday language. Journal of Philosophy 77: 453486. 1999 Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books. 2003 Metaphors We Live By (with a new Afterword). 2nd ed., Chicago: University of Chicago Press.

Image alignment in multimodal metaphor

211

Langacker, Ronald W. 2008 Cognitive Grammar: A Basic Introduction. Oxford: Oxford University Press. Teng, Norman Y. 2006 Metaphor and coupling: An embodied, action-oriented perspective. Metaphor and Symbol 21: 6785. Teng, Norman Y., and Sewen Sun 2002 Grouping, simile, and oxymoron in pictures: A design-based cognitive approach. Metaphor and Symbol 17: 295316. Turner, Mark 1990 Aspects of the invariance hypothesis. Cognitive Linguistics 1: 247 255. 1991 Reading Minds: The Study of English in the Age of Cognitive Science. Princeton: Princeton University Press. 1996 The Literary Mind: The Origins of Thought and Language. Oxford: Oxford University Press.

Chapter 10 Visual metaphoric conceptualization in editorial cartoons Joost Schilperoord and Alfons Maes

Abstract
Typical of metaphors in editorial cartoons is that they not only require somehow the mapping of features from one object or domain to another, as all metaphors do, but their interpretation also includes a critical stance towards a particular socio-political situation, event or person. We will argue that the full interpretation of editorial cartoon metaphors can best be accounted for on the basis of the combination of two cognitive interpretation strategies, i.e., schematic vs. taxonomic reasoning, following the two types of source domains distinguished by Shen (1999). In the chapter, we argue that schema-based reasoning tends to trigger the rich variety of features to be mapped from source to target domain, and that taxonomic reasoning is often the crucial trigger in interpreting the critical stance expressed in editorial cartoons. Keywords: editorial cartoons, visual metaphor, schema theory, critical viewpoint, categorical metaphor interpretation

1. Introduction Metaphors are employed to pursue a range of communicative goals, for example to explain things to people (a heart is like a motor), to persuade people (this car is as fast as a leopard) or to carry away readers (my thoughts are playing in my head like young lions in the evening sun). Since the seminal work of Lakoff and Johnson (1980), metaphors are considered a conceptual rather than a linguistic phenomenon. But, thus far, language has been the privileged input modality when it comes to studying metaphoric conceptualization. Over the last decades, cognitive linguists, cognitive psychologists and philosophers have studied the ways in which different types of

214

Joost Schilperoord and Alfons Maes

metaphor are conventionalized in language, what meaning operations are involved, how new metaphors emerge and how language users process metaphors triggered by linguistic cues (e.g., Bowdle and Gentner 2005; Deutscher 2005; Gentner, Bowdle, Wolff, and Boronat 2001; Gibbs 2005, 2006; Lakoff and Johnson 1980, 1999). Despite many different theoretical positions and claims with respect to metaphor interpretation, there is broad agreement on the basic template of metaphor. It is considered a rhetorical figure in which one particular entity (an object or domain), i.e., the target, is conceptualized in terms of another entity (object or domain), i.e., the source. The processing of a metaphor thus implies the understanding or experiencing of the target in terms of the source. This understanding and experiencing involves a dual process: the two entities have to be identified as source and target respectively, and correspondences between the entities originating from two distinct domains have to be found in order to establish what aspects of the source also apply to the target. The latter process is usually called cross-domain mapping.This process underlies for example Gentners career of metaphor theory (Bowdle and Gentner 2005) as well as Glucksbergs (2001) class inclusion idea. The conceptual nature of metaphor implies that not only language can be used as a stepping stone for metaphoric conceptualization. Pictures, sound and gestures can trigger metaphors as well (e.g., Cienki and Mller 2008, Forceville 2006/this volume, Zbikowski 2002). In recent years, we have witnessed a rapidly growing research interest in visual rhetorical figures, with a special focus on visual metaphors and similes (Forceville 1996, 2002; Kenney and Scott 2003; McCarthy and Mothersbaugh 2002; McQuarrie and Mick 1992, 1996, 1999, 2003a, 2003b; McQuarrie and Phillips 2005, 2008; Mothersbaugh, Huhmann, and Franke 2002; Phillips 2000, 2003; Phillips and McQuarrie 2002, 2004; Scott 1994; Scott and Batra 2003; Teng 2006; Teng and Sun 2002; Tom and Eves 1999; Van Mulken 2003; Van Mulken, Van Enschot-van Dijk, and Hoeken 2005). The research efforts include analyses of the form-meaning relationships involved in visual rhetoric, proposals for a typology of visual rhetoric, studies into readers responses associated with different types of visual rhetoric, and reflections on the medium (in)dependence of visual rhetoric. To our knowledge, the better part of the research into visual metaphors takes commercial advertisements as the main context of study. Obviously, commercial advertisements abound in metaphorical images (Phillips and McQuarrie 2002) and therefore offer a natural habitat and a rich and promising resource of data on visual metaphors, on how they are shaped and combined with verbal rhetoric. But editorial cartoons, too, are a metaphor-

Visual metaphoric conceptualization in editorial cartoons

215

rich communicative area. For example, Wikipedia observes that most [editorial cartoons] use visual metaphors and caricatures to explain complicated political situations, and thus sum up a current event with a humorous picture.1 The political cartoonist for the Daily Telegraph, Nicholas Garland (cited in Plumb 2004: 432) tellingly portrays the force of an editorial cartoon as being derived from the vehicle itself which, besides caricature, requires some or all of a mixture of caricature, metaphor, distortion, surrealism, deliberate misunderstanding and mockery. The communicative functions of editorial cartoons and advertisements differ considerably, which makes it interesting to investigate whether and how these differences are reflected in the metaphoric conceptualizations found. In this chapter, we will discuss a number of building blocks which we consider crucial in constructing a cognitively plausible interpretation of metaphors in editorial cartoons. Section 2 briefly characterizes editorial cartoons. In section 3, the method of selecting and defining metaphors in editorial cartoons will be discussed. Section 4 discusses four cases of editorial cartoons, representing different structural types of visual metaphor. In section 5, we will present two interpretative ingredients which we consider crucial in metaphoric cartoons and which enable us to account not only for their metaphoric conceptualization, but also for their critical stance. On the one hand, metaphoric cartoons tend to evoke scenarios and thus require a schematic interpretation of the source domain, in which clusters of (rather than individual) attributes or relations are mapped from source to target domain. On the other hand, they almost exclusively propagate a critical stance towards a certain topic, which as we will argue requires a categorical interpretation of the source domain. We conclude the chapter with a discussion on the multimodal nature of editorial cartoons.

2. Metaphorical editorial cartoons According to Wikipedia, an editorial cartoon is an illustration or comic strip containing a political or social message, that usually relates to current events or personalities. For our purposes, we would specify this definition in that the social message implies that a certain socio-political event or personality is depicted and commented on. That way, an editorial cartoon can be said to have a descriptive and an evaluative communicative function. Editorial cartoons are widely studied, especially in different areas of sociology and journalism. Most of these studies focus on cartoon selections related to particular political situations (such as presidential elections, world wars,

216

Joost Schilperoord and Alfons Maes

or the cold war, e.g., Becker 1996; Edwards 2001), particular countries (such as the US, Japan or Austria, e.g., El Refaie 2003; Feldman 1995) or particular societal issues (such as the role of women in society, e.g., Gilmartin and Brunn 1998; Morrison 1992), or they study the work of a particular cartoonist (e.g., Plumb 2004). Although the role of metaphor is often acknowledged in these studies, and some of them extensively explain metaphors within the pragmatic (political) context in which they thrive (e.g., Edwards 2001; El Refaie 2003), to our knowledge there has been no attempt to account for the generic structure of metaphoric conceptualization as it takes shape in editorial cartoons. As already indicated, editorial cartoons differ considerably from commercial advertisements. Cartoons aim at affecting states of minds, beliefs, points of view, and perspectives on socio-political affairs, rather than at changing or influencing behavior. In addition, the processing and interpretation of cartoons requires a complex mix of political, cultural, historical, and contextual knowledge. Most importantly, however, they almost invariably express a particular critical, if not radically negative stance towards the topic. Whereas commercial ads intend to evoke positive attitudes and feelings with respect to the target, the reverse often holds for editorial cartoons. One goal of this paper is to demonstrate the necessity of making this point of view explicit in order to properly analyze the way metaphors operate in cartoons (cf. Groarke 2002).

Figure 1. The Mobil oil advertisement (Forceville 1996).

Figures 1 and 2 may serve to illustrate these points. Figure 1 (taken from Forceville 1996) shows a bottle of Mobil motor oil, positioned upside down, with liquid pouring out of it. Together with the header text Intensive Care, the following metaphoric X IS Y scheme can be derived: MOBIL OIL IS INTRAVENOUS DRIP. Since the message is about Mobil motor oil, convention tells

Visual metaphoric conceptualization in editorial cartoons

217

us that this entity should be identified as the target. The ad thus expresses the message that the product is the life-blood of ones car (Forceville 1996: 159). The viewer/reader is invited to map positively oriented elements like professional care and expert treatment from source to target. The context of medical care is attributed to the maintenance of ones car: MOBIL OIL IS TO A CAR WHAT AN INTRAVENOUS DRIP IS TO THE HUMAN BODY.

Figure 2. The Mobil oil cartoon (Bernhardt Willem Holtrop, De Nieuwe Linie, 1977)

Now look at figure 2: an editorial cartoon drawn by the Dutch artist Bernhardt Willem Holtrop (Willem), which was published in the magazine De Nieuwe Linie in 1977. The cartoon shows us Ian Smith, leader of the former Rhodesian apartheid regime (presently Zimbabwe). Smith is portrayed in quite a deplorable physical condition, sitting in a wheelchair and connected to several bottles of intravenous drips. On these bottles the names of various Dutch and multinational companies are shown, amongst others Mobil. We can paraphrase the metaphor in this cartoon as INTERNATIONAL SUPPORT OF SMITH IS INTRAVENOUS DRIP, or, specified to Mobil as one of the supporters: MOBIL IS INTRAVENOUS DRIP, i.e., Mobils support of the Smith-regime is to that regime what an intravenous drip is to the human body. Now suppose a viewer, presented with this image, interprets it by mapping the same elements to the target object as intended by the Mobil oil ad. That is, s/he infers that Mobil oil is like a doctor to the Smith regime, providing professional care and expert treatment. S/he would then clearly misunderstand the essence of this cartoon. The central tenet of the message seems to be that the

218

Joost Schilperoord and Alfons Maes

Rhodesian apartheid regime can only survive by (an overkill of) external treatment, as it is no longer able to stand on its own, and to take care of its existence itself. In other words, whereas the Mobil advertisement in figure 1 aims at cross domain mappings that should lead to a favorable attitude towards Mobil, figure 2 aims at precisely the opposite. In our view, this short analysis suggests a systematic interpretative difference between editorial cartoons and commercial advertisements and brings to light that the same visual metaphor serves widely distinct purposes in these two communicative genres.

3. Sample and selection of editorial cartoons We used two main sources to collect the sample of editorial cartoons that we discuss in this chapter. Most of the cartoons were taken from collections of cartoons published as separate books (containing work) by various famous Dutch cartoon artists, such as Fritz Behrendt, Opland, Willem, Tom Janssen, Jos Collignon and Peter van Straaten. Our decision to focus mainly on Dutch cartoons is practical as it offers the best guarantee that we are able to understand as precisely as possible the public events and persons the cartoons refer to. In addition, we consulted a number of websites with collections of (international) editorial cartoons. In collecting cartoons, we did not systematically control cartoons per artist, time period, publication medium or type of public event satirized, as we are not interested in (representative analyses of) particular periods, artists or event types. Rather, we intend to characterize the dominant generic structure of metaphoric conceptualizations in editorial cartoons and to analyze the way visual metaphors are employed to make a point or express a viewpoint. Therefore, our main criterion in selecting cartoons is that they should contain a visual metaphor, or a combination of visual and verbal metaphor.2 That is, the image is to refer to (at least) two domains X and Y that have a metaphorical relationship (rather than a literal analogy) X IS Y, where one domain (X) serves as target and the other (Y) as vehicle.3 This way, we sampled a corpus of 117 Dutch and 27 international cartoons, large enough to make observations and cautious hypotheses about representative characteristic features and generalizations. Many of the selected cartoons obviously contain more than just a metaphor. For example, many also contain caricatures (Bush-as-elephant, Russia-as-bear) or other rhetorical figures, such as hyperboles. However, our analyses will primarily focus on metaphorical conceptualizations of target domains.

Visual metaphoric conceptualization in editorial cartoons

219

It is obvious that the interpretation of cartoon metaphors is to a certain extent driven by genre-related, pragmatic considerations. For example, viewers will more or less expect cartoons to criticize or ridicule situations or persons, and they will therefore most probably arrive at an interpretation of a cartoon to that effect. Also, the interpretation of cartoons crucially depends on the viewers ability to recognize persons, objects, and situations and on their knowledge of the relevant facts, such as news events, historical facts, cultural habits, and the like. This knowledge is indispensable for a full interpretation of the intended meaning associated with metaphors in cartoons. Hence, in analyzing cartoons, we take the position of a fully informed viewer, that is, a viewer who not only knows the conventions of the genre, but also recognizes the relevant persons, objects and situations and has sufficient factual knowledge to understand the gist of the cartoon. This being said, in analyzing and discussing the cartoon metaphors, we take a cognitive rather than a pragmatic point of view. We intend to account for the metaphoric conceptualization and critical stance of editorial cartoons in terms of what is actually shown in the image, rather than in terms of the cultural and general discourse context. This means that we focus on the information contained in cartoons that viewers are to process, rather than on the prior, contextual/pragmatic knowledge they bring to bear on interpreting the cartoon. Clearly, such an approach has its flaws. For example, as El Refaie (this volume) demonstrates, contextual/pragmatic knowledge varies from viewer to viewer, and this might cause interesting and complex interpretation problems problems that are fully worth to be studied in their own right. Nevertheless, we will ignore such variation, and concentrate on unraveling the source-to-target mapping mechanism, and the accomplishment of the evaluative position, thereby assuming (rather than assessing) all available and relevant background knowledge. That way, we hope to detect generic meaning ingredients and interpretation mechanisms related to metaphoric conceptualization in editorial cartoons.

4. Analyzing the cartoons: Four case studies For each cartoon, we carried out the following analytical steps. We defined the conceptual content of a cartoon by paraphrasing the metaphor using the X IS Y template, thus determining target and source domain and the aspects to be mapped.

220

Joost Schilperoord and Alfons Maes

We determined the way in which the two domains are realized. Following Forceville (1996), Philips and McQuarrie (2004) and Teng and Sun (2002), we tried to distinguish cases in which only one domain is expressed visually (so called replacements), cases in which the two domains are expressed separately (i.e., juxtapositions), and cases in which the two domains are visually integrated (i.e., fusions). Following the practice of Groarke (2002) in analyzing the argumentative structure of editorial cartoons, we reconstructed as objectively as possible the point of view expressed. In this section we will illustrate these analytical steps by discussing four cartoons. We will show that the three structural classes are relevant in detecting relations and attributes to be mapped and argue that cartoons need an additional interpretation step in which the critical point of view is extracted.

Figure 3. FOOTBALL IS June 2006).

SUN

(BAS, Tachydromos, www.cartoonweb.com, retrieved

Figure 3 shows us a solar system of two celestial bodies. On the smaller object latitude and longitude lines are projected, a conventional way of visualizing the earth, which suggests that we are to interpret the large object as the sun. This qualifies the cartoon as an example of replacement in that the sun is replaced by a football. Hence, the metaphorical objects involved here can be paraphrased as FOOTBALL and SUN. The cartoon dates back to 2002, the year in which the world champion soccer tournament took place in Japan and South Korea, so its topic is football, or the tournament. Hence, the object FOOTBALL is to be labeled the target object, and SUN the source, and the metaphorical structure can be rephrased as FOOTBALL IS SUN. What comment does this analogy express? The sun very often appears as source object in commercial ads, for example in ads for fruit juices, where it

Visual metaphoric conceptualization in editorial cartoons

221

attributes positively oriented notions like health, giving energy, and providing light and warmth to the product. Evoking such positive attitudes is not what this cartoon intends to do. Rather, the cartoon ironically comments on the worldwide impact of the championship, suggesting that it is the most important thing in the world. Two pictorial details are important in this interpretation. First, the complete replacement of the sun by a football may make the impact of the latter object stronger than a fusion or juxtaposition of these two domains would have done. If the objects were juxtaposed or fused, the viewer might infer that at least the sun would still be there, so to speak, whereas now it has completely disappeared from the scene. Second, the proportions of a solar system with a tiny earth and a large football moreover hint at the ironical stance towards the topic, expressing the artists opinion that, to many people, football has become as important as the sun is to life on earth.

Figure 4. TSUNAMI IS ary 2007)

SHARK

(Tom Janssen, www.tomjanssen.net, retrieved Janu-

Figure 4 is a case of fusion: two objects are fused into one hybrid visual image which expresses the metaphor TSUNAMI IS SHARK. The cartoon appeared right after the news of the disastrous tsunami in Indonesia in December 2004. Although natural events are hardly suitable to form an opinion about, the metaphor expresses the horror of the event by fusing the tidal wave and shark teeth. The cartoon thus exploits the source entity (SHARK) as a prototypical member of the set of horrible things. As we will discuss further in section 5.2, this requires a categorical interpretation: by using sharks as a source, the cartoon expresses the message that from now on we should also include tsunamis as members of the set of horrible things. Figure 5 shows a case of juxtaposition. We see a dog and a television both tied to a tree, with the accompanying text (to be translated as And now

222

Joost Schilperoord and Alfons Maes

we repeat last years summer broadcasting programs). Note that the two objects DOG and TV-SET are placed next to each other, and this way of grouping them invites readers to solve the cognitive dissonance caused by the apparent differences between the objects and to look for relevant connections between the objects (cf. Teng and Sun 2002). Distilling the critical comment in this cartoon requires some contextual knowledge about the two objects. First, summer broadcasting in the Netherlands (and elsewhere) is often criticized for mainly repeating old programs. Second, many people have the deplorable habit of going on holiday unwilling to take proper care of their pets. Pets are often left on their own in just the way this cartoon depicts. This knowledge enables us to identify the metaphorical objects as SUMMER TV PROGRAMMING and LEFT-BEHIND PET, and to determine the crucial element which connects the two objects: they represent comparable examples of contemptuous human behavior. Although the phenomena are equally deplorable, only the image of the dog refers literary to a real-world state of affairs. We may conclude that this should be the source object, and that the metaphorical structure can be rephrased as SUMMER TV PROGRAMMING IS (LIKE) LEFT BEHIND PET. By metaphorically grouping these objects, the artist thus expresses the idea that Dutch broadcast organizations who are responsible for summer TV programming relate to their audience the way dog owners who leave behind their dogs relate to their pets.

Figure 5. TELEVISION IS (LIKE) January 2007)

DOG

(Tom Janssen, www.tomjanssen.net, retrieved

As a final example, we consider a cartoon (published in the 1960s) in which there is more than one metaphor present within the same image (see figure 6). This cartoon refers to the situation in a class room where an angry teacher punishes a bad school boy by putting him in the corner. The name Cuba projected on the back of the boy allows us to identify the first meta-

Visual metaphoric conceptualization in editorial cartoons

223

phor as CUBA IS LITTLE BAD BOY, whereas the stars on the sleeve of the pointing figure tell us the second metaphor is AMERICA IS ANGRY TEACHER. What is special about this cartoon is that the two metaphors together trigger a scenario or scheme, as is in fact the case with a large number, if not all of the cartoons in our corpus (see section 5). In this case the cartoon not only maps attributes of bad boys and angry teachers onto the target domains Cuba and America respectively, but it also maps certain typical roles. As such the cartoon expresses the idea that Cuba relates to America like a little bad boy relates to his angry teacher.

Figure 6. CUBA IS LITTLE BAD BOY, AMERICA IS ANGRY TEACHER (www.politiekespotprenten.nl, retrieved March 2003).

It is not easy to distill the critical stance of the cartoon, as it only seems to tell the reader: This is what the political situation of Cuba and the US is like. So the image merely seems to signify some complex political situation to the reader, rather than to comment on it. However, note that the metaphorical scenario evokes the inherently unequal teacher-pupil relationship, emphasized by the visual perspective taken in the cartoon. This may hint at its critical stance, i.e., the US acting wrongly because it treats Cuba like a naughty boy. In international affairs there is no actor who is inherently superior to the other. All nations are equal, so if one nation (the US) considers itself as superior with respect to another nation (Cuba), then that first nation is wrong. Like the cartoon in figure 3, it is the very choice of the metaphoric scenario, together with pictorial details, that accounts for the cartoons critical stance. To conclude this section, the four cases show that the domains involved in visual metaphors can be realized or represented differently: by the replacement of one visual domain by the other or by a combination of two

224

Joost Schilperoord and Alfons Maes

visual domains, integrated (fused), or juxtaposed. Apart from that, we suggested two characteristics which indicate how in a cartoon a visual metaphor is involved in visual argument, both in signifying a certain state of affairs and in commenting on it. Pictorial details in cartoons may trigger schema or scenario domains involving prototypical relations between objects (the little earth and the large football; the pupil and the teacher), and they may trigger category domains including prototypical members (i.e., sharks and left pets). The next section discusses these characteristics in more detail.

5. Two basic interpretation ingredients 5.1 Signifying states of affairs: schematic source domain interpretation Especially the Cuba-US cartoon suggests a pervasive characteristic of editorial cartoon metaphors, i.e., their scenario character. The majority of cartoons in our corpus introduce a familiar scenario as a basis of the visual metaphor. A typical example is given in figure 7.

Figure 7. Hospital scenario metaphor (Fritz Behrendt, De Telegraaf, 1980; www. socsci .kun.nl/ped/owk/activeworlds/prenten/, retrieved January 2007).

This cartoon was published in 1980, during the heyday of the former Polish free labor union Solidarnosc (Solidarity), the first free labor union under a communist regime. It depicts union leader and Solidarnosc founder Lech Walesa, the former Polish leader Wojciech Jaruzelski, and Karl Marx, who gather around the sickbed of a patient, the personification of Poland. Walesa is holding a syringe and Marx a copy of his book Das Kapital. The cartoon triggers an entire conceptual domain that we may call a hospital scenario.

Visual metaphoric conceptualization in editorial cartoons

225

The scenario contains several elements: persons (staff, patients), roles (doctor, patient), relations (doctors and medical instruments) and objects (instruments, a drip), and attributes (being sick). Hence, with respect to the topic of this cartoon the scenario serves as a supplier of various metaphorical relations:
WALESA IS DOCTOR JARUZELSKI IS DOCTOR MARX IS DOCTOR POLAND IS PATIENT DAS KAPITAL IS MEDICINE SOLIDARNOSC IS MEDICINE POLITICAL DOMAIN/PUBLIC AREA IS HOSPITAL ROOM

Not only do scenario elements serve as source objects, but the scenario also imposes an additional conceptual structure onto the image by virtue of the conceptual relations that hold between the scenario elements. According to Lakoff and Johnson (1980), scenario metaphors can be seen as Idealized Conceptual Models, i.e., cognitive networks with causal, temporal and other sorts of relationships between persons, roles, locations and attributes which all are more or less fixed, and conventionally known by all members of the cultural community in which they appear. A more fine-grained account of this type of metaphor can be found in Shens (1999) notion of schematic source domain. With this notion, Shen provides an explanation why, in processing metaphorical messages, certain features are mapped from source to targets in preference to others. For instance, as also noted by Gentner et al. (2001), in comprehending a metaphor like plant stems are drinking straws, people tend not to map properties like tubular or hollow from source to target. Rather, their understanding of the metaphor is likely to involve the notion of a drinking device, and the relation between the person/object that drinks and the liquid being drunk. Shen accounts for this preference by proposing that the source domain of the drinking straw evokes a schema that not only contains the object itself, but also entities such as the person using the straw, the liquid being drunk, the receptacle, and more in general the affordances, experiences, and functions associated with these objects grouped together in this particular constellation. Besides sources-as-schemas, he also distinguishes sources-as-taxonomic categories. We return to this type of source domains in the next section where we attempt to account for the critical stance of editorial cartoons.

226

Joost Schilperoord and Alfons Maes

A schematic source domain thus represents a cognitive schema, i.e., a knowledge structure consisting of a set of entities which are connected to each other via relations of contiguity, such as thematic, causal, spatial and temporal relations (Shen 1999: 1633). In interpreting metaphors on the basis of this schema paradigm, the mapping of relations is preferred over the mapping of isolated attributes, and higher order relation mappings are preferred over lower order relation mappings, an idea which is consistent with Gentners structure-mapping theory (Gentner et al. 2001).4 The hospital or medical scenario-as-metaphor is a recurrent one. In our sample, eight cartoons exploit this scenario (see also figure 2). Equally frequent is the cooking-scenario, an example of which is given in figure 8.

Figure 8. Cooking scenario metaphor (Tom Janssen, Trouw, April 2004)

The cartoon was published in April 2004. At that time, the former Dutch minister of administrative innovation Thom de Graaff was deeply involved in defending and implementing his proposal for a new system for electing mayors and Dutch congress representatives. The cartoon shows us De Graaff, who is busy cooking the new election system (see the Dutch label nieuw kiesstelsel on the kitchen pot). Promoting the reorganization of the Dutch election system was the main reason for founding De Graaffs party D66 in 1966. As in the medical scenario, there are different roles here that may serve as individual metaphors:
MINISTER IS COOK POLICY IS DISH MEASURE IS INGREDIENT MEASURE IS COOKING EQUIPMENT

Visual metaphoric conceptualization in editorial cartoons

227

Once the schema is triggered, it is not difficult to infer other metaphors such as PEOPLE ARE CONSUMERS OF DISH. Other frequently encountered metaphorical scenarios in our corpus are marriage, funerals, and (boxing) games. The marriage scenario is often employed to portray coalitions or political treaties. The sports game scenario is well suited to metaphorically depict political opponents or election campaigns. An example is figure 9, published during the 2004 US presidential election campaign, depicting George Walker Bush and John Kerry as boxers in a boxing ring.

Figure 9. Boxing game scenario metaphor (KAL, The Economist, www.cartoon web.com, retrieved, June 2006)

Finally, there are some prototypical source objects (not scenarios) that we observed on several occasions, especially ships and animals. Ships are often used as source domains for nations, states or political parties. In one case a ship (quite a marked one, in fact) is even used for a person, i.e., the Titanic as a source domain for former US president Bill Clinton. Laws, treaties or other products of politicians are often portrayed as animals. The use of animals in cartoons justifies a study in its own right, as they are used metaphorically to evoke a wide range of semantic categories (see also figure 4). For example, we found two Dutch cartoons, dating from March 2005, both commenting on the special anti-terrorist act of the Dutch minister of Justice by portraying the act and terrorism as two equally big, blood-thirsty dogs, thus expressing the opinion that in this case the cure may be worse than the illness. Another animal that is frequently used as metaphorical source do-

228

Joost Schilperoord and Alfons Maes

main is the snail, often employed to express the opinion that some measure or action has been undertaken too late, or comes too slowly. 5.2 The critical viewpoint: categorical source domain interpretation As we already noted, editorial cartoons differ from commercial advertisements in that they often express some critical point of view. A full account of visual metaphor in cartoons requires this point of view to be made explicit. If we look again at figure 8, it is striking that De Graaffs kitchen is a mess and that black smoke evaporates from his kitchen pot. These pictorial aspects suggest that there is something wrong with both the dish and the way he is preparing it. For example, De Graaff is portrayed as a child, which suggests that he is not able to cope with his job. Such details allow us to conclude that the cartoon expresses a critical standpoint towards both the minister and his policy. But on what grounds precisely do we conclude this? In this section, we discuss an interpretation mechanism, which, we believe, enables us to explain the critical stance of editorial cartoons. It is based on the second type of source domain proposed by Shen (1999), i.e., the domain as category, which is reminiscent of Glucksbergs (2001) class inclusion idea. According to Shen, sometimes source domains require a taxonomic or categorical interpretation, which is based on a set-member relationship, rather than a part-whole (i.e., schematic) relation. The metaphorical expression Bush is a hawk, for example, is based on the taxonomic interpretation of a hawk as a particular type of bird. That is, the interpretation is based on how hawks relate to the set of birds, rather than how they relate to other objects or events in a scenario. Animals often evoke this type of categorical interpretation in editorial cartoons. The categorical interpretation is guided by the so-called diagnosticity principle, stating that the mapping of properties with high diagnostic value is preferred over the mapping of properties with low diagnostic value, an idea which is congruent with Ortonys salience imbalance theory of metaphor understanding (Ortony 1979). Applied to the Bush is a hawk metaphor, the categorical attributes of a hawk with a high diagnostic value in the given context are its ferocity and invulnerability. In the remainder of this section, we will reanalyze some of the cartoons discussed above, in an attempt to substantiate the plausibility of the relationship between categorical reasoning and the critical stance in cartoons. Consider again the difference between the Mobil metaphors in figures 1 and 2. The tenet of the Mobil ad in figure 1 seems to rely primarily on the underly-

Visual metaphoric conceptualization in editorial cartoons

229

ing medical scheme: by using Mobil oil a car owner takes care of his car like a doctor takes care of his patient. The medical schema evokes entities like doctor, medicine and patient, and the interpretation of the metaphor is based on the standard relations of contiguity between these entities in the medical scheme: actor, patient, instrument, and so on. Obviously, a drip may also be seen as member of the category of things used to treat sick patients, and even as a member with a high diagnostic value, but this adds little to what the viewer already knows, namely that he is to form positive attitudes towards the advertised product. Now consider figure 2. This cartoon triggers exactly the same medical schema. But in order for the viewer to distill the critical tenet of the cartoon, additional interpretation is required, otherwise, s/he would simply feel invited to understand the international support of the Smith regime in terms of medical care. In our view, this additional interpretation is critically dependent on categorical reasoning about the medical source domain. In particular, the medical care domain has to be viewed as a member of the set of external forces or assistance from outside. With respect to the target domain (political leaders or regimes), this categorical interpretation has a rich diagnostic potential. Political leaders or statesmen are good to the extent that they are supported by internal forces and assistance from inside, for example their own governments and people. Conversely, political leaders who can only stay in power due to external support are considered bad. Hence, the diagnostic potential of the categorical interpretation of medical care as a member of the set of external assistance offers the crucial trigger for the critical point the cartoon attempts to make (i.e., international support for the Smith regime is bad). Note that pictorial details are in line with this interpretation (e.g., the facial expression and posture of Smith, the relative large size of the drips, and the visual perspective of the dangling wheelchair). A similar categorical interpretation can explain the critical position expressed in the Poland cartoon (figure 7). A strictly schematic interpretation (as provided in section 5.1) actually does little more than signifying the state of affairs in Poland in 1981: Walesa, Jaruzelski, and Marx relate to Poland like a doctor relates to a patient. The critical tenet of the cartoon, however, rests on a multiple categorical interpretation of the source domain element doctor. Two opposite diagnostic properties of the class of doctors (or the superset of caregivers) are crucial. A doctor can either be a member of the set of caregivers who help and cure other people or he can be a member of the set of caregivers who mistreat patients and destroy their lives. So who belongs to which category in this cartoon? Our knowledge of the political circumstances in Poland by the time, and the political points of view of the

230

Joost Schilperoord and Alfons Maes

cartoonist will lead us to infer that Walesa is the good doctor. But this same conclusion is also warranted by pictorial elements, especially by the different facial expressions of the three doctors. So, the intended critical interpretation, we believe, is carried by the diagnostic potential of the categorical interpretation of the source domain element doctor. Furthermore, metaphorical entailment allows us to make the following inference: Because good doctors have good medicines, and bad doctors have bad medicines, Solidarnosc is a good medicine, and Das Kapital a bad one. Similar observations can be made with regard to the cooking source domain. The cartoon in figure 8 rests on a cooking schema to the extent that it clarifies how this minister relates to his policy and the people affected by it. However, only a categorical interpretation evokes the diagnostic property which reveals the critical stance of the cartoon: the cook is to be seen as a member of what can be termed the set of bad cooks, i.e., cooks who are not up to the task of organizing their work in the kitchen and preparing a proper meal. This categorical interpretation is supported by a number of visual details, such as the messy kitchen, or the black smoke, and reveals the relevant properties to be mapped to the target domain. In sum, it is evident that factual pragmatic knowledge is an essential condition to come up with these fine grained interpretations of cartoons. But the point that we want to pursue is that a generic cognitive mechanism (i.e., setmember classification) is at work which accounts for the evaluative aspect of visual metaphor in cartoons as it is triggered by pictorial details. We are only beginning to understand the kind of pictorial devices that can be interpreted in terms of point of view. Bodily expressions and the scaling (i.e., the relative size) of objects are good candidates. For example, the bad cook idea in figure 8 seems to rely on the relative size of the cook and his cooking pot, in that the small cook cannot be supposed to master his much too large cooking pot. Figure 9 is another example. At first sight, the cartoon just seems to portray the presidential election as a boxing game with Bush and Kerry as boxers in the ring, and thus to require only a schematic interpretation. But a closer look reveals a bigger glove on Kerrys right hand with the word Iraq projected on it. This enlarged glove suggests a categorical interpretation, i.e., that it is a member of the set of decisive weapons, which results in the interpretation that Kerry has the better weapon; he can attack Bush with the war in Iraq and therefore he is likely to win the elections (or at least, he himself believes this to be so). So, again, the gist of the message, and the critical position of the author is accounted for by a categorical aspect of the source domain, highlighted by pictorial details.

Visual metaphoric conceptualization in editorial cartoons

231

The recognition of this set-member mechanism does not necessarily mean that the critical stance in any cartoon is based on this same mechanism. Many more cartoons have to be analyzed to justify this conclusion. But the analysis shows that categorical reasoning can be used successfully as a heuristic in finding the critical stance in editorial cartoons. If indeed categorical interpretations are responsible for the (often negative) critical stance of editorial cartoons, they may be rare in commercial ads in which such a critical position is absent. Again, an empirical account of this hypothesis requires a large scale study of metaphors in commercial ads. For the time being, we contend that categorical source domain interpretations are relatively rare in commercial ads, but certainly not absent. As we discuss in more detail in Maes and Schilperoord (2008), there seem to be at least two types of commercial ads for which a categorical source domain interpretation is crucial, but not related to a critical stance. The first type is represented by ads in which one attribute of a product is highlighted instead of a network of relations and attributes within a schema. Take for example an ad for a four-wheel drive vehicle representing the car together with some hippos in the water, thus evoking the idea of imperturbable robustness. In cases like these, the artful deviation of the image is based on the activation of an ad hoc class, in this case the class of robust things, with a hippo as a prototypical member. Another type of advertisements that seem to require a categorical interpretation are the so-called real thing ads. Commercial ads for products like orange or tomato juice, mineral water or canned fish often express the message that the products are as fresh and tasty as the real thing. Such ads attempt to counter the commonly held opinion by consumers that such products are just a surrogate. In our view, the real thing idea is based on a categorical source domain interpretation. The viewer is stimulated to believe that the juice, salmon, or mineral water is a member of the set prototyped by the real thing. In sum, the notion of categorical source domains seems to offer a suitable interpretation mechanism for explaining the critical stance expressed in editorial cartoon metaphors. The mechanism is assumed to come on top of a basic schematic source domain interpretation, as is illustrated in the analyses above. In a particular schema, specific roles trigger the construction of ad hoc categories enabling viewers to find the diagnostic property which is responsible for the critical stance of the cartoon. The interpretation mechanism is not exclusive for editorial cartoons and can be used to trigger negative as well as positive diagnostic properties. But in interpreting editorial cartoons,

232

Joost Schilperoord and Alfons Maes

it most probably is the dominant mechanism accounting for the critical evaluative stance.

6. Conclusion In this chapter we have argued that most schematic source domains of metaphors in editorial cartoons require an additional categorical source domain interpretation, which results in the detection of the critical stance which is typical of editorial cartoons as a genre. The ultimate message a cartoon communicates is criticism, or a particular political stance towards a certain public topic, and this is, we believe, best accounted for by the categorical aspect of the source domain 6.1 The persuasive force of message form In its essence, metaphor is argumentative, so if we are to properly characterize the role metaphor plays in editorial cartoons, the notion of persuasion should be focused on. Socio-psychological theories of persuasion mostly stress the importance of exchanging information in persuasion processes. Fishbein and Ajzens (1975) famous model of persuasion is essentially an information processing model of influencing attitudes. In their view, attitudes have an informational base. However, from a rhetorical point of view, persuasion is first and foremost the process of directly trying to regulate the behaviors or points of view of an audience. This raises the question of the role metaphor plays in persuasion. Metaphor has long been considered an ornamental addition to the thing that really matters in communication: i.e., its content. Ever since Lakoff and Johnson (1980), we know that metaphor is the content of the message and this is, we believe, rhetoric in its most essential appearance. Applied to our rhetorical case here, this means that especially the choice of the source domain is responsible for the type of conclusion we are supposed to draw on the basis of the image. One famous Dutch cartoon (one we already alluded to in section 5) comments on the measures that the then Dutch minister of Justice, Donner, took upon him to combat terrorism in the Netherlands. The cartoon depicts the minister as a small figure, standing between two large, black, blood-thirsty dogs, one that has the word terrorism on its back and the other the word anti-terrorism. Because the opposite forces in the target domain are linked to the same source domain (of blood-thirsty, dangerous

Visual metaphoric conceptualization in editorial cartoons

233

dogs), the cartoon expresses the message that in this case the cure is at least as harmful as the pain. In addition, by depicting the owner of the antiterrorism dog as much smaller than his dog, Donner is presented as someone who is not capable of mastering his own dog (let alone the other one). Certainly, there is a message involved in this cartoon (with his new antiterrorism policy, the minister is playing with fire), but much more important here is the persuasive impact of the message form (sameness, big versus small) and the conclusions we have to draw from it. These are determined by the rhetorical choices, especially the strategic choice of the source domain, and the visual details providing us with the conceptualization of the problem and the critical position towards it. That way, we think that the interpretation of metaphors in editorial cartoons nicely illustrates the pervasive force of message form in establishing meaning. 6.2 The multimodal nature of cartoon metaphors In the preceding, we did not explicitly address the issue whether metaphors in editorial cartoons are a multimodal phenomenon. El Refaie (this volume) does address this issue. She contends that in the cartoons she has analyzed, the viewer should probably succeed in identifying the target objects from the caricatured features of the politicians appearing in her cartoons. Moreover, viewers will also draw on broader contextual information in doing so. In both cartoons, she states, the verbal messages also play a role in pointing the viewer towards the precise targets. She has in mind the verbal signs that often appear on persons or objects depicted in a cartoon, as the names of the multinationals in figure 2, the surname Thom and the noun new voting system in figure 8, etc. So, if indeed these verbal signs point the reader towards the target entities, this would suggest the following metaphors: MULTINATIONALS ARE DRIPS, THOM DE GRAAF IS CHILD, NEW VOTING SYSTEM IS COOKING POT. To address the question whether these metaphorical relations are real multimodal metaphors, we should first ask ourselves what multimodal metaphors are. Forceville (2006) states that multimodal metaphors are metaphors whose target and source are each represented exclusively or predominantly in different modes. His example is the cueing of the metaphor CAT IS ELEPHANT in an animation film. The designer could do this by depicting a cat and have it make a trumpeting sound. In this case, the target would be triggered visually, and the source by means of sound. And because vision and

234

Joost Schilperoord and Alfons Maes

sound are different modes, the resulting metaphor would be truly multimodal. If we apply this view to the cartoons in this chapter one may simply conclude that the above-mentioned cartoons with verbal cues are multimodal metaphors, because visual and verbal codes are different representational modes. But this is probably too simple. Look at the little cook in figure 8. If the viewer misses the necessary factual knowledge and does not know who secretary De Graaff is, the surname label (Thom) on the trousers will not be helpful, and will not make his metaphoric conceptualization multimodal. So, we should read and apply Forcevilles characterization in such a manner that metaphors are multimodal only if the identification of a metaphorical term really depends on the presence of that label. This is exactly the case in the CAT IS ELEPHANT example. The removal of one of the modes (either the image or the sound) would simply cause the metaphor to evaporate. All that would remain is a picture of a cat or the sound of an elephant. If, on the other hand, the proper name Thom is removed from the trousers of the cooking child, this would not destroy the metaphor DE GRAAFF IS CHILD. So, it would seem that the visual metaphors with proper noun label are not automatically multimodal. All the proper name does, or is intended to do, is to provide the viewer with an additional aid to make sure that the person is recognized. Not all verbal labels have the same function. Take the MEASURE/POLICY IS COOKING POT metaphor in figure 8. At first sight, this metaphor appears to be truly multimodal (albeit not a very exciting one), as there seems to be only one way of identifying the source (the visual sign) and the target (the verbal label sign on the cooking pot). Nonetheless, we believe there are at least two reasons to doubt this characterization. The first has to do with the metaphorical scenario itself. As soon as the viewer identifies the image as metaphorical (an aspect of interpretation that is evidently triggered by pragmatic considerations and knowledge of the genre) s/he would have little trouble identifying the cooking pot as some policy, e.g., on the basis of a straightforward entailment: if this secretary is portrayed as a cook, then the cooking pot and the stuff he is preparing must be his policy. The exact identification will, of course, depend on various external sources of knowledge, such as the time of publication and broader contextual information, for example the context of the ongoing public discussion in news media about the new voting system. This pragmatic knowledge should direct the viewer at the precise target (the new voting system), but not at the type of target (policy making), since the latter is to be figured out on the basis of the scenario evoked. And since that immediate context is evoked

Visual metaphoric conceptualization in editorial cartoons

235

visually as well, the metaphor NEW VOTING SYSTEM IS COOKING POT is at least not entirely multimodal. Our second reason for doubting the multimodal nature of this cartoon has to do with the possibility that the cooking scenario is actually a broad visual manifestation of one basic conceptual metaphor in the sense of Lakoff: IDEAS ARE FOOD, which instantiates the more general conceptual metaphor IDEAS ARE PHYSICAL OBJECTS. Verbal expressions that relate ideas to food are omnipresent and well documented (see e.g., Gibbs 2005), which suggests that this type of metaphoric conceptualization is cognitively anchored. We are only beginning to ask questions regarding the way basic and complex visual metaphors are anchored cognitively, but it does not seem too farfetched to hypothesize that making food (cooking), so often employed in cartoons, is particularly well suited to metaphorically conceptualize abstract concepts such as political coalitions and policy making. If so, it may well be that the verbal label in figure 8 is merely a secondary supporting feature designating the exact type of policy hinted at, rather than the verbal part of the multimodal metaphor. To conclude, we are convinced that there is such a thing as multimodal metaphor, but its precise delineation awaits further (preferably empirical) research. At the very least, it seems too hasty to conclude that since metaphors in editorial cartoons use both visual and verbal signs, they are multimodal.

Notes
1. 2. 3. 4. See en.wikipedia.org/wiki/Editorial_cartoons, consulted 2007-02-19. The difference between these two types is not unproblematic, as we suggest in the concluding section. As we will discuss in the conclusion, not all of these metaphoric cartoons fit in with Forcevilles (2006) definition of multimodal metaphor. In the analyses to follow, we adopt the theoretical framework proposed by Shen (1999) where two kinds of source domains are distinguished (see next sections). This does not mean that Shens framework is the only one suitable for the analysis of metaphorical conceptualizations in editorial cartoons. One obvious alternative is Blending Theory (e.g., Coulson and Oakley 2001; Fauconnier and Turner 2002). Many cartoons can in fact be considered a conceptual blend of (at least) two so-called input spaces, representing disparate conceptual domains. For example, the Cuba cartoon in figure 6 blends the conceptual domains of the schools (teachers, pupils) and that of international political relationships into one integrated conceptual domain. Although Blending theory is not a theory of metaphor proper, but a general theory of

236

Joost Schilperoord and Alfons Maes cognitive conceptualization, it has been shown to be fruitfully applicable to (visual) metaphors as well (Coulson in press). The blended conceptual domain consists of structural analogies between the two input spaces to the effect of establishing partial correspondences between distinct conceptual domains (such correspondences are modelled to constitute a so called generic space). The crucial structural analogy between the input spaces in the Cuba cartoon pertains to the relation between pupils and teachers in the school input space, and the Cuba and US relation in the international politics input space. The same kind of conceptual integration applies to the other cartoons as well. The football cartoon, for example, blends the conceptual domain of celestial systems and of international football tournaments, while the cartoon in figure 7 can be seen as a blend of the conceptual domains of hospitals and politics. Our reason for nonetheless adopting Shens framework is strictly practical, as it emphasizes the role of conceptual/cognitive schemata in the process of interpreting (visual) metaphors. As we hope to make clear, his idea of source domains as cognitive schemas may be very useful in order to account for the way metaphor is involved in depicting certain political and sociocultural events and persons, while his concept of categorical source domains allows us to account for the way the artist evaluates such events or persons. However, in most cases it is not difficult to envisage an analysis grounded in Blending Theory, because both frameworks strongly employ Gentners systematicity principle (see Gentner et al 2001), e.g., the fact that in order to establish correspondences between disparate conceptual domains, people will primarily look for structural analogies between the various elements and entities within the two domains.

References
Becker, Jonathan A. 1996 A disappearing enemy: The image of the United States in Soviet political cartoons. Journalism and Mass Communication Quarterly 73: 609619. Bowdle, Brian F., and Dedre Gentner 2005 The career of metaphor. Psychological Review 112: 193215. Cienki, Alan, and Cornelia Mller (eds.) 2008 Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins. Coulson, Seana in press Whats so funny: Conceptual blending in humorous examples. In The poetics of cognition: Studies of Cognitive Linguistics and the Verbal Arts, Vimala Herman, (ed.). Cambridge: Cambridge University Press.

Visual metaphoric conceptualization in editorial cartoons

237

Coulson, Seana, and Todd Oakley 2001 Blending basics. Cognitive Linguistics 11 (34): 175196. Deutscher, Guy 2005 The Unfolding of Language: An Evolutionary Tour of Mankinds Greatest Invention. New York: Metropolitan Books. Edwards, Janis L. 2001 Running in the shadows in campaign 2000: Candidate metaphors in editorial cartoons. American Behavioral Scientist 44 (12): 2140 2151. El Refaie, Elisabeth 2003 Understanding visual metaphor: The example of newspaper cartoons. Visual Communication 2 (1): 7595. this vol. Metaphor in political cartoons: Exploring audience responses. Fauconnier, Gilles, and Mark Turner 2002 The Way We Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Feldman, Ofer 1995 Political reality and editorial cartoons in Japan: How the national dailies illustrate the Japanese prime minister. Journalism and Mass Communication Quarterly 72: 571580. Fishbein, Martin, and Icek Ajzen 1975 Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Reading, MA/London: Addison-Wesley. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame work: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379 402. Berlin/New York: Mouton de Gruyter. Gentner, Dedre, Brian F. Bowdle, Philip Wolff, and Consuela Boronat 2001 Metaphor is like analogy. In The Analogical Mind: Perspectives from Cognitive Science, Dedre Gentner, Keith J. Holyoak, and Boicho N. Kokinov (eds.), 199253. Cambridge, MA: MIT Press. Gibbs, Raymond W., Jr. 2005 Embodiment in metaphorical imagination. In Grounding Cognition. The Role of Perception and Action in Memory, Language and Thinking, Rolf A. Zwaan and Diane Pecher (eds.), 6592. Cambridge: Cambridge University Press. 2006 Metaphor interpretation as embodied simulation. Mind and Language 21: 434458.

238

Joost Schilperoord and Alfons Maes

Gilmartin, Patricia, and Stanley D. Brunn 1998 The representation of women in political cartoons of the 1995 World Conference on Women. Womens Studies International Forum 21: 535550. Glucksberg, Sam 2001 Understanding Figurative Language: From Metaphors to Idioms. Oxford: Oxford University Press. Groarke, Leo 2002 Toward a pragma-dialectics of visual argument. In Advances in Pragma-Dialectics, Frans van Eemeren (ed.), 137151. Amsterdam: Sic Sat. Kenney, Keith, and Linda M. Scott 2003 A review of the visual rhetoric liturature. In Persuasive Imagery. A Consumer Response Perspective, Linda M. Scott and Rajeev Batra (eds.), 1756. Mahwah, NJ/London: Lawrence Erlbaum. Lakoff, George, and Mark Johnson 1980 Metaphors We Live by. Chicago: University of Chicago Press. Lakoff, George, and Mark Johnson 1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York, NY: Basic Books. Maes, Alfons, and Joost Schilperoord 2008 Classifying visual rhetoric. Conceptual and structural heuristics. In Go Figure! New Directions in Advertising Rhetoric, Edward F. McQuarrie and Barbara J. Phillips (eds.), 227253. Armonk, NY: ME Sharpe. McCarthy, Michael S., and David L. Mothersbaugh 2002 Effects of typographic factors in advertising-based persuasion: A general model and initial empirical tests. Psychology and Marketing 19: 663692. McQuarrie, Edward F., and David G. Mick 1992 On resonance: A critical pluralistic inquiry into advertising rhetoric. Journal of Consumer Research 19 (2): 180197. 1996 Figures of rhetoric in advertising language. Journal of Consumer Research 22 (4): 424438. 1999 Visual rhetoric in advertising: Text-interpretive, experimental, and reader-response analyses. Journal of Consumer Research 26: 3754. 2003a The contribution of semiotic and rhetorical perspectives to the explanation of visual persuasion in advertising. In Persuasive Imagery. A Consumer Perspective, Linda M. Scott and Rajeev Batra (eds.), 191221. Mahwah, NJ/London: Lawrence Erlbaum.

Visual metaphoric conceptualization in editorial cartoons 2003b

239

Visual and verbal rhetorical figures under directed processing versus incidental exposure to advertising. Journal of Consumer Research 29: 579587. McQuarrie, Edward F., and Barbara J. Phillips 2005 Indirect persuasion in advertising. Journal of Advertising 34: 720. McQuarrie, Edward F., and Barbara J. Phillips (eds.) 2008 Go figure! New Directions in Advertising Rhetoric. Armonk, NY: ME Sharpe. Morrison, Susan S. 1992 The feminization of the German Democratic Republic in political cartoons 198990. Journal of Popular Culture 25: 35. Mothersbaugh, David L., Bruce A. Huhmann, and George R. Franke 2002 Combinatory and separative effects of rhetorical figures on consumers effort and focus in ad processing. Journal of Consumer Research 28: 589602. Ortony, Andrew 1979 Beyond literal similarity. Psychological Review 86:161180. Phillips, Barbara J. 2000 The impact of verbal anchoring on consumer response to image ads. Journal of Advertising 29: 1524. 2003 Understanding visual metaphor in advertising. In Persuasive Imagery. A Consumer Perspective, Linda M. Scott and Rajeev Batra (eds.), 297310. Mahwah, NJ/London: Lawrence Erlbaum. Phillips, Barbara J., and Edward F. McQuarrie 2002 The development, change, and transformation of rhetorical style in magazine advertisements 19541999. Journal of Advertising 31: 1 14. 2004 Beyond visual metaphor: a new typology of visual rhetoric in advertising. Marketing Theory 4 (12): 113136. Plumb, Steve 2004 Politicians as superheroes: The subversion of political authority using a pop cultural icon in the cartoons of Steve Bell. Media, Culture and Society 26: 432439. Scott, Linda M. 1994 Images in advertising: The need for a theory of visual rhetoric. Journal of Consumer Research 21: 252273. Scott, Linda M., and Rajeev Batra (eds.) 2003 Persuasive Imagery: A Consumer Response Perspective. Mahwah, NJ/London: Lawrence Erlbaum. Shen, Yeshayahu 1999 Principles of metaphor interpretation and the notion of domain: A proposal for a hybrid model. Journal of Pragmatics 31: 16311653.

240

Joost Schilperoord and Alfons Maes

Teng, Norman Y. 2006 Metaphor and coupling: An embodied, action-oriented perspective. Metaphor and Symbol 21: 6786. Teng, Norman Y., and Sewen Sun 2002 Grouping, simile, and oxymoron in pictures: A design-based cognitive approach. Metaphor and Symbol 17: 295316. Tom, Gail, and Anmarie Eves 1999 The use of rhetorical devices in advertising. Journal of Advertising Research 39: 3943. Van Mulken, Margot 2003 Analyzing rhetorical devices in print advertisements. Document Design 4: 114128. Van Mulken, Margot, Renske Van Enschot-van Dijk, and Hans Hoeken 2004 Puns, relevance and appreciation in advertisements. Journal of Pragmatics 37: 707721. Zbikowski, Lawrence M. 2002 Conceptualizing Music: Cognitive Structure, Theory, and Analysis. Oxford: Oxford University Press.

IV Metaphors of Emotion in Comics, Manga, and Animation

Chapter 11 Anger in Asterix: The metaphorical representation of anger in comics and animated films Bart Eerden

Abstract
This chapter analyses anger in two Asterix comics albums and two Asterix animated films. This will on the one hand yield, enrich, and qualify Forcevilles (2005a) earlier findings on the visual representation of the Idealized Cognitive Model of anger in the Asterix album La Zizanie and on the other hand enable insights into medium-specific representations of the ICM of anger. In investigating the structural part of the emotion metaphors, handbooks for comics and animated films will be taken into account as well, thus allowing for comparisons between theories developed in metaphor scholarship and the practices of animation artists. The methodological framework constructed for the analysis in this paper is applicable to (a) other emotions and (b) other pictorial manifestations of emotions in static and moving images. Keywords: Idealized Cognitive Model, anger, emotions, comics, animation

1. Introduction In cognitive metaphor research, metaphorical expressions are the verbal manifestations through which Idealized Cognitive Models (ICMs) or Folk Models (Lakoff and Johnson 1980, 1999; Lakoff 1987) can be charted. Zoltn Kvecses has systematically analyzed the ICMs of various emotions (Kvecses 1986, 1990, 1995, 2000, 2002 and 2005). Based on verbal evidence in several languages, Kvecses argues that people structurally conceptualize emotions metaphorically. For example he is doing a slow burn and he spat fire can be traced back to the concept ANGER IS FIRE. By contrast, he has a ferocious temper and he unleashed his anger draw on ANGER IS

244

Bart Eerden

A CAPTIVE ANIMAL

(Kvecses 2000: 21). Over the past decade the research of visual and multimodal metaphors within the cognitivist paradigm has taken shape in the work of Carroll (1996), Forceville (1996, 2002, 2005b, 2006/this volume, 2008), Kennedy (1993), Khordoc (2001), Whittock (1990), but most of this work focuses on creative rather than on structural metaphors. Kvecses model, however, provides a good starting point for the investigation of structural emotion metaphors in non-verbal and multimodal representations. Elaborating on findings by Kennedy (1982, 1993), Forceville (2005) examines visual representations of anger in an Asterix comic. Comics provide good source material for research, because of their rich use of pictorial metaphors to convey a vast array of emotions (see also Fein and Kasher 1996; Khordoc 2001). Forceville (2005) introduces various pictorial signs such as red face, spirals and bulging eyes that are frequently used in the Asterix comic to depict anger. The nature and use of these signs appear to confirm that these are not just creative metaphors in the sense of Black (1979) and Forceville (1996) but indeed manifestations of structural metaphors (like the he spat fire example) in the sense of Lakoff and Johnson (1980, 1999). Whereas Forceville confined himself to anger in one comics album, this chapter, building on Eerden (2004), will analyze anger in two other Asterix comics albums and two Asterix animation films. This will on the one hand enrich and qualify Forcevilles earlier findings and on the other hand enable insights into how yet another medium (animated film) can represent the ICM of anger. In investigating the structural part of the emotion metaphors, handbooks for comics and animated films will be taken into account as well, thus allowing for checking theories from the realm of metaphor scholarship against the practices of comics and animation artists. The methodological framework constructed for the analysis in this paper is intended to be subsequently applicable to (a) other emotions and (b) other pictorial manifestations of emotions in static and moving images.

2. Non-verbal metaphor The perception psychologist Kennedy is one of the first to mention cartoons as a source of non-verbal metaphors (Kennedy 1982). Consider the wavy lines used by a cartoonist to depict smoke. These lines realistically represent a visible phenomenon. Now imagine the same wavy lines used in a cartoon to represent the strong smell of garbage. These lines are not realistic but

Anger in Asterix

245

exemplify a metaphorical representation of (invisible) smell (example from McCloud 1994: 128). In the same way, comics use realistic and non-realistic signs to depict emotions. Much of the importance of Kennedys work resides in the fact that he argues how the signs used in comics need not always be derived from verbal metaphors: there may be pictorial devices which are metaphoric but which have no clear equivalent in language (Kennedy 1982: 600). Kennedy introduces the term pictorial rune for non-realistic visual metaphors. According to Kennedy, pictorial runes are often used to depict abstract concepts which are difficult to depict literally. Emotions are a good example of such abstract concepts: States such as anxiety and pain are difficult to depict []. Cartoonists often turn to pictorial runes to show these states (Kennedy 1982: 600). Kennedys observation of the representation and understanding of abstract concepts through concrete perceptible phenomena in fact reflects one of the central principles of Cognitive Metaphor Theory. Kennedys concept of pictorial runes is further fine-tuned by Forceville (2005a). Forcevilles analysis of the non-verbal representation of anger in the Asterix album La Zizanie (Goscinny and Uderzo 1970) introduces two categories of visual representations of anger, based on a metonymic relation between sign and emotion. One category consists of pictorial runes, a term reserved for non-realistic (not perceptible in real life) signs. The second category is formed by indexical signs. The indexical sign differs from the pictorial rune in that it is a realistic sign (although often exaggerated). Apart from its realism, the indexical sign seems to function much like a pictorial rune. Both pictorial rune and indexical sign signify anger through a metonymic relation (as opposed to arbitrary signs or literal depiction).

3. Anger in comics Before presenting the results of the analysis of anger in two Asterix comics, it is important to look at the results of Forcevilles (2005a) earlier analysis. Forcevilles analysis of anger in La Zizanie will be the reference point for the current analysis. La Zizanie contains 103 angry characters, and anger is expressed through twelve different signs (Forceville 2005a: 7577). The twelve angry signs can be further divided in two categories: pictorial runes (ex-mouth,1 spirals around the head, smoke above a head, bold face and jagged text-balloon lines) and indexical signs (bulging eyes, tightly closed

246

Bart Eerden

eyes, wide mouth, tightly closed mouth, red face, arm/hand position and shaking.
Table 1. Signs of anger in percentages of total number of angry characters per album. (Percentages in the table are rounded to whole percents. For example, bulging eyes occur 47 times in La Zizanie on a total of 103 angry characters. This means that in 46 (45,63) percent of all angry characters in La Zizanie bulging eyes is present.) La Zizanie Pictorial runes Smoke/steam Helmet Ex-mouth Bold face Jagged line Spirals head Indexical signs Bulging eyes Closed eyes Wide mouth Tight mouth Red face Hand/arm Shaking body 46 37 4 0 1 3 1 7 5 0 6 42 50 34 15 32 59 5 59 41 48 4 7 70 1 1 13 38 31 44 2 3 21 34 16 50 16 48 4 35 Asterix Lgionnaire Asterix et Latraviata

The pictorial runes in La Zizanie, Forceville argues, are metonymically motivated, just as the indexical signs (Forceville 2005a: 82). More importantly, the runes in La Zizanie, in some cases, appear to be direct manifestations of the ICM of anger, thus confirming Kennedys claim that pictorial metaphors do not necessarily have a verbal counterpart (Kennedy 1982: 600). However, Forceville acknowledges the possibility that the runes under scrutiny here have become conventionalized as, somehow, visual translations of verbal manifestations of ICMs (Forceville 2005a: 83). Both indexical signs and pictorial runes are commensurate with the important central conceptual

Anger in Asterix

247

metaphor ANGER IS A HOT FLUID IN A PRESSURIZED CONTAINER (as discussed by Kvecses 1986). In this chapter, two other Asterix comics, Asterix et Latraviata (Uderzo 2001) and Asterix Lgionnaire (Goscinny and Uderzo 1967) are analyzed to test the insights of Forceville (2005a).2 Asterix et Latraviata is a relatively recent album whereas Asterix Lgionnaire was published in 1967, three years before La Zizanie. With almost twenty-five years between the two albums, they represent a cross-section of the Goscinny/Uderzo oeuvre. The signs depicting anger in Asterix Lgionnaire and Asterix et Latraviata correspond largely, in form and in frequency, to the signs in La Zizanie. See table 1 for the results, including Forcevilles. As is noticeable from table 1, the two eyes, mouth, and the hand/arm categories are the most frequently used signs in the three Asterix albums. Helmet is a new pictorial rune, which does not appear in Forcevilles list. A helmet flying off a head as well as a character with smoke or steam around its head appear only three times but these signs are certainly worth mentioning.3 They are good examples of the depiction of different categories or stages of anger. The helmet flying off is a sign that occurs with the outburst of anger. Smoke as a sign occurs right before the explosion of anger when the emotion is (still) suppressed. This stage of anger is described by Forceville as typically [comprising] a tightly closed mouth, often emphasized by clamped teeth and the hidden arm/hand position (Forceville 2005a: 8383). The hand/arm sign is a rather large category of different arm positions occurring in different stages of anger. Forceville (2005a) uses a threefold distinction to differentiate the arm/hand category: (i) fisted hands; (ii) hands/arms emphatically close to the body; and (iii) pointing with the index finger. Positions (i) and (ii) are particularly associated with attempts to control anger whereas position (iii) suggests the eruption of anger. A closer look at the hand/arm category in the two Asterix albums reveals two additional elements of this sign, upright position and stretching forward. The first element (upright) seems to appear with Forcevilles description of arm/hand position (i) and (ii), with fisted hands or arms close to the body (as if to keep the anger inside the body-container). These positions of arm/hand often occur in combination with an upright position of the body in which the head is tilted with the nose pointing upwards.4 The second element (stretching forward) typically co-occurs with the outward pointing arm/hand position (iii) in an outburst of anger. The forward stretched body appears to convey the eruption of the pressurized container. Moreover, stretching forward seems to be preceded by the upright position, thus conveying a

248

Bart Eerden

sequence of anger consisting of two stages, (a) the rising or suppression of anger, followed by (b) the explosion of anger. However, in the static images of a comics album, upright and stretching forward are difficult to recognize. This further distinction of the hand/arm sign will become relevant in the discussion of animation film in section 5.

4. Handbooks for comics and animation films Kvecses (2000, 2005) and Forceville (2006) as well as Lakoff and Turner (1989) and Gibbs and Steen (1999) emphasize the influence of culture and context on (the representation of) ICMs and the interpretation of source domains, which is an important issue for the current analysis. After all, only three comics albums by the same artist have been examined in this chapter. Previous research (Kvecses 2005) shows evidence of quite fundamental differences between for example the verbal manifestations of the Chinese and Western container metaphor for anger. Yu (1998) describes how the Chinese container metaphor, contrary to its Western equivalent, does not involve a hot fluid but a gas which produces pressure on the body container and which is not associated with heat. Shinohara and Matsunaka (2003) demonstrate that in Japanese the metaphor ANGER IS GASTRIC CONTENTS IN THE DIGESTIVE SYSTEM is saliently present. But within Western culture, differences between verbal (and visual) metaphors in different languages seem minor, suggesting that the most important conceptual metaphors of an ICM transcend at least national/language borders. Handbooks for comics and animated films in western culture can prove to be helpful in further investigating the structural part of the anger representations. The results of a short survey of handbooks (Blair 1994; Maestri 1999, 2002; Thomas and Johnston 1981; Williams 2001) not only show a great similarity between the advice proffered by the different handbooks among themselves, but also between the handbooks and the findings on anger in the three Asterix comics investigated. The use of, and emphasis on, the same signs to represent anger in different examples suggests the existence of embodied concepts that are, to a certain extent, cross-culturally represented through the same signs in comics and animation. Blair (1999) describes a number of prototypical characters and facial expressions pertaining to standard emotions. His description of the standard anger face strikingly resembles Forcevilles description of the visualization of anger in La Zizanie. According to Blair, a prototypical angry character has fisted hands, clamped teeth, V-shaped heavy brows and small, black

Anger in Asterix

249

pupils located in the corner of the eyes (Blair 1999: 24, 52, 53). All these elements appear in Forcevilles analysis as well as in the current analysis of two new Asterix albums (arm/hand (i), tight mouth and bulging eyes). My earlier claim (Eerden 2004) about the existence of strong signs which prototypically represent a certain emotion, and weak signs which can only represent a certain emotion in combination with other signs, is consistent with Thomas and Johnstons (1981) views. They also emphasize that emotions more often than not are represented by a certain combination of signs. While a combination of signs visually represents an emotion, say anger, an isolated sign from such a combination, or such a sign in combination with other signs, might represent a headache (spirals) or a person who has eaten spicy food (red face) rather than anger. According to Thomas and Johnston, emotions are most importantly expressed via the eyes, and again the similarities with angry eyes in the analyzed Asterix comics are evident. As a final point, Thomas and Johnston describe how Disney Studios initially tried to copy facial expressions from real actors. This method proved unsuccessful and Disneys designers turned back to their drawing boards to experiment with and try out other ways to represent emotions visually. This seems to confirm a metonymically motivated connection between sign and emotion instead of a sign iconically depicting an emotion. In digital animation, to conclude this brief survey of handbooks, the same few pictorial signs of anger can be found. Williams (2001) and Maestri (1999, 2002) yet again emphasize the eyes (with brows) and the mouth as the main sites for the expression of emotions. These are among the most differentiated signs in the Asterix albums. Although the fact that the handbooks corroborate the theoretical findings should not lead to sweeping conclusions, it does present data in support of the idea that pictorial anger signs metonymically instantiate conceptual metaphors, specifically the ICM of anger. Moreover, the handbooks present eyes as the most important part of the face for expressing an emotion, which ties in with eyes being the largest and most differentiated category in all three Asterix albums under scrutiny here. Interestingly, Kvecses (1990) gives numerous examples of eyes as the source domain of emotion metaphors in English and Hungarian. The verbal expressions found by Kvecses, however, do not correspond with the signs found in the Asterix albums. Whereas verbal expressions seem to focus on the presence and kind of emotions (love showed in his eyes, I could see the fear in his eyes), consistent with the idea of eyes as mirror of the soul (Kvecses 1990: 173), visual signs, at least of anger, appear to focus on representing the intensity of an emotion in, and also around, the eyes (pouches, brows, lines).

250

Bart Eerden

Although different in appearance, both visual and verbal expressions of emotions in eyes can be traced back to the conceptual metaphor EYES ARE CONTAINERS FOR THE EMOTIONS (Kvecses 1990; Lakoff and Johnson 1980). These examples substantiate the hypothesis that visual metaphors can be direct manifestations of an ICM instead of merely being visual translations of verbal metaphors. 5. Animated films This section examines the visual representation of anger in Asterix animation films. The films under scrutiny here are Asterix et la Surprise de Csar (Brizzi and Brizzi: France 1985) and Asterix chez les Brtons (van Lamsweerde: France 1986). The first animated Asterix film is an adaptation of the album Asterix Lgionnaire (Goscinny and Uderzo 1967). Since neither of the two other albums exists as animated film, the second film was selected more or less randomly. Although based on the original comics album Asterix Lgionnaire (Goscinny and Uderzo 1967), Asterix et la Surprise de Csar is very different from the original. Not only has the story been altered but, more importantly for present purposes, there are salient differences in the visual representation of anger due to the fact that animated film, unlike comics, can actually show moving characters and moreover include sound. The analysis of the representation of anger through different media is an important way to determine the influence of a medium on its message, in this case the pictorial metaphors. In the following paragraphs I will discuss the most salient differences between these two media, and some consequences for the analysis of anger. First of all, comics are static and films are animated. This obvious fact has a great influence on the appearance and interpretation of certain anger signs. For example, the famous pictorial runes speed lines and shape changes, used to indicate speed or motion in static images (Kennedy 1982: 501), are much rarer in animated films. The fact that animated films can convey motion also sheds new light on certain anger signs. For example, the hair of the characters turned out to be a difficult sign in comics. It is for instance hard to see if the braids of Obelix (one of the main characters in Asterix) swing up to express an emotion or just because they follow the movement of his head. Hair as a sign is for that reason not included in the analysis of the comics. In the animated films, by contrast, it is easy to see how Obelix braids and moustache sometimes unrealistically swing up when he expresses anger. Another example of different visualization in animated

Anger in Asterix

251

films is the position or posture of a characters body. In section 3 the new categories stretched forward and upright were introduced as subcategories of Forcevilles hand/arm sign. But in the context of comics these posture signs are difficult to define. A change in posture that is caused by an emotion is difficult to recognize in static images, where every panel shows a more or less new situation. In contrast, animated films consist of shots and sequences instead of panels, which makes it easy to detect a change in body position. A second big difference is related to the moving aspect of animation, too. Since a comics album consists of a sequence of separate panels, readers can decide for themselves how long they want to look at a panel. In fact the panel can be seen as the smallest unit of a comic. For the average viewer the smallest unit of an animated film is one shot (since normally a viewer can not discern a single frame but only the succession of frames). This makes it difficult to analyze and compare the two animated films because the length of shots in Asterix et la Surprise de Csar and Asterix chez les Brtons varies from less than one second to more than ten seconds. This means that mutually exclusive signs (in comics) such as wide mouth and tight mouth can easily appear multiple times in a single shot with a single character. Moreover, the comparison between a sign in a panel and the same sign in a shot is difficult to express in numbers. If bulging eyes appears in a panel it counts as one appearance, but how to count bulging eyes in a shot of half a second or ten seconds, and what if bulging eyes appears five times in a single shot? The inevitable conclusion is that reliable quantitive comparison between both media is impossible. However, it is possible to analyze the use of emotion signs across the different animated films and to compare similarities and differences between emotion signs used in different media in a more general sense. One last important difference has to do with the multimodality of the different media. According to Forcevilles (2006) list of modes, comics contain pictorial signs, written signs, and gestures. Animated films contain the same three modes plus spoken signs, sounds, and music. Since the present study focuses only on visual representations of anger, spoken signs, sounds, and music will not be included in the analysis.

6. Signs of anger in animated films The new indexical sign stretched forward returns in the expression of anger in the two animated films. This sign typically appears in combination

252

Bart Eerden

with the other newly introduced sign upright. These signs usually appear one after the other, expressing the build-up, followed by the release of anger. The upright position is a manifestation of the building up of pressure in the container, usually in combination with arms close to the body and/or clenched fists, both associated with keeping the pressure in the container (Forceville 2005a: 81). The stretched forward sign expresses the release of pressure from the body container. Stretched forward usually appears in combination with stretched arms and pointing or fisted hands. The outward pointing arms also suggest the release of pressure. For examples of both signs, see figures 1 and 2.5

Figure 1. Upright (hand-traced still from Asterix et la Surprise de Csar).

Apart from stretched forward and upright, which could already be seen in comics, although not very clearly, some other completely new signs materialize in the animated films. These signs either do not appear in comics at all or vary substantially from their equivalents in comics. In the following analysis these signs will be examined more closely. An important aim of the analysis is to find out if these new signs can be attributed to a conceptual metaphor and if so, to which one. 6.1 Wide mouth As with bulging eyes the wide mouth sign in animated films also appears to differ from the same sign in comics because of the multimodality (especially the presence of the sonic modality) and the motion aspect of the animated film. Because animated film is not static, it is even better able to build up a sign through different stages. In the Asterix comics, a mouth counts as wide if at least two of the following are visibly present: (i) the tongue; (ii) teeth; (iii) (a) line(s) running over the cheek from the nose to the corners of the mouth (Forceville 2005a: 76). In animated films, by contrast, the wide mouth requires the presence of only one of these three features to

Anger in Asterix

253

Figure 2. Stretched forward (hand-traced still from Asterix et la Surprise de Csar).

communicate anger, and moreover is not restricted to the expressed anger stage. This has to do with animated films ability to express the building up of an emotion through time in a continuous sequence. This feature of animated film along with the presence of the sonic modality makes it much easier to identify the emotion anger in a sequence. A wide mouth typically represents the release of pressure from the body-container. An example of wide mouth in animated film is provided in figure 3.

Figure 3. Wide mouth (hand-traced still from Asterix et la Surprise de Csar).

6.2 Bulging eyes The two animated films show a wide variety of bulging eyes. All signs, however, include a V-shaped brow with pouches and frowns as already described by Forceville in the Asterix comics album (Forceville 2005a: 75). The medium animated film and its specific features (moving images) possibly plays an important role in the variation in pictorial signs of anger. The animated films show a greater range in the depiction of bulging eyes. The two animated films for example have bulging eyes wide open but also with one eye closed and the other wide open or all the traits of bulging eyes with

254

Bart Eerden

the eyelids still slightly closed. Bulging eyes are a manifestation of the (release of the) internal pressure of the body-container.

6.3 Shaking This sign expresses anger in the same way as does the shaking sign in comics which consists of (i) multiple superimposition of a character and/or (ii) a non-moving character depicted as loose from the ground (Forceville 2005a: 76). There is, however, a difference in form. Shaking can appear in animated film without the superimposition described for the comics version of shaking, and in other cases the head of the character is the only shaking part. The absence of superimposition or extra lines inevitably means that shaking is only recognized in a sequence of frames, but not in a single (freeze)frame. Shaking is a manifestation of the (immense) internal pressure of the container. The shaking sign appears especially at moments where suppressed anger turns into an outburst or when an outburst ends. An earlier study of pictorial signs representing love in animated films suggested that shaking is also used as a sign when love strikes a person or when a person instantly falls out of love (Eerden 2004: 53). This particular example of the shaking sign is limited to the head only. The shaking of the head can be traced back to the folk theory of emotions. Kvecses (1990, 2000, 2005) describes the folk theory according to which an emotion can be characterized as a five-stage scenario. In this cognitive model the emotion affects the self as a force that causes a change of state. As a result the self loses control and at a final stage responds to the emotion with emotional behavior (Kvecses 2000: 5859). The new subcategory of shaking, which focuses specifically around the head, marks the sudden entering of one of the final stages of an emotion. The character instantly loses control.

6.4 Ex-mouth In comics this sign has straight lines emitting from the mouth (Forceville 2005a: 77). The lines express something forcefully coming out of the mouth. This could be explained as the release of pressure, which makes ex-mouth a pictorial rune. On the other hand, the straight lines might simply represent

Anger in Asterix

255

inadvertent spitting as an expression of anger. In that case ex-mouth should fall in the category of indexical signs. The ex-mouth sign appearing in the animated films obviously has something forcefully coming out of the mouth but without the straight lines visible in comics albums. These lines, it seems, are replaced by sound (shouting) and an unrealistic stream of air emitting from the mouth. The stream of air is perceptible in the waving of clothes or hair of the victim who is shouted at. These unrealistic traits confirm the idea of ex-mouth being a pictorial rune. There is no example of spitting with anger to be found in either animated film. As with shaking, this is again an example of a sign which loses all its speed lines or superimpositions when translated from comic to animation. Probably the waving of clothes is a motion hard to depict in comics, hence support is needed in the form of lines to accentuate the release of pressure. This is a good example of how pictorial signs have medium-specific manifestations. In animated films the lines disappear and are replaced by sound. For an example of ex-mouth in animated films, see figure 4.

Figure 4. Ex-mouth and the waving movement of clothes in the Roman soldiers cape (hand-traced still from Asterix et la Surprise de Csar).

6.5 Upright As an indexical sign for anger, upright has an erect position of the body with the back of the head usually pressed against the neck and the nose pointing upwards. The upright position of the body signals the build-up of internal pressure in the body-container.

256

Bart Eerden

6.6 Stretched forward Stretched forward is particularly associated with the outburst of anger and is often preceded by the upright sign. Stretched forward usually appears with arms pointing outwards and clenched fists or pointing hands. This sign can be interpreted as the violent release of pressure from the container.

6.7 Low angle At first glance, it appears as if the low angle-shot functions as an arbitrary sign for anger.6 This shot (see figures 5 and 6) is often used to show angry or threatening characters. Film theorists such as Bordwell and Thompson stress, however, that framing does not automatically possess absolute or general meaning (Bordwell and Thompson 2004: 263). According to Bordwell and Thompson the context of a film will determine the function of a low angle shot. If low angle is considered neither a Peircean symbol sign nor a Peircean iconic sign, the question will be whether low angle can be explained as the visual representation of a conceptual metaphor of anger, making it an indexical sign. Possibly, low angle is the representation of the conceptual metaphor ANGER IS A DANGEROUS ANIMAL (an animal to which a character looks up from a low angle). Yet Kvecses (1990: 63) does not derive metaphors from this concept that confirm the above hypothesis. His outline of ANGER IS A DANGEROUS ANIMAL focuses on the growing or height of the angry person whereas low angle seems to convey the experience of the victim watching a huge angry character. Perhaps the sign low angle does not express anger, but fear. But then again the analysis resulted in only one example of a point of view shot from a frightened character. Possibly, the low angle sign represents a part of the ICM of anger that is not, or at least differently, expressed in language. Since ICMs have hitherto mainly been charted on the basis of verbal expressions, the analysis of visual expressions might result in new representations of known conceptual metaphors or possibly even new conceptual metaphors. Kvecses (2005), following Grady, offers a possible explanation for cultural variations between conceptual metaphors and their verbal representations. According to Grady, complex conceptual metaphors are built up from a combination of simpler primary metaphors. In this way the complex metaphor ANGER IS A HOT FLUID IN A CONTAINER is for example based on a combination of primary metaphors such as INTENSITY IS HEAT, INTENSITY IS

Anger in Asterix

257

QUANTITY and INTENSITY IS SPEED (Kvecses 2005: 27). Although the primary metaphors seem universal, combinations of primary metaphors vary between different cultures, thus resulting in different complex metaphors, which accounts for cultural variation in verbal expressions of anger. This same phenomenon seems to be relevant for different modes of communication. I suggest that different modes of communication can account for variations in conceptual metaphors in much the same way as cultural context does.

Figure 5. Low angle (hand-traced still from Asterix et la Surprise de Csar).

Figure 6. Low angle (hand-traced still from Asterix et la Surprise de Csar).

7. Results Many of the signs found in the analyzed comics and films are commensurate with the results of previous research (Forceville 2005a, Eerden 2004). The analysis of comics and animated films reported in this chapter has also resulted in the identification of some new signs of anger. Some of these occurred in the two Asterix comics albums, but were not reported by Forceville

258

Bart Eerden

(2005a); others appear to be specific for the medium of animation. The question here is whether different signs of anger downplay or highlight different parts of the ICM of anger in verbal, comics, and animated form, respectively. Further analysis is necessary to chart the conceptual metaphors which are expressed through the newly found animated signs. The eyes and mouth signs, followed by arm/hand are omnipresent in both comics and animated films. The analysis of handbooks confirms the important role of the three largest categories of signs and the rich variations existing within each category (Blair 1999; Maestri 1999, 2002; Thomas and Johnston 1981; Williams 2001). These three categories are the most used and most differentiated signs in the comics, animated films, and handbooks examined. But however important these signs are in visual representations, verbal equivalents seem hard to find. An example of the differences between verbal and visual representations can be found in the source domains for eyes. The verbal expressions largely pertain to EYES AS CONTAINER FOR EMOTIONS with the emotion being visible in the eyes (I could see the fear in his eyes, his eyes were filled with anger, and love showed in his eyes). Comics and animated films, by contrast, are able to express not only the presence of a certain emotion, but also the intensity of an emotion like anger, as well as its stage. This again confirms the idea that complex conceptual metaphors might be constructed and expressed differently in different media and modes of communication. The representation of anger in animated films results in the identification of at least one new sign (low angle) which does not seem to fit in with the central conceptual metaphor of anger (ANGER IS A HOT FLUID IN A PRESSURIZED CONTAINER) as derived from verbal expressions. While all the other signs can be explained as referring to embodied metaphors, low angle has no relation to the body of the angry person, which explains why it is not compatible with the container metaphor. Kvecses also presents verbal expressions that do not refer to the container metaphor but to other conceptual metaphors, such as ANGER IS A DANGEROUS ANIMAL or ANGER IS AN OPPONENT. However none of these examples seem to be related to the low angle sign. The low angle sign originates from the unique possibilities (such as framing and motion) of the animated medium. Also low angle seems to focus more on the perspective of a victim experiencing or witnessing the anger expressed by another character. When a victim-character is not present in the story, it is the viewer who experiences the anger through a virtual point-of-view-shot.

Anger in Asterix

259

Animated films and comics in general contain yet other examples of signs conveying emotions that are not related to the body. One might think of a light-bulb or rain cloud above someones head. Certainly this area of pictorial metaphor needs more research. Shinohara and Matsunaka (this volume) give examples of such external signs as thunder, wind, or flowers often occurring in Japanese Manga. They also give examples of background scenes of panels which are used to express the emotional state of the character in the panel. A similar version sometimes appears in the Asterix comics. This is not included in the current analysis, as the signs do not seem to appear structurally, but further research is important here. One such sign is green text balloons, which appears in La Zizanie. Forceville labels this sign as arbitrary and thus excludes it from his analysis, using the sign as an independent indicator of anger, since the green text balloons appear in over 50 percent of the anger panels (Forceville 2005a: 75). The green text balloons seem even more arbitrary because they do not appear in the next two albums. However, a closer look at the emotion anger in Asterix et Latraviata and Asterix et la Surprise de Csar shows many unrealistically colored backgrounds in panels with angry characters. These background and text balloon signs might be arbitrary, but in light of Shinohara and Matsunakas research it is possible that these signs actually express certain conceptual metaphors. The results in table 1 present hand/arm and eyes as the largest two categories of signs, followed by mouth signs. On average, eyes appears in 85 percent of anger panels, for hand/arm the average is 63 percent, and the category mouth can be found in 51 percent of the anger panels. The three categories are not only the largest in numbers but also the most differentiated signs, both in comics and animated films. The three signs are at the very least commensurate with the conceptual metaphor ANGER IS A HOT FLUID IN A PRESSURIZED CONTAINER. I would venture the stronger claim that the container metaphor is at the center of the representation of anger in the analyzed comics as well as the animated films. Kvecses emphasizes the role of the container concept as central to the concept of anger. It seems warranted to conclude that certain complex and central embodied concepts such as CONTAINER and FORCE play a central role in metaphors, irrespective of medium or mode. However, based on verbal evidence, the concept of anger consists of a number of other important metaphors such as ANGER IS A BURDEN, ANGER IS AN OPPONENT IN A STRUGGLE and ANGER IS A CAPTIVE ANIMAL (Kvecses 2000: 21). These metaphors are good examples of concepts that are not primarily based on embodiment but seem more related to behavioral aspects. It seems that visual representations of anger focus almost entirely on embodied container

260

Bart Eerden

concepts whereas verbal expressions regularly draw on non-embodied, behavioral concepts. Since culture and context influence behavior rather than physical aspects, it appears that verbal expressions more strongly reflect culture-specific expressions of anger. The visual part of comics and animated films, which draws heavily on embodied concepts, seems less influenced by culture-specific metaphors. The current analysis of pictorial signs in comics and animated films supports Forcevilles (2005a) and my own earlier (Eerden 2004) assumption that pictorial signs provide medium-specific representations of the ICM of anger. Presumably, the charting of ICMs through verbal expressions alone creates blind spots. To achieve a complete insight in an ICM means one has to study the metaphorical representations in every mode of communication. The emphasis on embodied concepts and the temporal development of an emotion are examples of new insights from analyses of clearly defined corpora, in this case three comics albums and two animated films. Future research into other modes of communication will certainly yield new aspects of ICMs that have hitherto been downplayed in verbal (and visual) contexts. Tying in with the blind spots of the charting project is the idea that a particular medium draws on specific medium-related possibilities and impossibilities to represent a certain emotion. Such medium-specific representations appear to occur on two levels. A specific medium draws heavier on certain unique combinations of primary metaphors to construct complex conceptual metaphors. Since animated film (as opposed to comics) can depict movement, it will draw more on those primary metaphors concerning the progression of an emotion through different stages. Moreover, some conceptual metaphors are in fact limited to a specific mode of communication, or are at least very rare in other modes. Consider for example an indexical sign that is tied to animated film such as low angle. This sign is all-pervasive in the analyzed films and is an important sign in the representation of anger. This metaphorical sign, however, does not seem to have a verbal equivalent.

8. Further research Since research into non-verbal and multimodal metaphor is relatively new, further research can move into largely unexplored areas. In addition, it is important to verify or falsify the results of previous research. This is one of the main reasons for me to use a clearly defined corpus of research. The more eclectic approach of Kvecses or Lakoff and Johnson makes it more

Anger in Asterix

261

difficult to compare results. Considering the current analysis and results, some important research projects come to mind. First of all, more research into the visual representation of anger and other emotions is needed to substantiate and verify the results reported in the current chapter. Research should, however, not be limited to comics and animation alone. Other forms of communication and especially multimodal communication should be included. Forceville (2006) distinguishes nine different modes of metaphorical representation, including at least pictorial signs, written signs, spoken signs, gestures, sounds, music, smells, tastes and touch. Further research into these modes can present important insights. The current analysis already suggests that certain medium-specific signs in comics appear as sonic signs in animated films. A particularly interesting area is the representation of emotions in various new media. One can think of the representation of emotions through emoticons and different fonts in internet communication or the use of emotionspecific avatars; and of the representation of emotions in Massive Multiplayer Online Role Playing Games (MMORPGS) such as Second life. MMORPGS can prove especially interesting for research into cultural variations because MMORPGS create a new cross-cultural online community. Finally, Gibbs and Steen (1999) as well as Kvecses (2005) underline the important influence of culture on ICMs. This suggests cross-cultural differences in the conceptualization and representation of emotions. Complex nonwestern media such as manga and anime are important source material for further investigation in cultural variations (Shinohara and Matsunaka 2003, this volume). Especially for this kind of research and its comparison with earlier findings, the use of a clearly defined corpus is important. Moreover, the use of empirical research and verifiable experiments is crucial if we are to present evidence about the structural use of metaphorical signs and conceptual metaphors.

Notes
1. 2. This sign shows straight lines emitting from the mouth. The albums used here are Dutch translations (since language is not a part of the analysis) of Asterix Lgionnaire (Goscinny and Uderzo 1967) and Asterix et Latraviata (Uderzo 2001). Both signs occur in Asterix Lgionnaire and steam/smoke occurs once in Latraviata. The helmet can be seen on page 19 and 23 and the steam/smoke occurs on page 15 and 16 and in Latraviata on page 17.

3.

262
4. 5.

Bart Eerden Forceville (2005a) also recognizes an upward tilted head in combination with anger and suggests that it might correspond with pride. For copyright reasons and in order to emphasize the relevant details, all figures are hand-traced stills from Asterix et la Surprise de Csar (Brizzi and Brizzi: France 1985). Symbol is the term used by Charles Sanders Peirce. A symbol is characterized by the arbitrary link between sign and referent; for example words in a language are symbolic signs.

6.

References
Black, Max 1979 More about metaphor. In Metaphor and Thought, Andrew Ortony (ed.), 1943. Cambridge: Cambridge University Press. Blair, Preston 1994 Cartoon Animation. Laguna Hills: Walter Foster. Bordwell, David, and Kristin Thompson 2004 Film Art: An Introduction. 7th ed. Boston: McGraw-Hill. Carroll, Noel 1996 A note on film metaphor. In Theorizing the Moving Image, 212 223. Cambridge: Cambridge University Press. Cienki, Alan 1998 Metaphoric gestures and some of their relations to verbal metaphoric expressions. In Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (ed.), 189204. Stanford: Center for the Study of Language and Information. Eerden, Bart 2004 Liefde en woede: De metaforische verbeelding van emoties in Asterix. [Love and anger: The metaphorical visualization of emotions in Asterix.] MA thesis, Department of Media Studies, University of Amsterdam, The Netherlands. Fein, Ofer, and Asha Kasher 1996 How to do things with words and gestures in comics. Journal of Pragmatics 26: 793808. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 1999 The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 17998. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114.

Anger in Asterix 2005a

263

Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2005b Cognitive linguistics and multimodal metaphor. In Bildwissenschaft zwischen Reflexion und Anwendung, Klaus Sachs-Hombach (ed.), 264284. Cologne: Halem. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379 402. Berlin/New York: Mouton de Gruyter. 2008 Metaphor in pictures and multimodal representations. In The Cambridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr. (ed.), 465485. Cambridge: Cambridge University Press. Gibbs, Raymond W., Jr., and Gerard J. Steen (eds) 1999 Metaphor in Cognitive Linguistics. Amsterdam/Philadelphia: Benjamins. Goscinny, Ren, and Albert Uderzo 1967 Astrix Legionnaire. Paris: La Hachette. 1970 La Zizanie: Une Aventure dAstrix. Neuilly-sur-Seine: Dargaud. Khordoc, Catherine 2001 The comic books soundtrack: Visual sound effects in Asterix. In The Language of Comics: Word and Image, Robin Varnum and Christina T. Gibbons (eds.), 156173. Jackson: University Press of Mississippi. Kennedy, John M. 1982 Metaphor in pictures. Perception 11: 589605. 1993 Drawing and the Blind: Pictures to Touch. New Haven/London: Yale University Press. Kvecses, Zoltn 1986 Metaphors of Anger, Pride and Love. Amsterdam/Philadelphia: Benjamins. 1990 Emotion Concepts. New York: Springer. 2000 Metaphor and Emotion: Language, Culture, and Body in Human Feeling. Cambridge: Cambridge University Press. 2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press. 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Lakoff, George 1987 Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago: University of Chicago Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press.

264

Bart Eerden

Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books. Lakoff, George, and Mark Turner 1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. Maestri, George 1999 Digital Character Animation: Volume 1 Essential Techniques. Indiana: New Riders. 2002 Digital Character Animation: Volume 2 Advanced Techniques. Indiana: New Riders. McCloud, Scott 1994 Understanding Comics: The Invisible Art. New York: HarperPerennial. Shinohara, Kazuko, and Yoshihiro Matsunaka 2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen: Journal of Yokohama Linguistic Circle 4: 118. Thomas, Frank, and Ollie Johnston 1981 Disney Animation: The Illusion of Light. New York: Abberville Press. Uderzo, Albert 2001 Astrix et Latraviata. Paris: Les Editions Albert Ren. Whittock, Trevor 1990 Metaphor and Film. Cambridge: Cambridge University Press. Williams, Richard 2001 The Animators Survival Kit. London: Faber and Faber. Yu, Ning 1998 The Contemporary Theory of Metaphor: A Perspective from Chinese. Amsterdam/Philadelphia: Benjamins.

1999

Chapter 12 Pictorial metaphors of emotion in Japanese comics Kazuko Shinohara and Yoshihiro Matsunaka*

Abstract
In this chapter, we analyze pictorial metaphors of emotion in Japanese comics (manga) within the framework presented by Forceville (1994, 2005) and support his argument that many of the metaphors expressed through the verbal modality and those expressed through other modalities appear to share the same fundamental motivation, that is, mappings between conceptual domains (Lakoff and Johnson 1980). Our analysis of the data including emotion types such as anger, happiness, love, anxiety, surprise, and disappointment shows that (1) the conceptual metaphor ANGER IS A HOT FLUID IN A CONTAINER is shared by verbal and visual modalities, that (2) differences can emerge due to the different properties of each modality though basic conceptual mappings are shared by both modalities, and that (3) culture-specific aspects found in verbal emotion metaphors in Japanese may also be found in pictorial emotion metaphors in Japanese comics. We can conclude that metaphor is a matter of concept and cognition not limited to language. Keywords: pictorial metaphor, indexical signs, emotion, manga, conceptual mappings

1. Introduction Since the foundation of Conceptual Metaphor Theory (CMT) by Lakoff and Johnson (1980), verbal metaphors that are motivated by cross-domain conceptual mappings have attracted researchers attention and the theory itself has experienced revision and elaboration (e.g., Lakoff 1990, 1993; Grady 1997; Lakoff and Johnson 1999; Kvecses 2000, 2002, 2005). This theory has had a great impact on the view of metaphor, changing its fundamental understanding from a rhetorical to a cognitive one. Though metaphor tends to be thought of as a device of poetic ornamentation that is added to ordinary

266

Kazuko Shinohara and Yoshihiro Matsunaka

language, CMT claims that metaphor is pervasive in everyday life, not just in language but in thought and action, and that [o]ur ordinary conceptual system, in terms of which we both think and act, is fundamentally metaphorical in nature (Lakoff and Johnson 1980: 3). In CMT, a conceptual metaphor is defined as the systematic mapping between two different conceptual domains, and it is assumed that surface linguistic expressions of such a metaphor emerge from that conceptual mapping. The central tenet of CMT is that metaphor, by its very nature, not only affects surface linguistic expressions but also characterizes cognitive/conceptual structure. This tenet of CMT, however, has not been fully explored, although some researchers have started to apply this idea to the study of multimodal metaphors. Forceville (1994) suggests that metaphors are not limited to linguistic ones, and he maintains that to further validate the idea that metaphors are expressed by language, as opposed to the idea that they are necessarily linguistic in nature, it is necessary to demonstrate that, and how, they can occur non-verbally and multimodally as well as purely verbally (2006: 381/this volume, emphasis in original). Thus, in recent developments of the study of metaphor, researchers attention is being drawn to the new field of multimodal metaphors. Metaphors expressed through the verbal modality and those expressed through other modalities appear to share the same fundamental motivation, and thus they cannot be regarded as totally separate phenomena. Although, of course, multimodal metaphors that are based on nonverbal sources have unique properties that can be traced back to the nature of the mediums modalities (Forceville 1994, 1996, 1999, 2005; Carroll 1994), they share the underlying mapping between conceptual domains with verbal metaphors. Thus, previous studies on multimodal metaphors strongly support the argument that metaphor is a matter of concept and cognition not limited to language. This chapter builds upon these previous studies, especially the one by Forceville (2005), which deals with pictorial metaphors of anger in a French comics album. Following Forcevilles view, we try to demonstrate that at least some manifestations of anger are found in visual no less than in verbal metaphors. We also try to extend Forcevilles view to other types of emotions such as happiness, love, anxiety, surprise, or disappointment. To attain this goal, we examine pictorial metaphors of emotion in Japanese comics. The source of this type of metaphor is pictorial or visual, where meanings are conveyed via pictorial or visual representations. The target is emotion, which belongs to a more abstract domain of psychological experience. In this metaphor, what is expressed as a picture can be interpreted as representing some emotion. Data are taken from some of the present day popular Japa-

Pictorial metaphors of emotion in Japanese comics

267

nese comic (manga) books written by several different authors. Innumerable comic books are published in Japan now, but despite the diversity, the data and the analysis we are providing in this chapter will clearly show that there are pictorial metaphors that manifest conceptual metaphors that are also expressed verbally. 2. Previous studies and issues 2.1 Emotion metaphors in Japanese Since Lakoff and Johnson (1980) established CMT in cognitive linguistics, efforts have been made to establish generalizations for various emotion metaphors. It has been argued that there are structural correspondences between the source and the target domains of each metaphor. For anger metaphors, in particular, it has been suggested that the correspondences are mainly based on folk theories of anger that are commonly accepted within a speech community.1 For example, Lakoff (1987: 381382) describes the common folk theory of anger in English-speaking communities as follows:
The physiological effects of anger are increased body heat, increased internal pressure (blood pressure, muscular pressure), agitation, and interference with accurate perception. As anger increases, its physiological effects increase. There is a limit beyond which the physiological effects of anger impair normal functioning.

Many of the anger metaphors suggested by researchers are based on this or similar folk theories. One such metaphor is ANGER IS A HOT FLUID IN A CONTAINER (Kvecses 2000: 21), which is manifested in expressions of anger.2 The following are examples from English.
(1) a. b. You make my blood boil. She is blowing off steam.

This is not, of course, limited to English. In Japanese, we can find examples like the following.
(2) a. atama-kara yuge-ga deru head-from steam-Nom. come-out Steam comes out of (ones) head. harawata-ga niekurikaeru. gut-Nom. boil-up Gut boils up.

b.

268

Kazuko Shinohara and Yoshihiro Matsunaka

This emotion metaphor is observed cross-linguistically. Kvecses (2000, 2002) examines several basic emotions in English, Hungarian, Chinese, and Japanese, and arrives at a single underlying master metaphor: EMOTION IS FORCE. These four languages and cultures are remarkably similar with regard to their basic structure of emotion metaphors (Kvecses 2000: 146), and this similarity is explained in terms of the same physiological processes all humans experience when they are in the state of a certain emotion (Kvecses 2000: 156). At the same time, it is observed that the general metaphor is elaborated in different ways at a more specific level of metaphor in each language. In Kvecses argument of the master metaphor EMOTION IS FORCE, a great deal of attention is paid to the internal pressure in a container, which is assumed to be the source concept of force. Previous studies on Japanese metaphors confirm that Japanese also has such emotion metaphors (Matsuki 1995, Shinohara and Matsunaka 2003).
(3) a. kanashimi-de mune-ga harisakeru. sorrow-by chest-Nom. tear Chest tears with sorrow. yorokobi-de mune-ga ippai-ni natta. joy-by chest-Nom. full-Loc. became Chest was filled up with joy. nikushimi-ga mune-ni uzumaku. hatred-Nom. chest-Loc. billow Hatred billows in chest.

b.

c.

As these examples show, emotions such as sorrow, joy and hatred are thought of as contents in the container (the chest). They either exert a kind of force upon the container, which makes it tear apart or become full, or they stay in the container in an unstable state, having a certain impact on the container. Matsunaka and Shinohara (2001b) call this kind of force the inner force, since the force affects the container from inside. In addition to this, Matsunaka and Shinohara demonstrate that there are at least some examples of metaphors of emotions in Japanese that imply an outer force rather than an inner force.
(4) a. kanashimi-ni uchi-nomes-areru. sorrow-by beat-flat-Passive (Self) is beaten by sorrow. kanashimi-ni mune-ga shime-tsuker-areru. sorrow-by chest-Nom. screw-put-Passive Chest is screwed up by sorrow.

b.

Pictorial metaphors of emotion in Japanese comics c. huan-ga mune-ni noshi-kakaru. anxiety-Nom chest-Loc. make.flat-hang Anxiety hangs over chest. kyouhu-de mune-ga oshi-tsubus-areru. terror-by chest-Nom. push-squash-Passive Chest is squashed by terror. huan-ga mune-ni oshi-yoseru. anxiety-Nom. chest-Loc. push-approach Anxiety inundates chest. kyouhu-ni osow-areru. fear-by attack-Passive (Self) is swept by fear.

269

d.

e.

f.

These examples show that the metaphorical force of emotion can exist outside the self or the part of the body that is regarded as the container of emotion. The force of emotion affects the container from the outside, making its shape change, or even breaking it down. Though previously it was assumed that only very intense emotions are conceptualized as outer forces such as natural forces (Kvecses 2000: 72), it has been found that weak or mild emotions can also be conceptualized in terms of natural or meteorological phenomena in Japanese. The following Japanese examples illustrate that both intense and mild emotions can be expressed in terms of natural or meteorological phenomena (Matsunaka and Shinohara 2001b, Shinohara and Matsunaka 2001).
(5) a. kokoro-ni honokana hikari-ga sasu. heart-Loc. faint light-Nom. shine A weak light brightens heart. (One feels relieved a little bit.) kumo-yuki-ga ayashii. cloud-go-Nom. strange The weather is getting squally. (A person is getting badtempered.) kuro-kumo-ga mune-ni ooi-kabusaru. black-cloud-Nom. chest-Loc. cover-lap A black cloud (=anxiety) covers chest. kimochi-ga harebaresuru. feeling-Nom. become.fine.weather My heart clears up. (I feel very happy.) kokoro-wa doshaburi da. heart-Topic heavy.rain be It rains heavily in (my) heart. (I am very sad.)

b.

c.

d.

e.

270

Kazuko Shinohara and Yoshihiro Matsunaka f. ikari-no tsunami-ga oshi-yoseru. anger-Gen. tsunami-Nom. push-approach Tsunami of anger inundates (self). kokoro-ni arashi-ga hukiareru. heart-Loc. storm-Nom. bluster Storm (of anger) blusters in (my) heart. kaminari-ga ochiru. thunder-Nom. fall Thunder falls. (A person gets very angry and scolds another.)

g.

h.

As the above examples show, the intensity of natural phenomena corresponds to the intensity of the emotion. Severe natural phenomena such as tsunami or storms are mapped onto intense emotions. Less intense natural phenomena like clouds, rain, or sunshine are likely to be mapped onto less intense emotions. In general, anger may be intense and sadness or anxiety may be less intense, but each emotion seems to have a range of intensity. Through analyses of these examples, Shinohara and Matsunaka (2003) suggest the existence of an emotion metaphor that they name EMOTION IS EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF, at least in the Japanese language. Here, the mappings between natural/meteorological phenomena and emotions seem to be experientially motivated. Changes of weather can affect mental or physical states of human beings. For example, depression can be caused by low atmospheric pressure or by lack of sunshine. This is called seasonal affective disorder (NihonSeikishoogakkai 1992: 4243). Moreover, this metaphor may also be attributed to the socio-cultural background of the Japanese. As Yamanaka (2003) argues, Japan has a long tradition of regarding the heart as a microcosm, which appears in many old Japanese poems. In such poems, the outside natural phenomena reflect inner emotions of the poet, or the inner emotions are regarded as natural/meteorological phenomena.
ikadekawa tori-no nakuran hito-shirezu omou-kokoro-wa mada yo-hukaki ni (anonymous) [Why does a morning bird chirp, even though my heart that yearns for you in private is still deep in night?] sabishisa-ya omoi yowaru-to tsuki mireba kokoro-no sora-zo aki hukaku-naru (by Kujou, Yoshitsune) [Feeling sad, in despair, and looking at the moon, my hearts sky is deepened into autumn.]

Pictorial metaphors of emotion in Japanese comics takitsuse-ni hito-no kokoro-o miru-koto-wa mukashi-ni ima-mo kawarazarikeri (by Go-suzaku-in) [It is unchanged in all ages to regard river rapids as ones heart/mind.]

271

These wakas (Japanese old-style poems) date back to the 11th or 12th century. Since that time, our hearts have been thought of as experiencing night, seasons, or landscapes such as rivers. This cultural tradition of Japan may be one of the background factors that motivate EMOTION IS EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF in Japanese. If this emotion metaphor is culture-specific, then it may be due to this type of tradition in Japanese society. However, the mappings between natural/meteorological phenomena and emotions do not seem to be totally arbitrary or unmotivated, but rather, they seem to be supported by the biometeorological tendencies mentioned above. This tendency may not be so universal as the rise of blood pressure and temperature during the experience of anger and thus its physiological motivation for the conceptual metaphor of emotion may perhaps be rather weak. Consequently, metaphors based on this tendency may be culture-specific. To sum up this section, we have argued that (1) the general schema of the anger metaphor ANGER IS A HOT FLUID IN A CONTAINER is applicable to Japanese emotion metaphors; (2) there is an elaborated distinction between inner and outer force in Japanese emotion metaphors; (3) the source domain of natural force is highly elaborated in Japanese emotion metaphors and displays various levels of intensity; and (4) some Japanese emotion metaphors have mappings from natural or meteorological phenomena to emotions, which are basically experientially motivated but allow for the possibility of culture-specificity. As the discussion proceeds, we will examine whether these findings from a cognitive linguistic viewpoint also occur in pictorial/visual metaphors of emotion in Japanese manga.

2.2 Semiotic characteristics of pictorial metaphors In the previous section, we have reviewed the cognitive linguistic analysis of emotion metaphors, which are verbal in nature. In recent developments of the study of pictorial and multimodal metaphors analyses of the expressions of emotion in comics have provided intriguing data and insights. An important study in this line is the one by Forceville (2005). He takes data from a French comic album, La Zizanie, and analyzes the pictorial expressions of anger. The data are classified into two categories. Category I (straightfor-

272

Kazuko Shinohara and Yoshihiro Matsunaka

ward indexical signs) includes bulging eyes, tightly closed eyes, wide mouth, tightly closed mouth, pink/red face, arm/hand position, and shaking. These are indexical signs since we recognize them as symptoms accompanying anger from our everyday experience (Forceville 2005: 77). Category II (pictorial runes, a term coined by Kennedy 1982) includes spirals, ex-mouth, smoke, bold face, and jagged line. These are not perceptible in real life, and their indexicality is therefore less evident than those in Category I (2005: 77). Through the analysis of the collected examples of expressions of anger in La Zizanie, Forceville concludes, the pictorial runes signaling anger appear indeed to be Peircean indices rather than Peircean symbols, since they are motivated rather than arbitrary signs (2005: 82). Here, motivatedness is an important notion. In CMT, it has been repeatedly argued and demonstrated that the most basic linguistic metaphors are motivated in such a way that the motivation can be traced back to our perceptual or bodily experience. One of the theses of cognitive linguistics is that language is embodied and therefore our mind is fundamentally embodied. The above-mentioned study of pictorial metaphors by Forceville and other researchers along this line provide strong support for this view of mind. If we can find additional evidence that demonstrates parallel structures between linguistic metaphors and pictorial and multimodal metaphors, this will strengthen the claim that metaphorical mappings are not merely a matter of language but reside in a deeper layer in our mind, thus enabling manifestations in multimodality. Eerden (this volume) demonstrates that Forcevilles arguments also apply to animated films. This constitutes evidence to support this line of ideas. In our study we aim at the same goal, but use a different kind of data. We will add evidence taken from works of Japanese comics written by several authors.

2.3 Issues and goals What we try to do in this study is two-fold. First, we will analyze pictorial emotion metaphor in Japanese manga using the two categories suggested by Forceville, Category I (indexical signs) and Category II (pictorial runes). Our analysis will basically support Forcevilles (2005: 82) discussion that the pictorial runes signaling anger are Peircean indices rather than Peircean symbols. In addition to this, some of our examples will show how pictorial runes can deviate from indexicality (being related to physical states of a person in anger that actually occur). Some of the visual signs of anger in

Pictorial metaphors of emotion in Japanese comics

273

Japanese manga, which are experientially motivated, can be drawn in physically impossible places (e.g., veins in the air). Such cases of deviation sometimes cannot be verbally expressed, or they become hard to understand if they are directly translated into words. Though they share cross-domain mappings with the verbal expressions of conceptual metaphor ANGER IS A HOT FLUID IN A CONTAINER, these pictorial anger metaphors have a broader range of use than verbal expressions of this metaphor. Thus, we can see that some pictorial metaphors have different and novel ways of representing the conceptual metaphor of anger from verbal ones. Second, a possibly culture-specific Japanese metaphor, EMOTION IS EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF (Shinohara and Matsunaka 2003) is not restricted to verbal metaphors but is also instantiated pictorially in Japanese comics. Thunder, wind, rain, cloud, fog, flowers, birds, and other naturally occurring or existing phenomena are used as the background-scene of panels, and can express emotional states of a person described in a panel. The sources of data that are used in this study are as follows.
(i) Kashimashi: Girl Meets Girl Vol. 1, by Akahori [story] and Katsura [art], 2005. (ii) Azumanga Daiou, Vol.1, by Azuma, 2000. (iii) Azumanga Daiou, Vol. 4, by Azuma, 2002. (iv) Yotsubato, Vol. 5, by Azuma, 2006. (v) Ichigo Mashimaro, Vol. 1, by Barasui, 2003. (vi) Ichigo Mashimaro, Vol. 4, by Barasui, 2005. (vii) Black Jack, Vol. 3, by Tezuka, 1993. (viii) Crayon Shinchan, Vol. 1, by Usui, 1992a. (ix) Crayon Shinchan, Vol. 3, by Usui, 1992b. (x) Crayon Shinchan, Vol. 16, by Usui, 1997.

The first three authors (Akahori, Azuma, and Barasui) have been popular among young people in recent years, and though originally written as comics for girls, their readership extends beyond gender differences. Most of their works selected for this study are about young heroines school and family lives. The last two authors, Tezuka and Usui, and their pieces used in this study, are well known long-selling comic artists for broader age groups in Japan. Usuis works, which describe the daily life of a kindergartener and his parents, are more vulgar and comical than Tezukas serious medical drama.

274

Kazuko Shinohara and Yoshihiro Matsunaka

3. Anger metaphors in Japanese manga 3.1 Category I: Indexical signs As described in 2.2, Forcevilles (2005) Category I is defined as indexical signs, which are recognizable as symptoms that we often perceive in daily life as accompanying the emotion of anger. Bulging eyes, tightly closed eyes, wide mouth, tightly closed mouth, pink/red face, arm/hand position, shaking, and ex-mouth/spit are reported to have been observed in La Zizanie (2005: 77). These are metonymic rather than metaphorical, since we can perceive them when we see an angry person. For example, a red face stands for anger of the person metonymically. So does a shaking body. We find such indexical signs of anger in the Japanese manga we have investigated. Some of them are included among the items that Forceville finds in La Zizanie, and others are not, but they are all indexical in the sense that they are recognizable physical symptoms of anger.

Figure 1. Slanted open eyes representing anger (Yotsubato vol. 5: 80, Kiyohiko Azuma).

A typical pictorial expression of an angry person in Japanese manga is slanted eyes (figure 1) or slanted eyebrows (figure 2). These eyes can be either wide open or narrow/closed. They correspond to Forcevilles bulging eyes and tightly closed eyes respectively, but the angle (slant) seems more noticeable than size, since half-open slanted eyes can also express anger. The slanted shapes of eyes and eyebrows are so expressive that, as figure 1 shows, the detailed parts of eyes like the pupils are sometimes omitted but still the reader can recognize that the person is angry. Slanted eyes and eyebrows are sometimes accompanied by wrinkles near their inner edge or be-

Pictorial metaphors of emotion in Japanese comics

275

tween them, which symbolizes the knitting of eyebrows in anger. Figures 1 and 2 also show wide mouth, as found by Forceville.

Figure 2. Slanted eyebrows, tightly closed eyes, and wide mouth representing anger (Yotsubato vol. 5: 85, Kiyohiko Azuma).

Shrunken or round pupils are also typical pictorial expressions of anger in Japanese comics. In figure 3, shrunken round pupils are used to express the persons anger. A tightly closed mouth and shaking body also appear in this picture, like the ones that Forceville (2005) found in La Zizanie. (The girl in figure 3 has also Y-shaped signs on her forehead and cheek, which will be analyzed in 3.2 as pictorial runes.)

Figure 3. Shrunken pupils, slanted eyes, tightly closed mouth, and shaking body representing anger (Yotsubato vol. 5: 82, Kiyohiko Azuma).

Arm/hand position, as suggested by Forceville, is also expressive of anger. In figure 4, the girl on the left side raises her arm and clenches her fist. Even

276

Kazuko Shinohara and Yoshihiro Matsunaka

though her face is not drawn, readers can easily understand that she is angry. This may be because the final stage of the scenario of the folk theory about anger is that an angry person loses control of herself and vents her anger in the act of retribution (Lakoff 1987: 397398), and the raised hand and fist represent this act, showing that the angry girl is about to hit, or at least threatening to hit, the other girl.

Figure 4. Raised arm/fist representing anger (Azumanga Daiou vol. 1: 112, Kiyohiko Azuma).

The types of pictorial expressions of anger shown in this section are, as mentioned earlier, metonymic in nature. That is, they are perceivable physical states of a person who is angry. They may be, in that sense, not purely metaphorical. However, as figures 13 show, the pictorial expressions are in most cases exaggerated and schematized to some extent. For example, the actual shapes of eyes of an angry person are not like the ones in the figures, but the pictures can invoke the typical physical change in the shape of eyes or eyebrows of an angry person. Thus, what are included in Category I (indexical signs) constitute examples of pictorial metaphor of anger.

3.2 Category II: Pictorial runes The other category examined by Forceville is called pictorial runes. It is Kennedys (1982) label but Forceville uses it in a broader sense. According to Forcevilles definition, pictorial runes are not perceptible in real life, and their indexicality is therefore less evident than those in the first category (2005: 77). Examples found in La Zizanie are spirals, ex-mouth, smoke or fire, bold face, and jagged line. Forceville (2005: 82) explains them as follows.

Pictorial metaphors of emotion in Japanese comics

277

The fan-shape of the spirals surrounding the characters head appears to convey the effect of its risen temperature, its almost bursting with the exertion of either trying to suppress the anger, or with its expression. If the something coming out of the mouth is understood as signaling loud noise, or indeed as a non-physical phenomenon, this is a pictorial rune, which is explicable as the release of pent-up pressure built up within the body-container in the HOT FLUID metaphor. Smoke (in other comics also often fire) is clearly an effect of the heating up of the fluid or gas in the body-container. Bold face and larger fonts [i.e., in the letters in the text balloons] can be seen as equivalent to saying he spoke very, very loudly. The large fonts and bold face, then, cue loudness via a more generic metaphor; and loudness is metonymically associated with (expressed) anger. The angularity of the jagged line connection [i.e., linking the text balloon to the speaking character] is a less obvious cue for anger, but if we characterize it as non-smooth as opposed to the rounded and hence smooth way of connecting balloon to character that is the default, we may hypothesize that it fits in with a whole category of tense behaviors.

In the Japanese manga we have investigated, spirals were not found. Instead, radial straight lines are often drawn as emanating from the front of the entire angry person, as seen in figure 5.

Figure 5. Radial straight lines representing anger (Azumanga Daiou vol. 1: 30, Kiyohiko Azuma).

These straight lines may have the same function as Forcevilles ex-mouth, that is, signaling loud voice or the release of pent-up pressure in the body-

278

Kazuko Shinohara and Yoshihiro Matsunaka

container in the HOT FLUID metaphor (2005: 81). Whether it is temperature or voice, emission of some kind of energy seems to be expressed by these radial lines. Thus, it shares motivations with verbal anger metaphors. Smoke (steam) is also seen in Japanese manga. There is only a little variation in the shape of smoke (steam), as in figures 6, 7, 8.

Figure 6. Steam representing anger (Crayon Shinchan vol. 3: 16, Yoshito Usui).

Figure 7. Steam representing anger (Black Jack vol. 3: 98, Osamu Tezuka).

Figure 8. Steam representing anger (Ichigo Mashimaro vol. 4: 136, Barasui).

Pictorial metaphors of emotion in Japanese comics

279

The small cloud-like shape and the line(s) under it represent that the angry person is blowing off steam from the head. It seems to be steam rather than smoke, since the same shape can be used to express steam coming out of a boiling pan. Indeed, we have a verbal metaphor of anger using the word for steam (example (2a) in Section 2.1, repeated here).
(6) atama-kara yuge-ga deru head-from steam-Nom. come-out Steam comes out of (ones) head.

As often argued in CMT, the use of smoke or steam in this metaphor is motivated by the folk theory that assumes the rising of temperature in an angry persons body. Fire is quite another kind of sign in Japanese manga. It comes out the whole body, as if the persons body is in flames. An example is shown in figure 9.

Figure 9. Fire representing anger (Crayon Shinchan vol. 1: 98, Yoshito Usui).

We have verbal metaphorical expressions corresponding to the concept of fire in Japanese as in (7). Again, the motivation for the use of fire is provided by the folk theory about anger.
(7) Taro-wa ikari-ni moe-teiru Taro-Topic anger-Loc. burn-State Taro is burning with anger.

Figures 2, 5, 8 and 9 include examples of bold face as well. It may represent a loud voice and is often accompanied by an exclamation mark. As Forceville argues, there may be a mediating generic metaphor LOUDER IS BIGGER, and a loud voice may metonymically stand for anger.

280

Kazuko Shinohara and Yoshihiro Matsunaka

As for jagged line, we have a similar kind of signal in Japanese manga, but not exactly the same as the one Forceville presented as an example. In the examples taken from La Zizanie, what is jagged is mainly the connecting line between the balloon and the person, but most cases in Japanese manga, it is the balloon itself that is jagged.3 We can see some examples in figures 2, 5, 7, and 8. Though Forceville (2005: 82) suggests that the jagged line is related to tense behaviors in general, these Japanese cases seem to represent a loud voice or shouting, as radial straight lines seem to do. Since loud voice is not physically jagged, this is a kind of metaphor, and loud voice metonymically stands for anger. We have so far examined the items of pictorial runes suggested by Forceville (2005) and demonstrated that the same or similar kinds of runes are observed in Japanese manga. Though several differences were identified between Forcevilles findings and ours, those differences do not exceed the scope of the conceptual metaphor ANGER IS A HOT FLUID IN A CONTAINER (Kvecses 2000) or ANGER IS THE HEAT OF A FLUID IN A CONTAINER (Lakoff 1987), nor the scenario in the folk theory of anger. However, we find in addition a very peculiar pictorial rune in Japanese manga which is not found in Forcevilles study. This rune has a very simple shape that represents pressurized veins on ones temples. Figure 10 illustrates two variants of this rune.

Figure 10. Two variants of pressurized veins.

This sign is very commonly used in Japanese manga to express anger. The most basic use is to put this sign on a characters temple.4 Physically, when a persons blood pressure is very high, the veins near the temples sometimes stand out. This may be seen with old or middle-aged people, whose facial skin and hypodermic tissues have decayed so that their veins are no longer hidden. Indeed, there is a verbal metaphor in Japanese that refers to ones veins in the brain, as illustrated by example (8). This is an example of ANGER IS A HOT FLUID IN A CONTAINER.
(8) kekkan-ga kire-sou veins-Nom. cut-likely My veins are just about to snap (Im very angry).

Pictorial metaphors of emotion in Japanese comics

281

Being based on the physically occurring states, this sign might be classified as an indexical signal of anger. However, the shape of the sign, especially the cross-shape, does not coincide with the shape of the veins in the temple. Moreover, this sign is indiscriminately used even for young girls or children, whose temple veins are not likely to stand out. Thus, it seems reasonable to classify this sign as a pictorial rune. The sign of pressurized veins can be seen in most of the above figures of manga. However, it may be noticed that the location of the sign varies. This displacement is a very interesting phenomenon that deserves discussion. We will examine it more closely in the next section.

3.3 Deviation of pictorial runes As mentioned in the previous section, one of the most typical and productive pictorial runes of anger in Japanese manga is a sign of pressurized veins. This pictorial rune can be detached from its original (physically occurring and perceivable) place, that is, ones temples, and can be drawn in places that seem to be physically impossible. For example, in figures 3 and 6, it is drawn on the forehead and cheek. In figure 11, the signs are on the hands of the angry mother who is pinching the boys face.5 In figure 12, it appears on the neck (or the lower part of the cheek). In figure 13, four of the people waiting in the queue have this sign on their head but three of them have no detailed face. So it is not certain where the pressurized vein is, but we can easily conclude that they are all angry from the narrative and the utterances.

Figure 11. Displaced sign of anger (Crayon Shinchan vol. 3: 16, Yoshito Usui).

282

Kazuko Shinohara and Yoshihiro Matsunaka

Figure 12. Displaced sign of anger (Crayon Shinchan vol. 3: 16, Yoshito Usui).

Figure 13. Displaced sign of anger (Crayon Shinchan vol. 1: 34, Yoshito Usui).

In all the above cases, the signs of the pressurized veins appear on a persons face, except in the case of hands. However, this sign can also be witnessed in unexpected places. In figures 4 and 8, it is on the back of the head, or on the hair. It is definitely impossible to see pressurized veins standing out on ones hair. At this stage, the sign seems to have been freed from its feature of appearing on a persons skin. In figure 5, it is in the air, just above the head. This indicates that the sign can even become detached from the human body. Finally, in figures 9 and 11, it is in the balloon, beside the words, as if indicating the intonation or tone of voice with which the words are uttered. Thus, even in places other than temples, the sign of pressurized veins can express anger. Though this sign can be traced back to its original status as a metonymy, it seems that it has become a sort of independent sign representing anger and thus can enjoy free displacement and deviation of a kind that is not seen in verbal metaphors. In verbal metaphors, example (8) implies hemorrhage in the brain and nowhere else in the body. To verbally express the location of the sign such as on hands, on the back of the head, in the air, and

Pictorial metaphors of emotion in Japanese comics

283

so forth, would make no sense. It is only in the visual mode that this sign can be displaced. It seems that the free deviation of this pictorial rune is made possible by the visual properties of the medium in which it is used. This suggests that pictorial metaphor is not a mere substitution for or equivalent of verbal metaphor, though these two obviously share experiential motivations and the same cross-conceptual mappings. As Forceville (2006: 281) maintains, an exclusive or predominant concentration on verbal manifestations of metaphor runs the risk of blinding researchers to aspects of metaphor that may typically occur in multimodal representations only. Displacement of the pictorial rune of pressurized veins may be an instantiation of the aspects that Forceville argues. It is a visual manifestation of a conceptual metaphor of anger and its meaning cannot be fully covered by the corresponding verbal metaphor. So far, we have seen that there are the same or similar kinds of pictorial runes in Japanese comics as Forceville finds in the French album La Zizanie. They are all compatible with the verbal expression of the ANGER IS A HOT FLUID IN A CONTAINER metaphor or its background folk theory and the scenario of anger events. The pictorial runes are not straightforward expressions of physically perceivable states of an angry person, so they are not straightforward indexical signs in Peircean semiotic theory. However, as Forceville (2005: 82) argues, they are not symbols in Peircean sense either, since they are not arbitrary but are motivated. Considering the importance of the notion of motivation in the study of metaphor, we agree with Forceville in maintaining that pictorial runes should be classified as indexical signs. For this discussion, Forceville provides examples from a French comics album. The present study provides further supports for this claim by giving evidence from Japanese comics. Motivated pictorial runes are not confined to French, but are realized in at least two language communities.

3.4 Atmosphere and natural phenomena As mentioned in 2.1, Japanese emotion metaphors have an elaborated distinction between inner and outer force, and there is a conceptual mapping we call EMOTION IS EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF. In this conceptual mapping, the metaphorical force of emotion is conceptualized as being outside the self or the body-part that is regarded as the container of emotion. Not only intense emotions like anger, but also weak or mild emotion types such as happiness, sadness, disappointment, or anxiety can be conceptualized in terms of natural or mete-

284

Kazuko Shinohara and Yoshihiro Matsunaka

orological phenomena. To our knowledge, English does not have as rich a repertoire of this kind of highly elaborated mappings between natural phenomena and emotional states of a person as Japanese does.6 As Matsunaka and Shinohara (2003) argue, it seems that the rich realization of this emotion metaphor in Japanese is motivated by the traditional Japanese view that the heart is a microcosm and the outside natural phenomena reflect inner emotions, or inner emotions are regarded as natural/meteorological phenomena (see 2.1). Instantiations of this possibly culture-specific conceptual metaphor are not restricted to verbal ones; we also find them in pictorial expressions. In Japanese manga, meteorological phenomena such as thunder, wind, rain, cloud, or fog, and some life forms in nature like flowers or birds can be used to express a persons emotional state. They are drawn as the background scene of a panel, by which readers can recognize how the person in that panel is feeling. The following figures illustrate some of such uses.

Figure 14. Cloud or fog representing anger (Yotsubato vol. 5:79, Kiyohiko Azuma).

Figure 15. Thunder representing anger (Yotsubato vol. 5: 84, Kiyohiko Azuma).

Pictorial metaphors of emotion in Japanese comics

285

Figure 16. Thunder, wind, and rain representing anger (Crayon Shinchan vol. 1: 100, Yoshito Usui).

In the background scene of figure 14, a cloud or fog-like substance is described. Since the setting of this scene is in the living room at sunny lunchtime, this cloud or fog cannot be a physical occurrence. It expresses the girls anger, which is getting more and more intense. In figures 15 and 16, thunder appears, which expresses that the angry person is releasing his/her anger, and in typical cases, shouting out. Thunder may be associated with emission of intense energy, perhaps with a loud noise, which may be metaphorically mapped onto the loudness of the persons shouting voice. Figure 16 has wind and rain besides thunder. The slant lines and the onomatopoeia written on the left side (which reads Gooooo) represent strong wind. The lines may also represent rain, together with the drops between them. They all combine to represent a storm. Since there is no actual storm in this scene, the storm is here a metaphorical expression of the persons emotion. The image of storm may be associated with intense energy and destructive power that will affect the person nearby. It may be noticed that this storm is drawn only around the mother, not around the boy. This clearly means that the mother is very angry but the boy does not care at all. The effect of the storm does not reach the boy. In this way, visual representation of meteorological phenomena can be used very effectively and expressively in manga. So far, we have observed meteorological or natural phenomena in pictorial metaphors that express the emotion of anger. These kinds of pictorial metaphors are, however, not limited to anger. Other types of emotions like love, happiness, surprise, disappointment, or anxiety can also be expressed by meteorological or natural phenomena that are not physically present. The following are some examples.

286

Kazuko Shinohara and Yoshihiro Matsunaka

Figure 17. Lightning representing surprise (Ichigo Mashimaro vol. 4: 58, Barasui).

Figure 18. Darkness representing disappointment/anxiety (Azumanga Daiou vol. 4: 137, Kiyohiko Azuma).

Figure 19. Birds representing joy/happiness (Azumanga Daiou vol. 4: 158, Kiyohiko Azuma).

Pictorial metaphors of emotion in Japanese comics

287

Figure 20. Flowers representing joy/happiness (Ichigo Mashimaro vol. 1: 51, Barasui).

Figure 21. Flowers representing love (Kashimashi vol.1: 152, Yuukimaru Katsura).

Figure 22. Flowers representing love (Crayon Shinchan vol. 16: 12, Yoshito Usui).

288

Kazuko Shinohara and Yoshihiro Matsunaka

Figure 23. Dropping petal of a flower representing disappointment (Ichigo Mashimaro vol. 1: 42, Barasui).

In figure 17, two girls are holding their heads in surprise or shock. This emotional state is emphasized by the background lightning. The motivation behind this pictorial metaphor may be that the impact of surprise or shock on ones heart is similar to that of lightning on ones eyes, in temporal aspect (abrupt, unexpected occurrence), and its intensity. As a more lasting and static environment, darkness is expressed by narrow vertical lines in figure 18. This is expressive of the persons emotional state of disappointment or anxiety. In this scene, the girls are talking about their exams, in which they have no confidence. The relation between disappointment/anxiety and darkness is observed also in verbal metaphors. That these negative emotions are experienced as dark may be experientially motivated by a biometeorological tendency, as mentioned in 2.1, or just by what we tend to feel in actual darkness. Figures 1923 differ from figures 17 and 18 in that they use natural life forms instead of meteorological phenomena to express emotional states. In figure 19, the girls are happy and full of joy, and this is symbolized by the birds surrounding them. Since this is an outdoor scene, the presence of birds is not totally implausible. This is an example of what Forceville (2006) calls a realistically motivated metaphor. Consistent with what he argues, the

Pictorial metaphors of emotion in Japanese comics

289

metaphor is subtler in this case than the case of thunder, whose presence is very unlikely. Since birds can fly in the air, the conceptual mapping HAPPY IS UP may motivate this use of birds. Figure 20 represents a girl drinking beer after taking a bath. She is very relaxed and happy. This state of emotion is expressed by surrounding flowers. The relationship between happiness and flowers is obvious, since it is a typical reaction for us to feel relaxed and happy when we see flowers. Thus, this use of flowers is experientially motivated. In figures 21 and 22, flowers represent love. This is also observed in verbal metaphor in Japanese, in which a rich vocabulary about flowers is associated with love. This may hold in other languages as well. Figure 23 has a more complex structure. This pictorial metaphor consists of two successive panels. In the first panel, the flower behind the girl has a complete shape. However, after she hears what other persons say, which is shown in the two balloons, one petal of the flower drops, as in the second panel. This change of state of the flower represents the girls emotional state, that is, how she became disappointed when she heard those utterances. The petal dropping indicates incompleteness of the flower, deficiency, or lack of something. Since flower represents a happy state, this deficiency means destroyed happiness. It is, however, not a severely destructive experience because the flower is still a flower even when it has lost one of the petals. Thus, this pictorial metaphor is highly expressive of such dynamic state of ones emotion, and it illustrates that dynamic or complex use of visual signs is possible in this kind of media. To sum up, meteorological or natural phenomena used in pictorial emotion metaphors we have examined are experientially motivated. Flowers can represent happiness or love, but they do not represent anxiety or shock. What makes pictorial metaphors of this kind comprehensible and interpretable resides in the motivation or grounding that supports their mappings. Considering these properties, pictorial metaphors in Japanese manga should be classified as Peircean indices rather than Peircean symbols. However, it should be noted that the use of meteorological or natural phenomena to express emotional states may be, at least to some extent, culture-specific. As already mentioned, to regard natural scenes as reflections of ones internal or emotional states may be an aspect of Japanese sociocultural tradition which may not be shared by other cultural groups. We have argued elsewhere that experiential motivation does not imply universality. Even when a conceptual mapping has physical motivation, it could still be culture-specific. Pictorial runes of meteorological or natural phenomena

290

Kazuko Shinohara and Yoshihiro Matsunaka

in multimodal emotion metaphors may constitute an instantiation of this argument. 4. Conclusion In this chapter, we have examined some works of Japanese manga to demonstrate that there are pictorial metaphors that are motivated by the same conceptual mappings as verbal metaphors. We have shown that the conceptual metaphor ANGER IS A HOT FLUID IN A CONTAINER is shared by verbal and visual modalities. Readers/viewers of the pictorial expressions of anger such as the ones discussed in this chapter require no verbal information, either in the form of talk by a character in the panel, or in the surrounding narrative context, to assess the emotion depicted. That is, readers know what kind of emotion is expressed from only pictorial signs of the types we have presented. (Emotions expressed by meteorological/natural phenomena are sometimes ambiguous and thus need to be aided by facial expressions of a character, though these do not require verbal information either.) The parallel structures between verbal and visual metaphors indicate that metaphors are not merely a surface linguistic phenomenon but rooted more deeply, structuring our cognition and concepts. However, we have also observed that verbal and visual metaphors are not completely equivalent. Some examples of the pictorial metaphors are specific to the visual mode. Even though basic conceptual mappings are shared by both modalities, it seems that differences can emerge due to the different properties of each modality. Moreover, culture-specificity may be observed in both verbal and visual metaphors. By examining EMOTION IS EXTERNAL METEOROLOGICAL/ NATURAL PHENOMENON THAT SURROUNDS THE SELF, we have argued that culture-specific aspects found in verbal emotion metaphors may also be found in pictorial emotion metaphors. This point, however, needs more investigation. Researchers of multimodal metaphors, especially of other languages and cultures, are invited to this promising field. Notes
* We would like to acknowledge Charles Forceville and Eduardo Urios-Aparisi for their constructive comments. We are also indebted to Kotaro Yamamoto for his role in collecting many of the panels we used as the data.

Pictorial metaphors of emotion in Japanese comics 1.

291

2.

3. 4.

5.

6.

Discussions on anger metaphor within the field of cognitive linguistics can be found e.g., in Geeraerts and Grondelaers (1995), Gevaert (2002), Kotze (2005). In this metaphor, fluid is mainly assumed to be blood, as Kvecses (2000: 159) says [I]t is reasonable to assume that it is mainly blood (but perhaps some other body fluids as well) that accounts for the fluid component in the container metaphors. Other than blood, qi (gas) in Chinese (Kvecses 2000, Yu 1998) and gastric contents in Japanese (Matsunaka and Shinohara 2001a) have been suggested as instantiations of the fluid. It is suggested that jagged balloons sometimes appear in also European comics as well (Forceville, personal communication). As McCloud (2006: 97) points out, this sign is one of the symbolic expressions which vary from culture to culture, though it has found its way to English language comics. The veins on ones hands may stand out when he/she strains the muscles of the hands. In that case, this example may be an indexical sign of physically occurring states. There are a few similar cases of this kind in English (Kvecses 2000: 71), but they are confined to strong emotions.

References
Akahori, Satoru, and Yuukimaru Katsura 2005 Kashimashi [clamorous]: Girl Meets Girl. Vol. 1. Tokyo: Media Works. Azuma, Kiyohiko 2000 Azumanga Daiou [Azumanga the Great]. Vol. 1. Tokyo: Media Works. 2002 Azumanga Daiou [Azumanga the Great]. Vol. 4. Tokyo: Media Works. 2006 Yotsubato [With Yotsuba]. Vol. 5. Tokyo: Media Works. Barasui 2003 Ichigo Mashimaro [Strawberry Marshmallow]. Vol. 1. Tokyo: Media Works. 2005 Ichigo Mashimaro [Strawberry Marshmallow]. Vol.4. Tokyo: Media Works. Carroll, Noel 1994 Visual metaphor. In Aspects of Metaphor, Jaakko Hintikka (ed.), 189218. Dordrecht: Kluwer. Forceville, Charles 1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Activity 9: 129.

292

Kazuko Shinohara and Yoshihiro Matsunaka

Pictorial Metaphor in Advertising. London/New York: Routledge. The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 179198. 2005 Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2006/this vol. Non-verbal and multimodal metaphors in a cognitivist framework: agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. Geeraerts, Dirk, and Stefan Grondelaers 1995 Looking back at anger: Cultural traditions and metaphorical patterns. In Language and the Cognitive Construal of the World, John Taylor and Robert MacLaury (eds.), 153179. Berlin/New York: Mouton de Gruyter. Gevaert, Caroline 2002 The evolution of the lexical and conceptual field of ANGER in Old and Middle English. In A Changing World of Words: Diachronic Approaches to English Lexicology and Semantics, Javier E. Daz Vera (ed.), 275299. Amsterdam: Rodopi. Grady, Joseph 1997 Foundation of Meaning: Primary Metaphors and Primary Scenes. Ph.D. diss., University of California at Berkeley. Kotze, Zacharias 2005 Humoral theory as motivation for anger metaphors in the Hebrew Bible. Southern African Linguistics and Applied Language Studies 23 (2): 205209. Kvecses, Zoltn 2000 Metaphor and Emotion. Cambridge: Cambridge University Press. 2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press. 2005 Metaphor in Culture. Cambridge: Cambridge University Press. Lakoff, George 1987 Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago: University of Chicago Press. 1990 The Invariance Hypothesis: Is abstract reason based on imageschemas? Cognitive Linguistics 1: 3974. 1993 The contemporary theory of metaphor. In Metaphor and Thought. 2nd ed. Andrew Ortony (ed.), 202251. Cambridge: Cambridge University Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press.

1996 1999

Pictorial metaphors of emotion in Japanese comics 1999

293

Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books. Matsuki, Keiko 1995 Metaphors of anger in Japanese. In Language and the Cognitive Construal of the World, John Taylor and Robert MacLaury (eds.), 137151. London: Methuen. Matsunaka, Yoshihiro, and Kazuko Shinohara 2001a ANGER IS GASTRIC CONTENTS: Japanese anger metaphor revisited. Paper presented at 4th International Conference on Researching and Applying Metaphor, Tunis, Tunisia. 2001b Emotion and outer force schema: A perspective from Japanese. Paper presented at The First Seoul International Conference on Discourse and Cognitive Linguistics, Seoul, Korea. 2003 Clouds and sunshine in mind: Meteorology-based Japanese emotion metaphors. Paper presented at 5th International Conference on Researching and Applying Metaphor, Paris, France. McCloud, Scott 2006 Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels. New York: HarperCollins Publishers. Nihon-Seikishoogakkai [Japanese Society of Biometeorology] 1992 Seikishougaku no Jiten [Encyclopedia of Biometeorology]. Tokyo: Asakura Shoten. Shinohara, Kazuko, and Yoshihiro Matsunaka 2001 Is emotion really force? Japanese metaphor of sorrow. Paper presented at The Seventh International Cognitive Linguistics Conference, Santa Barbara CA, USA. 2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen [language and human being]: Journal of Yokohama Linguistic Circle 4: 118. Tezuka, Osamu 1993 Black Jack. Vol. 3. Tokyo: Akita Shoten. Usui, Yoshito 1992a Crayon Shinchan. Vol. 1. Tokyo: Futabasha. 1992b Crayon Shinchan. Vol. 3. Tokyo: Futabasha. 1997 Crayon Shinchan. Vol. 16. Tokyo: Futabasha. Yamanaka, Keiichi 2003 Waka no Shigaku [The Poetics of Classical Japanese Poetry]. Tokyo: Taishuukan Shoten. Yu, Ning 1998 The Contemporary Theory of Metaphor: A Perspective from Chinese. Amsterdam/Philadelphia: Benjamins.

V Metaphor in Spoken Language and Co-Speech Gesture

Chapter 13 Words, gestures, and beyond: Forms of multimodal metaphor in the use of spoken language Cornelia Mller and Alan Cienki

Abstract
This chapter offers a systematic account of the forms that mono- and multimodal metaphors may take in face-to-face communication. The account is based on the relation of source and target domains expressed either in one modality only (thus forming a monomodal metaphor) or in two modalities (forming a multimodal metaphor). We will then illustrate the inherent dynamic nature of metaphors when used in spoken interaction, pointing out more specifically how metaphors are being elaborated within and across modalities. We will focus particularly on metaphors that are realized in speech and/or gesture, but point out the relevance of studying metaphors in other articulatory forms such as stress and intonation. The different forms of multimodal metaphors are systematically based on different relations between metaphoric and gestural expressions. Finally, implications for metaphor theory and for the dynamic aspects of thinking for speaking are discussed, suggesting that multimodal metaphors in spoken language are products of the process of creating metaphoricity (by a speaker/gesturer and ideally also by a listener/perceiver), which is essentially independent of modality and expressive form. Keywords: Activation of metaphoricity, gestures, gestural metaphors, thinking for speaking and gesturing, multimodal metaphor, monomodal metaphors, verbal metaphors, verbo-gestural metaphors

1. Introduction The situation which has been most influential for the form that spoken languages have is arguably the face-to-face encounter. We take it as a scenario that has been described by sociologists, social psychologists, anthropologists and linguists, and which rather unanimously has been characterized as a communicative situation that is inherently multimodal.

298

Cornelia Mller and Alan Cienki

Adam Kendon, inspired by Goffman (1967), was interested in the interactional aspects of this situation and he has devoted many of his early studies to finding responses to Erving Goffmans (1967) call for a study of the ultimate behavioral materials of interaction. [] That is, [] the glances, gestures, positions and verbal statements that constitute the stuff of face-toface encounters (Kendon 1990: ix). David Efron, one of the pioneers of gesture studies, directed attention to the fact that the hand gestures people seem to use unwittingly and very regularly when they converse with each other are so deeply intertwined with spoken language that in his empirical investigation of cross-cultural differences between Jewish and Italian immigrants to New York City he distinguishes spatio-temporal aspects, i.e., gesture simply considered as movement, from their referential aspects, i.e., gesture envisaged as language (Efron 1972: 67, emphasis in original). Herb Clark, in turn, has described the face-to-face situation of communication as the canonical encounter of human beings:
From the social psychologists viewpoint, man is a social animal, who enjoys, perhaps even needs, to interact socially with other people. What are the characteristics of the most usual interaction between two people, John and Mary? For our purposes, the most important property is that they will be facing each other a short distance apart. It is in this position that John and Mary are situated for the optimal perception of messages both verbal and nonverbal from the other person. John is in Marys positive perceptual field, and Mary is in Johns. If John and Mary were side by side, or back to back, or back to front, or in any other position, these conditions would no longer be optimal. It is no accident that normal conversations are carried out face-to-face. This face-to-face situation is what I would like to refer to, for convenience, as the canonical encounter (Clark 1973: 3435).

Granting that the canonical encounter as described above does not imply that co-participants literally face each other,1 it is a form of interaction which is extremely common in the cultures of many areas of the world and which does imply reciprocal audibility and visibility (although to varying degrees). This face-to-face encounter between two people will be the context in which we will treat the use of spoken language in this chapter. As Clark indicates, the use of spoken language in this context is inherently a process of multimodal communication, involving not only the oral production of sound and its aural reception, but also the production of various kinds of bodily motion in space, which the addressee can perceive visually. The multimodal nature of spoken communication has been especially emphasized in recent years by those researching spontaneous gesture with speech, suggesting that gesture

Words, gestures, and beyond

299

and speech are visible and audible actions that form one single utterance (e.g., Kendon 2004) or proposing that gesture and speech are dynamically based in different forms of thought but constitute one integrated system (e.g., McNeill 1992, 2005). Given that spoken language involves multiple modalities, it makes sense that metaphor should have the potential for multimodality when used in this form of communication; and indeed over the past years quite a substantial body of research on metaphor in gesture, speech, and sign language has been carried out (cf. Bouvet 1997, 2001; Calbris 1998, 2000, 2003; Cienki 1998, 2005b; Cienki and Mller 2008a, 2008b; McNeill 1992; Mittelberg 2006; Mittelberg and Waugh, this volume; Mller 1998, 2008; Nez 2004; Nez and Sweetser 2006; Webb 1996; Wilcox 2000, 2004). The topic we want to explore here in particular is the different forms that multimodal metaphors may take in face-to-face communication. We will specifically concentrate on the kinds of relations between metaphors that are realized in speech and/or gesture. It is not by accident that the study of metaphor is increasingly taking data from gesture studies into account (e.g., Cienki and Mller 2008a, 2008b; Mller 2008) and this chapter offers a systematic account of the forms of metaphors that occur either in speech or in gesture or in both modalities at the same time. However, we would also like to point out that beyond gesture, there are additional properties of spoken communication which have received much less or no attention in terms of their implications for the expression of metaphor, including prosodic features, such as stress and intonation, and the time course in which all of these expressive forms are used during acts of speaking (for the latter point see Mller 2007, 2008). In order to clarify what we are discussing, we will restrict the term modality to two dimensions of face-to-face communication: one will refer to what is expressed orally and perceived primarily aurally as sound (the oral/aural modality), and the other will refer to bodily forms and movements in space which are primarily perceived visually (the spatial/visual modality). In this sense, we will see that gesture/word combinations can constitute multimodal metaphors. Within each modality, there are various forms which can be used for expressive purposes. In the oral/aural modality, intonation and stress can be discussed separately from each other and separately from the words being articulated. We will refer to these as different articulatory forms within this modality. Similarly within the spatial/visual modality, eye gaze, body shifts, manual gestures, etc., can all be considered different expressive forms. Our understanding of articulatory form partially overlaps with Forcevilles use of the term mode. In his critical stance towards giving a satisfactory definition of mode or of compiling an exhaustive list of modes,

300

Cornelia Mller and Alan Cienki

Forceville argues that this fundamental difficulty is no obstacle for postulating that there are different modes and that these include, at least, the following: (1) pictorial signs; (2) written signs; (3) spoken signs; (4) gestures; (5) sounds; (6) music; (7) smells; (8) tastes; (9) touch (Forceville 2006: 3823/this volume). In short, spoken words and gestures are articulatory forms or modes, which are realized in an aural/oral or spatial/visual modality. After presenting an overview of what appear to be the most common ways in which the use of metaphor can play out in the oral/aural and spatial/visual modalities and articulatory forms, we will point out the inherent dynamic nature of metaphors when used in spoken interaction. Eventually we will suggest that these observations indicate that multimodal metaphors are products of the process of creating metaphoricity (by a speaker/gesturer and ideally also by a listener/perceiver), which is essentially independent of modality and articulatory form, if metaphoricity is a matter of understanding one idea (or domain) in terms of another. However, we will also argue that the different modalities and forms that are involved in spoken interaction afford the use of different expressions for metaphors. What one can express via a given modality and expressive form will have an effect on what one will express using that modality. We will conclude by considering the implications this has for how we can think with metaphors while we are speaking, or attending to someone who is speaking. 2. Monomodal and multimodal metaphors in words and gestures To begin with we need to clarify our understanding of mono- and multimodal metaphors. Following Forceville (2006: 383), we will consider as monomodal those metaphors whose target and source are exclusively or predominantly rendered in one mode. This means we will distinguish monomodal verbal metaphors from monomodal gestural metaphors. We will consider as multimodal those metaphors whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006: 384), and for the present chapter this means that we will document and discuss verbogestural metaphors. In fact, and even more precisely, this means that we are actually talking about verbal and gestural, or verbo-gestural metaphoric expressions, since the phenomenological level we are concerned with in our analysis is that of verbal, gestural, or verbo-gestural utterances. If, and if so how, these expressions relate to a general level of conceptual metaphors (such as LOVE IS A JOURNEY) remains unaddressed. We do, however, con-

Words, gestures, and beyond

301

sider the meaning of those metaphoric expressions be they verbal, gestural, or a combination of both to be conceptual (adhering to the cognitive linguistic assumption that meaning in general is conceptual). Thus for the sake of brevity only we will henceforth use the terms metaphor and metaphoric expressions as synonyms in this chapter, unless otherwise indicated. Second it is central to illustrate what we mean by gesture. Here we will be focusing on visible, effortful movements of parts of the body whose primary purpose is apparently not that of self-adjustment (for example, as with grooming behavior) or object manipulation (such as lifting a cup to take a drink). One could focus on many different parts of the body, such as head gestures, eye movements, foot gestures, body shifts, shoulder shrugs, and so on. Here we will focus on positions, orientations, and movements of the hands and forearms; these are what we will mean henceforth with the term gesture, unless specified otherwise. Which gestures will be considered metaphoric? Here we will restrict the discussion to gestures whose primary function can be identified as abstract reference. Mller (1998: 1101) notes that referential gestures can refer either to physical objects, properties, actions, or relations, or to abstract notions in terms of such physical means. Thus the same two-handed gesture with thumb and index finger on each hand forming a 90 degree angle could be used when talking about a picture frame or when describing the framework of a theory. Abstract referential gestures are inherently metaphoric by virtue of rendering a non-physical idea in terms of a physical, spatiotemporal representation. We can note that a concrete referential gesture can also be metaphoric in certain contexts (e.g., when someone imitates an animal referring to a person in a derogatory way), but we will not focus on such usage here, as we have found it extremely rare in our research to date. We should also mention our criteria for identifying verbal expressions as metaphoric. For this we rely on the procedure developed by the Pragglejaz Group (2007), with language-specific adaptations, as appropriate. To put it briefly, the procedure has been designed for the identification of (a) when a word is being used in a given context with a meaning which is different from another physically more basic meaning that it may have, and (b) when the contextual meaning is interpreted via comparison with the more basic meaning. It is a maximally inclusive procedure, intended to identify words which may even potentially be understood metaphorically in the given context of use. For a detailed account of the procedure see Pragglejaz Group (2007). In the examples that follow, we will indicate words so identified via this procedure with underlining. The examples below are from our qualitative analyses of videorecorded conversations from four different languages: American

302

Cornelia Mller and Alan Cienki

English, German, Cuban Spanish, and Russian. The conversations in each language were elicited from pairs of native speakers, and were on abstract topics so as to increase the likelihood of use of metaphoric expressions. The English and Russian conversations were between pairs of university students who were talking about how they take exams at their universities (in the United States and Russia, respectively). The German data come from middle-aged German women who were talking with the researcher (the first author) about their first love relationship. The Cuban data were all recorded in Havana. Participants were asked to give accounts of important events in their lives (a wedding) or to talk about life in Cuba in general. They were roughly of the same age (in their thirties, with one exception of a speaker who was 54 years old), and mixed with regard to gender. Each example cited here comes from a different conversation, unless noted otherwise. Each line in the transcript indicates a new intonation unit (see Chafe 1994 on intonation units as units of analysis for spoken discourse). A comma (,) indicates an intonation unit with a terminal pitch that signals continuation, a period/full stop (.) marks a final intonation unit falling to a low pitch at the end, and a double dash (--) shows that the intonation unit was truncated. A longer pause by the given speaker is indicated with three dots (), and a shorter one with two (..). In the transcription of the examples, each separate gesture is indicated by a number (G1, G2, etc.); hands involved in gesturing are noted as rh, lh, bh, indicating right, left or both hands; the onset of gesture notation is synchronized in the transcript with the beginning of the gestural movement and bold face indicates the entire duration of gestural movement (preparation, stroke, retraction, cf. Kendon 2004; McNeill 1992). Building on the possible relations that Cienki (1998) has found between metaphors expressed in words and in gestures and on Mllers (2008) discussion of different realms of metaphor, we will now give an overview of what is known about the manners in which metaphor may be expressed in speech in real time, that is: in one modality (spoken words or gestures) or in a combination of the two. Put differently, we will offer a sketch of what appear to be common forms of mono- and multimodal metaphors in words and/or gestures. 3. Monomodal metaphors: source and target within one modality First, we can confirm that we often find the use of metaphoric verbal expressions without co-occurring metaphoric gestures. For example, one American

Words, gestures, and beyond

303

student talks about how people may verbalize certain beliefs about honesty, but sometimes might not behave in accordance with them. At this point she says (Example 1): Example 1:
Just because of the pressure, the peer pressure,

The word pressure was coded as metaphoric in this context because peer pressure normally involves behaviors other than physical pressing the more basic physical meaning of the word. Even if peer pressure involved physical contact, the word still can be understood with the abstract sense of coercive behavior (it has potential metaphoricity). Although the word was coded as metaphorically used, the speaker made no gestures while saying it either time, keeping her hands resting on her leg as she was sitting. We might refer to such a use of metaphor purely on the verbal level as monomodal metaphor or as verbal metaphoric expression. We also find the converse monomodal pattern of metaphor use: metaphors expressed in gestures without metaphors in the co-occurring speech, that is, gestural metaphoric expressions that are used concurrently with speech. Example 2 comes from one of the Russian students talking about how they take exams at their university. An English translation is provided below the transliterated transcript of the Russian. The student is trying to characterize the Russian concept of chestnost, which may be translated as honesty. Example 2 (from Russian):
Dlia menia chestnost eto nekaia absoliutnaia kategoriia. For me chestnost is a kind of absolute category. G1 preparation bh raised in front of torso, flat in vertical plane, fingers pointing out Kogda vot iest situatsiia, When theres this situation, G1 stroke bh move straight downward slightly seichas postupit chestno tak. then [you need] to act honestly like this.

The student begins seated, hands at rest in his lap, and starting when he says situatsiia [= situation], he lifts his two hands in front of his torso, the right

304

Cornelia Mller and Alan Cienki

hand somewhat higher than the left, palms and fingers flat in the vertical plane, fingers pointing forward. He holds his hands there until he gets to the word tak [= like this], at which point he moves them both slightly downward in unison, maintaining their position relative to each other and flat in the vertical plane. Here the speaker explicitly uses the gesture to make reference if you dont see the gesture, you dont know what he thinks it means to act honestly (chestno). The speaker uses a verbal deictic particle to direct the attention of his co-participant to the gesture: like this points to the gesture, and his hands move in temporal coordination with the verbal deictic. Thus verbally he makes clear that the gesture contains relevant information, but there is no verbal mention of a metaphoric source. In this instance, manner of behavior (honest) is expressed gesturally as a physical form (flat/straight) with a certain motion (brief and straight). (See Cienki 1999 for further discussion of this and related examples.) Note that this is a particularly interesting case, since not only are the gestures source and target independent from any verbal metaphoric expression (in fact there is none in the concurrent speech), but the gestural metaphoric expression is used in place of words. Thus we might speculate whether this is an instance of a multimodal utterance consisting of a monomodal gestural metaphoric expression which is being inserted into a verbal utterance. Another type of gestural metaphoric expression that is very common among different cultures involves gestures which perform a speech-act or more generally a communicative activity. These are gestures that recur in form and function over a large amount of contexts and therefore we term them recurrent gestures (cf. Bressem and Ladewig in prep.; Ladewig in prep.; Teendorf in prep. a, b). Examples are the palm-up-open-hand gesture (cf. Kendon 2004; Mller 2004; Streeck 1994), the ring gesture (Fatfouta in prep.; Kendon 2004; Morris 1977; Neumann 2004), or the brushing aside gesture (Mller and Speckmann 2002; Speckmann 1999; Teendorf in prep. a, b). These gestures all share a common origin, in that they are all metonymic derivations of everyday actions (cf. Mittelberg 2006; Mittelberg and Waugh, this volume; Mittelberg and Mller in prep.; Mller 1998, in prep. a, b; Mller and Haferland 1997; Streeck 1994): presenting, offering or receiving something (the palm-up-open-hand gesture); picking up small objects with the index finger and thumb (the ring gesture); or brushing aside small objects. What we observe in these gestures is a two-step semiotic process as identified and described by Mittelberg and Waugh (this volume), in which the metonymic target of the sign-formation process turns into the source of the metaphoric gesture (see also Mittelberg 2007). The targets of the metonymic process in our cases are the modulated actions: i.e., part of

Words, gestures, and beyond

305

the action stands for the action as a whole, thus consituting a classical instance of a synecdochic relation (Mller in prep. a, b). This modulated action is used now as a metaphoric source for symbolizing abstract issues such as presenting a discourse object on the palm-up-open-hand, indicating the preciseness of arguments, or brushing aside unpleasant topics. For instance the brushing aside gesture is widely used to express negative assessments, and this is what we will see in the next example. Example 3 comes from free conversations recorded in Cuba (Mller and Speckmann 2002; Speckmann 1999). (For a detailed analysis of the brushing aside gesture used by speakers of the Iberian Peninsula, see Teendorf in prep. a, b). In Example 3 the speaker thinks out loud about the possible consequences of what it would be like to have four instead of two TV-channels in Cuba. He is convinced that this would disturb family life by causing endless discussions about which program to watch. He describes the big arguments this would raise in a very lively way, and in doing this becomes himself part of such an imaginary situation: he imagines himself standing in the living room, and he indicates three different places, each of them relating to a different person voting emphatically for another program: Yo quiero vel aquello yo quiero ver lo otro yo quiero ver esto (I want to see this one, I want to see that one, I want to see the other one). The more programs to choose from, the more arguments you have in your family this is the moral of the speakers imagined scenario. It is clearly not desirable to have four channels on Cuban TV and correspondingly he concludes his discussion with a negative assessment performed gesturally as a brushing aside gesture. Example 3 (from Cuban Spanish):
G1 rh point to the right yo quiero vel aquello, I want to see that one G2 lh points straight yo quiero ver lo otro, I want to see the other G3 G4 rh points straight rh brushes aside yo quiero ver esto (.) I want to see this one (.)

306

Cornelia Mller and Alan Cienki

The first three gestures the speaker uses relate to the propositional content of the utterance; G1, G2, and G3 point to three different places in the imagined apartment, localizing three different persons with three different wishes. The brushing aside gesture (G4) is located in a micro-pause at the end of this utterance and clearly assesses this imagined situation as an undesirable one. There is no verbalization of a negative assessment; the gesture takes over the entire communicative burden. It is noteworthy that the gesture is placed at the end of the phrasal unit, precisely where a verbal evaluative particle could have been placed. Instead the speaker pauses and produces a gesture with similar content. It seems as if the brushing aside gesture would do the same job as a verbal particle would (which is why Mller and Speckmann, suggested the term gestural particle). It gives a negative evaluation of a situation being described, and the gestural meaning is derived from the negative connotation of the practical action. (For a detailed cognitive semiotic analysis of this process, see Teendorf in prep. a, b.) What the brushing aside gesture shares with the other recurrent gestures mentioned before is that it has a performative or (more general pragmatic) function rather than a referential one, and it is obvious that metaphor plays a different role here than in example 2. In the second example the communicative function is metaphorical reference, whereas in example 3 the gestures function is the performance of a communicative action. Hence the first use of the metaphoric gestures belongs to the realm of semantics while the other one belongs to pragmatics. The difference is not a simple matter, but for the sake of brevity it might be characterized as a difference between gestures contributing information to the propositional content of the utterance and gestures contributing meta-communicative information. While in the second example the metaphoric gestures expressed aspects of the propositional content (honesty as a physical form and movement) in the third example the metaphoric gestures are used for meta-communicative purposes (they qualify the propositional content), telling us how the propositional content (the choice of various programs) is being assessed by the speaker. Thus while metaphor as well as metonymy are clearly involved in these gestures, they come in at the level of the semiotic process of sign formation rather than on the level of communicative function. We may conclude that monomodal metaphors are frequent in words but they also can be found in gestures. As for gestural metaphoric expressions we have found two different kinds: on the one hand there seem to be gestures that are more likely to be created on the spot (such as example 2, honesty with a straight gesture), and others that appear to recur with a relatively stable form and function (the brushing aside case, example 3). These two

Words, gestures, and beyond

307

kinds of gestural metaphors furthermore seem to fall into two different functional groups: one of them expressing parts of the propositional content, the other one performing meta-communicative acts in the widest sense. It seems that the latter ones show a tendency for conventionalization, which is why we are able to put together repertoires of those forms but not of the spontaneous referential gestures, created ad hoc.

4. Multimodal metaphors: source and target in two modalities Turning to multimodal metaphor, it is not only interesting to note that once again we encounter quite some variation with regard to which articulatory mode expresses which aspects of the metaphor. The kind of variation and distribution of duties over the two modalities involved tell us something about the nature of the collaboration of words and gestures in spoken language, and it offers insights into the cognitive activation of metaphoricity during speaking. Of the three theoretically possible variations, two are common, whereas one is extremely rare. We find that the same source and target in two modalities, as well as different source and same target in two modalities, are very common forms of multimodal metaphors in words and gestures, whereas same source and different target appears to be rare. 4.1 Same source and same target in two modalities A very common form of gesture word collaboration in expressing metaphor is, as one might expect, when the source domain of a conceptual metaphor appears simultaneously in both verbal and gestural form. Consider example 4, in which a young woman is describing how her teenage love became more and more clingy (klebrig) and intense (heftig): Example 4:
G1 open palms touching each other repeatedly also da hab ich schon gemerkt naja So there I had already realized, well, G1 continued des is ganz schn klebrig. this is pretty clingy,

308

Cornelia Mller and Alan Cienki G1 continued (..) oder heftig. (..) or intense.

Here clingyness is being expressed verbally and gesturally. The speakers flat hands repeatedly touch each other, moving apart and then back to sticking together. It is as if the palms were sticky and it was hard to separate them. The gesture enacts the source domain of the verbal metaphoric expression, indicating that metaphoricity of this expression was activated or in the foreground of the speakers attention. However, the semantic coexpressiveness that we observe in this example does not imply that the gestural and the verbal parts of such a multimodal metaphor must also be expressed simultaneously in real time. As is widely known from gesture studies, gestures often precede words they are co-expressive with; sometimes they are held up and wait for speech, and sometimes they continue past the expression. This is precisely what happens here. The gestural metaphor enacting clingyness begins with the first line in the transcript. Well I already realized and it is held through the pause in the third line and recycled while she offers another metaphoric qualification of the relation (strong, intense). Put differently, even before the speaker actually verbalizes the metaphoric expression clingy she begins to enact clingyness gesturally. This is a case of a verbo-gestural metaphor in which the gesture enacts the source domain of the verbal metaphoric expression, but it does so significantly before the verbal part of the metaphor is uttered. In fact the temporal overlap of the verbo-gestural metaphor is surrounded by an ongoing gestural enactment of the metaphor. Put differently, gestures may dynamically foreshadow and maintain verbal metaphors over longer stretches of discourse. Moreover, they indicate activation of metaphoricity of conventional and transparent metaphors (Mller 2003, 2007, 2008). Gestures expressing the source domain of a verbal metaphoric expression therefore indicate that at this very moment in the production of a verbo-gestural metaphor, the speaker had activated metaphoricity, and we may therefore characterize this metaphor as waking for that very speaker at this moment in time. These cases appear to be widespread and to occur frequently, and we will consider another, similar example below.2 In Example 5 a young man challenges his co-participants opinion on the future implications of ones first job after graduating from university. He thinks that the first job one takes on determines the path of ones future career, and in order to make his point he uses a German idiomatic expression die Weichen stellen, literally setting the tracks. Note that when he is

Words, gestures, and beyond

309

using this idiom for the first time, he does not gesture. Gesturing begins with his elaboration and illustration of this metaphoric argument. Example 5:
nein es is nich so, no this is not the case, aber es stellt natrlich Weichen. but it obviously sets tracks. das is das Problem. this is the problem. G1, 2 joined flat hand point towards left es is schon ne Weiche- it does set tracks

The speaker begins to develop his alternative viewpoint with a very common rhetorical pattern in German conversations, the nein aber (no but) pattern, in which a preceding suggestion is first confirmed and then challenged. The confirmation in our example is verbalized in the first line: no, this is not the case, hereby confirming his interlocutor's point of view, which is then followed by the counterargument in line two, beginning with but: but it obviously sets tracks. He verbally formulates his alternative viewpoint, and he does this metaphorically: it obviously sets the tracks. No gesture is produced along with this first formulation of his counterargument; he only begins to gesture with his first reformulation of the verbal metaphoric expression. Having had no ratifying reaction from his co-participant he begins to elaborate his argument. And with this elaboration he performs a pointing gesture towards his left. Note that the pointing gesture is one in which the extended palms, held vertically, are used to indicate a certain direction. Note that there is a systematic variation of form and function in pointing gestures. Kendon and Versante (2003) show that in Neapolitan conversations speakers use the index finger to point out objects, whereas the flat hand is used to indicate directions. In our case the vertical open palms of the two hands are joined to indicate one direction of a future career. In short, we see here another example of a source being expressed in words and in gestures; the goal of a track is to lead the train into a certain direction, and the gesture visualizes and spatializes this aspect of directionality of the source: gesturally the future career is located to the left hand side of the speaker. That this pointing gesture is a metaphoric one only becomes clear when considering the words with which it is co-expressive, and these entail a verbal metaphoric expres-

310

Cornelia Mller and Alan Cienki

sion Weiche (tracks). Words and gestures share source and target of a metaphoric expression, and these cases are what Mller (2008) terms verbo-gestural metaphors (cf. Forcevilles 1996, 2002 concept of verbopictorial metaphors). 4.2 Different source and same target in two modalities Here we encounter two different types of multimodal metaphor: one in which there is a gestural metaphoric expression with a target that is verbalized in a non-metaphorical fashion, and another one in which a gestural metaphoric expression goes along with a verbal metaphoric expression. Thus in both cases the target is shared, but only in one case is it metaphorically conceptualized in both modalities. We begin with an example in which the target of the gestural metaphoric expression is verbalized non-metaphorically. The example comes from conversations between American students in which they discussed honesty as a moral value in the context of taking exams, and the student describes honesty as a kind of abstract thought. Example 6:
y'know, G1 bh in front of chest, palms facing self, fingers curled a- as far as an abstract thought of honesty is,

As soon as the speaker says far she lifts her two hands up and places them next to each other with the palms of her hands basically facing herself, and turned slightly towards each other. Both hands are cupped, with the fingers tense and curled inward halfway towards the palms. The shape is as if her hands were surrounding a medium-sized ball that she were squeezing. Given that she holds her hands in this position for the entire phrase abstract thought of honesty is (making rhythmic beats on the syllables far, stract, and hon-) we argue that this is a way in which she physically characterizes this abstract thought in gesture. We therefore find the metaphoric target domain in her words and the source domain (a solid form like that of a round object) in the gesture. Note that there is no metaphoric expression on the verbal level. We sometimes see verbal and gestural metaphoric expressions being uttered at the same time, each using a different source to express the same target. In example 7 there is a color metaphor expressed verbally with a

Words, gestures, and beyond

311

spatial metaphor expressed gesturally (see also the discussion of this example in Cienki 2008, and Cienki and Mller 2008b). Here the speaker from Example 6 above continues the thought which was begun there describing honesty as something that does not have gradations: instead it is characterized by clear oppositions: right or wrong, black or white. Example 7:
I mean-y'know, y'know, G1 bh in front of chest, palms facing self, fingers curled a- as far as an abstract thought of honesty is, y'know, G2 bh palms together, flat in horizontal plane, lh palm up, rh sweeps left to right across palm of lh ther- there is no gradations. G3 lh flat and palm up, rh flat, outer edge taps palm of lh (v marks tap), alternating slightly to the left (L) and to the right (R) vL vR vL vR Either you're right you're wrong y'r black 'r white y'know.

While verbally describing these oppositions (G3), she moves her left hand out in front of her, palm up and open. She holds her right hand above it, flat, with the palm held vertically, and taps the right edge of her palm against her open left hand in time with the speech as she says each of the words right, wrong, black, and white. Her right hand taps the left hand first slightly on the left side of her palm (while saying right), then slightly to the right side (while saying wrong), and repeats these left and right taps when saying black and white, respectively. In one sense the gesture appears to be the dividing line, separating the space on the palm of her left hand into two parts (left and right spaces); but at the same time it indicates those very spaces, the left and right sides of the palm of her hand, by tapping them. Whereas the gesture indicates each member of the two sets of opposing categories as two spaces, the words invoke an opposition between black and white. While colors (or the lack of them) would be difficult to represent in an

312

Cornelia Mller and Alan Cienki

iconic way with gestures, spatial concepts are easily rendered, and consequently the metaphor used in gesture (different spaces) is different than the one used in words (different colors). We see here how the specific characteristics of the expressive modality may inform the type of metaphors expressed, leading in this case to multimodal metaphoric expressions that have different sources but share the same metaphoric target (different categories of behavior). These expressions might be tentatively termed verbo-gestural metaphoric compounds. They differ from verbo-gestural metaphors (source and target are shared) in that they work together in expressing the same target metaphorically but do so with different means, i.e., by using different sources. 4.3 Same source and different target in two modalities It is interesting to note that, although theoretically possible, this variant does not appear to be used at all at least insofar as our sets of data are concerned. One could picture a situation where somebody talks about brushing off crumbs of potato chips (crisps) from ones sweater while doing a dismissive brushing aside gesture, characterizing this as a negative aspect of eating potato chips.3 Cases like these seem to be extremely rare. What we do find instead is that the gestures appear to have a tendency to follow the semantics and pragmatics of the verbal utterance, if they can (i.e., if the verbalized content is gesturable, recall the color metaphor example). It seems, therefore, that when the source is shared the target is also shared. This is theoretically interesting because it puts the source information rendered gesturally into a specific light. We will return to this highlighting of source information through gestures later. 4.4 Discussion If we consider the examples of metaphoric expression in gesture, described above, we find certain aspects which are qualitatively different than what we see in metaphoric expression in words. First, words are part of the symbolic system of a language. We generally accept that individual words and the phrases they comprise have ascertainable meanings motivating their use. While within a given culture there may be recurring forms which many gestures take, or certain parameters that they share, most gestures do not have highly codified symbolic form-meaning pairings. (The well-known exceptions are the emblem gestures which can substitute for words, such as the

Words, gestures, and beyond

313

thumbs-up gesture used as a positive response to something, at least in many European cultures.) In this way they contrast with the manually produced signs of a signed language. Because of the spontaneously determined form, placement, and duration of their use, and obviously because they consist of physical forms and movements in space, we find they have properties when serving as expressions of metaphor source domains which are different from those of words expressing metaphoric source domains. In addition, it is well known that many gestures present abstract ideas, which are being mentioned in the speech, as concrete entities in front of the speaker: the gestures indicate particular spaces and locations for the idea, or the hands appear to hold an idea, as if it were an object. But this reification is not a simple reification of ABSTRACT AS CONCRETE (what Lakoff and Johnson 1980 called ontological metaphors). The gestures also show us certain properties of the objectified ideas or topics their size, relative location as imagined by the speaker in the space before him/her, perhaps even their metaphorical evaluation as good or bad by their placement in a high or low space (respectively). For example, in Example 6 when the speaker says abstract thought of honesty, she holds her two hands out in front of her, the pinkie-finger sides next to each other with her palms facing herself, the hands half curled in a tense position, as if the hands were cupping and holding an object about the size of a grapefruit. Here the abstract thought is shown to be something quite concrete, discrete, and of a size comparable to that of objects we manipulate with our hands every day. As this description shows, tidy characterization of these metaphors for research purposes is problematic, particularly when using a theory like Conceptual Metaphor Theory, in which the formula X IS Y (e.g., GOOD IS UP) is the standard way of analyzing metaphoric mappings. We will return to this dilemma towards the end of the chapter. In addition, we find that metaphors do not necessarily occur as single units, but they can also extend over time, and can add up to complex structures, and as this volume demonstrates they may appear in a broad range of media and modes among them film, photography, painting, and sculpture (Forceville 2006; Gilot and Lake 1964; Mittelberg 2002; Mller 2007). 5. The dynamic nature of metaphoric expressions in the flow of discourse Metaphors can be successively elaborated and specified. This holds for monomodal verbal metaphors as well as for multimodal metaphors realized

314

Cornelia Mller and Alan Cienki

in spoken language as well as for those instantiated in other media, such as for instance film or cartoons (Forceville 1999, 2005, 2006; Mller 2007, 2008). Example 8 is a continuation of Example 5 and it shows such a successive multimodal elaboration of a verbal metaphoric expression. In this example the co-participants are discussing their differing viewpoints regarding the future implications of ones first job after graduating from university. The current speaker rejects his co-participants position and argues for the important consequences that the first job may have for the path of ones future career. Recall that he uses a German idiomatic expression die Weichen stellen, literally, setting the tracks, when expressing his point verbally. He only begins to gesture as he offers a succession of examples which illustrate and elaborate his understanding of setting the tracks by choosing a specific job after graduation. Example 8:
nein es is nich so, no this is not the case, aber es stellt natrlich Weichen. but it obviously sets tracks. das is das Problem. this is the problem. G1 2 joined flat hand point towards left es is schon ne Weiche- it does set tracks G2 flat hands point forward es is wieder ne Weiche-it sets another track G3 flat hands point upward wenn de sachst ich studiere Medizin-when you say I will study medicine G4 2 flat hands point towards left oder Germanistik-or German studies

Words, gestures, and beyond G5 2 flat hands point towards right oder Landwirtschaft-or agriculture G6 2 flat hands point upwards and clap during pause oder (..) werde Tennislehrer. or become a tennis coach des is schon ne Weichenstellung. this is a kind of setting the tracks

315

G7 1 flat hand points forward twice, held through pause nachm Studium mute dir wirklich berlegen welche--(...) after graduating [from university] you really have to think carefully which...

In the first part of his response he expresses his alternative viewpoint with a verbal metaphoric expression: it does set tracks. We see no gesture going with this first formulation; rather this counterargument is highlighted verbally through a meta-comment: this is the problem. These first verbal moves set the stage for a sequence of verbal illustrations and gestural enactments of the verbal metaphoric expression. Subsequently the verbally expressed metaphoric concept of setting the tracks is illustrated by listing three job alternatives medicine, agriculture, tennis each one being gesturally situated in a different direction: medicine is the path to the left, agriculture to the right, and tennis is located in the upward direction. The gestures visualize the source of the metaphoric expression setting the tracks, they embody directionality, and they locate the different future career paths in three alternative directions in the gesture space (left, right, up). But this is not the end of the speakers argument. After verbalizing three alternatives and enacting three different directions for three different jobs, he summarizes and comes back to the verbal metaphoric expression that he had used initially: this is a kind of setting the tracks; once again this verbal metaphoric is not accompanied by a gesture. By returning to his initial expression he retrospectively frames his verbo-gestural elaborations as examples for the metaphoric expression he had used to challenge his co-participants argument while at the same time preparing his last and now fully explicit reformulation of his counter-argument: after graduating you really have to think carefully which-- (). This last re-formulation ends with a gestural expression of the idiom that replaces the words and is inserted into the

316

Cornelia Mller and Alan Cienki

speech-pause. He uses yet another pointing gesture, but this one is performed with one hand only and it is directed forward a direction which has not yet been occupied by any of his preceding examples. Moreover the gesture is highly articulated in shape: it is supported by the left hand, directed towards the recipient, repeated twice, and held through the speech pause at the end of the turn; and with this gesture the counter-argument and the turn end. To sum up, in this segment of talk we find a verbal metaphoric expression at the onset which is further verbally illustrated with concrete examples, enacted and elaborated in gestural metaphoric expressions, and completed with a final gestural metaphoric expression at the end of this counterargument. This example nicely illustrates that metaphoricity is a dynamic feature which may trigger metaphoric elaborations in multiple modalities successively in time, and which may provide grounds for the ad hoc creation of new metaphoric gestures, doing different jobs. We may argue that when the verbal metaphoric expression was uttered first, metaphoricity was not in the foreground of the speakers attention; we find no indication that metaphoricity was particularly active for the speaker at that point in time. Put differently, at this moment the metaphor was sleeping; only as the speaker is moving on is he building his elaborations in words and gestures on this sleeping metaphoric expression, thus using it as source. Doing this makes clear that metaphoricity becomes successively more active, as he moves along with his argument, such that we may now speak of waking metaphors. Formulated in McNeills terms, what we may find here is a metaphorical growth point that structures a whole unit of discourse (McNeill 1992; McNeill and Duncan 2000).4 For conceptual metaphor theory this raises questions about how to account for metaphoricity as a dynamic property, which can be more or less highlighted (Mller 2008). Again, the formula of TARGET IS SOURCE problematically reifies the two domains as static entities.

6. Metaphors beyond words and gestures But there is more to multimodal metaphor in spoken language use than words and gestures. Let us return to the various expressive forms involved in the oral/aural modality. The metaphoric possibilities of prosodic expression have received less attention in the literature. However, some of the existing research on prosody (e.g., Pierrehumbert and Hirschberg 1990) may be reinterpreted in terms of revealing the potential of intonation for metaphoric significance. As an example, we can take Pierrehumbert and Hirschbergs

Words, gestures, and beyond

317

discussion of a well-known role played by intonation in speech, that of expression on the metanarrative level. Thus the speakers belief about whether or not s/he shares information mutually with the hearer may be expressed metaphorically via low or high pitch accents, respectively. Looking at these findings from the perspective of metaphor studies, might they ultimately be grounded in the metaphorical patterns of reasoning (known since Lakoff and Johnson 1980) concerning what is KNOWN AS DOWN and what is UNKNOWN AS UP? (In fact these themselves are secondary metaphors based on our Western metaphorical understanding of pitch along a vertical scale, but we will not dwell on that here.) Other research shows a connection between the lexical semantics of words rated as positive or negative, and the relative pitch with which they were produced in experimental settings, correlating with the metaphors GOOD IS UP and BAD IS DOWN. For example, Herold (2006) found in her study that words with positively rated meanings (like happy and yummy [tasty]) were produced with a higher fundamental frequency (pitch) than words with negatively rated meanings (like sad and weak). In terms of metaphoric expression in speech which is independent of metaphoric verbal semantics, think of vocalizations that are not lexical words and how their interpretation can differ by the intonation with which they are uttered. In response to a question about whether someone likes something or not, one (at least an American English speaker) can utter Mmm starting with a high pitch accent, and then letting the pitch fall, to indicate a positive reply. But one could also say Mmm with a level low tone, which could indicate a negative reaction, or at least non-confirmational uncertainty. Thus perhaps metaphorical mappings such as POSITIVE IS UP and NEGATIVE IS DOWN may appear in the use of intonation, even without accompanying words that have corresponding lexical meanings. One study (Cienki in prep.) provides some evidence about the degree to which individuals interpret the quality of prosodic features (stress and intonation contours) in metaphoric terms. The study involved having 20 participants categorize a series of phrases which they simultaneously heard and read using a set of image schemas as descriptors. The term image schemas is being used here in the sense of Johnson (1987) to refer to simple patterns which frequently recur in various aspects of peoples everyday experience (especially visual, tactile, and force-dynamic experience). The set of image-schema names from which participants could choose was limited to the following: container, cycle, force, object, and path, plus the alternative of other. For comparison in this study, another 20 participants performed the same task of having to characterize the same phrases using the given set of

318

Cornelia Mller and Alan Cienki

image schema names, but this second group only read the phrases and did not hear the recordings of the speakers voices uttering them. The experiments were actually conducted as controls for another experimental setting in which the participants saw and heard the video clips in which the speakers uttered these phrases and made co-verbal gestures (Cienki 2005a). Since the utterances were chosen because they were ones which occurred with gestures of various kinds, the words and phrases themselves were rather random, ranging from more substantive ones, such as their tests are difficult and its like youre performing, to comments and interjections, such as no, not really and like. After completing the categorization task, the participants in the first group were asked to write a sentence or two explaining how they used the image schemas to categorize the phrases they heard. The results revealed that they sometimes categorized some of the phrases according to their acoustic properties, rather than referring to the meanings of the lexical items. Consider the following response as an example: a phrase where the tone rose and fell back again seemed cyclical, whereas when the tone steadily rose it seemed like a path. We see how metaphor may play a role in interpreting how an utterance was spoken. As a side note, this could be important for metaphor researchers in terms of setting up stimuli for experiments on metaphor interpretation. The findings underscore the importance of considering the mode of presentation of experimental stimuli (in oral versus written form) because of the effect it may have on the interpretation of the same linguistic expressions. It is worth noting with these examples of metaphor in intonation that we are not dealing with verbal semantics, but with metaphor on the pragmatic level what the speaker meant with the use of a given intonation contour. Interestingly, we find a parallel phenomenon of metaphor on the pragmatic level in gesture. The primary function of some gestures appears to be to highlight interactive or interpersonal relations, to parse the discourse, or to accomplish a performative act (Kendon 2004: ch. 9). Mller, referring to unpublished observations by Jrgen Streeck, discusses the pragmatic functions of the palm-up open-hand (PUOH) gesture, which can serve to present an abstract, discursive object as a concrete, manipulable entity (2004: 233). The gesture can indicate that what the speaker is saying is to be interpreted as an idea to be discussed, a proposal, or a question (Kendon 2004: 159). In terms of conceptual metaphor theory, we might say that this gesture uses the pragmatic metaphor of INTRODUCING AN IDEA IS PRESENTING AN OBJECT. Here as in the other recurrent gestures discussed above, the metaphor does not simply work on the level of what the speakers words express

Words, gestures, and beyond

319

semantically. Rather, it works on the pragmatic level, in that the source is expressed in the gesture, and the target is what the speaker is doing with his/her words as well as with his/her gestures.

7. Implications for metaphor theory A major conclusion we can draw from the fact that metaphors can be realized in multiple modalities is that metaphoricity is modality-independent. It documents that the establishment and creation of metaphoricity is a cognitive process with products in various modalities, thus offering strong support for Lakoff and Johnsons initial idea of moving metaphor(icity) out of the realm of literary discourse into the mundane world of everyday thought (Mller 2003, 2007, 2008). However, this also has critical implications for metaphor theory in that it calls for refined empirical methodology as well as for a new theoretical understanding of the different forms of multimodal metaphors and their constitutive semantic relations. It also directs our attention to the necessity of including a cognitive-semiotic analysis of metaphoric, as well as of metonymic, processes (see Mittelberg 2006, 2007; Mittelberg and Mller in prep. a; Mittelberg and Waugh this volume). A major implication of the insights gained through the analysis of multimodal metaphors in the use of spoken language is the fact that as spoken language is inherently dynamic, so is multimodal metaphor. As already indicated above, the study of metaphors as expressed in the dynamic processes of speaking presents us with metaphoric source domains which are themselves contingent on time for their realization. This raises a problem, given the traditional means of conceptual metaphor analysis, namely that it involves the static verbal formula of TARGET IS SOURCE (an issue raised long ago by the anthropologist Bradd Shore, personal communication). Various authors in recent research have suggested alternatives to try to overcome the limitations of this analytic device. For some types of source domains, one solution is to characterize them by using schematic images. An example described in Cienki (2005b) is that when Al Gore was a candidate for U.S. president in 2000 he used the same gesture at several points during the televised debates: a gesture with one or both hands palm up and cupped slightly with the fingers slightly curved, as if he were holding a small ball. This gesture occurred with phrases such as enable us to project the power for good, shepherds that economic strength, the power of example is Americas [greatest power] in the world (with square brackets indicating the timing of the gesture in the last

320

Cornelia Mller and Alan Cienki

example). We would argue that in the examples from Gore, the gesture serves basically the same purpose as the PUOH gesture discussed by Mller (2004), but that there is an added element here indicated by the cupped shape of the hand. In a physical situation, such a hand shape would be used not only to support a small object in the hand, but also to prevent it from falling off the extended hand, thus protecting it in a way. Thus the gesture not only suggests that the speaker is treating AN IDEA AS AN OBJECT, which he is presenting to the addressee (the moderator of the debate and, by extension, the television audience), but that he is also showing something about his attitude toward the idea he is presenting, perhaps that it is something good which he wants to support (all three utterances expressed positive ideas which Gore espoused). In light of the meaning added by the cupped hand shape, the manner of presenting is significant, and (as argued in Cienki 2005b) could be indicated by a diagram or schematic image see the ones Efron (1972) used in his analysis of the linguistic properties of the gestures used by Italians as compared to Eastern-European Jewish immigrants to New York City. See also Calbris (2003) schematizations of gesture hand shapes and motions in diagrammatic form. Finally, the increasing use of digital publishing (online or on CDs or DVDs) allows for video characterization of source domains which are dynamic in nature, in that they can be presented as moving schematic images, for example as small animations.

8. Implications for thinking for speaking and gesturing Slobin (1987, 1996) argues that there is a special form of thought which is mobilized in the process of talking, which he calls thinking-for-speaking. As he describes it, Thinking for speaking involves picking those characteristics [of a perceived event, CM and AC] that (a) fit some conceptualization of the event, and (b) are readily encodable in the language which the speaker is using at the moment (Slobin 1987: 435). Thus the lexical and grammatical means of expression available in a language are used by speakers already as they are anticipating how to utter what they want to utter. McNeill and Duncan (2000) suggest that gesture needs to be taken into account in this process as well. They discuss how the idea units which we are continually developing and unraveling for expression while we talk, what McNeill (1992) has called growth points, combine both imagery and linguistic-categorical content. In the process of thinking while speaking, which McNeill and Duncan (2000: 157) note is perhaps a more accurate way to refer to the phenomenon, the imagistic content receives partial expression in the gestures that the

Words, gestures, and beyond

321

speaker produces. Indeed, which imagery is expressible in gesture is a factor in how the verbo-gestural utterance is produced. Consequently we (Mller and Cienki 2006) have discussed the process as thinking-for-speaking-andgesturing. Both the nature of the available linguistic forms as well as the expressive potential of hand-gestures which one can use in the expression of ones thoughts while speaking are significant for what thoughts ultimately get expressed. This means that there are important connections between which single or multiple modality/ies are at ones disposal for expression and the kind(s) of metaphoric ideas which one ultimately conceptualizes and expresses either monomodally or multimodally.

9. Conclusion The insights gained through the analyses of multimodal metaphors in language use have rather far-reaching consequences for a theory of metaphor. Not only do they underline the stance of Conceptual Metaphor Theory with regard to the principally modality-independent nature of metaphoricity (this means metaphor as a cognitive mechanism); they uncover that the hitherto static view on metaphor in thought and language must be supplemented by a dynamic view on metaphor in thinking, speaking and gesturing (cf. Mller 2003, 2007, 2008; Mller and Cienki 2006;). Such a dynamic view takes into consideration the procedural nature of meaning creation in situations of face-to-face communication, including the elaboration of metaphoric expressions in the discourse as well as the dynamic activation of metaphoricity, which for a specific and often short moment in time may turn sleeping metaphors into waking ones (Mller 2003, 2007, 2008). Put differently, these observations indicate that we need a theory that distinguishes between products and processes (Gibbs 1992, 1998, 1999; cf. also Cameron and Low 1999), and also between system and use (Mller 2008; Steen 2006; Steen and Gibbs 1999) or one that cross-cuts these dichotomies as Steen (2007) has recently proposed; that one distinguishes metaphors in grammar and usage, be they approached as symbolic structure or as forms of behavior. Further consequences of realizing the multimodal nature of metaphoric expression in the use of spoken language include that in co-speech gesturing as well as in prosodic features of speech we may see manifestations of the imagistic or, more generally, the embodied nature of many metaphoric source domains. We also see that the metaphoric process is not a unidirectional one, one in which a preconfigured thought is being translated into gesture, word, or sound; rather we must conceive of it as an interactional

322

Cornelia Mller and Alan Cienki

process which takes into consideration the nature and the expressive potential of the respective modalities (colors do not lend themselves to expression in gesture, but for spatial relations the opposite holds true). Compare for instance what is known about metaphoric expression in another use of the manual modality, namely in sign language. In both cases, gesture and sign language, the iconic nature of visual/manual expression affords different potentials than aural/oral expression does (Mller in prep.; Taub 2001), although gestures with speech are normally co-verbal, as opposed to constituting linguistic signs in and of themselves. However, in the process of communication or to put it in Wallace Chafes (1994) terms, in the flow of discourse these modality-specific properties can be exploited to varying extents in any given event of speaking. A dynamic approach to linguistic theory (such as that proposed by McNeill 2005) or to metaphor theory (as in Mller 2008) which can accommodate the multimodal potential of language production and reception can provide a more complete picture of the complexity of this form of human behavior than the static views of language, metaphor, and thought which currently dominate the field of cognitive linguistics and beyond. In conclusion, for researchers of spoken language, moving beyond the level of the words can uncover many facets of metaphoricity that had previously lain hidden. Acknowledgements
We are grateful to Charles Forceville, Irene Mittelberg, and Linda Waugh for insightful comments on an early draft of this chapter.

Notes
1. In his work on F-Formations (facing formations) Kendon (1990) indicates that facing formations may actually show different forms, varying with the amount of participants involved but also within dyadic encounters: In Fformations of two indidviduals, for example, we may see arrangements that vary from a direct face-to-face pattern, to an L-shaped pattern, or even a sideby-side pattern (Kendon 1990: 250). Kendon (personal communication) has also observed that Australian aboriginals conduct conversations in certain situations all facing the same direction, i.e., in a side-by-side configuration. Tzeltal speakers appear to quite commonly choose a side-by-side configuration in dyadic situations (Stephen Levinson, personal communication). For

Words, gestures, and beyond

323

2. 3.

4.

the interactive structure of establishing such F-formations, see Mller and Bohle (2007). For a more detailed account of this and other similar examples, see Mller (2008). This example is inspired by a case reported by Teendorf, in which the brushing aside movement is used to brush aside crumbs of potato chips, functioning here as an object manipulation. For the metonymic and metaphoric links necessary to transform this action into a pragmatic gesture see Teendorf (in prep. a, b). For further discussion and more examples of dynamic metaphoricity in gesture and speech see Mller (2003, 2007, 2008) and Cienki and Mller (2008a).

References
Bouvet, Danielle 1997 Le corps et la mtaphore dans les langues gestuelles: A la recherche des modes de production des signes. Paris: LHarmattan. 2001 La dimension corporelle de la parole: Les marques posturo-mimogestuelles de la parole, leurs aspects mtonymiques et mtaphoriques, et leur rle au cours dun rcit. Paris: Peeters. Bressem, Jana, and Silva Ladewig (eds.) in prep. Handmade patterns: Recurrent Forms and Functions of Gestures. Amsterdam/Philadelphia: Benjamins. Calbris, Genevive 1990 The Semiotics of French Gestures. Bloomington, IN: Indiana University Press. 1998 Geste et images. Semiotica 118 (1-2): 105120. 2003 From cutting an object to a clear cut analysis: Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture 3: 1946. Cameron, Lynne, and Graham Low (eds.) 1999 Researching and Applying Metaphor. Cambridge: Cambridge University Press Chafe, Wallace 1994 Discourse, Consciousness, and Time: The Flow and Displacement of Conscious Experience in Speaking and Writing. Chicago: University of Chicago Press. Cienki, Alan 1998 Metaphoric gestures and some of their relations to verbal metaphorical expressions. In Discourse and Cognition: Bridging the

324

Cornelia Mller and Alan Cienki

Gap, Jean-Pierre Koenig (ed.), 189204. Stanford, CA: Center for the Study of Language and Information. 1999 Metaphors and cultural models as profiles and bases. In Metaphor in Cognitive Linguistics, Raymond. W. Gibbs, Jr. and Gerard J. Steen (eds.), 189203. Amsterdam/Philadelphia: Benjamins. 2005a Image schemas and gesture. In From Perception to Meaning: Image Schemas in Cognitive Linguistics, Beate Hampe (ed.), 421441. Berlin/New York: Mouton de Gruyter. 2005b Metaphor in the Strict Father and Nurturant Parent cognitive models: Theoretical issues raised in an empirical study. Cognitive Linguistics 16: 279312. 2008 Why study metaphor and gesture? In Metaphor and Gesture, Alan Cienki and Cornelia Mller (eds.), 525. Amsterdam/Philadelphia: Benjamins. in prep. The (image) schematicity of concrete versus abstract referential gestures and their accompanying words. Cienki, Alan, and Cornelia Mller (eds.) 2008a Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins. Cienki, Alan, and Cornelia Mller 2008b Metaphor, gesture, and thought. In The Cambridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr. (ed.), 483501. Cambridge: Cambridge University Press. Clark, Herbert 1973 Space, time, semantics, and the child. In Cognitive Development and the Acquisition of Language, Timothy E. Moore (ed.), 2763. New York: Academic Press. Efron, David 1972 Gesture, Race, and Culture. (First published (1941) as Gesture and Environment). Berlin/New York: Mouton de Gruyter. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge 1999 The metaphor COLIN IS A CHILD in Ian McEwans Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 17998. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2005 Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988 2006 Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco J. Ruiz de Mendoza Ibez (eds.), 379 402. Berlin/New York: Mouton de Gruyter.

Words, gestures, and beyond

325

Gibbs, Raymond W., Jr. 1993a Process and products in making sense of tropes. In Metaphor and Thought (2nd ed.), Andrew Ortony (ed.), 252276. Cambridge: Cambridge University Press. 1998 The fight over metaphor in thought and language. In Figurative Language and Thought, Albert N. Katz, Cristina Cacciari, Raymond W. Gibbs, Jr., and Mark Turner (eds.), 119157. New York/Oxford: Oxford University Press. 1999 Researching metaphor. In Researching and Applying Metaphor, Lynne Cameron and Graham Low (eds.), 2947. Cambridge: Cambridge University Press. Gilot, Francoise, and Carlton Lake 1964 Life with Picasso. New York: McGraw-Hill. Herold, Debora 2006 Acoustic correlates to word meaning in infant directed speech. Ph.D. diss., Emory University. Johnson, Mark 1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University of Chicago Press. Kendon, Adam 1990 Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge: Cambridge University Press. 2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. Kendon, Adam, and Laura Versante 2003 Pointing by hand in Neapolitan. In Pointing: Where Language, Culture and Cognition Meet, Sotaro Kita (ed.), 109137. Hillsdale, NJ: Lawrence Erlbaum. Ladewig, Silva in prep. Metonymic relations in a recurrent polysemous gesture. In Metonymy in Gesture and Signed Languages, Irene Mittelberg and Cornelia Mller (eds.), special Issue of CogniTextes. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press. McNeill, David 1992 Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press. 2005 Gesture and Thought. Chicago: University of Chicago Press. McNeill, David, and Susan Duncan 2000 Growth points in thinking-for-speaking. In Language and Gesture, David McNeill (ed.), 141161. Cambridge: Cambridge University Press.

326

Cornelia Mller and Alan Cienki

Mittelberg, Irene 2002 The visual memory of grammar: Iconographical and metaphorical insights. Metaphorik.de 2/2002: 6989. 2006 Metaphor and Metonymy in Language and Gesture: Discourse Evidence for Multimodal Models of Grammar. Ph.D. diss., Cornell University. 2007 Internal and external metonymy in co-verbal gesture: Jakobsonian and cognitivist views on contiguity relations within and across modalities. Paper presented at the 10th ICLC, Krakow, July 2007. Mittelberg, Irene, and Linda Waugh this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. Mittelberg, Irene, and Cornelia Mller (eds.) in prep. Metaphor and Metonymy in Gesture and Sign Languages. Special Issue. CogniTextes. Morris, Desmond 1977 Manwatching. London: Jonathan Cape. Mller, Cornelia 1998 Redebegleitende Gesten. Kulturgeschichte Theorie Sprachvergleich. Berlin: Berlin Verlag Arno Spitz. 2003 Gestik als Lebenszeichen toter Metaphern. Tote, schlafende und wache Metaphern. Zeitschrift fr Semiotik 1-2: 6172. 2004 Forms and uses of the Palm Up Open Hand: A case of a gesture family? In The Semantics and Pragmatics of Everyday Gestures, Cornelia Mller and Roland Posner (eds.), 23356. Berlin: Weidler. 2007 A dynamic view of metaphor, gesture and thought. In Gesture and the Dynamic Dimension of Language, Susan Duncan, Justine Cassell, and Elena Levy (eds.), 109116. Amsterdam/Philadelphia: Benjamins. 2008 Metaphors Dead and Alive, Sleeping and Waking: A Dynamic View. Chicago: University of Chicago Press. in prep. a Creative devices in gestures and sign languages. In Cornelia Mller, Ellen Fricke, Silva Ladewig, Irene Mittelberg, Sedinha Teendorf (in prep.) Gestural Modes of Representation as Mimetic Devices Revisited. Gesture. in prep. b How hand movements turn into gesture: Gestural modes of representation as metonymic resources of gesture creation. In Metaphor and Metonymy in Gesture and Sign Languages. Special Issue of CogniTextes, Irene Mittelberg and Cornelia Mller (eds.). Mller, Cornelia, and Ulrike Bohle 2007 Das Fundament der Interaktion: Zur Vorbereitung und Herstellung von Interaktionsrumen durch krperliche Koordination. In Koordination: Analysen zur multimodalen Interaktion, Reinhold Schmitt (ed.), 129165. Tbingen: Narr.

Words, gestures, and beyond

327

Mller, Cornelia, and Alan Cienki 2006 Verbal to gestural, and gestural to verbal, metaphoric expression: Activation and development of metaphoricity. Paper presented at the sixth conference on Researching and Applying Metaphor (RaAM 6), Leeds, UK, April 2006. Mller, Cornelia, and Harald Haferland 1997 Gefesselte Hnde. Zur Semiose performativer Gesten. Mitteilungen des Germanistenverbandes 3: 2953. Mller, Cornelia, and Gerald Speckmann 2002 Gestos con una valoracin negativa en la conversacin cubana. De Signis 3: 91103. Neumann, Ragnhild 2004 The conventionalization of the ring gesture in German discourse. In The Semantics and Pragmatics of Everyday Gestures, Cornelia Mller and Roland Posner (eds.), 217224. Berlin: Weidler. Nez, Rafael 2004 Do real numbers really move? Language, thought and gesture: The embodied cognitive foundations of mathematics. In Embodied Artificial Intelligence, Fumiya Iida, Rolf Pfeifer, Luc Steels, and Yasuo Kuniyoshi (eds.), 5473. Berlin: Springer. Nez, Rafael, and Eve Sweetser 2006 With the future behind them. Convergent evidence from Aymara language and gesture in the crosslinguistics comparison of spatial construals of time. Cognitive Science 20: 149. Pierrehumbert, Janet, and Julia Hirschberg 1990 The meaning of intonational contours in the interpretation of discourse. In Intentions in Communication, Philip R. Cohen, Jerry Morgan, and Martha E. Pollack (eds.), 271311. Boston, MA: MIT Press. Pragglejaz Group 2007 MIP: A method for identifying metaphorically used words in discourse. Metaphor and Symbol 22: 139. Slobin, Dan I. 1987 Thinking for speaking. Proceedings of the 13th Annual Meeting of the Berkeley Linguistics Society, 435445. 1996 From thought to language to thinking for speaking. In Rethinking Linguistic Relativity, Stephen C. Levinson and John Gumperz (eds.), 7096. Cambridge: Cambridge University Press. Speckmann, Gerald 1999 Sprachbezug und Konventionalitt von Gesten am Beispiel kubanischer uerungen. MA diss., Freie Universitt Berlin.

328

Cornelia Mller and Alan Cienki

Steen, Gerard J. 2006 Metaphor in applied linguistics: Four cognitive approaches. Documentao de Estudos em Lingstica Terica e Aplicada D.E.L.T.A. 22 (1): 2144. 2007 Finding Metaphor in Grammar and Usage: A Methodological Analysis of Theory and Research. Amsterdam/Philadelphia: Benjamins. Steen, Gerard J. and Raymond W. Gibbs, Jr. 1999 Introduction. In Metaphor in Cognitive Linguistics, Raymond W. Gibbs, Jr. and Gerard J. Steen (eds.), 18. Amsterdam/Philadelphia: Benjamins. Streeck, Jrgen 1994 Speech-handling: The metaphorical representation of speech in gestures. A cross-cultural study. Ms. University of Texas at Austin. Sweetser, Eve 1998 Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrs International des Linguistes (CDRom). Amsterdam: Elsevier. Teendorf, Sedinha in prep. a From everyday action to gestural performance: Metonymic motivations of a pragmatic gesture. In Metaphor and Metonymy in Gesture and Sign Languages. Special Issue of CogniTextes, Irene Mittelberg and Cornelia Mller (eds.). in prep. b Pragmatic and recurrent: The case of the brushing aside gesture. In Recurrence in Coverbal Gestures Investigating Form and Function, Jana Bressem and Silva Ladewig (eds.). Amsterdam/ Philadelphia: Benjamins. Taub, Sarah 2001 Language from the Body: Iconicity and Metaphor in American Sign Language. Cambridge: Cambridge University Press. Webb, Rebecca 1996 Linguistic Features of Metaphoric Gestures. Ph.D. diss., University of Rochester, NY. Wilcox, Phyllis P. 2000 Metaphor in American Sign Language. Washington, DC: Gallaudet University Press. 2004 A cognitive key: Metonymic and metaphorical mappings in ASL. Cognitive Linguistics 15: 119147.

Chapter 14 Metonymy first, metaphor second: A cognitivesemiotic approach to multimodal figures of thought in co-speech gesture Irene Mittelberg and Linda R. Waugh
Abstract
Based on spoken academic discourse and its accompanying gestures, this chapter presents a cognitive-semiotic approach to multimodal communication that assigns equal importance to metaphor and metonymy. Combining traditional semiotics with contemporary cognitivist theories, we demonstrate how these two figures of thought jointly structure multimodal representations of grammatical concepts and structures. We discuss Jakobsons view of metaphor and metonymy, and particularly his distinction between internal and external metonymy, thus discerning various principles of sign constitution and indirect reference within metaphoric gestures (whether or not the concurrent speech is metaphorical). We then introduce a dynamic two-step interpretative model suggesting that metonymy leads the way into metaphor: in order to infer the imaginary objects or traces that gesturing hands seem to hold or draw in the air, a metonymic mapping between hand (source) and imaginary object (target) is a prerequisite for the metaphorical mapping between that very object (source) and the abstract idea (target) it represents. Keywords: metaphor, metonymy, gesture, semiotics, cognitive theory

1. Introduction Work done by scholars in many disciplines has shown that metaphor and metonymy rely on general cognitive processes of conceptualization and association that may materialize in modalities other than spoken and written words, e.g., in gesture. While the chapters in this volume contribute to a unified approach to the role of metaphor in multimodal representations, we will show here that it is both metaphor and metonymy that, by working together in multimodal communication, function to convey complex meanings,

330

Irene Mittelberg and Linda R. Waugh

just as they motivate, to various degrees, all processes of signification (Jakobson 1990 [1956]). Thus, we agree with many other scholars that more attention should be paid to metonymy: it is equal in importance to metaphor but crucially different from it in its contribution to communication (e.g., Barcelona 2000; Croft 1993; Dirven and Prings 2002; Gibbs 1994; Goossens et al. 1995; Lakoff 1987; Panther and Radden 1999; Panther and Thornburg 2003; Turner and Fauconnier 2002; Wilcox 2004). We will show this by focusing on a multimodal combination of verbal and bodily communication namely, spoken academic discourse and accompanying manual gestures (see also Mller and Cienki, this volume). The discourse here is classroom lectures by linguists about grammar and syntactic theory and, as we will demonstrate, both metaphor and metonymy play a crucial role in presenting these abstract phenomena. Multimodal representations of knowledge domains such as grammar have a long-standing tradition and visual metaphor, in particular, has widely been used to illustrate linguistic explanations (see Mittelberg 2002 on Early Modern printed images of grammar). Besides figurative language, pictorial metaphor has been studied as a way of expressing an analogy between two entities (e.g., Carroll 1994; Forceville 1994, 1996, 2002, 2005; Kennedy 1982). However, if we compare gesture to solid sculptures or pictorial signs, we realize that, just as speech in the auditory domain is inherently dynamic and fleeting, so with gestures the interplay of time, space, and motion typically engenders transient rather than lasting images that have to be understood quickly in time. This difference is crucial for an understanding of gesture. In the gestural representations of grammar we will investigate here, the hands and arms serve as a resource for making meaning as they correlate with the on-going speech through co-speech gestures (McNeill 2005). Cognitive linguists have provided a detailed picture of the different ways in which the human body serves as a (re)source for a large array of metaphors (e.g., Kvecses 2002; Lakoff and Johnson 1980, 1999). We are primarily interested here in the ways in which the gesture modality generally can provide insights into situated cognition because it helps to externalize concepts, structures and practices through the use of both space and the body thereby creating an ex-bodiment of these concepts and structures through motor actions and other forms of interaction with the physical and social environment (Mittelberg 2006). Thus, this work rests on the premise that concepts (and other aspects of conceptual structure such as image schemata) are embodied and are themselves multimodal, including kinesthetic features (Hampe 2005; Johnson 1987; Lakoff 1987; Lakoff and Johnson 1980, 1999; Sweetser 1990, 1998). One of our aims here is to present ways in which co-

Metonymy first, metaphor second

331

speech gesture provides additional evidence for conceptual metaphor and metonymy by revealing in a dynamic fashion dimensions of schemata not necessarily expressed in the concurrent speech (Calbris 2003; Cienki 1998, 2005; Cienki and Mller 2008; McNeill 1992, 2005; Mittelberg 2008; Mller 1998, 2008; Sweetser 1998, 2007; Taub 2001). Another is to show that some of the claims about the relationship between source and target domains made on the basis of multimodal representations without gestures fail to account for gestures satisfactorily, since the metaphorical meaningmaking processes cannot be elucidated without taking into consideration how they interact with metonymic modes. We will also show that metonymy assumes distinct functions regarding, for instance, gestural sign constitution through synecdoche and indirect reference, as others have suggested (see Bouvet 2001; Gibbs 1994; Mller 1998; Wilcox 2004; Wilcox and Morford 2007). Building on these considerations, we present an approach to multimodal communication, and in particular to multimodal metaphor and metonymy, that has been shaped by both theoretical motivations and by the nature of our data. Our approach combines contemporary cognitivist theories (Gibbs 1994, 1999; Johnson 1987; Lakoff 1987, 1993; Lakoff and Johnson 1980, 1999; Sweetser 1990) with the older but still relevant work of C. S. Peirce (1960, 1991, 1992, 1998) and Roman Jakobson (1956, 1987a, b, 1990; Jakobson and Pomorska 1983). The Jakobsonian and Peircean semiotic approaches continue to provide rich perspectives for multimodal research and lend themselves to being applied to gesture, for they are broader in scope than linguistic theories and have been used to analyze a wide variety of dynamic, discursive phenomena such as theatre, cinema, myths, rituals, music, poetry, etc. Since we have treated in detail elsewhere Peirces theory (Waugh 1992, 1998; Waugh et al. 2004) and Peircean perspectives on the gestural sign (Mittelberg 2006, 2008; see also Fricke 2007; McNeill 1992), our focus here will be on Jakobsons approach and its relevance for gesture research from a cognitivist perspective. Our rationale for combining these theoretical perspectives rests on the fact, firstly, that Cognitive Metaphor Theory (CMT) and Peirces semiotic share central assumptions about the link between image-schematic structures and metaphorical (and diagrammatic) projections, as well as about habitual patterns of experience and interpretation (Danaher 1998; Hiraga 1994, 2005; Mittelberg 2006, 2008). Second, Jakobson emphasized the importance of metaphor and metonymy as different semiotic and cognitive strategies that structure both verbal and non-verbal signs and messages (Jakobson 1956, 1987a, b; Lodge 1977; Waugh 1998, 2000; Waugh and Monville-Burston

332

Irene Mittelberg and Linda R. Waugh

1990; Waugh et al. 2004). Thirdly, both Jakobson and Peirce stress the point of view of the interpreter of a given message: in similar fashion, our aim will be to explicate how gestures are interpreted by the viewer (e.g., the student, the gesture analyst) through both metaphor and metonymy.

2. Characterizing the data: Semiotic idiosyncrasies of metalinguistic discourse and its accompanying gestures Connecting the body, language, cognition, society and culture, gestures can provide a window into thought processes and their pragmatic and ecological anchorage (Goodwin 2003; Ochs et al. 1996; Streeck 2002). From a cognitivist viewpoint, i.e., taking the perspective of the speaker, research has shown how hands (and arms) may reveal, consciously or unconsciously, thoughts and attitudes that speech might conceal (McNeill 1992: 246). We, on the other hand, are trying to see how the gestures help the viewer to understand the conceptualization of abstract ideas that the speaker/gesturer is communicating. The spontaneous gestures we will be analyzing here are not part of an elaborated sign system but are created by the speaker as he/she speaks, and thus gesture and speech can produce very different effects, including juncture or disjuncture, redundancy, complementation, or mismatch (GoldinMeadow 2003; McNeill 2000, 2005). A gesture may disambiguate linguistic information and thus make meaning more precise (for instance, by pointing at a concrete referent that is linguistically only referred to via an unspecified pronoun), or it may add components of meaning not expressed in the speech it accompanies (Kendon 2000).1 Often, however, spontaneous gestural signs tend to be polysemous and need a contextual support to be correctly interpreted; thus, discourse-pragmatic factors and concurrent speech help to disambiguate them (Calbris 1990; Kendon 2004; McNeill 2005; Mller 1998). As Jakobson noted, a pointing gesture at a package of cigarettes could be interpreted to mean this package in particular, or a package in general, one cigarette or many, a certain brand or cigarettes in general, or, still more generally, something to smoke. The viewer does not know if the pointer is simply showing, giving, selling, or prohibiting the cigarettes. The only way to know is through the accompanying speech (Jakobson 1953: 567). A single gesture could also fulfill several functions at once: e.g., from representational to deictic, or from accentuating the rhythm of the speech to attracting attention and managing interaction between the interlocutors. Gestures are thus visuo-spatial motor signs (Jakobson 1987a: 474) that derive their

Metonymy first, metaphor second

333

locally-situated meaning from the very human body that articulates them, the speech they accompany, and the socio-cultural and material environment the person interacts with. Consequently, a gestural sign does not exist, and cannot be analyzed, detached from either the human body or the here and now of the speech event (the origo in Bhlers terms, see Fricke 2007). This means that in order to understand the gestures under discussion here, we first need to characterize the speech they accompany in terms of its genre and functions. As indicated above, our data come from one specific spoken genre: metalinguistic academic discourse in lecture format, from a corpus of such lectures by four professors (three women and one man), all native speakers of American English, while they were teaching introductory linguistics to undergraduate and graduate students at two major American universities. The lectures were videotaped in a naturalistic setting, that is, regularly scheduled classes where neither the teacher nor the students knew about the purpose of the taping (in particular, they did not know that the analysis was to focus on gesture). Thus, the assumption is that the gestures used by the professors were not affected by the videotaping (for a detailed description of methods of collecting, editing and transcribing the data, including the coding and annotation systems used, see Mittelberg 2006, 2007). Now, in the typical classroom setting there are other visual modalities: e.g., black/green/white boards with writing and other visuals on them, handouts, slides and power point projections. However important these are for the communication of information in the classroom, what is unique to gestures is that they are conveyed by the body of the lecturer and correlated with the speech that is emanating from that same body.2 The speech that is at issue here is highly complex. It has multiple functions: it conveys information about language that reflects the beliefs of the speaker and is directed at the audience (the students in the class) with the aim that the students will gain at least an understanding of, and perhaps also a belief in, the concepts being discussed. The gestures have the same complex multifunctionality as they contribute to the communication and understanding of the lectures. Our focus will be on what Mller (1998: 110113) calls referential gestures, that is, gestures that depict objects, attributes of objects and people, actions, or behaviors, whether concrete or abstract (Mller 1998; see also Cienki 2005). More specifically, the gestures analyzed here are all attempts at making fairly abstract grammatical concepts and aspects of the syntactic structure of sentences more understandable for the listener/viewer, by turning them into (partial) visuo-spatial and embodied manifestations of these concepts.

334

Irene Mittelberg and Linda R. Waugh

3. Jakobson: Metaphor and metonymy (internal and external), similarity and contiguity, selection and combination Roman Jakobsons view of metaphor and metonymy has been successfully utilized in the analysis of a wide variety of monomodal and multimodal texts (Bradford 1994; Jakobson 1956; Jakobson and Pomorska 1983; Shapiro 1983; Whittock 1990) and is particularly valuable and compatible with contemporary, cognitively-oriented accounts of metaphor and metonymy since it is in fact one of their predecessors. In recent publications (e.g., Dirven and Prings 2002), cognitive linguists revisit and offer a great deal of evidence for Jakobsons theory. This chapter attempts to show that adopting his balanced approach and exploring the interplay of these two different mental strategies of conceptualization (Dirven 2002: 75) has the potential to illuminate the semiotics of gesture and of multimodal communication more generally.3 Jakobson (1956) contends that metaphor and metonymy are two different modes of association that structure both linguistic and non-linguistic signs. While until not too long ago metaphor consistently received much more scholarly attention than metonymy, Jakobson paid equal attention to both tropes. In his view, metonymy is not a sub-type of metaphor, but the two are in opposition with each other and thus create a fundamental polarity that is at the root of all symbolic processes, cultural manifestations, and human thought in general. Thus, studies concerned with metaphor ought to pay (more) attention to its interaction with metonymy, a view that is also present in the work of quite a large number of cognitive linguists. In defining the difference between metaphor and metonymy, Jakobson was particularly inspired by Peirces famous trichotomy of signs: icon, index and symbol. According to Peirce, similarity is at the root of iconic relationships between the sign and the object it represents and he includes metaphor as a specific sub-type of icon (Peirce 1960, 1992, 1998). Contiguity, on the other hand, is inherent to the index, deictic categories, and, as Jakobson (1956, 1966) also emphasized, metonymy. For Jakobson, similarity and contiguity are bipolar opposites, representing the two essential structural relations between signs that permeate all of language (Shapiro 1983: 194). Thus, similarity is the basis for metaphor, as well as synonymy, paraphrase, antonymy, analogy, etc, and contiguity underlies metonymy, as well as spatial and temporal neighborhood (both proximity and remoteness), causeeffect relations, etc. In addition, Jakobson differentiated between two major subtypes of metonymy: (1) external metonymy (metonymy proper), in which the name of an object is replaced by the name of an attribute, or of

Metonymy first, metaphor second

335

an entity related in some semantic way (e.g., cause and effect; instrument; source) (Wales 2001: 252), e.g., the term the White House when referring to the President of the U.S. (place for person); and (2) internal metonymy(synecdoche), in which the name of the referent is replaced strictly by the name of an actual part of it (Wales 2001: 252) or by the name of the whole of which it is a part; e.g., part stands for whole and whole for part (e.g., all hands on deck, in which hands stands for the whole body). That is, Jakobson integrated synecdoche as an important sub-type of metonymy, and as we will see, these two types of metonymy are crucial to the study of gesture. Most importantly, Jakobson insisted that similarity and contiguity and metaphor and metonymy are not mutually exclusive: just as signs can exhibit both similarity and contiguity in differing hierarchies (Jakobson 1966: 411), so the nature of a given sign is dependent on the preponderance of one of the two modes over the other (see Jakobson 1956: 130). According to Jakobson (1956: 117, see also Waugh and MonvilleBurston 1990), the similarity/contiguity relations between signs are different from the basic types of operations by which any linguistic utterance is constructed by the speaker. Any act of utterance formation involves the selection of certain linguistic entities from the code (e.g., words) and their combination into linguistic units of a higher degree of complexity (e.g., phrases and sentences). Understanding by the addressee implies the reverse order of operations: the combination of units of greater complexity has to be dissolved into the individual linguistic entities selected. Both modes of arrangement (Jakobson 1956: 119) reflect the structural reality of language: selection relies on the organization of the linguistic system, while combination is evidenced in the fact that every sign is made up of constituent signs (sentences, words, morphemes, phonemes, features) and serves as the context for other signs. Jakobson (1956: 119) referred to this kind of semiotic contextualization as contexture, e.g., the process by which any linguistic unit at one and the same time serves as a context for simpler units and/or finds its own context in a more complex linguistic unit. [ C]ombination and contexture are two faces of the same operation. In the case of multimodal messages, signs from more than one mode are selected and combined to constitute the contexture for one another: for example, gesture combined with speech. Such combinations may be concurrent and/or sequential: so, a given gesture is concurrent with the simultaneously occurring words, and the way in which gestures unfold in time (with or without speech) is an example of sequential combination.

336

Irene Mittelberg and Linda R. Waugh

4. Conceptual metaphor and metonymy in gestural representations of grammar Offering new insights into multimodal instantiations of conceptual metaphor, previous work on metaphoric gestures has shown that they are not random, unsystematic hand movements, but exhibit recurrent forms and formmeaning mappings (Bouvet 2001; Calbris 1990, 2003; Cienki 1998, 2005; Cienki and Mller 2008; McNeill 1992, 2005; Mittelberg 2006, 2008; Mller 1998, 2008; Nuez and Sweetser 2006; Parrill and Sweetser 2004; Sweetser 1998; Webb 1996). For each metaphorical meaning construal, it is necessary to determine locally whether the underlying metaphor is materialized in the speech and/or in the manual modality and how the relationship of source and target domains can be defined. For example, language and gesture do not necessarily exhibit the same metaphorical understanding, and gesture and speech may be motivated by different but compatible metaphors (see also Cienki and Mller 2008; Mller and Cienki this volume). Moreover, gesture may reveal metaphorical understandings even if the concurrent discourse is non-metaphorical (Mittelberg 2008; Mller 2003, 2008). These observations attest to the importance of gesture as a rich data source in cognitive linguistics in general (Sweetser 2007) and for embodiment theory in particular (Gibbs 1994, 2003, 2006; Lakoff and Johnson 1999; Taub 2001). Much less work has been done on metonymy in gesture. The objective of this section is two-fold. First, we will show that the interpretation of the gestures in our corpus is anchored in metonymy. In referring to an abstract notion, for instance, a metaphorical gesture relies on metonymic principles of sign formation: for example, via synecdoche, the hands may depict only the locally essential elements (parts) of the object or action (whole) in question (cf. Bouvet 2001; Mittelberg 2006; Mller 1998). Secondly, we suggest that due to its spontaneous and ephemeral nature co-speech gesture allows insights into the dynamics of figurative thought, and our analysis contributes to existing views of the definition of multimodal metaphor and the relationship between, e.g., source and target domain (for implications regarding static versus moving images, see Forceville 2003, 2005, 2006/this volume). 4.1 Non-metaphorical discourse and ad-hoc metaphorical visualizations in gesture As indicated earlier, spontaneous referential gestures tend to be polysemous and often need contextual support to be correctly interpreted; for example,

Metonymy first, metaphor second

337

one and the same gestural form may potentially refer to either a concrete or an abstract entity. Depicting via metonymy contextually pertinent features of objects or actions, referential gestures may either portray predominantly iconic sign-object relationships (representing concrete objects or movements), or they may rely on metaphorical sign-object relationships (involving abstract entities) and thus call forth a metaphorical interpretation. For example, a gesture with two hands may trace the frame of a painting or the frame of a theory. In both interpretations, the gesture is synecdochic since it provides only some aspects of the frame by rendering the parts that are pragmatically salient in the given discourse context. When used nonmetaphorically, the synecdochic gesture can be interpreted as referring to a spatial, physical structure (e.g., the essential panels of the frame itself, not the other elements that hold the painting in place). In the case of a metaphorical interpretation, the synecdochic gesture further represents, in Peirces (1960: 157) terms, some sort of parallelism or similarity between the form and function of a physical frame and the form and function of an abstract frame structure (Kller 1975). Adopting a cognitivist perspective, we can say that the gesture is interpreted with respect to the metaphorical concepts IDEAS ARE OBJECTS and CONCEPTUAL STRUCTURE IS GEOMETRIC PHYSICAL STRUCTURE (Lakoff and Johnson 1980; Sweetser 1998). Stated in McNeills (1992) terms, such images of abstract ideas (originally called metaphorics) represent both source and target domain information (in this example, the gesture would be regarded as representing both the conceptual frame and the physical frame). However, as we will see, the metaphorical gestures we will be discussing here are not directly iconic of the concrete source domain they involve. In fact, what is common to all of the metaphorical interpretations is that they rest on a first interpretation of the gesture through metonymy: e.g., the traces in the air have to be interpreted as meaning a frame of some sort; only then can the metaphorically-motivated object be accessed. Let us look at some examples from the data to determine how metaphor and metonymy are manifested in gesture and how source and target domains play out in the two modalities. The gesture represented in figure 1 is an example of a frequently occurring form that has several potential interpretations. Looking simply at the morphology of the gesture, we see that it consists of two, relatively relaxed, open hands held fairly far apart with palms facing each other (the right hand is partly closed because it contains a piece of chalk). If the speaker was referring linguistically to the length of a physical object such as a large box, the gesture would receive a concrete interpretation through metonymy, as if the speaker were holding an elongated object

338

Irene Mittelberg and Linda R. Waugh

like a box between his hands. However, in this case, the speaker is referring to a sentence and represents the sentence metonymically by the hands, which are assumed to be marking the beginning and end of its projection in space. The sentence is conceptualized metaphorically as bounded space or a large, elongated object. Thus, the gesture may be said to reflect some basic metaphorical concepts proposed in the cognitive linguistic literature: e.g., IDEAS ARE OBJECTS; CONTENTS ARE CONTAINERS; CATEGORIES ARE CONTAINERS; CONCEPTUAL STUCTURE IS GEOMETRIC PHYSICAL STRUCTURE (Lakoff and Johnson 1980; Sweetser 1998). To get to the idea of a sentence, however, the viewer first has to take a metonymical path from the hands to the space or the imaginary object. This is a case of external metonymy, because the speaker is holding the imaginary object between his hands, which are external (i.e., adjacent) to the object. And then to get from the object (or the space extending between the hands) to the sentence, the viewer has to take a metaphorical path from the imaginary concrete entity (or space) to the abstract entity (the sentence).4 (1)
(sentences) ... Sentences, \ G1 pvoh-bh far apart (..) [while theyre made up of words, _ G1 being held (...) arent made up of words, \]

Figure 1. A sentence as an elongated object held (or space extending) between two hands

It should be noted here that while the term sentence is non-metaphorical, its gestural portrayal is first metonymical in nature and then interpreted metaphorically. In other words, there are two interpretative moves needed to get to the imaginary object: (1) the hands represent, via (external) meton-

Metonymy first, metaphor second

339

ymy, the object held between them; (2) the object is a metaphorical representation of a sentence (which is a non-metaphorical linguistic expression). The imaginary object being held is metonymically inferred through the gesture itself (ACTION FOR OBJECT INVOLVED IN ACTION, Panther and Thornburg 2004). But the underlying metaphorical mapping, involving the target domains CONCEPTUAL STRUCTURE, IDEA, or CATEGORY and the source domains PHYSICAL STRUCTURE, OBJECT, or CONTAINER respectively, can only be inferred by a metaphorical interpretation of the metonymically conveyed object. Forceville suggests that [b]y contrast to monomodal metaphors, multimodal metaphors are metaphors whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006: 384). For this example, this definition holds: the target domain (sentence) is expressed linguistically and the source domain (object) is conveyed manually. The gesture above recurs in the data in slight variations referring to linguistic units of different degrees of complexity (words, phrases, constituents, sentences, etc.). By contrast, single words, units below the word level, and grammatical categories such as noun and verb are often represented by a single hand, for example by an open hand with the flat palm turned upward, thus forming a surface on which to present something to the addressee (see Mller 2004 for a detailed discussion of this gesture type). According to our analysis, the gesture is interpreted metonymically to mean that there is an object on the hand, and then, through metaphor, that object is interpreted as a word, a morpheme, a noun, or a verb. In example 2 (figure 2), this open hand is combined with a closed fist. The speaker, who is talking about morphological structure, illustrates the fact that the English noun teacher consists of two morphemes by forming two closed fists held next to each other. His left fist seemingly contains the lexical morpheme teach- and his right fist, which opens up into a relaxed palm-up open hand during the demonstration, contains the grammatical morpheme -er. Although the interpretation of the linguistic expressions relies on neither metaphor nor metonymy, the two figures of thought again are involved in a two-step process in this semiotically complex instance of indirect gestural reference. Here, however, there is no direct similarity (i.e., image iconicity) between the form of the gesture and the objects it refers to (as in the frame example discussed above). Instead, the enclosed fist is interpreted metonymically as containing, and the open hand as holding, small physical objects, e.g., LOCATION FOR OBJECT; ACTION FOR OBJECT INVOLVED IN ACTION; REPRESENTATION FOR REPRESENTED (Panther and Thornburg 2004; Wilcox 2004). Thus, the left hand serves as a CONTAINER and the right hand as a SUPPORT

340

Irene Mittelberg and Linda R. Waugh

structure for the imagined objects; they evoke, independently of the speech content, these two basic image schemas (cf. Johnson 1987; Mandler 1996).5 And in both cases, these imagined objects are metaphorically construed as being the two morphemes (IDEAS ARE OBJECTS, Lakoff and Johnson 1980). (2)
(the teach-er) our understanding of this is as speakers of English you know G1.1 1.2 1.3 [that the teacher] consists [of the] [and teacher (..)] 1.4 1.5 1.6 and [teacher] [consists of teach] [and er] 1.7 [not] 1.8 1.9 1.10 1.11 [the teacher con]sists [of the] [and teach] [and er].

Without going into the theoretical views regarding the differences between metaphor and metonymy currently debated in the cognitive linguistics literature, it should be noted that one of the received understandings holds that whereas metaphor is based on cross-domain mappings, metonymy consists of mappings within the same experiential domain (cf. Barcelona 2000a; see also Croft 1993; Radden 2000).6 In light of this domain-based definition of metonymy, we can say that both manual actions constitute common experiential domains of holding objects, and thus the gesturer can expect the viewers to easily relate to the action from their own experience and to build the basis for accessing the metaphorically construed objects. According to the two interpretative moves we introduced above, metonymy again comes first: the gestural vehicles (e.g., the hand configurations) serve as visible metonymic sources, that is, reference points (Langacker 1993); they point to the invisible target concepts (teach- and -er, sitting in/on the hands) that are mentioned in the concurrent discourse. These are instances of external metonymy, since the imagined objects are adjacent to (contained in or sitting on), but external to, the hands. The gestural form embodies the source, thus making it perceivable and present in the immediate context and pointing to the unperceivable target. So while the associative relation between visible source and associated invisible target is based on conceptual contiguity, the abstract notions are metaphorically construed as imaginary objects. Meton-

Metonymy first, metaphor second

341

ymy is also based on the fact that the two hands positioned somewhat near each other hold associated objects the metaphorically inferred lexical and grammatical morphemes referred to in the speech that together make up a word (PART FOR WHOLE, PART FOR PART, Jakobson 1956, 1963).

Figure 2. Morphemes as small objects on open hand or in closed fist

We are now in a position to define the relationship between source and target domains more clearly. The perceivable, manual modality triggers cognitive access to the abstract target via two interrelated mappings in which chunks of space extended between two hands or the imagined physical objects serve as a juncture between metonymy and metaphor. In figure 2, the imaginary object, presented on the right hand of the speaker, serves as both the target of the metonymic mapping (the hand stands for the object resting on it) and the source of the metaphoric mapping (a morpheme is a small object). The same holds for the gesture shown in figure 1, in which the speaker seems to be holding a large object between his two hands. Again, the hands (metonymic source) point to the object or the space (metonymic target) extending between them; or, put differently, the action of holding an object (metonymic source) stands for the object (metonymic target) itself. Thereby, the metonymically accessed imaginary object or, the chunk of space extending between the two hands is both the metonymic target and the metaphoric source, since it stands for the sentence (metaphoric target). The assumption that some metaphors are grounded in metonymy (Barcelona 2000a: 33; Geeraerts 2002; Goossens 1995: 171; Jakobson 1956, 1960; Lodge 1977: 111; Radden 2000: 93) holds in all the examples in our corpus. Whereas in these cases the speech itself is for the most part non-figurative (i.e., consists of technical grammatical terminology), figurative principles guide the interpretation of these dynamic multimodal representations: first metonymy and then metaphor contribute to the meaning-making processes

342

Irene Mittelberg and Linda R. Waugh

linking the manual modality to cognitive processes of association and imagination via contiguity and similarity. As we hope has become clear in our discussion, in order to interpret these spontaneous expressions conveyed through the manual modality, we rely on our embodied cognitive and sensory-motor abilities and schemata to see and feel the contiguity between the hands and the objects they seem to manipulate (or the hands and different amounts of space extending between them), which stand metaphorically for the ideas concurrently mentioned in the discourse. In other words, in order to arrive at the meaning of these gestures, the viewer can be assumed to perform an act of pragmatic inferencing (see work on metonymy in language by Panther and Thornburg 2003, 2004). To conclude this section, what is important to note here is that although, in the examples discussed so far, the concurrent speech is non-metaphorical (sentence, teach-, er), the gesture depicts the image schema (OBJECT) underlying the metaphorical projection (IDEAS ARE OBJECTS). The bodily modality thus spontaneously and dynamically expresses a metaphorical understanding of abstract entities as imaginary graspable objects. In the next section, we will examine multimodal representations of more elaborate theoretical constructs based on a well-defined set of conventionalized metaphors. 4.2. Metaphorical discourse and theory-based metaphorical visualizations Having discussed some of the most basic gestural forms that recur in the data, we will now turn to more complex multimodal representations of syntactic structure that are based on a specific model of linguistic structure, namely generative grammar. In these cases, there are ready-made metaphorical visualizations provided by the theory (tree structure diagrams to depict syntactic structure) that can then be referred to by the gestures. For example, when explaining dependent clauses in English, a speaker employing this framework used the right hand to sketch a branch of a tree structure diagram extending toward the lower right of her body. Figure 3 shows such a diagonally descending movement that is meant to represent an embedded clause. The speaker illustrates the idea of subordination (G1) by repeatedly moving her right hand first up to eye-level and then downward to her right side, thereby making a wave-like movement by tilting the hand from side to side. This can be assumed to roughly sketch out, through synecdoche, an elaborated tree structure, which is a diagrammatic metaphor used in generative grammar for the structure of complex sentences with subordinate clauses. More importantly, such tree structures are used in linguistic textbooks for

Metonymy first, metaphor second

343

learners and in research articles by scholars and in the case discussed here, there is a tree diagram behind the speaker on the white board. The fact that the speaker (who is left-handed) is talking about English, a right-branching language, may motivate the use of her right hand for the gesture even further. (3)
(wavy embedded clauses) G1 (G1 repeated) (G1 repeated) rh diagonal wavy line from head downward to the right [but this is gonna be another one with embedded sentences (G1 repeated) G2 rh extended arm and index finger point toward ground coming in verb phrases] [all the way down].

Figure 3. An embedded clause as a wavy diagonal line

The gesture is synecdochic (i.e., an instance of internal metonymy), but its proper interpretation entails a metaphorical interpretation of the metonymically given object, which is then inferred to be the same as the metaphors in the textbook or on the board. In the next gesture in the same utterance (G2) the speaker uses in her speech the metaphor (also tied to the tree diagram) all the way down to indicate the fact that in certain cases embedded sentences may continue almost without stopping. At this point the speaker extends her right arm towards the floor and points with her index finger straight to the ground in a deictic gesture (for a Peircean approach to deictic gestures see Fricke 2007). Without the background knowledge of the theory

344

Irene Mittelberg and Linda R. Waugh

and its canonical metaphors and diagrams, the gestures could not be interpreted correctly. This is different from the more intuitive examples in section 4.1, in which the speech was non-metaphorical (sentence, morphemes) and the gesture rendered a metaphorical understanding of abstract entities as objects or chunks of space without any ready-made visualization to fall back on. Since linguistic theories are often built on many specific metaphors, interactions of more than one metaphorical understanding can also be observed in the data. The subordination gesture (G1) in example (3) represents, as we just saw, the notion of embedded sentence (mentioned in the concurrent speech) as a wavy line descending in a diagonal toward the floor. Subordinated (embedded) entities are thought of in generative grammar as below the ones that dominate them. This indicates that the theory the speaker has in mind when talking about sentence structure motivates the form the gestures take. Moreover, the theory of syntactic structure proposed within generative grammar rests on a combination of spatial metaphors (i.e., the tree diagram) and power relations (i.e., dominance, control, etc.). The question that arises is whether, and if so, how, these two different source domains are made manifest in the verbal and/or manual modalities. In the example above, relations of dominance are not alluded to linguistically, but let us look at another sequence where the same speaker makes reference to the idea of dominance in the speech modality. Just as in figure 3, the gesture derives its meaning from the movement and the virtual traces left in the air. As shown in figure 4, the speaker draws a tree chunk in the form of a triangle in the air, with both hands starting out at the center top (the node) and then tracing diagonals outward and downward to either side of the body. The gesture is a synecdochic depiction that is metaphorically interpreted as meaning several technical terms (nodes alpha and delta, domination, and branching). Outside of this theoretical model, these terms do not necessarily entail spatial relationships, or if they do, then they might not be represented in exactly the same way (e.g., branching does not necessarily have a downward orientation). An interesting moment occurs when the speaker realizes that she was talking about a node dominating elements without actually having introduced the idea of dominance. In the speech modality, she quickly changes from the hierarchical understanding of dominance back to the spatial tree metaphor involving a node being on top of two things. Thus, the speech here is metaphorical in two compatible ways; compatible because spatial and social hierarchies both draw on spatial relations such as UP and DOWN, with certain values attached to each location in the corresponding system (e.g., POWER IS UP, see Lakoff and Johnson

Metonymy first, metaphor second

345

1980). Although in the speech modality there is, for a moment, a slight hesitation about which metaphor to use, the gesture modality consistently and repeatedly represents the spatial features of the tree model and thus is motivated by the corresponding spatial metaphor, which is a conventional part of the theory. (4)
(branching, domination) No=des, alpha and delta, / G1 bh, branch triangle, branching movement x2 [branch, \ (...7)] okay? / so thats a technical term, _ G2 bh, branch triangle, branching movement x2 (..) [when the nodes]--, / G3 rh branch G4 bh triangle, branching movement x2 [a node] [dominates--], / (..) woops I said a technical term too soon, G5 rh draws triangle G6 bh triangle branching (..) when [the node is on top] [of two things] or more, / (..) it branches. \ /

Figure 4 represents (1) the very beginning of the branching gesture (hands are joined at the top, the node, of the triangle) and (2) the repeated downward movement that reinforces the idea of an active branching process. As for the underlying conceptual metaphors, these gestural diagrams all reflect the metaphor SYNTACTIC STRUCTURES ARE GEOMETRIC PHYSICAL STRUCTURES, based on CONCEPTUAL STRUCTURE IS GEOMETRIC PHYSICAL STRUCTURE (Sweetser 1998) discussed above. Given its specific semiotic affordances, gesture, a semiotic system exploiting space, provides a spatial projection of compatible metaphors stemming from the domains of physical structures and social hierarchies with a built in up-down orientation (POWER/HIGH STATUS IS UP; HAVING CONTROL IS UP; BEING SUBJECT TO CONTROL IS DOWN; LOW STATUS IS DOWN; Lakoff and Johnson 1980; Sweetser 1998). Following canonical tree diagrams, these gestures depict

346

Irene Mittelberg and Linda R. Waugh

logical relations between entities (which is one of the central functions of diagrams; see Peirce 1960; Mittelberg 2008; Waugh et al. 2004). While within the domain of meta-grammatical discourse dominance is indeed used metaphorically, in the manual modality the theory-driven spatial conceptualization of linguistic structure seems to be the predominant, overriding metaphorical understanding that motivates the representation of the behavior and relationship of elements in a sentence. As in the examples discussed in section 4.1, we can also discern a double mapping here: (1) there is external metonymy between the hand (metonymical source) and the trace left in the air (metonymical target), whereby the hand and the line drawn are part of the same experiential domain of drawing a tree structure (whether it is on paper, on a blackboard, or in the air); and (2) through metaphorical projection these spatial tree structures depicted by the gesture (metaphorical source) simultaneously represent the abstract conceptual structure (metaphorical target). The difference between the examples discussed in this section and section 4.1 is that whereas in 4.1 the speech is non-metaphorical, here it is metaphorical and the metaphors referred to are associated with theory-based canonical visualizations of abstract structure that can be easily mimicked by gesture. Also, the speaker had probably drawn many such tree diagrams on blackboards or paper before depicting them gesturally.

Figure 4. Sentence structure as a laterally branching tree chunk

5. Concluding remarks: Cross-modally achieved, intertwined figures of thought What all the examples in this paper show is that whether the metaphorical interpretation of the metonymic gesture is simple and easily accessible (e.g., a sentence, morphemes) or complex and only understandable in the context

Metonymy first, metaphor second

347

of a given linguistic theory (e.g., a tree structure, branching, dominance), they still adhere to the general principle of metonymy first, metaphor second, or to say it in another way, metonymy, whether external (adjacency/contact) or internal (synecdoche), leads the way into metaphor. Due to the abstract nature of the subject matter in linguistics courses, the objects in question are conceptualized via metaphor. But metonymy is needed to access the metaphor: e.g., external metonymy (contiguity through adjacency/contact) between, for example, the fist and the small object it seems to enclose, or between the hand drawing a line in the air and the imaginary trace that this movement leaves behind. In the latter case, there is also synecdoche between the diagram on the blackboard, for instance, and the sketchy hand movement representing it. Using the terms of contemporary metonymy theory, we have also claimed that the hands and the actions they perform constitute a common experiential domain and that the imaginary objects or traces are pragmatically inferred from the performed actions. Taking the material side of gestures as a point of departure, we thus identified associative processes involving two intertwined mappings leading from the form of the gesture to the metaphorically construed entity it stands for. The metonymic mapping functions as follows: in the case of the closed fist that co-occurs with the mention of the morpheme teach, the perceivable fist serves as a metonymic source triggering cognitive access to the imaginary object inside of it, that is, the metonymic target. In the ensuing metaphoric process, the metonymic target, i.e., the object, becomes the metaphoric source that is mapped onto the metaphoric target, that is, the linguistic unit teach. Both of these figurative, multimodally achieved efforts are needed to make abstract entities and conceptual structures visible (e.g., when the teacher is unable to point to words or diagrams written on the board), thus grounding them in the immediate teaching context and making them graspable for the student audience. In order to arrive at what is referred to, the addressee of these dynamic multimodal representations needs to interpret a combination of not only speech and gesture, but also metonymy and metaphor, in that order. We also saw that metaphorically motivated gestural forms do not always coincide with metaphorical speech. Whereas in the examples of the sentence or the morpheme teach-, discussed in section 4.1, the speech is technical rather than metaphorical, the gestural illustrations are metaphorically motivated, featuring imaginary physical objects or assigning meaning to chunks of space extending between the manual articulators. These ad-hoc gestural metaphors stand in contrast to the sequences discussed in section 4.2, in which the speech is metaphorical; however, the metaphors used in the

348

Irene Mittelberg and Linda R. Waugh

speech are given by the theory of generative grammar, which also provides conventional ways of diagramming syntactic structure in the form of inverse tree diagrams. Accordingly, the gestures that depict aspects of embedded sentences or dominance are more or less sketchy (i.e., synecdochic) renditions of those ready-made visualizations. In the light of the importance that those who work on multimodal manifestations of figurative thought place on the specific materiality and logic of each modality (e.g., Cienki and Mller 2008; Mller and Cienki, this volume; Forceville 2006; Kress and van Leeuwen 2006; Kress et al. 2001; Mittelberg 2002, 2006), it is interesting to realize that making sense out of what a speaker-gesturer is trying to convey involves our imaginative abilities as much as our visual and auditory senses. Interpreting gestures entails combining perceivable visual and verbal materialized information; but the manual configurations and movements also appeal to our capacity during the process of interpretation to assign meaning to empty space and to fill in missing information, for example, when inferring objects and actions from gestures involving closed fists, open hands, or lines drawn in the air. In the multimodal manifestations of metaphor and metonymy examined above, source and target meanings are not always neatly distributed across the two modalities (see Forceville 2006), and gesture may be the only modality in which the metaphor is expressed (especially when it is spatial metaphor). Of course, there are also instances in the data in which the speech is metaphorical but there is no gesture. As we saw, source and target domains of a mapping are not necessarily co-present in a given instance of multimodal representation: they may need to be inferred by interpretative hypotheses (Peirce 1991, 1992, 1998) from the discourse and/or physical context (neighborhood/contexture; Jakobson 1956), or the knowledge of the linguistic theory talked about. Since gesture is a largely unconscious, spontaneous means of expression, the multimodal metaphors discussed here can hardly be compared with elaborated and consciously chosen metaphorical messages in cartoons or advertisements (see Forceville 1996, 2002, 2005; El Refaie 2003, this volume; Yus, this volume; Schilperoord and Maes, this volume). And the question of whether the linguistic explanations and the linguistically expressed metaphors could be recognized and understood by the audience without the gestural support is not answerable on the basis of our data. However, the interplay between metaphor and metonymy deserves, as has been shown already (e.g., Bouvet 2001; Forceville 2005; Gibbs 1994; Whittock 1990), a more detailed scrutiny in other forms of multimodal communication. The Jakobsonian (and Peircean) notions, combined with contemporary cognitivist

Metonymy first, metaphor second

349

approaches, are a way to account for not only the materialized dimensions of figures of thought motivating multimodal discourse, but also for their cognitive and imaginative dimensions.

Acknowledgements
We wish to thank the editors of this volume, Charles Forceville and Eduardo Urios-Aparisi, as well as Alan Cienki and Cornelia Mller for insightful comments on an earlier version of this chapter. We are also grateful to Allegra Giovine, Joel Ossher, and Daniel Steinberg for their help with data coding and to Yoriko Dixon for providing the artwork.

Notes
1. 2. This is very similar to Barthes concept of anchoring (see for discussion Forceville 1996: 71). For reasons of space, the functions fulfilled by the additional modalities and artifacts in the class environment and by the teachers facial expressions or gaze cannot be included in the analysis here (see Kress et al. [2001] and Ochs et al. [1996] for work on multimodality in the science classroom). See Furuyama (2001) regarding Jakobsons concept of the poetic function in gesture. The abbreviations used in the transcript are to be read as follows: puoh stands for palm-up open hand, pvoh stands for palms vertical open hand, bh stands for both hands. As for gesture-speech synchrony, the speech segments that coincide with a gesture are set off by square brackets, speech segments highlighted in bold face represent the gesture stroke (the peak of a gestural expression), and underlined speech segments indicate a post-stroke gesture hold. G1 in example 1 stands for Gesture 1. For more details see Mittelberg (2006, 2007). According to Johnson (1987: xiv), an image schema is defined as a recurring, dynamic, pattern of our perceptual interactions and motor programs that gives coherence and structure to our experience. See Mittelberg (2006, forthcoming) for a complete list of the image-schematic and geometric schemata that emerged from the data. Barcelona (2000a: 4) gives the following definition: Metonymy is a conceptual projection whereby one experiential domain (the target) is partially understood in terms of another experiential domain (the source) included in the same common experiential domain (italics in original).

3. 4.

5.

6.

350

Irene Mittelberg and Linda R. Waugh

References
Barcelona, Antonio 2000a The cognitive theory of metaphor and metonymy. In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 128. Berlin/New York: Mouton de Gruyter. Barcelona, Antonio (ed.) 2000b Metaphor and Metonymy at the Crossroads: A Cognitive Perspective. Berlin/New York: Mouton de Gruyter. Bradford, Richard 1994 Roman Jakobson: Life, Language, Art. London/New York: Routledge. Bouvet, Danielle 2001 La Dimension Corporelle de la Parole. Les Marques PosturoMimo-Gestuelles de la Parole, leurs Aspects Mtonymiques et Mtaphoriques, et leur Rle au Cours dun Rcit. Paris: Peeters. Calbris, Genevieve 1990 The Semiotics of French Gestures. Bloomington: University of Indiana Press. 2003 From cutting an object to a clear cut analysis: Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture 3 (1): 1946. Carroll, Noel 1994 Visual metaphor. In Aspects of Metaphor, J. Hintikka (ed.), 89218. Dordrecht: Kluwers. Cienki, Alan 1998 Metaphoric gestures and some of their relations to verbal metaphoric expressions. In Discourse and Cognition: Bridging the Gap, Jean-Pierre Koenig (ed.), 189204. Stanford: CSLI Publications. 2005 Image schemas and gesture. In From Perception to Meaning: Image Schemas in Cognitive Linguistics, Beate Hampe (ed.), in cooperation with Joseph Grady, 421442. Berlin/New York: Mouton de Gruyter. Cienki, Alan, and Cornelia Mller (eds.) 2008 Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins. Croft, William 1993 The role of domains in the interpretation of metaphor and metonymy. Cognitive Linguistics 4: 335370. Danaher, David 1998 Peirces semiotic and Cognitive Metaphor Theory. Semiotica 119 (12): 171207.

Metonymy first, metaphor second

351

Dirven, Ren 2002 Metonymy and metaphor: Different mental strategies of conceptualisation. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.), 75111. Berlin/New York: Mouton de Gruyter, Dirven, Ren, and Ralf Prings (eds.) 2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New York: Mouton de Gruyter. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2003 Bildliche und multimodale Metaphern in Werbespots. Zeitschrift fr Semiotik 25 (12): 3960. 2005 Visual representations of the Idealized Cognitive Model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37: 6988. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. Fricke, Ellen 2007 Origo, Geste und Raum Lokaldeixis im Deutschen. Berlin/New York: Mouton de Gruyter. Furuyama, Nobuhiro 2001 De-syntacticizing the theories of reference maintenance from the viewpoint of poetic function of language and gesture: A case of Japanese discourse. Ph.D. diss., University of Chicago. Geeraerts, Dirk 2002 The interplay of metaphor and metonymy in composite expressions. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.), 435465. Berlin/New York: Mouton de Gruyter. Gibbs, Raymond W., Jr. 1994 The Poetics of Mind: Figurative Thought, Language, and Understanding. Cambridge: Cambridge University Press. 1999 Speaking and thinking with metonymy. In Metonymy in Language and Thought, Karl-Uwe Panther and Gnter Radden (eds.), 6176. Amsterdam/Philadelphia: Benjamins. 2003 Embodied experience and linguistic meaning. Brain and Language 84: 115. 2006 Embodiment and Cognitive Science. Cambridge: Cambridge University Press.

352

Irene Mittelberg and Linda R. Waugh

Goldin-Meadow, Susan 2003 Hearing Gesture: How Hands Help Us Think. Cambridge, MA/London: The Belknap Press of Harvard University Press. Goodwin, Charles 2003 The semiotic body in its environment. In Discourses of the Body, Justine Coupland and Richard Gwyn (eds.), 1942. Basingstoke/New York: Palgrave-Macmillan. Goossens, Louis 1995 Metaphtonymy: The interaction of metaphor and metonymy in figurative expressions for linguistic action. In By Word of Mouth: Metaphor, Metonymy and Linguistic Action in a Cognitive Perspective, Goossens, Louis, Paul Pauwels, Barbara Rudzka-Ostyn, A.-S. Simon-Vandenbergen, and Jan Vanparys (eds.), 159174. Amsterdam/Philadelphia: Benjamins. Hiraga, Masako 1994 Diagrams and metaphors: Iconic aspects in language. Journal of Pragmatics 22: 521. 2005 Metaphor and Iconicity: A Cognitive Approach to Analysing Texts. Basingstoke/New York: Palgrave-MacMillan. Jakobson, Roman 1953 Results of a joint conference of anthropologists and linguists. In Selected Writings II: Word and Language, 554567. 1956 Two aspects of language and two types of aphasic disturbances. In On Language, Linda Waugh, and Monique Monville-Burston (eds.), 115133. 1960 Linguistics and poetics. In Language in Literature, Krystyna Pomorska, and Stephen Rudy (eds.), 6294. 1963 Parts and wholes in language. In On Language, Linda Waugh and Monique Monville-Burston (eds.), 110114. 1966 Quest for the essence of language. In On Language, Linda Waugh, and Monique Monville-Burston (eds.), 407421. 1971 Selected Writings II: Word and Language. The Hague: Mouton. 1987a On the relation between auditory and visual signs. In Language in Literature, Krystyna Pomorska and Stephen Rudy (eds.), 467473. 1987b Language in Literature, Krystyna Pomorska and Stephen Rudy (eds.). Cambridge, MA: Harvard University Press. Jakobson, Roman, and Krystyna Pomorska 1983 Dialogues. Cambridge, MA: MIT Press. Johnson, Mark 1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University of Chicago Press.

Metonymy first, metaphor second

353

Kendon, Adam 2000 Language and gesture: unity or duality? In Language and Gesture, David McNeill (ed.), 4763. Cambridge: Cambridge University Press. 2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. Kennedy, John M. 1982 Metaphor in pictures. Perception 11: 589605. Kller, Wilhelm 1975 Semiotik und Metapher. Untersuchungen zur grammatischen Struktur und kommunikativen Funktion von Metaphern. Stuttgart: Metzler. Kvecses, Zoltn 2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press. Kress, Gunter, and van Leeuwen, Theo 2006 Reading Images: The Grammar of Visual Design. 2nd ed. London/New York: Routledge. Kress, Gunter, Carey Jewitt, Jon Ogborn, and Charalampos Tsatsarelis 2001 Multimodal Teaching and Learning: The Rhetorics of the Science Classroom. London/New York: Continuum. Lakoff, George 1987 Women, Fire, and Dangerous Things: What Categories Reveal About the Mind. Chicago: University of Chicago Press. 1993 The contemporary theory of metaphor. In Metaphor and Thought, 2nd ed., Andrew Ortony (ed.), 202251. Cambridge: Cambridge University Press. Langacker, Ronald W. 1993 Reference-point constructions. Cognitive Linguistics 4: 138. Lodge, David 1977 Two Modes of Modern Writing: Metaphor, Metonymy, and the Typology of Modern Literature. Ithaca, NY: Cornell University Press. Mandler, Jean 1996 Preverbal representation and language. In Language and Space, Paul Bloom, Mary A. Peterson, Lynn Nadel, and Merrill F. Garrett (eds.), 365384. Cambridge, MA: MIT Press. McNeill, David 1992 Hand and Mind: What Gestures Reveal about Thought. Chicago: Chicago University Press. 2005 Gesture and Thought. Chicago: Chicago University Press. McNeill, David (ed.) 2000 Language and Gesture. Cambridge: Cambridge University Press.

354

Irene Mittelberg and Linda R. Waugh

McNeill, David, Francis Quek, Karl-Eric McCullough, Susan D. Duncan, Nobuhiro Furuyama, Rolbert Bryll, and Rashid Ansari 2001 Catchments, prosody and discourse. Gesture 1 (1): 933. Mittelberg, Irene 2002 The visual memory of grammar: Iconographical and metaphorical insights. Metaphorik.de, 02/2002: 6989. 2006 Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar. Ph.D. Diss., Cornell University. 2007 Methodology for multimodality: One way of working with speech and gesture data. In Methods in Cognitive Linguistics, Monica Gonzalez-Marquez, Irene Mittelberg, Seana Coulson, and Michael Spivey (eds.), 225248. Amsterdam/Philadelphia: Benjamins. 2008 Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar. In Metaphor and Gesture, Alan Cienki and Cornelia Mller (eds.), 115154. Amsterdam/Philadelphia: Benjamins. forthc. Geometric and image-schematic patterns in gesture space. In Language, Cognition, and Space: The State of the Art and New Directions, Vyvyan Evans and Paul Chilton (eds.). London: Equinox. Mller, Cornelia 1998 Redebegleitende Gesten. Kulturgeschichte Theorie Sprachvergleich. Berlin: Berlin Verlag A. Spitz. 2003 Gesten als Lebenszeichen toter Metaphern. Zeitschrift fr Semiotik 25 (12): 6172. 2004 Forms and uses of the Palm Up Open Hand: A case of a gesture family? In The Semantics and Pragmatics of Everyday Gesture: The Berlin Conference, Cornelia Mller and Roland Posner (eds.), 233 256. Berlin: Weidler. 2008 Metaphors. Dead and Alive, Sleeping and Waking. A Cognitive Approach to Metaphors in Language Use. Chicago: Chicago University Press. Mller, Cornelia, and Alan Cienki this vol. Words, gestures, and beyond: Forms of multimodal metaphor in the use of spoken language. Nez, Rafael, and Eve E. Sweetser 2006 Aymara, where the future is behind you: Convergent evidence from language and gesture in the cross-linguistic comparison of spatial construals of time. Cognitive Science 30: 149. Ochs, Eleanor, Patrick Gonzalez, and Sally Jacoby 1996 When I come down Im in the domain state: Grammar and graphic representation in the interpretive activity of physics. In Interaction and Grammar, Eleanor Ochs, Emanuel Schegloff, and Sandra Thompson (eds.), 329369. Cambridge: Cambridge University Press.

Metonymy first, metaphor second

355

Panther, Klaus-Uwe, and Gnter Radden (eds.) 1999 Metonymy in Language and Thought. Amsterdam/Philadelphia: Benjamins. Panther, Klaus-Uwe, and Linda L. Thornburg, (eds.) 2003 Metonymy and Pragmatic Inferencing. Amsterdam/Philadelphia: Benjamins. 2004 The role of conceptual metonymy in meaning construction. Metaphorik.de 06/2004: 91113. Parrill, Fey, and Eve E. Sweetser 2004 What we mean by meaning: Conceptual integration in gesture analysis and transcription. Gesture 4 (2): 197219. Peirce, Charles Sanders 1960 Collected Papers of Charles Sanders Peirce (19311958), vol. I: Principles of Philosophy, vol. II: Elements of Logic, Charles Hartshorne and Paul Weiss (eds.). Cambridge: The Belknap Press of Harvard University Press. 1991 Peirce on Signs: Writings on Semiotic, James Hoopes (ed.). Chapel Hill: University of North Carolina Press. 1992 The Essential Peirce: Selected Philosophical Writings, vol. 1 (18671893), Nathan Houser and Christian Kloesel (eds.). Indiana: Indiana University Press. 1998 The Essential Peirce: Selected Philosophical Writings, vol. 2 (18931913), Nathan Houser and Christian Kloesel (eds.). Indiana: Indiana University Press. Radden, Gnter 2000 How metonymic are metaphors? In Metaphor and Metonymy at the Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 93 98. Berlin/New York: Mouton de Gruyter. Schilperoord, Joost, and Alfons Maes this vol. Visual metaphoric conceptualization in editorial cartoons. Shapiro, Michael 1983 The Sense of Grammar: Language as Semeiotic. Indiana: Indiana University Press. Sweetser, Eve 1990 From Etymology to Pragmatics: Metaphorical and Cultural Aspects of Semantic Structure. Cambridge: Cambridge University Press. 1998 Regular Metaphoricity in Gesture: bodily-based models of speech interaction. Actes du 16e Congrs International des Linguistes (CDROM), Amsterdam: Elsevier. 2007 Looking at space to study mental spaces: Co-speech gesture as a crucial data source in cognitive linguistics. In Methods in Cognitive Linguistics, Monica Gonzalez-Marquez, Irene Mittelberg, Seana Coulson and Michael Spivey (eds.), 201224. Amsterdam/Philadelphia: Benjamins.

356

Irene Mittelberg and Linda R. Waugh

Streeck, Jrgen 2002 A body and its gestures. Gesture 2 (1): 1944. Taub, Sarah 2001 Language from the Body: Iconicity and Metaphor in American Sign Language. Cambridge: Cambridge University Press. Turner, Mark, and Gilles Fauconnier 2002 Metaphor, metonymy, and binding. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.), 469487. Berlin/New York: Mouton de Gruyter. Wales, Katie 2001 A Dictionary of Stylistics, 2nd ed. London: Longman. Waugh, Linda R. 1998 Semiotics and language: The work of Roman Jakobson. In Hi Fives: A Trip to Semiotics, Roberta Kevelson (ed.), 85102. New York: Peter Lang. 2000 Jakobson, Roman (18961982). In Concise Encyclopedia of Pragmatics, Jacob Mey (ed.) Oxford: Pergamon. Waugh, Linda R., and Monique Monville-Burston 1990 Roman Jakobson: His life, work and influence. In Jakobson on Language, Linda R. Waugh and Monique Monville-Burston (eds.), 145. Cambridge, MA: Harvard University Press. Updated version published as 2002, Introduction to Roman Jakobson: Selected Writings, I, third edition, v-xcviii. Waugh, Linda R., Norma Barletta, Susan Smith, Elizabeth Specker, Shawn Steinhart, and Jue Wang 2004 Peircean theory, diagrammatic iconicity and academic texts: Global structure, abstracts, and the role of narrative. Logos and Language. Journal of General Linguistics and Language Theory 6: 3962 (special issue on Aspects of Iconicity in Contemporary Linguistics). Webb, Rebecca 1996 Linguistic Features of Metaphoric Gestures. Ph.D. Diss., University of Rochester. Whittock, Trevor 1990 Metaphor and Film. Cambridge: Cambridge University Press. Wilcox, Phyllis P. 2004 A cognitive key: Metonymic and metaphorical mappings in ASL. Cognitive Linguistics 15 (2): 197222. Wilcox, Sherman, and Jill Morford 2007 Empirical methods in signed language research. In Methods in Cognitive Linguistics, Monica Gonzalez-Marquez, Irene Mittelberg, Seana Coulson and Michael Spivey (eds.), 171200. Amsterdam/ Philadelphia: Benjamins.

VI Metaphor Involving Music and Sound

Chapter 15 Music, language, and multimodal metaphor Lawrence M. Zbikowski

Abstract
This chapter considers the topic of multimodal metaphor from the perspective of cross-domain mappings between the musical and the linguistic domains. Beginning with an example of what musicans call text painting (in which music is used to paint an image related to the text of a vocal work), I explore the different ways music and language structure thought. Examples of musical passages from Palestrina, Biber, Bach, Schubert, and Jerome Kern are used to demonstrate how music contributes to meaning construction and thus may serve as a source domain for a multimodal metaphor. I conclude with a brief discussion of how conceptual blending theory can be used for the analysis of text-music relations, and the multimodal metaphors that may result. Keywords: music, text-painting, conceptual blending, popular song, musical grammar, Bach, Schubert

1. Introduction Giving voice to an idea that took a number of forms in his later work, Ludwig Wittgenstein, near the end of his Philosophical Investigations, wrote, Understanding a sentence is much more akin to understanding a theme in music than one may think (Wittgenstein 2001: frag. 527). The close relationship between language and music suggested by Wittgensteins observation is borne out by similarities between the two: both are unique to the human species, both unfold over time, both have syntactic properties, and both make use of sound. There are also, of course, notable differences: musical meaning is on the whole much less precise than linguistic meaning; music often involves simultaneous events, where language does not; and there is more of a sense of play in ordinary music than there is in ordinary

360

Lawrence M. Zbikowski

language. On the one hand, these differences suggest that language and music belong to two different conceptual domains. On the other hand, the similarities between the two suggest that language and music may recruit some of the same cognitive resources, and that structure from one domain may be readily mapped to the other to create meaning. Understanding a sentence is like understanding a musical theme because both language and music offer possibilities for constructing meaning, possibilities that can be exploited through multimodal metaphors. As an example, consider the passage from the Credo of Giovanni Pierluigi da Palestrinas Pope Marcellus Mass (printed 1567) given in figure 1.1 The text Palestrina sets here is Qui propter nos homines et propter nostram salutem descendit de clis (Who for us men, and for our salvation, came down from heaven). With the first statement of the word descendit, each voice begins a scalar descent. Christs descent from heaven is thus represented with a cascading fall through musical space, a series of overlapping movements down the musical scale. This representation exploits the common construal of musical pitches as situated in vertical space, a construal that follows from the characterization of pitches as high or low with respect to one another.

Figure 1. Giovanni Pierluigi da Palestrina, Credo of the Pope Marcellus Mass, mm. 53-58.

Music, language, and multimodal metaphor

361

Although this characterization seems quite natural, it is actually rather arbitrary. For instance, when you play higher notes on the piano, you move your hand to the right; when you play higher notes on a cello, you move your hand down toward the ground.2 Describing musical pitches in terms of high and low is in fact a product of mapping structure from one domain of knowledge (relationships among points in vertical space) onto another (relationships among musical pitches). From the perspective provided by the contemporary theory of metaphor (Lakoff 1993), this mapping relies on the conceptual metaphor PITCH RELATIONSHIPS ARE RELATIONSHIPS IN PHYSICAL SPACE. Language, of course, facilitates this mapping. In that language belongs to one mode (that of either spoken or written signs) and music to another, the metaphorical description of pitches as high or low is multimodal in at least a minimal sense. If, however, we take a closer look at Palestrinas compositional choices, a somewhat more textured view of multimodal metaphors that involve music emerges. Were Palestrina interested only in portraying a move from high to low, he could have used a single falling interval rather than his stepwise descending scale. Using the resources offered by six voices, he also reinforced and inflected his sonic image of descent through a series of multiple, and subtly different, descending gestures, all of which culminate in the important cadential arrival of measure 58. What Palestrina gives us, then, is the sound of descent, realized as an orderly, stately process. Contrast his setting of descendit with that of Heinrich Biber at the corresponding point in the Credo from Bibers Missa Christi resurgentis, written a little over a hundred years after Palestrinas mass. As shown in figure 2, the path sketched by Biber is much more involved, comprising fifteen notes and proceeding through a series of twisting turns before reaching its goal. Palestrina, for his part, takes only eight notes, and never changes direction.3 The sonic image of descent offered by Biber is consequently quite different than that offered by Palestrina. Bibers descent

Figure 2. Comparison of melodic passages from Heinrich Biber, Credo of the Missa Christi resurgentis, mm. 51-57 and Giovanni Pierluigi da Palestrina, Credo of the Pope Marcellus Mass, mm. 53-55.

362

Lawrence M. Zbikowski

is a leisurely, almost caressing, affair in which the journey is at least as important as the goal. Indeed, when Biber approaches the cadence that articulates the end of this section of the Credo he leaves behind the melodic material of figure 2 and adopts a much more proclamatory style, effectively bracketing off the sensation of descent from its goal. Palestrinas and Bibers settings of descendit demonstrate, in a concise way, the different resources offered by language and music for the construction of multimodal metaphors. Language gives access to a rich network of conceptual frameworks. High and low, for instance, are used not only for orientation in physical space, but also as evaluative terms: Christs descent is from heaven, where everything is good, to earth, where it is not, a perspective that relies on the conceptual metaphor STATE OF BEING IS ORIENTATION IN VERTICAL SPACE.4 As another example of the conceptual frameworks to which language offers access, the continuum of vertical space can be divided up into a series of points with distances between them. When these points are mapped onto musical pitches (through the conceptual metaphor PITCH RELATIONSHIPS ARE RELATIONSHIPS IN PHYSICAL SPACE) it becomes possible to measure the distance between any two notes. I would emphasize that neither of these conceptual frameworks has immediate relevance for Palestrinas or Bibers setting of descendit, yet both are accessible through the conceptual networks to which the high and low of musical space is connected. Music, for its part, infuses the process of meaning construction with a crucial dynamic aspect. It is not simply the concept of descent through vertical space that we draw from the musical domains set up by Palestrina and Biber but particular kinds of descent, each with its own texture and shape. Where language provides us with only the bare prompt of the word descendit, a notion we fill out with our own experiences with the process of descent, music actively sketches different sorts of descents through carefully arranged sonic materials. In this chapter I want to explore in greater detail multimodal metaphors that involve language and music from both theoretical and practical perspectives. The theoretical perspective, which is developed in the first main section that follows, concerns the different functions of language and music in human culture. I should note that in this section, and indeed throughout this chapter, I shall be concerned with the ways musical materials actually give rise to meaning rather than the meanings with which they may become associated.5 The practical perspective, which is developed in the second main section, begins with a treatment of my opening examples in terms of the theoretical perspective for which I have argued and expands the discussion to consider relationships between text and music in greater detail. In a con-

Music, language, and multimodal metaphor

363

cluding section I shall return to Wittgensteins observations about understanding in language and music, and consider what multimodal metaphors can tell us about how we understand language and music.

2. Function and structure in language and music As was noted above, there are numerous similarities between language and music, and there are any number of cultural practices that blur the boundaries between the two modes of communication (Boiles 1967; Forns 2003). It is still the case, however, that we do not live our lives inside the equivalent of a grand opera or Broadway musical, with every utterance sung and every action accompanied by an orchestra. So why is it that human cultures have developed both language and music? Although there is any number of ways to approach this question, I take the position that language and music have different functions within human culture. In doing so I am influenced by the work of the developmental psychologist Michael Tomasello, who situates the emergence of language in our species within the broader development of human culture. In Tomasellos view, the primary function of language is to direct the attention of another person to objects or concepts within a shared referential frame (Tomasello 1999: chap. 5). I would argue that music is similarly part of a cultural framework unique to our species, but one whose primary function is to represent through patterned sound various dynamic processes that are common in human experience. Chief among these dynamic processes are those associated with the emotions (which, following recent work by Antonio Damasio, can be construed as sequences of physiological and psychological events that subtend feelings [Damasio 1999, 2003]) and the movements of bodies including our own through space. The difference in function between these two modes of communication is matched by a difference in the forms through which the functions are realized. In the case of language, this is accomplished through symbolic units that correlate with functions such as those represented by nouns and verbs, as well as with the many other parts of speech recognized by grammarians (Croft 2001: chap. 2). Through the use of these symbolic units we can direct the attention of another person to objects or concepts within a shared referential frame. In the case of music, the basic formal unit is what I call a sonic analog, which represents through patterned sound the central features of some dynamic process.6 Descent, for instance, is one such dynamic process: Palestrina, in the example shown above, provides one sonic analog for this

364

Lawrence M. Zbikowski

process; Biber, in the Credo from his mass, provides another. I would hasten to add that neither musical passage stands for descent in a direct or unequivocal fashion. Instead, the phenomenal properties of these passages are such that they can serve as analogs for the process of descent.7 Given this perspective, it should be clear that mappings between the domains of language and music will involve structures that are fundamentally different in kind. Language tends to focus on objects (whether real or imagined) and relationships between objects. Language can direct our attention to a process (that is, the noun descent picks out a dynamic process that involves a traversal of space), but it is less common for language to embody such a process. When it does when, for instance, we imitate the sound of a horses step with the words clip-clop, clip-clop language starts to become more like music. Music, for its part, does not tend to be involved with the rich symbolic systems typical of language.8 In those cases where music does exploit this sort of symbolic system as, for instance, through musical topics of the type employed by Mozart and Haydn (Ratner 1980; Allanbrook 1983; Agawu 1991, 1999) its dynamic aspect tends to recede in importance. In sum, then, mappings from language to music will tend to focus on static aspects of the musical domain; mappings from music to language will draw out the dynamic aspects of the domain of language. Although any attempt to determine a crisp boundary for what counts as language or what counts as music may be an endeavor destined to generate more heat than light, it seems clear that, at least in their characteristic usage in the contemporary world, language and music have different functions. While the range of language functions is broad, primary among these is the use of symbolic tokens to direct the attention of another person to objects and relations within a shared referential frame. Music, by contrast, provides sonic analogs for a wide range of dynamic processes that are marked in human experience, especially those associated with the regulation of emotions. Multimodal metaphors that involve language and music draw on both of these resources, as the analyses in the next section shall make clear.

3. Text Painting Because so much of our communication is done through language, the contribution of non-linguistic forms of communication to the construction of meaning often goes unnoticed. However, as work on the gesture that accompanies speech has shown (Goldin-Meadow 2003a; Goldin-Meadow 2003b; Kendon 2004; McNeill 1992, 2005; Mller and Cienki this volume; Mittel-

Music, language, and multimodal metaphor

365

berg and Waugh this volume), non-linguistic forms of communication are an important way humans shape their thought. Evidence for a similar role for prosody has been provided by a recent set of experiments that demonstrated the use of pitch inflection as an analog for motion through space, and rate of delivery as an analog for the speed of an object (Shintel, Nusbaum, and Okrent 2006). With regard to the latter, when subjects were asked to characterize the motion of a dot that moved horizontally across a screen at one of two speeds by saying either It is going left or It is going right, they spoke more rapidly if the dot was moving quickly than if it was moving slowly. Moreover, listeners were able to make accurate judgments about the speed of the dot when they heard recordings of these statements. In the case of multimodal metaphors that involve language and music, a consequence of the emphasis on language has been that mappings from language to music have received the most attention (Zbikowski 2008). Mappings from music to language which function, as do gesture and prosody, to shape thought have been, by comparison, rather neglected (although see Lidov 2005 for a recent corrective). The compositional technique of text painting provides an opportunity to explore both ways of mapping in more detail, and to develop a better understanding of multimodal metaphors that involve music. The basic idea of text painting is simple enough. When a particularly strong or compelling image occurs in the text for a musical work, the composer writes the accompanying music to suggest, or paint, the image. Thus if the text mentions a galloping horse, the music coincident with the text might imitate the sound and action of a horse proceeding at full speed. The passages from Palestrinas and Bibers masses that I discussed in my introductory comments are typical examples of text painting, but with one illuminating characteristic: there is in fact nothing in Palestrinas or Bibers music that imitates the sound of an actual descent. Indeed, the sounds associated with descents have little in common with a descending scale, and in some cases for instance, walking down a hill descent may be virtually soundless. Text painting is, in consequence, not so much about imitating some naturally-occurring sound as it is about providing sonic analogs for various dynamic processes. In Bibers and Palestrinas masses, the music provides a sonic analog for the dynamic process associated with descent (correlating descending pitch with a decrease in potential energy), and our understanding of this analog is structured by the accompanying text. In terms of the contemporary theory of metaphor, text painting occurs when concepts prompted by the text for a musical work (which serves as the source domain) are mapped on to a series of musical events (which serves as

366

Lawrence M. Zbikowski

the target domain). This mapping structures our understanding of the musical domain: we hear the sounds as descending. But as I suggested in my introductory comments, the sonic analog provided by the music also shapes our understanding of the text, for the music gives the delivery of the words a specific contour and duration.9 Again, given our tendency to give priority to linguistic domains, the notion that the seemingly indefinite and nonconceptual domain of music could be used to structure thought may seem at best little more than a passing curiosity, and at worst downright nonsense. But consider three situations: descendit spoken; descendit sung by Palestrinas singers; and descendit sung by Bibers singers. If there are any differences between these three utterances, they come from the structure music can impose on language. Two further examples of text painting can help to elaborate this point. The first of these comes from the fourth movement of Johann Sebastian Bachs Advent cantata Nun komm der Heiden Heiland (BWV 61). The text and music for each of the three preceding movements of the cantata have all focused on the coming of Christ (as befits the Advent theme). In the fourth movement Christ is suddenly before us, speaking words from the third chapter of Revelation: Behold, I stand at the door, and knock. If any man hear my voice, and open the door, I will come in to him, and will sup with him, and he with me. Bach sets this text as an accompanied recitative for baritone, with the strings playing pizzicato throughout; a shorthand version of the string parts and solo bass melody for the first four measures is given in figure 3.

Figure 3. Measures 1-4 of the fourth movement (Recitativo) from J. S. Bachs cantata Nun komm der Heiden Heiland (BWV 61).

Bachs text painting centers on the words und klopfe an that is, and knock. Bach uses three compositional techniques to paint this activity. First, he summons the repetitions we associate with the act of knocking by repeating the words, and by using three notes to set the first syllable of the initial klopfe (a device called a melisma) such that the articulation of the

Music, language, and multimodal metaphor

367

syllable is repeated. Second, he uses staccato marks on the three notes of the melisma, which place silences between these notes; these silences are similar to those that fall between knocks on a door. Third, he sets the words with a broken chord (or arpeggio). This places a kind of distance between each successive note but also allows us to hear all as belonging to a single connected gesture. Again, as an out-and-out imitation of the act of knocking Bachs setting of und klopfe an leaves something to be desired. Knocks are usually unpitched, for instance, but Bach gives us different pitches for each blow. Knocking is not usually accompanied, but here we have pizzicato strings pulsing in the background.10 The reason we hear these musical events as knocking (to the extent that we do) is that our understanding of them is structured by mapping concepts from language onto music. Again, once we hear these musical events as the sound of knocking our understanding of the text starts to change, for the music creates a rather specific analog for the dynamic process of knocking on a door. This sonic analog relies on the rhythmic and pitch resources of music: the delivery of und klopfe an is at first halting and then (when it is repeated) hurried; the rhythmic structure of the passage is coordinated with a notable expansion of pitch space (from the span of B3 to D#3 in measures 1-2 to the span of C4 to B2 in the first part of measure 3) which is then compressed by the final F#3-B3-F#3-G3. These resources in turn shape our understanding of the text. We not only know that someone is knocking at the door, but how they are knocking: first tentatively, and then with more urgency. My final example of text painting is from Franz Schuberts song Gretchen am Spinnrade, which takes as its text a scene from Goethes Faust. In the scene we overhear Gretchen as, alone in her room, she describes her love for or perhaps enchantment by Faust. The song begins, as shown in figure 4, with the briefest of introductions by the piano, a swirling sixteenth-note figure that circles around a D minor chord, a few sparse and repetitive bass notes sounding beneath. This pattern, or some version of it, continues throughout the entire 120 measures of the song, with but one interruption (to which we shall return in a moment). At first glance, the text (the first lines of which translate as My peace is gone, / My heart is heavy, / I will find it never / and never more) may seem to have little to do with this monotonous accompaniment. The link is provided by the title: this is Gretchen at the spinning wheel. Schuberts accompaniment is, of course, meant to evoke the sound of the wheel in action, with the swirling sixteenth notes summoning the wheel itself and the repetitive, off-beat accents in the middle voice representing the clack of the bobbin, but for modern listeners

368

Lawrence M. Zbikowski

this sonic image will not typically resolve itself until we have mapped conceptual structure from the domain of the text onto the domain of the music. The text thus serves a function similar to what Roland Barthes has called anchoring, rendering transparent an aural image that might otherwise remain opaque.

Figure 4. Measures 1-13 from Franz Schuberts Gretchen am Spinnrade, op. 2 (D. 118).

Again, Schuberts accompaniment does not, in any direct way, imitate the sound of a spinning wheel (which is unpitched) although it does provide a surprisingly accurate analog for the act of spinning. Typical treadle speeds start at about 60 treadles per minute, with the main wheel turning around once with each push of the treadle.11 Each complete pattern of sixteenth notes in Schuberts accompaniment (with two complete patterns per measure) takes a bit less than a second to complete at standard performance tempos, meaning that Schuberts wheel spins at approximately the same speed

Music, language, and multimodal metaphor

369

as does Gretchens. And, as does Gretchens wheel, Schuberts continues uninterrupted until, midway through the song, Gretchen seems to forget herself and the accompaniment momentarily breaks off. The resumption of the musical pattern and, presumably, Gretchens spinning is at first halting, getting under way only with the return to the music shown in figure 4. In my other examples of text painting, the mappings between language and music were more or less focused: a particular musical passage provided a sonic analog for descent, or for the act of knocking. In Gretchen am Spinnrade the mappings are rather less focused: the correlation is not between a word or cluster of words and a particular musical passage, but between the situation described in the title and the accompaniment pattern that permeates the song. As a result, the music informs our understanding of not just one word from the text but the text as a whole. Through the music we can hear Gretchens obsession with Faust (in the relentless patterns of the accompaniment) and sense its fevered intensity: over the course of the song Schubert makes sparing use of the sort of normative harmonic syntax used to suggest progress, and relies instead on quicksilver gestures toward various harmonic centers indicative of a mind that cannot settle down to anything. The curious situation evident in this last case of text painting, where a sonic analog informs our understanding of an entire scene rather than just a single word, points to an interesting feature of multimodal metaphors that involve language and music, and that can be illuminated by a brief consideration of directionality in metaphor. As has often been noted, metaphors are directional: the statement The hippopotamus is a ballerina is rather different from the statement The ballerina is a hippopotamus. In the first case, attributes associated with a ballerina are mapped on to the hippopotamus; in the second case, attributes associated with a hippopotamus are mapped on to the ballerina. One relatively straightforward explanation for the obvious differences between these two mappings views metaphor as a special case of analogy, in which the correlated domains are conceptually distant from one another (Holyoak and Thagard 1995: 213). From this perspective, the directionality of metaphor can be viewed as a consequence of mapping conceptual structure from a source domain to a target domain when there is relatively little conceptual overlap between the two; when the mapping is reversed, the large amount of new information introduced creates a second metaphor markedly different from the first. In the case of linguistic metaphors, the more abstract structure of the correlated domains is nonetheless retained: objects are mapped to objects (ballerina to hippopotamus) and relations to relations (dances to lumbers). Given the account of the different functions of language and music that I have offered, however, even this aspect of

370

Lawrence M. Zbikowski

correlation drops away in the case of multimodal metaphors that involve language and music: the objects and relations of language do not have an obvious correlate among musics sonic analogs. The directionality of mappings between the domains of music and language will, in consequence, take on an added dimension. When language is used to structure our understanding of music, the result will be a view of music that is static and reflective. (This is a view basic to much musical analysis, one which Hans Keller 1994 tried to counter with his wordless musical analyses.) When music is used to structure our understanding of language, the result will be a view of language that emphasizes moment to moment transitions and semiotic indeterminacy. In the preceding analyses of text painting I have deliberately tried to be even handed: I have described how language can structure our understanding of music, but I have also tried to describe how music can structure our understanding of language. While this strategy reflects my long acquaintance with music music for me does not simply accompany thought, but actively shapes it it raises two problems. First (and returning to my initial examples), are the concepts summoned by descendit and the passages by Palestrina and Biber really different from one another? The answer is yes and no. No in that the meaning structures activated by all share certain elements and relations. But yes in that the way these elements and relations are configured is decidedly different in each case. The second problem is one of source and target. Do the novel meaning structures created by these mappings really reflect mapping either from language to music, or from music to language, or do both domains contribute more or less equally to the process of meaning construction? The answer is, it depends. On just what it depends can be made clearer by exploring the perspective on cross-domain mapping provided by the theory of conceptual blending.

4. Conceptual blending and song As was contemporary metaphor theory, the theory of conceptual blending was developed primarily with respect to linguistic phenomena.12 Early on, however, its application was extended to non-linguistic areas (Zbikowski 1999; Cook 2001), in part because blending involved mental spaces (Fauconnier 1994) rather than conceptual domains (with the former understood as ephemeral and pragmatic and the latter as relatively stable and abstract.) In the following, I would like to explore a conceptual blend in which one of the input spaces is set up by language, but the other is set up by music. My

Music, language, and multimodal metaphor

371

focus here will not be on one moment, or even one basic image, but on concepts that are developed over the course of an entire song.

Figure 5. Lead sheet for Jerome Kern and Dorothy Fields The Way You Look Tonight Polygram International Publishing, Inc., and Aldi Music.

Jerome Kern and Dorothy Fields The Way You Look Tonight is typical of many of the tunes produced during what some have called the Golden Age of American song. As can be seen from the lead sheet given in figure 5, Kern chose an AABA form for the tune. That is, the music for the first sixteen measures (the A section) is re-used with little alteration for the second sixteen measures. Contrasting music (the B section, often called the bridge) is introduced in the third sixteen-measure unit (measures 33-48), and this is then followed in measure 49 by a return of the A section (in a slightly modified form, with a tag added at the end). Kerns musical form meshes with that of Fields lyrics, in which the third stanza contrasts with the others in both the length of its lines and its rhyme scheme. In the following, I would like to concentrate on the first verse and explore the mental space set up by

372

Lawrence M. Zbikowski

the lyrics, the mental space set up by the music, and then the blended space that results from their combination. Fieldss lyrics open with a characteristic space builder: Someday.13 Here, the space builder establishes a mental space focused on a future state of affairs rather than on the present. The followings lines fill out the picture: the speaker, beset with rather dire circumstances (When Im awfully low / And the world is cold) will be comforted by the remembrance of the object of his affections and, more specifically, by the way she looks on this particular night (I will feel a glow just thinking of you / And the way you look tonight). As is hinted at by the transformative effect of the appearance of the beloved (an effect confirmed by the second verses There is nothing for me but to love you), what is involved here is not simply a kind of passive looking, with one person gazing on another, but an intimacy of association that has both power and depth. The mental space established by the first verse thus develops into a scene in which what is of moment is not some future opportunity to look back to the present as a golden past but the centrality of the way you look tonight to a highly charged romantic relationship. The Someday space builder is thus somewhat misleading: what is important is not the future but the present. The melody for the song also begins with a space builder, but in this case the falling fifth A5-D4. Although these pitches could be understood in a variety of ways, the simplest interpretation (and one supported by the opening D major chord) is as the fifth and first notes in a D scale. In the music that follows the registral space between D4 and A5 is filled in by a sequence of arch-like figures that flesh out the musical topography with notes from the key of D major (rather than, for instance, D minor). These figures ultimately move past A5 (in measure 7) and arrive on the high note of the melody (D5) of measure 9. This arrival coincides with a return to the rhythm of the opening gesture, but with the A5-D4 falling fifth replaced by a D5-D4 descending octave. It is worth noting that while the D4 of measure 2 and the D4 of measure 10 are the same pitch, their context is quite different: the registral space above the D4 of measure 10 has been expanded through the sequential figures of measures 3-8, and the temporal space between the two whole notes has been filled in by the moving quarter-notes in these same measures. Measure 10 is followed by a final passage that owes something to the compositional strategies that filled out the musical space in measures 3-8, but which now lead directly to the D4 that concludes the melody of the A section in measure 13. From a musical perspective, then, the song opens with a space builder roughly equivalent to the Someday of the words, after which

Music, language, and multimodal metaphor

373

the space is filled out with notes instead of words to solidify and stabilize the conceptual realm prompted by the space builder. The melodic process that leads to the arrival on D4 in measure 13 is reinforced by rhythmic and harmonic processes that are to some extent independent of the melody.14 Over the course of measures 1-8 the number of shorter-duration notes increases until it reaches its maximum density in measures 7-8. This is followed (as has already been noted) by the whole notes of measures 9-10. The contrast between shorter- and longer-duration notes is then revisited in a more orderly fashion, with the quarter notes of measure 11, half notes of measure 12, and whole note of measure 13 (tied over into the quarter note of measure 14). While the harmonies used by Kern a four-measure pattern repeated (with slight variation) four times are quite typical in American popular music (what musicians would call a I-VIII-V pattern), two details contribute to the overall dynamic shape of the A section. First, although the D major harmony of measure 5 represents a return to the opening harmony, the music is kept moving by the melodic sequence that accompanies this harmony. Second, although D major returns once more in measure 9 (and supports long notes that recall the long notes of the opening measures), Kern destabilizes the chord by turning it into a dominant seventh. The overall effect of all of these processes melodic, rhythmic, and harmonic is to make measures 11-13 a goal for the A section, a goal whose culmination is the D4 of measure 13, the note which correlates with the arrival on the last syllable of The way you look tonight. Essential features of the mental spaces set up by the lyrics and the music for the first verse of The Way You Look Tonight are given in the conceptual integration network diagrammed in figure 6, with each mental space represented by a circle. Where the mental space set up by the text is concerned primarily with objects and relations (namely, the appearance of the beloved and its importance for romance) the mental space set up by the music is concerned with a set of coordinated processes that lead to the final phrase in the section. When these two spaces are correlated with one another (as they are in the song) aspects of their structure are projected into the blended space to yield a dynamic representation of the development of an intimacy of association. Guiding the integration of these concepts is the generic space (represented in the top circle of the diagram), which defines the core cross-space mapping and basic topography for the network. The generic space, which reflects the insights captured in the invariance principle (Lakoff 1990; Turner 1990), is centered on the idea that focused attention is a form of intimacy. This sort of attention is behind the fixed gaze of the lover, and it

374

Lawrence M. Zbikowski

is also behind our discomfort when confronted with the unbending stare of a stranger.

Figure 6. Conceptual integration network for the first verse of The Way You Look Tonight.

The development of this sort of intimacy was at the heart of the scene from the 1935 movie Swing Time (starring Fred Astaire and Ginger Rogers) where The Way You Look Tonight first appeared. In this scene Astaire is trying to win Rogers over: he has managed to get in to her hotel suite, and, even though she has repeatedly spurned him, gives it his best effort by singing this song (accompanying himself at the piano). Rogers, for her part, has locked herself in her room and has started shampooing her hair. As the song unfolds, however, she emerges from her room in a bathrobe with hair lathered, first smiling on Astaire and then soundlessly walking over to stand behind him at the piano, where she rests her hand on his shoulder. This gesture coincides with Astaires arrival at the music of measure 61 (the penultimate statement of The way you look tonight), and in response to it he sings the words one more time, turning to look into Rogers eyes. At the conclusion of the song his gaze becomes quizzical as he notices the lather on her head and she, observing this change, turns to a mirror and discovers her appearance. This comedic moment is, of course, a play on the way you look

Music, language, and multimodal metaphor

375

tonight, but Astaire and Rogers have just shared an encounter typical of a much more intimate relationship than they have enjoyed thus far. This intimacy is one that is worked out not only in the first verse of Kern and Fields song, but over its entire course. The second verse moves from generalities of appearance to specifics (With your smile so warm / And your cheek so soft), the bridge (with its music momentarily suspending the process enacted by the first two A sections) adds the detail of that laugh that wrinkles your nose, and the final verse finds the singer speaking directly to the beloved: Never, never change. From the perspective of conceptual blending theory, then, in some cases words and music will prompt the construction of two independent but correlated mental spaces. Both of these spaces contribute structure to a third mental space, in which concepts drawn from each of these two input spaces are blended. This new space typically serves as a site for the imagination. For the conceptual blend created by the words and music for The Way You Look Tonight, we might well imagine that the intimacy established between the lovers is one that would lead them to dance together, or to exchange loving words, or perhaps just to stare into each others eyes. None of these possibilities concludes the scene from Swing Time Rogers, aghast at her appearance, rushes back to her boudoir but subsequent scenes do make clear that Rogers and Astaires characters are now a pair. The mappings associated with a conceptual blend of this sort are different from those associated with a metaphor in two important ways. First, blending typically involves highly fluid and thoroughly pragmatic mental spaces rather than established domains. Metaphorical mappings often yield systems of metaphor; while blends may exploit such systems, they may also destabilize them by extending the system in novel ways, and thus push against the boundaries of the domain. Second, rather than one domain (the source) providing structure for the other (the target) a mapping that gives rise to the directionality of metaphor noted above in a blend correlated spaces each contribute to structure that is mapped onto the blend. But whether metaphor or conceptual blending is involved, I hope I have demonstrated the resources for meaning construction provided by these two different modalities. Most readers will find mapping from language to music to be simple enough, not the least because language is the primary means most humans use to structure their understanding of the world. Mapping from music to language might seem a stranger alternative, but I believe it is a real possibility, especially when language is for some reason ambiguous (as it is in the opening of Schuberts Gretchen am Spinnrade). Additional evidence that music can serve as a proper conceptual domain, and thus have at least the potential to

376

Lawrence M. Zbikowski

structure our understanding of language, comes from instances of conceptual blending where music provides one of the input spaces for a conceptual blend. (I discuss further instances in Zbikowski 2002: chaps. 2 and 6.)

5. Conclusion Although Wittgenstein believed that how we understand music provided important insights into how we understand anything in general and language in particular he himself struggled with what it meant to understand music. In a fragment that dates from 1948 he wrote, If I now ask So what do I actually experience when I hear this theme and understand what I hear? nothing occurs to me by way of reply except trivialities. Images, sensations of movement, recollections and such like (Wittgenstein 1980: 69e-70e). Multimodal metaphors that involve language and music have the potential to provide crucial insights into this question, for such metaphors make clear the different functions of language and music in human culture, and the different ways they construct meaning. Language gives us the means to represent symbolically objects and relations, and through these representations we can direct the attention of another person to things within a shared referential frame. Music, by contrast, provides us with sonic analogs for dynamic processes, processes that include movement through space (such as descent), physical gestures (like knocking), and emotional states (such as obsession or the development of intimacy). A place to begin understanding how we understand music is with such sonic analogs. Although the established models of cross-domain mapping can tell us much about the different contributions language and music make to multimodal metaphors, the methodology of conceptual blending, and the somewhat more fluid construct of a mental space basic to this methodology, offers one way to capture the unique contribution of each mode of communication to the process of meaning construction. Conceptual blending raises at least as many questions as it answers, but it has the potential to place language and music on an equal footing so that we may better understand the contribution of each to multimodal metaphor. As Wittgenstein suggested, understanding a sentence is much more akin to understanding a musical theme than we might first think. This is not to say that language and music accomplish their ends by the same means I have in fact argued just the contrary but that they draw on some of the same cognitive resources for constructing meaning. A key to understanding how this is possible is offered by multimodal metaphors that involve both

Music, language, and multimodal metaphor

377

language and music, for such metaphors give us compelling insights into the rich and varied world of meaning within which humans dwell.

Notes
1. 2. I discuss this passage at greater length in Zbikowski (2002: chap. 2). Further discussion of ways to characterize relationships between musical pitches can be found in Zbikowski (2002: chap. 2) and Ashley (2004). 3. For the sake of concision I have given only one of Bibers melodic strands. At this point in the Credo there are nine vocal parts, and together with the instrumental ensemble the melodic contour shown in example 2 is replicated no fewer than four times (in most cases with pairs of voices). The temporal span of Bibers setting of descendit de clis is also significantly longer than that of Palestrina: Bibers setting runs to forty-five seconds, where Palestrinas is only about twenty seconds long. 4. This conceptual metaphor is a variant of STATES ARE LOCATIONS discussed by Lakoff and Turner (1989). 5. With regard to the associational meanings of music, it is worth noting that one of the most well-known depictions of the pounding of horses hooves in American popular culture the use of a portion of Gioacchino Rossinis overture to Guillaume Tell for the theme music for the radio and television series The Lone Ranger does not have any associations with horses in its original context, but with general ideas about victory. 6. As I use the term, representation does not have to entail a full semiotic system. This perspective is similar to that adopted by Naomi Cumming (2000: chap. 3). A dynamic process may be provisionally defined as a coherent sequence of phenomena that is distributed over time and typified by parametric modulation or change. 7. Sonic analogs are akin to Charles Peirces notion of an icon, and in particular to the form of an icon he called a diagram, which represent the relations of the parts of one thing by analogous relations in their own parts (Peirce 1960, 1:277). 8. I draw my perspective on the unique symbolic aspect of language from Deacon (1997, 2003). 9. The rough equivalent in prosody would combine the two parameters studied by Shintel and her colleagues, yielding something like It is going dooooowwwwnnn spoken with a descending inflection. 10. A few writers have gone so far as to interpret the steady plucking of the orchestra in this movement as a further embodiment of knocking, but this seems something of a stretch. Not only are the attack points too widely spaced to sound much like knocking but the effect is far too persistent, more like Edgar Allen Poes telltale heart than a summons from the Savior. 11. This information was gleaned from http://www.hjsstudio.com/espinner.html (accessed 26 September 2006).

378

Lawrence M. Zbikowski

12. For an overview of the theory see Fauconnier and Turner (1998, 2002). For a review of the latter, see Forceville (2004). 13. Fauconnier describes a space builder as follows: Linguistic expressions will typically establish new spaces, elements within them, and relations holding between the elements. I shall call space-builders expressions that may establish a new space or refer back to one already introduced in the discourse (1994: 17). 14. Music scholars often analyze music in terms of three primary parameters: melody, harmony, and rhythm. In the same way that the very notion of music varies broadly across cultural practices, the manifestation of these parameters is not always obvious or unequivocal. In The Way You Look Tonight, however, both the harmony (indicated by the chord symbols above the staff) and the rhythmic frame (indicated by the notated durations of pitches and by barlines) are relatively clear-cut.

References
Agawu, Victor Kofi 1999 The challenge of semiotics. In Rethinking Music, Nicholas Cook and Mark Everist (eds.), 13860. Oxford: Oxford University Press. 1991 Playing with Signs: A Semiotic Interpretation of Classical Music. Princeton: Princeton University Press. Allanbrook, Wye Jamison 1983 Rhythmic Gesture in Mozart: Le Nozze di Figaro and Don Giovanni. Chicago: University of Chicago Press. Ashley, Richard 2004 Musical pitch space across modalities: Spatial and other mappings through language and culture. In Proceedings of the 8th International Conference on Music Perception and Cognition, Scott D. Lipscomb, Richard Ashley, Robert O. Gjerdingen, and Peter Webster (eds.), 6467. Adelaide, Australia: Causal Productions. Boiles, Charles L. 1967 Tepehua thought-song: A case of semantic signaling. Ethnomusicology 11 (3): 26792. Cook, Nicholas 2001 Theorizing musical meaning. Music Theory Spectrum 23: 17095. Croft, William 2001 Radical Construction Grammar: Syntactic Theory in Typological Perspective. Oxford: Oxford University Press. Cumming, Naomi 2000 The Sonic Self: Musical Subjectivity and Signification. Bloomington: Indiana University Press.

Music, language, and multimodal metaphor

379

Damasio, Antonio R. 1999 The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace. 2003 Looking for Spinoza: Joy, Sorrow, and the Feeling Brain. Orlando, FL: Harcourt. Deacon, Terrence W. 1997 The Symbolic Species: The Co-Evolution of Language and the Brain. New York: W.W. Norton. 2003 Universal grammar and semiotic constraints. In Language Evolution, Morten H. Christiansen, and Simon Kirby (eds.), 111139. Oxford: Oxford University Press. Fauconnier, Gilles 1994 Mental Spaces: Aspects of Meaning Construction in Natural Language. 2nd ed. With a foreword by George Lakoff and Eve Sweetser. Cambridge: Cambridge University Press. Fauconnier, Gilles, and Mark Turner 1998 Conceptual integration networks. Cognitive Science 22: 13387. 2002 The Way We Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Forceville, Charles 2004 Review of Gilles Fauconnier and Mark Turner, The Way We Think: Conceptual Blending and the Minds Hidden Complexities. Metaphor and Symbol 19: 8389. Forns, Johan 2003 The words of music. Popular Music and Society 26: 3751. Goldin-Meadow, Susan 2003a Hearing Gesture: How Our Hands Help Us to Think. Cambridge, MA: Harvard University Press. 2003b The Resilience of Language: What Gesture Creation in Deaf Children Can Tell Us About How All Children Learn Language. New York: Psychology Press. Holyoak, Keith J., and Paul Thagard 1995 Mental Leaps: Analogy in Creative Thought. Cambridge, MA: MIT Press. Keller, Hans 1994 The musical analysis of music. In Essays on Music, Christopher Wintle (ed.), 12628. Cambridge: Cambridge University Press. Kendon, Adam 2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. Lakoff, George 1990 The invariance hypothesis: Is abstract reason based on imageschemas? Cognitive Linguistics 1: 3974.

380

Lawrence M. Zbikowski

The contemporary theory of metaphor. In Metaphor and Thought, 2nd ed. Andrew Ortony (ed.), 20251. Cambridge: Cambridge University Press. Lakoff, George, and Mark Turner 1989 More Than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. Lidov, David 2005 Is Language a Music? Writings on Musical Form and Signification. Bloomington: University of Indiana Press. McNeill, David 1992 Hand and Mind: What Gestures Reveal About Thought. Chicago: University of Chicago Press. 2005 Gesture and Thought. Chicago: University of Chicago Press. Mittelberg, Irene, and Linda R. Waugh this vol. Multimodal figures of thought: A cognitive-semiotic approach to metaphor and metonymy in co-speech gesture. Mller, Cornelia, and Alan Cienki this vol. Words, gestures and beyond: Forms of multimodal metaphor in the use of spoken language. Peirce, Charles Sanders 1960 Collected Papers of Charles Sanders Peirce, Charles Hartshorne and Paul Weiss (eds.). Cambridge, MA: The Belknap Press of Harvard University Press. Ratner, Leonard G. 1980 Classic Music: Expression, Form, and Style. New York: Schirmer Books. Shintel, Hadas, Howard C. Nusbaum, and Arika Okrent 2006 Analog acoustic expression in speech communication. Journal of Memory and Language 55: 16777. Tomasello, Michael 1999 The Cultural Origins of Human Cognition. Cambridge, MA: Harvard University Press. Turner, Mark 1990 Aspects of the invariance hypothesis. Cognitive Linguistics 1: 247 55. Wittgenstein, Ludwig 1980 Culture and Value. Georg Henrik von Wright and Heikki Nyman (eds.), Peter Winch (trans.). Chicago: University of Chicago. 2001 Philosophical Investigations. 3rd ed. Gertrude Elizabeth Margaret Anscombe (trans.). Oxford: Blackwell. Zbikowski, Lawrence M. 1999 The blossoms of Trockne Blumen: Music and text in the early nineteenth century. Music Analysis 18: 30745.

1993

Music, language, and multimodal metaphor 2002 20023 2008

381

Conceptualizing Music: Cognitive Structure, Theory, and Analysis. (AMS Studies in Music.) New York: Oxford University Press. Music theory, multimedia, and the construction of meaning. Intgral 1617: 25168. Metaphor and music. In The Cambridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr. (ed.), 50224. Cambridge: Cambridge University Press.

Chapter 16
The role of non-verbal sound and music in multimodal metaphor1

Charles Forceville

Abstract
Any object or phenomenon that evokes clear-cut connotations for a community of users can function as the source domain of a metaphor, since these connotations qualify for mapping on the target domain. The activation of connotations, in turn, presupposes that the source domain is identified by means of one or more modes/modalities. While the linguistic mode, in its written and spoken varieties, has received ample attention, and the visual mode is also now theorized more broadly, non-verbal sound and music as (source) domain-cueing modes/modalities still await exploration. The present chapter demonstrates that sound and music can play a role in multimodal metaphor (1) by cuing a source domain; and/or (2) by triggering mappable connotations of a source domain signaled in a nonverbal mode. Ten examples of multimodal metaphors involving sound and music from two different genres (advertising and film) are discussed. The chapter ends with hypotheses that invite further testing in theoretical and experimental research. Keywords: non-verbal sound, music, genre, advertising, film

1. Introduction As defined in Forceville (2005, 2006/this volume, 2007, 2008), multimodal metaphors are metaphors whose target and source domains are predominantly or entirely presented in different modes, these modes including minimally visuals, written language, spoken language, non-verbal sound, and music. According to this definition, the verbo-pictorial metaphors in Forceville (1996: 14862) belong to this category, but so do verbal metaphors whose target and source are presented in written and spoken form respectively. The present chapter is an exploration of how non-verbal sound and

384

Charles Forceville

music can play a role in the construal and interpretation of multimodal metaphor. Its aim is to chart parameters that need to be taken into account in a full-fledged theory of multimodal metaphor by discussing ten case studies. Specifically, I will focus on (a) the role of sound and music to (help) identify the metaphors source domain; (b) the role of sound and music to (help) identify features that can be mapped from source to target. Five examples originate in advertising, with its clearly specifiable genre-convention of attempting to persuade an audience of positive qualities adhering to a certain product, brand, or service; and five are fragments from fiction films, a genre which is supposed let us say with Horace to delight, instruct, and move. The analyses are guided by the three questions that need at the very least be capable of being answered for something to count as a metaphor: (1) What are its two domains? (2) What is its target domain, and what its source domain? (3) Which feature or (structured) cluster of features can/must be mapped from source to target? (see Forceville 1996: 108) 2. Case studies Example 1 (Hi): MOBILE PHONE IS PIANO. A Dutch commercial from KPN for Hi, a mobile phone service, shows an attractive woman playing an instrument that looks and sounds like a piano, but on closer inspection turns out to be a visual hybrid between a piano and a mobile phone (figure 1). Playing the piano, the woman sings the following song fragment:
Je bent de liefde van mijn leven/ voor altijd verbonden. You are the love of my life/ connected forever. Jij hebt mij vrienden en vrijheid gegeven/ k kan niet meer zonder. You have given me friends and freedom/ I can no longer do without Ik bel en SMS, met heel hart en ziel/ Ik ben volslagen mobiel. I call and SMS, with [my] whole heart and soul/ I am completely mobile. (My translation, here and throughout the chapter, ChF)

The fragment (composed by Sander Baas and Iwan den Boestert) sounds like a hit song of the kind popular in Holland at the time of writing, but was in fact specifically composed for the commercial (see http://www. megamediamagazine.nl/mvtr.php [accessed 9 July 2008] under KPN Telecom Volslagen Mobiel).

The role of non-verbal sound and music in multimodal metaphor

385

Figure 1. An attractive woman plays on a mobile phone that is shaped like a piano (still from HI mobile commercial, The Netherlands).

After the last line, a female voice-over says, Turn completely mobile, too; turn Hi-subscriber. This month as from no more than 10. Look on Hi.nl, followed by an audiologo (So Hi iii!). It is to be noted that the original Dutch text has (near-)rhymes: leven/gegeven; zonder/verbonden; ziel/mobiel, contributing to the sense that this is a song. Since both domains (mobile phone and piano) are cued in the visual mode, neither written text, nor spoken text, nor music is necessary for their identification, and hence this specimen is strictly speaking a monomodal metaphor of the pictorial variety. However, non-verbal sound does play a role in the construal and interpretation of the metaphor. In the first place the fact that the text is sung rather than spoken adds emotion to the womans appeal to the mobile phone, so that her ode becomes aligned with numerous pop songs in which singers romantically address their lovers. Secondly, the music enhances the similarity that is created between mobile phone and piano. The visual homology between striking a pianos keys and pressing a mobile phones buttons is complemented by the effect of this action: singing a song while accompanying oneself on the piano (a source domain action), it is suggested, in the target domain transforms mobile-phoning into a playful, artistic act with aesthetic effects. The commercials makers could hardly have spelled out these mappable features verbally on penalty of appearing ridiculous; but as nonverbalized features they strengthen and enrich the MOBILE PHONE IS PIANO metaphor. Example 2 (Tuc): COOKIE IS PERSON DOING FITNESS EXERCISES. A commercial for a salty cookie called Tuc shows a Tuc cookie moving from left to right, and going up and down (figure 2). We interpret this movement as self-propelled due to the commands of the male voice-over (And left!

386

Charles Forceville

Two, three, right! And up! . And relax). This text evokes the domain of doing fitness exercises, but so does the rhythmic beat accompanying the voice-over. Thus the target domain is cued visually alone. If the sound is switched off, and all the verbal cues are (mentally) eliminated, the personification of the cookie might still be inferred by some viewers: after all it seems clear that the cookie jumps to and fro entirely of its own accord. However, the specification of the source as not simply a person but as a person doing fitness exercises is cued by the voice-over text as well as by the rhythmic beat. Each of these would suffice alone for this specification, although no doubt their combination facilitates and probably quickens comprehension of the metaphor COOKIE IS PERSON DOING FITNESS EXERCISES.

Figure 2. A Tuc cookie jumps up and down like a fitnesser. Text: But Tuc is most of all a lot more tastier (still from Tuc Commercial, The Netherlands).

Example 3 (Shell): CARS ARE FISH. The images in the commercial are those of beautifully colored, animated fish apparently swimming just above the bottom of the sea (reminiscent of the animation film Finding Nemo, USA 2003). We see a school of fish all stopping in mid-swim (figure 3), then moving on, and a small fish darting away just in time from a swordfish trying to stab it. The sounds we hear are traffic sounds revving motors, screeching tires, claxons, a siren. The voice-over reinforces the traffic domain:
What is the advantage, in everyday traffic, of a gasoline that has been developed in collaboration with the people at Ferraris? V-Power is a new gasoline that ensures better performance. Thanks to a better combustion. So that your car can respond faster when necessary. And where do you find a gasoline that guarantees such a good performance as V-Power? Shell. Where you stop, we go further.

The role of non-verbal sound and music in multimodal metaphor

387

Figure 3. Fish halt suddenly as if cars stopping before traffic lights, suggested by street sounds (still from Shell commercial, The Netherlands).

The traffic domain is thus cued by both non-verbal sound and spoken language. Once the audience has accessed this domain, some movements of the fish, for instance their completely synchronized stopping and moving, can be interpreted as signaling the traffic domain (here: stopping before a traffic light), but these visual cues alone would not suffice to ensure its recognition. Moreover, the sound adds liveliness and precision to the images: for instance, the little fishs escape from the swordfish is emphasized not just visually by the quick movement, and verbally by the brief pause before when necessary, but also by the revving sound. Since speed embodies a quality claimed to be facilitated by the product advertised (V-Power), this is meaningful. In view of the claim that in principle metaphors, irrespective of the modalities they draw on, do not allow for reversal of target and source (Forceville 1995, 2002), it is interesting that in this metaphor its distribution is not immediately clear. In the first instance, before the voice-over is audible, I suspect viewers hypothesize that the fish are, or belong to the domain of, the product promoted. That is, they might at this stage speculate, for instance, that this is a commercial for a zoo, or perhaps an amusement park with an aquarium. At the moment they hear the words traffic and gasoline, words that strongly connote the realm of cars and driving, the assumption that the fish are to be taken literally is probably discarded. Indeed, it is not until this moment that viewers will reinterpret the fish as the source domain of what is to be construed as a metaphor. I submit that just as inferences made on the basis of the visual track tend to prevail over those made on the basis of the non-verbal soundtrack, inferences on the basis of verbal lan-

388

Charles Forceville

guage override those originating in the visual track. It is thus the verbal track, supported by the traffic sounds, that makes viewers postulate the metaphor CARS ARE FISH. The target-status of cars is further reinforced once we hear the name of the brand advertised: Shell and see one of a series of shells on the sea bottom transform into the Shell logo. Example 4 (Iglo): CORNCOB AND FRENCH BEAN ARE BRIDE AND GROOM. An Iglo instant-meal commercial features a mini-corncob and a French bean, walking together toward an Iglo package (figure 4). Their movements alone already suggest anthropomorphizing, but it is the tune of the Wedding March which metaphorizes the two into bride and bridegroom (and makes us realize that the Iglo package actually resembles a church). A voice-over tells us:
In our newest Iglo dish there are mini-corncobs and French beans a combination of young and crispy. And what should of course never be missing on such a joyful occasion? Exactly! And together with chicken filet and soy sauce these make up the delicious new Iglo dish.

Figure 4. A min-corncob and a French bean walk as a married couple, the Wedding March being audible on the sound track (still from commercial for Iglo meals).

At the moment the male voice-over says exactly we hear a faint voice say yes, now!, followed by a kissing sound, and we see, under cheers and applause, rice showered upon the couple. The metaphor CORNCOB AND FRENCH BEAN ARE BRIDE AND GROOM is cued mainly by the Wedding March tune, since the anthropomorphizing of the two does not suffice to turn them into a wedding couple. If the sound were turned off, most viewers would not access the source domain on the basis of visual cues until the rice

The role of non-verbal sound and music in multimodal metaphor

389

shower, if then. Note that despite the reference to a joyful occasion, the verbal text alone, too, would not be sufficient to cue WEDDING at all. That is, there is no unambiguous verbal reference to the source domain. Both target and source are cued pictorially, but it is the Wedding March tune that turns the vegetables not just into humans, but into bride and groom even before we see the rice-shower. Example 5 (Senseo): COFFEE MAKER IS MOTORCYCLE. A Philips coffee maker, Senseo, shows the metallic machine first in a number of extreme close-ups defying recognition. We hear a throbbing motorcycle engine and a fragment of Steppenwolfs Born to be Wild featuring the line Get the motor running, hit it on the highway made famous by the opening sequence of Easy Rider (Dennis Hopper, USA 1969). The metaphor COFFEE MAKER IS MOTORCYCLE is further emphasized by concurrently showing a finger pushing the coffee makers on/off button and sounding a kick-starting motorcycle. The shiny black covering of the coffee machine, moreover, looks like the surface of a motorcycle and the dripping coffee resembles a drop of oil (figure 5). The following phrases are superimposed, one after the other, on the images: designed with a vision developed with passion makes of each moment a sensation that delights all senses. The metaphor in this commercial is cued at least as much aurally as visually. One mappable feature is clearly the high-tech design, but more importantly the music evokes connotations such as living life in the fast lane, freedom, unconventionality, youth, sixties counterculture a whole range of qualities nostalgically associated with Easy Rider motorbiking that are potentially mapped to making your coffee with a Senseo machine. That these connotations are considered important by the commercials maker is confirmed by the written pay-off line at the end of the commercial: three years old, and already a legend at least in the kitchen. Switching off the sound of the commercial presumably eliminates the most important cue for the source domain, Easy Rider motorbike. This commercial, therefore, is best classified as a multimodal metaphor. (I owe this example and part of the analysis to Victor 2004.) Watching commercials is governed by the strong genre-expectation that some product, service, or brand name is promoted, which enormously facilitates and constrains the preferred interpretation of everything visible and audible in them, including the metaphors. In most metaphors in commercials, the product is the target and the source is something else, which means that it is positive features that are mapped from source to target (Forceville 1996: 104). However, it is important not to theorize multimodal metaphor exclusively on the basis of case-studies exemplifying a single genre, since this might lead to a mistaken conception of pictorial and multimodal metaphors

390

Charles Forceville

prototypical characteristics (Forcevilles 2002 criticisms of Carrolls 1994, 1996 theory of pictorial/ visual metaphor expose this misconception). It is commendable, therefore, to consider specimens from a different genre as well. Metaphors in art are usually not amenable to the kind of clear-cut, singular intentions found in advertising. One crucial artistic convention is that it is, in the words of Siegfried Schmidt (1991), more poly-interpretable (see for discussion of Schmidts fact versus aesthetic and monovalence versus polyvalence convention Fokkema and Ibsch 2000: chap. 2 and 5; see also Forceville 1999a). Here are, therefore, five examples of multimodal metaphors from fiction films. (For discussions of multimodal metaphors in the cartoon genre, see El Refaie this volume; Yus this volume; Schilperoord and Maes this volume.)

Figure 5. A drop of what looks like oil an impression reinforced by the accompanying Born to be Wild song is in fact a drop of coffee (still from a commercial for Senseo, The Netherlands).

Example 6 (The General): REAL CANNON IS CIRCUS CANNON. Buster Keatons brilliant film The General (1927), set during the American Civil War, features Keaton as the train driver Johnnie Gray, whose locomotive is stolen by enemy Northern soldiers. Johnnie gets hold of another locomotive, hitches on to the loc a wagon with a cannon, and single-handedly goes in pursuit along the same railway track. When he has the Northerners in sight, he loads the cannon (figure 6), ready to fire over the cabin of his own loc. Just before Johnnie fires the cannon, we hear the soft-and-quick drum on the soundtrack that we recognize as a clich device to create tensive expectation, while the firing of the canon ball itself is accompanied by a very unrealistic popping sound (the music and sound of the version discussed here is by Konrad Elfers). As such, there is nothing much metaphorical about this, but the conven-

The role of non-verbal sound and music in multimodal metaphor

391

tional drums anticipating the firing will no doubt remind many viewers of a similar situation in circus acts, where a human cannon ball is fired. Hence construal of the metaphor REAL CANNON IS CIRCUS CANNON is invited. The mapped features in this case are the connotations evoked by a circus context, such as pleasurable excitement, risk-taking, the idea of watching a performance and of course the mapping onto real-cannon firing turns Johnnies action into slapstick. Incidentally, the fact that this film predates the sound era (Al Johnsons The Jazz Singer, 1927, is conventionally credited with being the first sound film), has an interesting consequence from the point of view of metaphors involving music and sound. In the pre-sound era, the typical program had musical accompaniment. In the more modest presentations, a pianist might play; in vaudeville theatres, the house orchestra provided music (Thompson and Bordwell 1994: 13). This, then, gave ad-libbing musicians opportunities to create multimodal metaphors involving sound where these may not have been envisaged by the films makers.

Figure 6. Johnnie (Buster Keaton) loads a cannon, while tensive circus-act music is audible (still from Buster Keaton, The General, USA 1927).

Example 7 (If ): PUBLIC SCHOOL IS ARMY. Lindsay Andersons If .... (UK 1968) is a satire on the perverse aspects of British public school life. The verbal and visual references to battle and war, combined with the violence and the expectation of blind discipline in a hierarchical system gives rise to the metaphor PUBLIC SCHOOL IS ARMY a conceptual metaphor that is reinforced throughout the film. Starting out realistically, the film gradually begins to show bizarre and surrealistic events that, commensurate with the counterfactuality suggested by the title, destabilize the status of the real. Indeed, the film increasingly literalizes the metaphor, resulting in a climactic

392

Charles Forceville

machine-gunning of the established order by a small band of rebels. The very last scene of the film can be construed as a multimodal metaphor in which music provides the source domain. The rebels, led by Mick (the first major role of actor Malcolm MacDowell), are positioned, heavily armed, on the roofs surrounding the colleges quadrangle. Because of a celebration, pupils, staff, parents, and various officials (tellingly: representatives of the royal family, the army, and the church) are gathered in the college chapel, until they are smoked out by the rebels. Outside they are gunned down helplessly (figure 7) until they manage to access a weapon depot in the college (!) and are able to retaliate. During the last images of cross-firing, the same melody is audible that had been sung in the chapel.2 This hymn-like melody metonymically connotes the rituality of a religious service, and given the films consistent focus on cruel, ritualized behavior (canings and other humiliating punishments such as cold showers and pupils being hung upside down in a toilet pot), it is possible to construe the multimodal metaphor MASSACRE IS RELIGIOUS RITUAL, potential mappable features being the circumstances that in a ritual event everything is anticipated, constitutes the performance of a script, and has profound meaning.

Figure 7. The massacre in the college quadrangle (still from If , Lindsay Anderson, UK 1968).

Example 8. (The Godfather I): MENTAL STATE IS FAST TRAIN. Michael (Al Pacino) in The Godfather I (Francis Ford Coppola, USA 1972) prepares to take revenge on his familys enemies, Sollozzo and McCluskey. He has planned for a gun to be hidden in the toilet in the restaurant where they will be having dinner. Since he is not an experienced killer at this stage of his life, this is an emotional moment for Michael. When he returns from the toilet, with the gun, the sound track features a rumbling sound, which be-

The role of non-verbal sound and music in multimodal metaphor

393

comes identifiable as that of a riding train. The sound is in this scene nondiegetic, that is, it does not emanate from the actual events portrayed at this moment. That this is so is made clear by the fact that the sound is not continuous: it is audible in the shots when we see Michael in the toilet, but not in the shots of Sollozzo and McCluskey left behind at the restaurant table. The sound is thus used to convey Michaels mental state, suggesting the metaphor MICHAELS MENTAL STATE IS FAST TRAIN. Possible mappings are such a trains unstoppability, the inexorable rhythm of its progress, the circumstance that its noise drowns out other sounds which in the target domain translate as, say, Michaels determination, or his refusal to reconsider his plan to kill. Lena Chatzigrigoriou (to whom I owe the example and part of the analysis) interprets the scene as follows: The sound of the shrieking train breaks gradually, overpowering Sollozzos voice. This sound tells us that Michael is not listening anymore, he is ready for action (Chatzigrigroriou 2006: 13). That the sound is a trains is confirmed in the DVD commentary track by Francis Ford Coppola, who observes that there was an elevator train in the vicinity of the restaurant where the scene was shot, and that it was, in fact, the sound of this train that provided the idea for its usage in the scene. This comment also suggests that we indeed already heard the same train sound earlier, but then diegetically, namely during the car trip to the restaurant. As in the If scene discussed in example 7, then, the sound used as source domain in the metaphor had been cued realistically in an earlier scene. Even if the analysis is accepted, it is nonetheless clear that presumably few spectators will consciously construe the concept TRAIN as a metaphorical source domain in this highly suspenseful scene. They may, instead, construe the sound more generically as swelling rumble or something like that. This leaves intact the claim that the sonic source domain brings to the fore to the audience what Michael thinks which is not directly made visible (although Al Pacinos facial expression arguably also helps the audience interpret what he is thinking). In Indurkhyas (1991) terms, this would be an example of a projective metaphor (Indurkhya 1991: 16), a type in which a source domain structures a largely unstructured target domain. But the example points up an issue that is particularly pertinent to multimodal metaphors whose source, for whatever reason, is difficult to verbalize unambiguously. Should we say that members of the audience who failed to cue TRAIN, but who did cue, say, SWELLING RUMBLE, and somehow mapped associations adhering to that source to the target MICHAELS MENTAL STATE have or that they have not processed the metaphor? Or have they processed differ-

394

Charles Forceville

ent metaphors? (See Bartsch 2002 and some responses to Bartsch in Forceville 2006.) This issue deserves sustained discussion.

Figure 8. Adam as dog, biting the hand of his creator (still from Adam, Peter Lord, UK 1991).

Example 9 (Adam and Remember to Keep Holy the Sabbath Day): HUMAN Animation films are rich in metaphors in which sound plays a role. A simple example is the juxtaposition of a human(oid) creature and a recognizable animal sound. Adam (Peter Lord, UK 1991) is a short claymation which playfully refers to the fact that the eponymous hero derives its name from Hebrew Adamah, meaning reddish clay. Its Godlike animator keeps ordering Adam about on his miniature planet earth. At one moment Adam, confused what behavior his creator expects of him, goes down on all fours, barks, and snaps at his creator (figure 8), so that for a moment the viewer is invited to entertain the metaphor ADAM IS DOG. A similar situation occurs in one of Phil Mulloys bleak animations, Remember to Keep Holy the Sabbath Day in the Ten Commandments series: Ezekiel Mittenbender kneels down and begins to bark. Note, incidentally, that the metaphor is used for different narrative purposes: in the first case, the metaphor is deployed to indicate the heros temporary confusion; in the second, to convey his insanity. Furthermore, while redundant to cue the source domain animal (the fact that both Adam and Ezekiel go down on all fours provides sufficient visual cues for that), the barking narrows down the animality to that of a dog thus potentially activating in the audience a whole range of connotations adhering to DOG and hence contributes information not available in the image track. Example 10 (Robin Hood): TENT IS TRAIN. In the animation film Robin Hood (Wolfgang Reitherman, USA 1973) at one moment a group of creatures in a tent scurries over a plain, only their feet being visible. The pheBEING IS DOG.

The role of non-verbal sound and music in multimodal metaphor

395

nomenon reminds us of a fast-moving, but unspecified vehicle, but the rhythmic percussion steers us into understanding it as a train an interpretation that is further supported when, upon the tents approach of a porch, we hear a train whistle. Without the sound, we might have opted for the metaphor TENT IS VEHICLE, but the sound specifies the vehicle as a TRAIN, and transforms the porch into a TUNNEL. I owe this example to Bensdorp and Vergeer (2004), who discuss many more sound metaphors in animation films. 3. Discussion and conclusions While it impossible to make sweeping generalizations on the basis of the examples discussed above, the case studies enable the formulation of observations and hypotheses that can be tested in further research, both theoretical and empirical, pertaining to sound in multimodal metaphor. Whereas language has the means to correlate verbal information with visual, sonic, or musical information via deixis (e.g., this man or the train), correlations between non-verbal modes depend on well-timed simultaneity. That is, in order to suggest a multimodal metaphor that does not draw on spoken or written language, the identity relationship between target and source must be triggered by making them visible/audible at the same moment. Such identity can, of course, be enhanced by many devices: a sound that is lip-synchronous with a character opening her mouth; visible movements that correspond with sonic rhythms (due to its abundant use in mainstream animation this is called Mickey mousing); or a montage pattern that has the same rhythm as the accompanying sound or music. The non-verbal and musical sounds that play a role in the metaphor tend to cue its source rather than its target domain. The visual component, that is, cues the literal target under discussion, whereas the sonic component metaphorically transforms this target. The Shell example suggests that the spoken language variety of the sound track, in turn, takes precedence over the visual track in cueing the target rather than the source (the same holds, more spectacularly, for the CMG commercial discussed in Forceville 2003). The sound is seldom alone responsible for the identification of the source domain. Usually, the pictorial and/or verbal information provides hints about the identity of the source domain (see also Forceville 1999b; Eggertsson 2006; Eggertsson and Forceville this volume). The sound, however, facilitates the identification of the source. In this respect Gibbs remark that even a cursory examination of theories of linguistic interpretation reveals a

396

Charles Forceville

tremendous diversity in the emphases on the different temporal points at which an utterance or text has supposedly been understood (1994: 115116) is no less pertinent for multimodal metaphors than for verbal ones. Time is an important factor here: it is likely that the sound in multimodal metaphors, even if it does not contribute mappable features, aids the speed with which a source domain is identified. This is particularly pertinent in films and commercial advertising, which do not, under normal viewing circumstances, allow for the kind of backtracking that for instance written verbal metaphors permit. In addition, as we have seen, in several examples the soundtrack provides information that steers identification of the source domain on a more specific level, or allows that identification to be made with greater confidence than on the basis of the visual and/or verbal track alone: in the Iglo example, it is the Wedding March that is the first cue that the viewer is not just to see the mini-corncob and the French bean as persons, but as bride and groom. A complete analysis of a multimodal metaphor involving sound as indeed of any kind of metaphor requires an assessment whether the source domain cued is diegetic or non-diegetic. If the source domain, or part of it, is diegetic (that is: it belongs to the events in the story world as presented at that moment) or quasi-diegetic, the metaphor is naturalized because there is a metonymic link between the source and the target domains. As a consequence, the metaphor is less obtrusive, and possibly less easily identifiable as such than when the source domain is plainly non-diegetic. It is noticeable, for instance, that in four of the commercial examples, the source domains are metonymically linked to the target domain, and thus in some quasi-logic literalized: in example 1 the piano and its music are not only the source domain for a mobile phone: they also help produce the love song supposedly addressed to the phone. In example 2, the Tuc cookie is not only, metaphorically, a fitnesser; fitnessers also do well, the voice-over tells us, to eat Tuc cookies because they are so light (i.e., presumably calories-poor). In example 3, the underwater world of fish leads naturally to a shot of a number of shells, one of which transforms into the Shell logo. In example 4 the rice showered over the wedding couple is part of the Iglo product promoted. Only in example 5, there is not really a diegetic motivation for the source domain, motorcycling, although the slogan already a legend is supposed to bridge the distance between Easy Rider-motorbiking and Senseo-coffeemaking. In the artistic metaphors, too, there is sometimes a degree of naturalization of the source domain. In Lindsay Andersons fragment from If , for instance, the organ music that helps metaphorize, in my reading of it, the final scene of the film, may not be particularly noticeable because the same organ song was used just before in a literal context. That is, in the earlier

The role of non-verbal sound and music in multimodal metaphor

397

scene it was diegetically motivated music, which in the latter scene has become non-diegetic (but recalls its diegetic use). The same holds true for the fragment from The Godfather, in which the elevator train had been diegetically audible in an earlier scene. The effect, I propose is similar to the effect in the commercials: it reduces the sense of artificiality necessarily associated with the presentation of a source domain that has no realistic motivation whatsoever. Bordwell and Thompson point out that sound can achieve very strong effects and yet remain quite unnoticeable (1997: 315). Inasmuch as sound is less consciously registered than images, metaphors with sonic source domains may exercise their persuasive or narrative influence more subtly than, for instance, metaphors whose terms are both presented in pictorial terms. Experimental research on multimodal metaphors with a sound dimension in which sound tracks are suppressed or altered (see Chatzigrigoriou 2007) is imperative to gain more insight into the working of sonic metaphors. Not all examples presented here are metaphorical with a high degree of explicitness. As always, the signals used to cue the source domain must be comprehensible by the envisaged audience for a metaphorical construal to be possible in the first place. In fact, one could venture that, because they never have a verbal is to explicitize them, multimodal metaphors tend to be less explicit than purely verbal ones (for more discussion on implicitly versus explicitly signaled metaphors, see Forceville 1999b). Finally, the genre to which a representation belongs steers the possible or most plausible interpretations of any element in it, including metaphor (see Charteris-Black 2004 for corpus analysis of metaphors in genres such as sports news, political manifestos, and religious texts in the Bible and the Koran; Caballero 2006 for an in-depth examination of metaphor use in architectural building reviews; and Caballero, this volume, for their use in wine-tasting notes). Systematic investigation of sonic metaphors in different genres (e.g., commercials, art films, horror films, video clips, computer games), or sequences within films (dreams, hallucinations, flash-backs) may reveal patterns specific for such genres or sequences. Notes
1. This chapter is a revised and expanded version of a paper that appeared earlier of Forceville (2004). I am indebted to Eduardo Urios-Aparisi for comments on an earlier draft of this new version.

398
2.

Charles Forceville The melody of the song is that associated with J. Threlfalls Hosanna, Loud Hosanna, but the words sung, so far as they can be deciphered, do not fit that hymns text. Lindsay Anderson himself wrote about the melody that he had originally asked Marc Wilkinson to write some music for the final onslaught, where Mick alone on the roof tries to hold at bay the attacking forces of Establishment, but we found when we played with the sound tracks in the cutting room that a simple organ version of the College song, which fortunately I had recorded when we were on location, fitted the sequence much better (Anderson 1975). Apparently, then, the hymns melody was used for a college song, whose text I have not been able to locate. I am indebted to Thomas Elsaesser, Erik Hedling, and Andrew Webber for help in my hunt for clues on the song.

References
Anderson, Lindsay 1975 Using music. [Typed draft, Library collection University of Stirling, UK] http://www.is.stir.ac.uk/libraries/collections/anderson/music. php (last accessed 8 July 2008). Bartsch, Renate 2002 Generating polysemy: Metaphor and metonymy. In Metaphor and Metonymy in Comparison and Contrast, Ren Dirven and Ralf Prings (eds.), 4974. Berlin/New York: Mouton de Gruyter. Bensdorp, Thomas, and Arjan Vergeer 2004 Geluid en de Populaire Animatiefilm: Een Cognitivistisch Onderzoek naar Betekenisvorming. [Sound and the popular animation film: A cognitivist approach to meaning formation.] MA diss., Department of Media Studies, University of Amsterdam, The Netherlands. Bordwell, David, and Kirstin Thompson 1997 Film Art: An Introduction. 5th ed. New York: McGraw-Hill. Caballero, Rosario 2006 Re-Viewing Space: Architects Assessment of Built Space. Berlin/ New York: Mouton de Gruyter. this vol. Cutting across the senses: Imagery in winespeak and audiovisual promotion. Carroll, Noel 1994 Visual metaphor. In Aspects of metaphor, Jaakko Hintikka (ed.), 189218. Dordrecht: Kluwer. 1996 A note on film metaphor. In Theorizing the moving image, 212 223. Cambridge: Cambridge University Press.

The role of non-verbal sound and music in multimodal metaphor

399

Charteris-Black, Jonathan 2004 Corpus Approaches to Critical Metaphor Analysis. London: Palgrave-MacMillan. Chatzigrigoriou, Eleni 2006 Take-home exam pictorial and multimodal metaphor. Manuscript. Department of Media Studies, University of Amsterdam, The Netherlands. 2007 Trans-Diegetic Sound: In-Between Spaces in Post-Classical Cinema. MA diss., Department of Media Studies, University of Amsterdam. Eggertsson, Gunnar Theodr 2006 Animal Horror: An Investigation into Animal Rights, Horror Cinema and the Double Standards of Violent Human Behaviour. MA diss., Department of Media Studies, Universiteit van Amsterdam. Eggertsson, Gunnar Theodr, and Charles Forceville this vol. The HUMAN VICTIM IS ANIMAL metaphor in extreme horror films. El Refaie, Elizabeth this vol. Metaphor in political cartoons: Exploring audience responses. Fokkema, Douwe, and Elrud Ibsch 2000 Knowledge and Commitment: A Problem-Oriented Approach to Literary Studies. Amsterdam/Philadelphia: Benjamins. Forceville, Charles 1995 (A)symmetry in metaphor: The importance of extended context. Poetics Today 16: 677708. 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 1999a Art or ad? the influence of genre-attribution on the interpretation of images. SPIEL 18: 279300. 1999b The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 17998. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2003 Bildliche und multimodale Metaphern in Werbespots [Tr. from English by Dagmar Schmauks] Zeitschrift fr Semiotik 25: 3960. 2004 The role of non-verbal sound and music in multimodal metaphor. In Words in their Places: A Festschrift for J. Lachlan Mackenzie, Aertsen, Henk, Mike Hannay, and Rod Lyall (eds.), 6578. Amsterdam: Faculty of Arts, VU Amsterdam. 2005 Cognitive linguistics and multimodal metaphor. In Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach (ed.), 264284. Cologne: Von Halem. 2006/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Ap-

400

Charles Forceville

plications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. 2007 Multimodal metaphor in ten Dutch TV commercials. Public Journal of Semiotics 1: 1951. http://semiotics.ca/. 2008 Pictorial and multimodal metaphor in commercials. In Go Figure! New Directions in Advertising Rhetoric, Edward F. McQuarrie and Barbara J. Phillips (eds.), 272310. Armonk, NY: ME Sharpe. Gibbs, Raymond W., Jr. 1994 The Poetics of Mind: Figurative Thought, Language, and Understanding. Cambridge: Cambridge University Press. Indurkhya, Bipin 1991 Modes of metaphor. Metaphor and Symbolic Activity 6: 127. Schilperoord, Joost and Alfons Maes this vol. Visual metaphoric conceptualization in editorial cartoons. Schmidt, Siegfried J. 1991 Literary systems as self-organizing systems. In Empirical Studies of Literature, Elrud Ibsch, Dick Schram, and Gerard Steen (eds.), 413424. Amsterdam/Atlanta, GA: Rodopi. Thompson, Kristin, and David Bordwell 1994 Film History: An Introduction. New York: McGraw-Hill. Victor, Paul 2004 The Metaphorical Use of Sound in Film. MA diss., Department of Media Studies, University of Amsterdam, The Netherlands. Yus, Francisco this vol. Visual metaphor versus verbal metaphor: A unified account.

VII Metaphor and Film

Chapter 17 Multimodal metaphor in classical film theory from the 1920s to the 1950s Mats Rohdin

Abstract
This chapter presents the discussion of multimodal metaphor in classical film theory from the 1920s to the 1950s. Many famous film metaphor examples from the silent era are multimodal since they involve not only moving images but also written texts (intertitles, that is, shots inserted in the film providing explanatory text or dialogue). The breakthrough of sound film in the late 1920s enhanced the construction of multimodal metaphors since speech, music and sound effects were added to images and written texts. Throughout this paper questions of identification and interpretation of film metaphor are discussed, and it is claimed that formal criteria are not sufficient to adequately describe the rich variety of multimodal metaphors. Keywords: Conceptual metaphor, film metaphor, film theory, monomodal metaphor versus multimodal metaphor.

1. Introduction Arthur Danto (1992: 74) has suggested that a possible explanation why art criticism never developed a concept for pictorial metaphor in ancient Greece was that scholars were too much concerned with mimesis. Dantos opinion is no less pertinent to film scholars during the first decades of the twentieth century, when the new medium was regarded as the culmination of the old mimetic tradition, almost as reality representing itself (Hake 1993: 277). The close connection between film and perceptual reality is probably one important reason why there is not any systematic account of metaphor in film theory until the late 1920s. Another reason is, of course, that many

404

Mats Rohdin

writers regarded the concept of metaphor as exclusively a verbal matter (e.g., Arnheim 1933: 260270; Stanford 1936: 95). In this chapter I will focus on multimodal metaphors in classical film theory from the 1920s to the 1950s. Multimodal metaphors are metaphors whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006: 384/this volume). In fiction film these modes include five different signs: (1) pictorial signs, (2) written signs, (3) spoken signs, (4) non-verbal sounds, and (5) music. Classical film theory refers to a theory of cinematic representation premised upon mechanical duplication (upon the existential relation between film and referent) and is therefore based upon an extensional semantic theory of meaning (Buckland 2000: 26). Principally I will elaborate on some tenets from my Ph.D. Vildsvinet i filmens trdgrd. Metaforbegreppet inom filmteorin [The wild boar in the garden of film. The concept of metaphor in film theory] (Rohdin 2003), where I examine film theory texts that explicitly include the term metaphor. For this occasion my investigation concerns the question whether some of these metaphors would qualify as multimodal metaphors. Worth remembering is that many examples discussed are difficult to fully describe due to the lack of information from the writers as to what guides the identification (what are the two terms? which are target and source, respectively?) and interpretation (what is mapped from source domain to target domain?) of the metaphor. Consequently some examples of identification and interpretation may be more rightly attributed to my inferences and guesses than to the author in question. As Forceville stresses, such verbalization is never neutral (2006: 390). When dealing with the identification and interpretation of multimodal metaphors I attempt to delineate three questions, which according to Forceville (1996: 6566, 2002: 23, 2005: 266), have to be answered: (1) Which are the two terms of the multimodal metaphor? (2) Which is the metaphors target domain and which is the metaphors source domain? (3) Which features are mapped from the source domain to the target domain? One reason why I have chosen to examine classical film theory is that the definitions and film examples presented by these theorists, especially in the 1920s and 1930s, have formed the canon to which film historians and theorists frequently return. These examples of metaphor have later been repeated extensively in film studies (e.g., in film theory and film history, as well as in film dictionaries), thus fulfilling a function similar to the classic examples in handbooks of rhetoric. Above all, these examples have illustrated for generations of university students the formal characteristics, which constitute criteria for defining film metaphor.

Multimodal metaphor in classical film theory

405

2. Classification of multimodal film metaphors up to the late 1950s In silent cinema during the 1920s experiments were done in different countries in order to find filmic equivalents of verbal metaphor. Soviet montage cinema, and especially Sergei Eisenstein, is particularly renowned in this respect. Eisenstein (1996: 222) even claimed that experiments dealing with montage tropes, including film metaphor, was Soviet montage cinemas most important contribution to film history. As a matter of fact, many of the canonized examples of film metaphor from the silent film period are multimodal, since they involve not only moving images (visual signs) but also intertitles (shots inserted between liveaction shots, providing explanatory text or dialogue). The breakthrough of sound film in the late 1920s further enhanced the possibility of multimodal metaphors since three more modes were added to the standardized film product: speech, sound, and music. When it comes to the question of how to identify a verbal metaphor, early theories often assumed that metaphor was a thing or an object in the text, which could be identified thanks to some sort of deviation or anomaly at the surface level, for example, semantic anomaly or logical absurdity when taken literally. In film theory the equivalent has often been a disruption of or deviation from a mimetic form of representation (for example by a superimposition, that is when two or more images are combined into one over the whole area of the frame by double printing or double exposure in the camera, Salt [1992: 329]) or against narrative codes (for example by using a nondiegetic insert in a mainstream fiction film, that is a shot or series of shots cut into a sequence, showing objects that are represented as being outside the world of the narrative [Bordwell and Thompson 2007: 480]). However, modern verbal and non-verbal metaphor theories often claim that there does not have to be any deviation or incoherence at all on the surface level of the representation in order to identify a metaphor (Black 1993: 34; Cohen 1997: 223225, 1999: 399; El Refaie 2003: 7980). In the writings of classical film theory between the 1920s and 1950s, a preliminary classification of different formal characteristics which are said to cue the spectator in identifying the film metaphor would, according to my analysis, look like this: - Superimposition (e.g., Arnheim 1933; Carroll 1996a) - Verbal image (e.g., Balzs 1930, 1952; Ejxenbaum 1981; Eisenstein 1987a, 1987b, 1996) - Montage (primarily the nondiegetic insert) (e.g., Arnheim 1997; Balzs 1930, 1952; Bazin 1967; Eisenstein 1974, 1996)

406 -

Mats Rohdin

Cinematography (e.g., Balzs 1930; Bazin 1981; Eisenstein 1991, 1996; Tynjanov 1981) No formal characteristics on surface level (e.g., Balzs 1952; Bazin 1971, 1973)

2.1 Superimposition In general, the superimposition is part of a dissolve, which is a transition between two shots during which the first image gradually disappears while the second image gradually appears; for a moment the two images blend in superimposition (Bordwell and Thompson 2007: 478, emphasis in original). This type of metaphor is often considered an anomaly in popular fiction film because two elements belonging to different categories are presented as a homospatially unified figure. In the early years of silent cinema, superimpositions were done in the film camera, which required a lot of skill from the photographer. A famous example of this is when Sergei Eisenstein in Strike (Stachka, 1925) blends an image of a police spy with that of a fox, thereby proposing the spectator to map aspects from the source domain, FOX, to the target domain, POLICE SPY, for example cunning and shrewdness. Thus the superimposition suggests the conventional metaphor MAN IS AN ANIMAL (figure 1). When the optical printer was invented in the late 1920s superimpositions could be produced in laboratories during postproduction, which was less complicated. However, at that time this type of film metaphor almost disappeared from popular fiction film. Another example of superimposition, discussed in Carroll (1996a), appears in Fritz Langs futuristic film Metropolis (1927), when the young protagonist visits the gigantic underground power plant. In a horrific vision, produced by a dissolve (thus including superimposition), he sees a machine transforming into a man-devouring monster. An intertitle explains that the monster is Moloch (figures 24). Thus the metaphor could be expressed as MACHINE IS MOLOCH: the machine is the target (pictorial signs) and Moloch is the source (pictorial and written signs). In the source the machine has been transformed in a number of ways, for example, the stairs have now become Molochs tongue and the former opening at the top of the machine is now the monsters mouth. The intertitle was probably considered necessary for the spectator in order for him or her to identify Moloch, the ancient deity from the Old Testament (Lev 20: 25; Jer 32: 35). There would of course still be a metaphoric relationship without the intertitle, namely, MACHINE IS MONSTER, but the source would in this case be confined to man-devouring mon-

Multimodal metaphor in classical film theory

407

sters in general, thereby losing specific connotations that are attached to the Moloch figure. Thus the pictorial signs and the intertitle, I would suggest, qualify the metaphor as multimodal. Mappable features from the source domain are, for example, how innocent people in ancient times were sacrificed into the fire-burning opening of Moloch, which can be compared to how workers in the film slave to death at the machines, and so on. Moloch is literally eating people, while modern industrial machines are metaphorically eating human beings, and thereby killing them.

Figure 1. Superimposition in Sergei Eisensteins Strike.

Figure 2. In Fritz Langs Metropolis ...

Figure 3. the intertitle helps the Figure 4. the man-devouring deity from the Old Testament. spectator to identify the transformation of the machine to Moloch

This metaphor could be read as reflecting the basic or conventional metaphor DEATH IS A DEVOURER (Lakoff and Turner 1989: 77), but the specific context triggers mappings not available from the devourer scheme as such,

408

Mats Rohdin

for example, extra-textual mappings dealing with metafilmic connotations from the intertitle of the source domain. That is to say, the graphics of the intertitle, oblique and vertical lines, resemble those used in another famous German Expressionist film, The Cabinet of Dr. Caligari (Das Cabinet des Dr. Caligari, Robert Wiene, 1920). This sort of genre mixing (science fiction, horror) facilitates new mappings from the source domain MOLOCH to the target domain MACHINE, for example, connotations like madness, tyranny, disorder, death, evil supernatural forces out of mans control, and so on. A good example of how metaphors can generate new interpretations is the way in which Janet Ward (2001: 164165) connects Moloch in Langs film to the institution of cinema during the Weimar period in German history: [T]he mouth of the devouring Moloch-machine into which workers are led parallels the electrically lit entrances of the movie palace, into which the new white-collar employees would stream, blissfully unaware, in their leisure time (164). Thus I will claim that we have a new metaphor: (THE INSTITUTION OF) CINEMA IS MOLOCH. This means that from the same source (MOLOCH) an implicit target domain (CINEMA) has been constructed, which is not part of the film. Although Ward does not stretch her interpretation any further, it is easy to map more features from the source domain MOLOCH to the new target domain CINEMA. For instance, the fact that it was mostly children who were sacrificed in the ancient Moloch rituals could be paralleled to the censorship debate in Germany during the early decades of 1900, which often centred on films allegedly detrimental influence not only on the working class but also on children (for early German film, see Hake 1993). Although Wards example is taken from a recently published book, and thereby does not belong to the period under study, it is illuminating since it shows how metaphors can build on earlier interpretations, underpin, challenge, revise, or revitalize them, etcetera, thereby changing meaning in the course of time. 2.2 Verbal image A verbal image is a filmic expression building on a verbal expression (metaphor, clich, pun, or proverb), which it visualizes literally (e.g., Carroll 1996b). Soviet montage cinema during the 1920s is famous for this sort of experiment (e.g. Ejxenbaum 1981; Eisenstein 1974, 1987a, 1987b, 1991, 1996; Tynjanov 1981). Often these verbal clichs and metaphors were appropriated from other media and art forms, for example, poster art and pup-

Multimodal metaphor in classical film theory

409

pet theatre, which were very popular at that time because of illiteracy amongst the majority of the people. Sergei Eisenstein often used these sorts of clichs when portraying capitalists, kulaks, priests, policemen, etcetera. Thus the film viewer could use experiences from other media and art forms, which shared codes and conventions with film. As a result, the spectators processing of the film narrative was facilitated considerably, which was important due to the typically fast editing of Soviet cinema at that time. Sergei Eisensteins film debut Strike includes many scenes built upon the visual expression of verbal metaphors and clichs. The famous slaughterhouse sequence at the end of the film, for example, is preceded by a scene in which the police chief gets furious and knocks a bottle of ink over a map showing the streets of the striking workers district, thus literalizing the metaphor the streets running with blood (Bordwell 1993: 5859) (figure 5). The verbal expression is target, while the ink on the map is source. The map metonymically stands for the streets, while the spilled ink metaphorically stands for the workers blood, which gives the metaphor STREETS RUNNING WITH BLOOD ARE MAP SPILLED WITH INK.

Figure 5. When the police chief in Sergei Eisensteins Strike rages and knocks a bottle of ink over a map, the verbal image streets running with blood is visualized.

A mappable feature from the source domain is the manner in which the spilled ink overflows the streets of the map, which correlates with how the next scene shows the massacre of the workers, as the streets running with blood. In order to understand the metaphor the interpreter has to construct the implicit target domain, STREETS RUNNING WITH BLOOD, which requires awareness of the metaphoric expression, either in spoken or written form.

410

Mats Rohdin

Thus the source is presented before the implicit target, but the target is then visualized in the following scene, a common method used by Eisenstein when dealing with verbal images. The Russian formalists, who also worked as film critics and scenario writers, were fond of using verbal images, which may not be surprising considering their literary background. Boris Ejxenbaum (1981: 79), for example, claimed that verbal images, or internal speech as he called it, were the only way to express metaphors in film. One advantage of visualizing verbal images, according to some critics, was that they revitalized conventional and worn out metaphors and clichs (Balzs 1952: 111112, 127), a method used in different media of today as well, for example in political cartoons (e.g., El Refaie 2003: 89). A change in the medium will also, of course, in one way or another, affect the content, which is always important to remember. Nevertheless, many critics and filmmakers (e.g., Clair 1972: 48) condemned this way of metaphorical filmmaking as too literary, a common accusation at a time when proponents of film claimed the medium to be an independent art form. Quite soon, indeed, filmmakers realized that metaphors in the form of verbal images would forever subordinate film to language. The result of this would be that novel metaphors could not be created in film since they had to exist verbally in order for the spectator to be able to identify them at all. For this reason, filmmakers began to seek other ways to express metaphors. 2.3 Montage (primarily the nondiegetic insert) Filmmakers in the early years soon realized how important montage was to achieve filmic signification. Some critics suggested that the film metaphor could be created within the story world (diegesis) by juxtaposing different objects or things by montage. For example, Balzs (1952) paid attention to this type of metaphorical montage, as he called it, which during silent cinema often was visual and monomodal, since the two domains belonged to the diegetic world. He exemplifies this subtype using a sequence from Eisensteins Battleship Potemkin (Bronenosets Potemkin, 1925), where close ups of the faces of the sailors are crosscut with the ships engines: Such repeated juxtaposition compels comparison. A visual parallel inevitably conjures up a parallel in the mind. The angry, resolute faces of the sailors transfer their own expression to the wheels and cranks. Yes, they are fighting side by side in a common struggle (Balzs 1952: 126). This suggests the conventional metaphor MACHINES ARE PEOPLE (Lakoff and Turner 1989: 132)

Multimodal metaphor in classical film theory

411

that attributes human properties to machines. Thus Balzs wanted to stress how even the mechanic world revolts against injustice by joining the revolutionaries. Other critics did not accept montage images such as these as metaphors, since they did not disrupt the mimetic model of the film representation. Instead they claimed that these expressions ought to be named similes, since the cut juxtaposed the two objects or things as the words is like do in verbal similes (e.g., Balzs 1930: 5153). However, when the montage juxtaposition included a nondiegetic insert it was often labeled a film metaphor. In Rohdin (2003) I claimed that it is the nondiegetic insert that is usually referred to when questions concerning film metaphor are discussed. The fact that the nondiegetic insert does not belong to the story world is the anomaly or conflict that cues the spectator to carry out a metaphorical interpretation. References are often made to Sergei Eisensteins films in the 1920s, for example, Strike (1925), October (Oktyabr, 1928) and Old and New (Staroe e novoe, 1929). The ending of Eisensteins Strike includes a well-known example of a nondiegetic insert when images of the militarys massacre of defenceless workers are intercut with scenes from the slaughter of an ox: WORKERS ARE SLAUGHTERED OXEN. Another often-quoted nondiegetic insert appears in Eisensteins October (1928), where shots of the politician Kerensky outside the Tsarinas bedroom in the Winter Palace are intercut with images of a mechanical peacock: KERENSKY IS A PEACOCK. Both these examples include intertitles and elaborate montage but could be referred to as monomodal since in both examples target and source, as well as their metaphorical coupling, can be identified visually. The breakthrough of sound film in the late 1920s stirred an array of debates. In Soviet film, for example, proponents of the montage cinema were worried that sound film would be detrimental to their own film style, built upon fast editing. In a manifesto Sergei Eisenstein, Vsevolod Pudovkin, and Grigori Alexandrov (1988) claimed that sound film could be justified only if sound was used contrapuntally: The first experiments in sound must aim at a sharp discord with the visual images (Eisenstein et al. 1988: 114, emphasis in original). An innovative use of sound in line with the Soviet directors beliefs, including a nondiegetic insert, appears in the comedy/musical The Million (Le Million, 1931) by the French director Ren Clair. When a couple of men in a theatre are fighting about a coat containing a valuable lottery ticket, it looks as though they are playing rugby. On the sound track a nondiegetic sound from a sport arena is heard, that is, noise from the rugby play as well as from the spectators (figures 67). Clair explained the metaphor as the men

412

Mats Rohdin

throwing the coat like a football, a writer would say (cited in Arnheim 1933: 264265). This, I would argue, qualifies as a multimodal metaphor: MEN CHASING COAT ARE MEN PLAYING RUGBY. Perhaps it would be possible to argue that this is a monomodal metaphor because the metaphor is possible to understand without the nondiegetic sound. Although the source domain men playing rugby is cued by both visual and aural elements, I believe that many viewers would miss this metaphor if the obtrusive nondiegetic sound were missing.

Figure 6. When a couple of men in Ren Clairs The Million are chasing a coat inside a theatre

Figure 7. a nondiegetic sound from a sport arena is inserted to give the impression that they are playing rugby.

Clair (cited in Arnheim 1933) characterized the experiment as a soundfilmic metaphor equivalent to a metaphor in literature (265). Rudolf Arnheim, the German gestalt psychologist and film theorist, by contrast, criticized the scene, which he found unacceptable to the spirit of the film medium: The result? Not a metaphor, but a fake (Arnheim 1997: 41). Arnheims reaction is a good example of his and other contemporary theorists view that the fiction film must obey a mimetic form of representation. Nondiegetic inserts, superimpositions, and so on, disturb the films representation of reality and thus, by definition, must be condemned. The nondiegetic sound in Clairs film helps the spectator to trigger the source faster, which may be important in film and its temporal development under normal viewing circumstances. Forceville (this volume) has observed that non-verbal and musical sound which influence the multimodal metaphor tend to cue its source rather than its target domain, which is also true of the example in The Million. Furthermore Forceville claims that a multimodal

Multimodal metaphor in classical film theory

413

metaphor including sound tends to be more unobtrusive and naturalized when part of the source domain is diegetic. This is also confirmed in the example from The Million, since the source is signaled both aurally and visually. Surprisingly, Arnheim, who is so negative about the nondiegetic sound insert, praises the scene as skilful: [T]he things that do not go together are fit together so cleverly that the viewer cannot help combining them psychologically (1997: 41). Hollywood also experimented with film metaphors of the nondiegetic type a couple of times, and two films made by former European directors are famous in this regard. In Fury (1936), Fritz Lang dissolves from a shot of gossiping women to a nondiegetic insert showing cackling fowl: GOSSIPING WOMEN ARE CACKLING FOWL. It could be considered a monomodal metaphor of the visual variety, since the visuals seem to suffice to cue both target (the gossiping women) and source (the cackling fowl). However, it is important to note that the transition to the nondiegetic insert has been prepared for in a number of ways in the film, by including other modes in addition to the visuals. For example, when the gossips start among the women in the village, nondiegetic music is heard simultaneously on the soundtrack, which mimics not only the voices of the women but also, as it will turn out later, the sound of cackling fowl. After a short while the womens voices fade out and become inaudible, while the volume of the mimicking music increases. Later, when another group of women is gossiping, the nondiegetic sound of cackling fowl can be heard on the soundtrack a moment before the nondiegetic insert appears (showing the fowl), which further emphasizes the connection between the womens chatter and the noise of the fowl. At the same time we hear the mimicking music in the background. In sum, four different modes are working together to create this metaphor; visuals, music, speech, and, finally, noise. The use of these various modes speed up recognition, but worth taking into consideration is also that without the aural cues it would presumably be unclear or less clear, at least instantaneously during film viewing, what is supposed to be the mappable feature. To sum up, this example from Fritz Langs Fury could indeed be considered a multimodal metaphor. Charlie Chaplins Modern Times (1936) is also interesting, because it raises important questions concerning the criterion governing the distinction between monomodal and multimodal metaphor. At the beginning of the film Chaplin uses a nondiegetic insert showing penned sheep, which is followed by a dissolve to people in a hurry coming up from the subway in the rush hour going to work (figures 8-10). Thus the two domains of the metaphor are SHEEP (source) and WORKERS (target); consequently, the metaphor can

414

Mats Rohdin

be expressed as WORKERS ARE SHEEP. Since the identification of both target and source can be made on the basis of the visual images alone, this metaphor would be monomodal. The mapping may be a comparison between the workers who are forced to take employment in industry in order to survive in the same way as the sheep are forced into the pen. Both workers and sheep can be seen as victims of modern large-scale industry and its inhuman assembly line production. The black sheep among the white flock is a visualization of the proverb to be a black sheep, and finds its counterpart in the little tramp (Chaplin), dressed in black, that cannot fit into the system among loyal industry workers and other law-abiding citizens in the film.

Figure 8 Charlie Chaplins Modern Times, a nondiegetic insert of penned sheep

Figure 9 dissolves into a shot showing workers

Figure 10 hurrying in the early morning rush.

But the metaphor, I propose, may also draw on the subtitles that precede the nondiegetic insert and the accompanying intense and aggressive music. The intertitles Modern Times. A story of industry, of individual enterprise

Multimodal metaphor in classical film theory

415

humanity crusading in the pursuit of happiness are superimposed on a clock whose second hand is moving relentlessly. Both the written text and the music may play important roles when deciding on mappable features to be transferred from source domain to target domain. For example, the religious connotations in the word crusading imply a war, and where there is a war there are victims. The sequence may also be read as an intertextual homage to Walter Ruttmans Berlin, Symphony of a Great City (Berlin, die Symphonie der Grostadt, 1927), and the controversial sequence where workers in the rush hour on their way to work are crosscut with a herd of cows being pushed through the city streets in the morning, heading for the slaughterhouse. Thus we have within a period of a few years, in films such as Metropolis, Berlin, and Modern Times, a number of remarkable metaphors that represent a fierce critique of modern industrial society and its inhuman treatment of workers. To summarize, in the 1920s and early 1930s the film metaphor is considered as one of the most interesting novelties of the fiction film. But in the middle of the 1930s things change, and Roman Jakobson writes that there is an increasing aversion to filmic metaphor (1987: 464), which reflects the general view among contemporary film critics and theorists. The main problem, according to me, is the insistence on a mimetic model of film representation, which makes it almost impossible to integrate metaphor into the narrative since metaphor, according to these theories, is characterized by an explicit anomaly or deviation disrupting the spatio-temporal and narrative continuity of traditional fiction film. Metaphors built on superimpositions or nondiegetic inserts supposedly ruin the impression of a closed diegetic reality in the fiction film. 2.4 Cinematography When filmmakers realized that nondiegetic inserts and superimpositions did not match with the norms of popular fiction film, they tried other solutions to create some deviation or anomaly on the surface level. A common way of producing these deviations was aided by cinematography, that is, all the manipulations of the film strip by the camera in the shooting phase and by the laboratory in the developing phase (Bordwell and Thompson 2007: 477). Devices commonly used include different sorts of camera angles, camera lenses creating perspective distortions, distance of framing, and so on. Balzs (1952: 111114), for example, paid much attention to this type of metaphor in his writings. Among other things, he points out how the camera

416

Mats Rohdin

set-up makes the film image look real in the diegetic story world, but how it also can express metaphoric meanings: In such metaphoric sequences the objects photographed are real the set-up merely gives them a deeper meaning, a second, symbolical significance, without depriving them of their own, real, normal meaning. The shot would be comprehensible, as the detail of an ordinary film scene, even to those who failed to grasp this second meaning (1952: 113). According to Balzs such metaphors show the skillfulness of the artist, the ability to take advantage of the film images capacity to embrace two different meanings at the same time. This type of metaphor could be based upon unusual camera angle and framing, as when Vsevolod Pudovkin in The End of St. Petersburg (Konets Sankt-Peterburga, 1927) shows the rulers of Russia with their heads cut off by the frame (figure 11). Balzs commentary is, as usual, concise, and not very informative: A pictorial metaphor whose meaning is obvious (Balzs 1930: 36, my translation, MR). However, the intra-textual context includes a lot of intertitles, which describe how Mother Russia calls the people to enter World War I with enthusiasm. Probably Mother Russia (the rulers) is target and the image of the rulers with their heads excluded is source, thus probably some sort of visualization of a verbal image: MOTHER RUSSIA IS HEADLESS PEOPLE. This ought to be labeled multimodal, since the intertitles are important to aid the spectators to process the rapidly changing images showing cheering people, marching soldiers, people sitting on chairs with their heads cropped by the frame, etcetera. The source domains image of the rulers of that time, with their heads cut off by the frame, indicates their headless politics when Russia entered Word War I, features which are mapped to the target domain Mother Russia. Another example of this type of metaphor cued by cinematography comes from Eisensteins Old and New (1929), were the director uses a close-up with wide-angle lens and low camera angle to criticize the Soviet bureaucracy. The intra-textual context is recounted with various shots and intertitles: the farmers are applying for a loan to buy a tractor in order to be able to bring in the harvest. But the local authorities deny them this, because the rules state that the farmers can only be given the loan after they have completed the harvest. Eisenstein wanted to disparage the system of bureaucracy, and he did this in different shots by showing objects (writing machine, pencil sharpener, law book) in gigantic close-ups made with a wide-angle lens, which dwarfs the clerks in the background (figure 12). In the shot in question a man is barely visible behind a gigantic book, perhaps a literal visualization of the verbal image, to hide behind bureaucracy. Thus the

Multimodal metaphor in classical film theory

417

metaphor can be expressed as follows: AUTHORITIES HIDING BEHIND BUREAUCRACY IS CLERK BEHIND BOOK. From the source domain CLERK BEHIND BOOK mappings are made to the target domain AUTHORITIES HIDING BEHIND BUREAUCRACY, for example, just as the clerk hides behind the book so we can barely see him, in the same way authorities hide behind bureaucracy, behind rules and regulations so that we get the impression that there are no humans any longer making decisions, and so on.

Figure

11. Cinematography (e.g., camera framing) cues the spectator to identify a metaphor in Vsevolod Pudovkins The End of St. Petersburg.

Figure 12. An example from Sergei Eisensteins Old and New where cinematography (e.g., close-up and camera angle) indicates a metaphor.

Pudovkins and Eisensteins examples show the importance of distance of framing, especially in the close-up. According to Eisenstein, the close-up makes an image more abstract than, for example, the long shot or medium shot. In Dickens, Griffith and ourselves, he speculates on the reasons which made the Russians realize the importance of the close-up: In Russian we talk about an object or a person being shot on a large scale [krupnym planom], i.e., large. An American says: closely (the literal meaning of the term close-up). Americans talk about the physical conditions of seeing. But we talk of the qualitative side of the phenomenon that is linked to its significance For an American, the term is associated with vision. For us, with the evaluation of what is seen (Eisenstein 1996: 226, emphasis in original). Thus Eisenstein seems to imply that there are verbal reasons which made the Russians connect the close-up with an evaluation of what is seen instead of linking it to the physical condition of seeing, as we do in the West. The result in the former case, according to Eisenstein, is that a metaphorical interpretation is facilitated. This example shows that a medium and its tech-

418

Mats Rohdin

nology, due to cultural circumstances like language, may be understood in different ways in different parts of the world, which is a healthy reminder of the extent to which metaphors are governed by culturally embedded rather than universal knowledge. 2.5 No formal characteristics on surface level This type of film metaphor challenges the old demand for a deviation on the surface level since there are no formal characteristics that cue the spectator to perform a metaphoric interpretation. The following example is taken from the writings of the French film critic Andr Bazin in the 1950s. Among film scholars he is frequently regarded as one of the most important proponents of a realistic film style in film history. Nevertheless, Bazin did speak favorably about cinema and its plastic reproduction of reality as an art of potential metaphor (1981: 151). A good example of his metaphoric interpretations occurs in an article when he writes about a scene at the end of Federico Fellinis The Swindle (Il bidone, 1955). The story is about an old, tired crook, who suddenly decides to become a better man and start anew. After the last swindle, he tries to cheat his accomplices out of a large sum of money. They find out his plans, however, beat him unconscious, and leave him to die on a mountain slope. The last thing he sees before he dies is a group of women and children passing by on the road, carrying bundles of sticks on their backs. Bazin interprets this as angels pass, which he sees as a typical element of Fellinis preoccupation with the angel metaphor (1971: 89) (figures 1314). The two domains are the PEASANT WOMEN (target) and ANGELS (source) respectively: WOMEN ARE ANGELS. Thus the source domain is absent, but is cued by the similarity between the womens bundles of sticks on their backs and angels wings. But it is also possible that Bazins religious interpretation rests upon his knowledge of the works of Dante, for example Vita Nuova and La Divina Commedia. The former deals with a mans wish to start anew, as is the case in Fellinis film, and so does the latter, which begins with the following words: In the middle of lifes road / I found myself in a dark wood. La Divina Commedia has a geographical composition (hell, purgatory, and paradise), which has a correspondence in the films journey from the hectic city nightlife to the top of the mountain near heaven. This can be compared to well-known conceptual metaphors like LIFE IS A JOURNEY, DEATH IS DEPARTURE, and DIVINE IS UP (Lakoff and Turner 1989: 9 11, 150151). Dante is guided into paradise by Beatrice in the same way as

Multimodal metaphor in classical film theory

419

the group of women lead the crook to heaven after his last words: Wait, Im coming. In early Christian belief, angels are regarded as messengers between humans and God, especially after death when every human is guided to heaven by angels (Nationalencyklopedin [The National Encyclopedia] 1996: 408409). The crook is ready to meet the Lord. Moreover, when he dies, non-verbal sounds of dry leaves scattering in the wind are heard, thus connoting his death. But what is most important in this religious interpretation is the sound of the wind, which activates ideas of God. In the Bible, God sometimes appears in disguise as the wind, for example in Genesis (8:1). To summarize, I would categorize this scene from The Swindle as a multimodal metaphor since speech and non-verbal sounds play such an important role in identifying the absent source domain, angels.

Figure 13. Federico Fellinis The Swindle ends, according to Bazin, with a metaphor

Figure 14. showing no formal characteristics on the surface level cueing a metaphoric interpretation.

It is interesting to note that this type of film metaphor is reconcilable with the norms of invisible storytelling of classical Hollywood cinema. Bazins metaphor example does not show any explicit deviation on the surface level from a realistic motivation. Instead his interpretation is based on different contextualizations which underpin the metaphoric interpretation, for example, Fellinis earlier films, thematic elements, stylistic expressions, graphic correspondences, and so on. What Balzs and Bazin have in common is that they connect the identification of the metaphor with the style of the film. Style has always been important in film due to the mediums technological base, which makes the history of film an ongoing story of technological changes in the form of improvements and innovations (e.g., sound film, film

420

Mats Rohdin

stock, moving camera, zoom lens, color film, cinemascope, stereo sound, etcetera), which gives each period and each film a particular appearance. Bazins way of using the concept of style is restricted and pertains mainly to formal characteristics, that is, the repeated and salient uses of film techniques characteristic of a single film or a group of films (Bordwell and Thompson 2007: 481). Bazins opinion about style is subtle, and can be compared to Arthur Dantos view on style attribution in The Transfiguration of the Commonplace: A Philosophy of Art: The structure of a style is like the structure of a personality. And learning to recognize a style is not a simple taxonomic exercise. Learning to recognize a style is like learning to recognize a persons touch or his character (1981: 207, emphasis mine, MR).

3. Summary and concluding remarks One of the most interesting findings in this chapter, I propose, is the important role of intertitles in the canonic examples of film metaphors in the silent cinema. Consequently, many of these examples qualify as multimodal metaphors (of the verbo-pictorial variety). This may be surprising considering that the film medium during this period often was characterized as moving images or visual Esperanto (e.g., Lindsay 1916: 177). Evidently, many film scholars seem to have proceeded from the conviction that film metaphors of silent cinema had to be monomodal, that is, both target and source were visuals. The reason for this is, of course, that the predominance of the visual mode in film was taken for granted, which made the written text auxiliary by definition. But many of the metaphors taken from the silent era listed above are to a high degree dependant on intertitles when spectators identify and/or interpret these metaphors. A possible explanation why early film theorists put so much emphasis on the visual mode is that, in order to prove that film could be accepted as an independent art, intertitles had to be downplayed. When early film theorists wrote about metaphors they often presented them out of context, thereby ignoring the intertitles, which often were of vital importance for identification and/or interpretation of the metaphor. At the same time it should not come as any surprise that, for example, metaphor experiments in Soviet montage cinema were dependent upon intertitles, since the spectator needed help to understand the logic of the story when suddenly superimpositions, nondiegetic inserts, and unfamiliar camera set-ups were invading the screen.

Multimodal metaphor in classical film theory

421

With the breakthrough of sound film in the late 1920s it was possible to include new modes in metaphorical compositions. Sound films like The Million, Fury, Modern Times, and The Swindle show that speech, sound, and music played an important role in both the identification and interpretation of multimodal metaphors. Often different modes interacted in order to make it easier for the spectator to quickly identify and interpret the metaphor, which of course was important considering the irreversibility of the spectacle (the spectator can not stop the film during projection in the theatre to think over or re-examine obscure passages). Sound could also produce meanings that were difficult to represent visually, for example in The Swindle, where the notion of God in the barren mountain landscape is conveyed by the sound of the wind. Nevertheless, elements such as speech, sound, and music are often neglected in early film theory when metaphors are discussed, as though the visuals alone are sufficient to make up the metaphor in question. Another pertinent question in this chapter is how target and source have been presented in these examples. In most of them, both target and source are visually present. An exception is the example from Metropolis, CINEMA IS MOLOCH, where the target is supplied by extra-textual information. Symptomatically, this is an interpretation from our time. My opinion is that this is quite uncommon in classical film theory texts, but much more accepted today, when reader-response theories have influenced criticism and interpretations in, for example, film studies at universities. In fact, much current metaphor interpretation deals with how to find a target to match a source in order to create a metaphor (Johnson and Malgady 1980: 266). This is also in line with the view of metaphor/figurative processing as a reader strategy in contexts which stress novelty and originality, for example, in academic writing (Gibbs 1994: 448449; see also Steen 1994). The only metaphor example discussed in this chapter that draws on an absent source, which is rare, is Fellinis WOMEN ARE ANGELS, where the angels are implied by extra-textual interpretation. An alternative interpretation could be that the metaphor should be reversed, that is, angels are women. This would correspond to the view that the source in linguistic metaphors is always supplied by the description (the text) (Indurkhya 1992: 1819). Another suggestion for how to decide on target and source is to choose as target the domain with the strongest denotation or which belongs to the story (Whittock 1990: 3132), that is, in the Fellini film, the women. However, both Indurkhya and Whittock put forth principles that are more prescriptive than descriptive, which blind them to the rich variety of manifestations of metaphor where target and source are decided on different

422

Mats Rohdin

grounds (for example, when extra-textual knowledge is involved). Various contexts make interpreters choose different ways of determining target and source, without giving priority to a certain mode of procedure determined in advance in the way Indurkhya and Whittock do. Therefore, in the Fellini film my suggestion is that it is the women who are the target, and not the angels, since we use the angels to predicate something about the women. This chapter also shows the rich variety of different features that are mappable from the source domain to the target domain. Since these examples deal with creative metaphors, the mappings may build on connotations and contextual factors of all sorts, for example, other films, genre expectations, socio-cultural knowledge, etcetera (for questions concerning what features are to be mapped, see Forceville 2005: 279). Notable in these examples from classical film theory is that a majority of the film examples are of the metaphor type CONCRETE A IS CONCRETE B. This makes the type different from conventional verbal metaphors, which, according to Lakoff and Johnson, are ABSTRACT A IS CONCRETE B. As scholars have pointed out novel metaphors in art often belong to the former type, that is, CONCRETE A IS CONCRETE B (e.g., Forceville 2006: 387). However, it is important to notice that the interpreter of a non-verbal multimodal metaphor may influence the categorizing when suggesting a verbal rendering of the metaphor. For example, in The End of St. Petersburg and Old and New I proposed two abstract targets, MOTHER RUSSIA and AUTHORITIES, instead of two more concrete targets that also would have been possible, for example, THE RULERS and THE CIVIL SERVANTS respectively. The reason for this did not rest upon any objective fact, but was motivated by my decision to adhere to the vocabulary of Cognitive Metaphor Theory. Another interesting finding is that many film metaphors in classical film theory could be summed up in the conceptual metaphor MAN IS ANIMAL, for example, spy and fox (Strike), workers and oxen (Strike), politician Kerensky and peacock (October), women and fowl (Fury), workers and sheep (Modern Times). But also other common source domains theorized in Cognitive Metaphor Theory (Kvecses 2002: 1620) are found in the film metaphors discussed in this chapter, for example, THE HUMAN BODY (The End of St. Petersburg), MACHINES and TOOLS (Metropolis), GAMES and SPORT (The Million), and MOVEMENT and DIRECTION (The Swindle). According to Kvecses the source domain of animals is extremely productive (2002: 17). Are we to look for embodied or for cultural explanations? Many of these examples come from the Soviet Union in the 1920s, where politicized art and ideology went hand in hand, influenced by poster art, caricatures, political cartoons, and so on. Also worth considering is that the fables of

Multimodal metaphor in classical film theory

423

writers like Aesop and La Fontaine have always been popular in authoritarian societies like Russia and later the Soviet Union. In the introduction of this chapter I posed three different questions, which, according to Forceville (1996: 6566; 2002: 23; 2005: 266), have to be answered: (1) Which are the two terms of the multimodal metaphor? (2) Which is the metaphors target domain and which is the metaphors source domain? (3) Which features are mapped from the source domain to the target domain? Although my classification in this chapter mostly deals with formal characteristics (for example, superimpositions, and nondiegetic inserts), it has also shown that the question of how to identify the metaphor, to quote Danto about style, is not a simple taxonomic exercise. Although some sort of deviation or surface anomaly frequently holds, especially in the early examples, it is not a necessary condition. Therefore, I agree with the philosopher Ted Cohen when he asserts: If we have learned anything since the explosion of interest in metaphor that began about twenty-five years ago, it is that the classical and standard definitions of metaphor are unreliable, and so every sensible person, I think, must suppose that we are dealing with formally undefined phenomena (Cohen 1999: 399, emphasis mine, MR). In contemporary metaphor theory, writers seem to agree that any linguistic material can be used to make up a metaphor (e.g., Kittay 1989: 103; Entzenberg 1998: xviiixxvii). In Rohdin (2003) I have demonstrated that the same holds true for any filmic material. This is in accordance, I argue, with Lakoff and Johnsons well known definition of metaphor: The essence of metaphor is understanding and experiencing one kind of thing in terms of another (Lakoff and Johnson 1980: 5, emphasis in original). However, this should not prevent us from examining which filmic expressions have been used in metaphorical interpretations throughout film history. Therefore I think that corpus-based studies of metaphors in different genres, movements, national cinemas, and among individual directors, would yield interesting results. Worth remembering when dealing with questions like these is Forcevilles critique of Nol Carrolls restricted metaphor classification: Categorizing and name-giving should follow analysis of phenomena, and not the other way round (2002: 9). Finally, when the history of non-verbal metaphors is outlined, references often start with I. A. Richards The Philosophy of Rhetoric: Thought is metaphoric, and proceeds by comparison, and the metaphors of language derive therefrom (1965: 94, emphasis in original). But it is interesting to note that the term metaphor already had been used for several decades in film theory since the 1910s (for example, Lindsay 1916: 172; Epstein 1974 vol. II, 68). It is important to remember that with film we have a new me-

424

Mats Rohdin

dium and art form whose development we have been able to monitor from its birth, in the late 1890s, which makes it exceptional compared to other, older art forms like theatre, literature or painting. This not only gives us important knowledge about the film medium itself, and how it has changed over time, but also demonstrates that metaphor, including what we today call multimodal metaphor, had been introduced, discussed, questioned, and, lastly, established in a non-verbal art such as film, from its earliest beginnings.

Acknowledgment
I would like to thank Charles Forceville for insightful comments on an earlier draft of this chapter.

References
Arnheim, Rudolf 1933 Film [1932]. L. M. Sieveking and Ian F. D. Morrow (transl.), London: Faber and Faber. 1997 Sound film by force [1931]. In Film Essays and Criticism [1977], 3942. Brenda Benthien (transl.), Madison/London: The University of Wisconsin Press. Balzs, Bla 1930 Der Geist des Films. Halle: Verlag Wilhelm Knapp. 1952 Theory of the Film: Character and Growth of A New Art [1945], Edith Bone (transl.), London: Dennis Dobson. Bazin, Andr 1967 The evolution of the language of the cinema [195055]. In What is Cinema? Vol. I [195859], Hugh Gray (ed. and transl.), 2340. Berkeley/Los Angeles/London: University of California Press. 1971 Cabiria. The voyage to the end of neorealism [1957]. In What is Cinema? Vol. II [196162], Hugh Gray (ed. and transl.), 8392. Berkeley/Los Angeles/London: University of California Press. 1973 The French Renoir. In Jean Renoir [1971], Franois Truffaut (ed.), W. W. Halsey II and William H. Simon (transl.), 7491. New York: Simon and Schuster. 1981 On LEspoir, or style in the cinema [1945]. In French Cinema of the Occupation and Resistance: The Birth of a Critical Esthetic [1975], Franois Truffaut (ed.), Stanley Hochman (transl.), 145 157. New York: Frederick Ungar.

Multimodal metaphor in classical film theory

425

Black, Max 1993 More about metaphor. In Metaphor and Thought. 2nd ed. Andrew Ortony (ed.), 1941. Cambridge: Cambridge University Press. Bordwell, David 1993 The Cinema of Eisenstein. Cambridge, MA/London: Harvard University Press. Bordwell, David, and Kristin Thompson 2007 Film Art: An Introduction. 8th ed. Boston: McGraw Hill. Buckland, Warren 2000 The Cognitive Semiotics of Film. Cambridge: Cambridge University Press. Carroll, Nol 1996a A note on film metaphor. In Theorizing the Moving Image, 212 223. Cambridge: Cambridge University Press. 1996b Language and cinema: Preliminary notes for a theory of verbal images [1980-1981]. In Theorizing the Moving Image, 187211. Cambridge: Cambridge University Press. Clair, Ren 1972 In the beginning was the image. In Cinema Yesterday and Today, R. C. Dale (ed.), Stanley Appelbaum (transl.), 4080. New York: Dover. Cohen, Ted 1997 Metaphor, feeling, and narrative. Philosophy and Literature 21 (2): 223244. 1999 Identifying with metaphor: Metaphors of personal identification. Journal of Aesthetics and Art Criticism 57 (4): 399409. Danto, Arthur C. 1981 The Transfiguration of the Commonplace: A Philosophy of Art. Cambridge, MA/London: Harvard University Press. 1992 Metaphor and cognition. In Beyond the Brillo Box: The Visual Arts in Posthistorical Perspective, 7387. Berkeley/Los Angeles/London: University of California Press. Ejxenbaum, Boris 1981 Problems of cinema stylistics [1927]. In Russian Formalist Film Theory, Herbert Eagle (ed.), 5580. Ann Arbor: University of Michigan. Eisenstein, Sergei 1974 Zur Komposition des Streik-Finale [1933]. In Schriften 1: Streik, Hans-Joachim Schlegel (ed.), 274276. Mnchen: Carl Hanser. 1987a Nonindifferent nature [1945]. In Nonindifferent Nature, Herbert Marshall (transl.), 216396. Cambridge: Cambridge University Press.

426

Mats Rohdin

Pathos [1947]. In Nonindifferent Nature, Herbert Marshall (transl.), 38199. Cambridge: Cambridge University Press. 1991 Montage 1937 [1937]. In Selected Works. Vol. II, Towards a Theory of Montage, Michael Glenny and Richard Taylor (eds.), Michael Glenny (transl.), 1158. London: British Film Institute. 1996 Dickens, Griffith, and ourselves [1942]. In Selected Works. Vol. III, Writings 193447, Richard Taylor (ed.), William Powell (transl.), 193238. London: British Film Institute. Eisenstein, Sergei, Vsevolod Pudovkin, and Grigori Alexandrov 1988 Statement on sound [1928]. In Sergei Eisenstein, Selected Works. Vol. 1, Writings 19221934, Richard Taylor (ed. and transl.), 113 114. London/Bloomington: BFI/Indiana University Press. El Refaie, Elisabeth 2003 Understanding visual metaphors: The example of newspaper cartoons. Visual Communication 2 (1): 7595 Entzenberg, Claes 1998 Metaphor as a Mode of Interpretation: An Essay on Interactional and Contextual Sense-Making Processes, Metaphorology, and Verbal Arts. Ph.D. diss., Uppsala University. Epstein, Jean 1974 crits sur le cinema 19211953. 2 vol. Paris: Seghers. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 2002 The identification of target and source in pictorial metaphors. Journal of Pragmatics 34: 114. 2005 Cognitive linguistics and multimodal metaphor. In Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach (ed.), 264284. Cologne: Von Halem. 2006/ this vol. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza (eds.), 379 402. Berlin/New York: Mouton de Gruyter. this vol. The role of non-verbal sound and music in multimodal metaphor. Gibbs, Raymond W., Jr. 1994 The Poetics of Mind: Figurative Thought, Language, and Understanding. Cambridge: Cambridge University Press. Hake, Sabine 1993 The Cinemas Third Machine: Writing on film in Germany 1907 1933. Lincoln, NE/London: University of Nebraska Press. Indurkhya, Bipin 1992 Metaphor and Cognition: An Interactionist Approach. Dordrecht: Kluwer.

1987b

Multimodal metaphor in classical film theory

427

Jakobson, Roman 1987 Is the film in decline? [1933]. In Language in Literature, Krystyna Pomorska and Stephen Rudy (eds.), 458465. Cambridge MA/London: Harvard University Press. Johnson, Michael G., and Robert C. Malgady 1980 Toward a perceptual theory of metaphoric comprehension. In Cognition and Figurative Language, Richard P. Honeck, and Robert R. Hoffman (eds.), 259282. Hillsdale, NJ: Lawrence Erlbaum. Kittay, Eva Feder 1989 Metaphor: Its Cognitive Force and Linguistic Structure. Oxford: Clarendon Press. Kvecses, Zoltn 2002 Metaphor: A Practical Introduction. New York: Oxford University Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press. Lakoff, George, and Mark Turner 1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chicago/London: University of Chicago Press. Lindsay, Vachel 1916 The Art of the Moving Picture. New York: MacMillan. Nationalencyklopedin 1996 Nationalencyklopedin [The National Encyclopedia] VITRY. Hgans: Bokfrlaget Bra Bcker. Richards, I(vor) A(rmstrong) 1965 The Philosophy of Rhetoric. Oxford: Oxford University Press. Rohdin, Mats 2003 Vildsvinet i filmens trdgrd. Metaforbegreppet inom filmteorin [The wild boar in the garden of film: The concept of metaphor in film theory] Summary in English 318329. Ph.D. diss., Stockholm: Edita Norstedts Tryckeri AB. Salt, Barry 1992 Film Style and Technology: History and Analysis. 2nd ed. Starword: London. Stanford, W(illiam) B(edell) 1936 Greek Metaphor: Studies in Theory and Practice. Oxford: Basil Blackwell. Steen, Gerard 1994 Understanding Metaphor in Literature: An Empirical Approach. London/New York: Longman.

428

Mats Rohdin

Tynjanov, Jurij 1981 On the foundations of cinema [1927]. In Russian Formalist Film Theory. Herbert Eagle (ed.), 81100. Ann Arbor: University of Michigan. Ward, Janet 2001 Weimar Surfaces: Urban Visual Culture in 1920s Germany. Berkeley/Los Angeles: University of California Press. Whittock, Trevor 1990 Metaphor and Film. Cambridge: Cambridge University Press.

Chapter 18 Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor in horror films Gunnar Theodr Eggertsson and Charles Forceville

Abstract
In this chapter, we analyze the structural metaphor human victim is animal in three modern horror films, and argue that the metaphor is characteristic of the horror genre. It is shown that its pervasive manifestations, in both monomodal and multimodal form, create what Musolff (2006) calls a metaphor scenario. In the concluding section we discuss some implications of our study for metaphor theory, genre, and animal rights. Keywords: structural metaphor, horror film, animal rights

1. Introduction Within the medium of film, or rather moving images in general, scholarly studies focusing specifically on non-verbal and multimodal metaphor are still relatively rare (Whittock 1990; Rohdin 2003, this volume; see also Carroll 1996; Forceville 1999a, 2003, 2005, this volume). These latter studies, moreover, discuss metaphors that are at the creative end (see Black 1979) rather than at the structural end of the continuum (Lakoff and Johnson 1980). However, assessing the validity of the Conceptual Metaphor Theory (CMT) paradigm launched by Lakoff and Johnson requires not only that non-verbal and multimodal expressions of conceptual metaphors are studied as such, but also that the extent of their systematic occurrence is investigated. After all, a central tenet of CMT is that human beings conceive certain phenomena systematically in terms of certain other phenomena, allowing for numerous different surface manifestations of a single conceptual metaphor. It is suggested in Forceville (2006a/this volume) that examining multimodal metaphors (as indeed many other discourse phenomena) is best

430

Gunnar Theodr Eggertsson and Charles Forceville

done by focusing on specific genres, since the meanings that can or must be gleaned from representations are strongly steered and constrained by the audiences awareness of the genre to which a discourse belongs (Forceville 1996, 1999b, 2006b). The current chapter aims (a) to highlight the importance of the structural metaphor HUMAN VICTIM IS ANIMAL in the modern horror film; (b) to demonstrate how this metaphor is expressed multimodally, and to formulate some tentative generalizations about the surface manifestations of the metaphor.

2. What should count as a multimodal metaphor? Before addressing the central metaphor itself, we want to clarify briefly our use of the labels multimodal and monomodal metaphor. According to Forceville (2006a), monomodal metaphors present target and source domains both in the same mode. In film, the list of modes includes at least: visuals, written language, spoken language, sound, and music. Although this is not an exhaustive list (Mittelberg and Waugh this volume and Mller and Cienki this volume extensively discuss the gestural mode), we will in the present chapter restrict ourselves to these. The best-known variant of monomodal metaphor is verbal metaphor, which until Lakoff and Johnson (1980) was considered the only type of metaphor anyway. Another variant of monomodal metaphor is the pictorial one, which cues both target and source visually (Forceville 1996). By contrast, multimodal metaphors are metaphors whose target and source are each represented exclusively or predominantly in different modes (Forceville 2006a: 384). A pure multimodal metaphor thus presents a target in one mode/modality, and one mode only, and the source in another mode/modality only. But such pure multimodal metaphors are distinguished for analytical purposes only: there is nothing inherently better about pure than about impure multimodal metaphors. Indeed, as acknowledged by Forceville (2006a: 385), probably by far the majority of multimodal metaphors in moving images cue target and/or source in more than one mode simultaneously (we could label these multimodal metaphors in the broad sense, as opposed to multimodal metaphors in the strict or narrow sense). The broad definition of multimodal metaphor is particularly pertinent if a metaphor is not a creative metaphor that surfaces only once in a multimodal text, but is an expression of a conceptual metaphor that keeps appearing throughout a narrative or argument. In Musolffs (2006) terminology, HUMAN VICTIM IS ANIMAL would be a scenario.1 In such a situation, the same conceptual metaphor can well be expressed in a

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

431

variety of ways in each individual instantiation. That is, in one instance it can surface as a monomodal metaphor of the verbal variety, in the next as a narrowly defined multimodal metaphor of the pictorial-sonic type, and in the third as a broadly defined multimodal metaphor with a target that is cued in visual terms only and a source that is cued primarily verbally, but with some musical underpinning as well. It would of course have been possible to assess for each individual manifestation of the HUMAN VICTIM IS ANIMAL metaphor in the three films to be discussed (a) whether it exemplifies a monomodal or a multimodal metaphor; (b) if the latter, whether it is a multimodal metaphor in the strict or in the broad sense; and (c) which mode(s) is/are involved in the target and the source domains, respectively. But since, as we claim, the HUMAN VICTIM IS ANIMAL is a structurally occurring metaphor, the modes in which target and source of the metaphor are cued are bound to vary across different instantiations (for a similar situation, see the discussion of the structural metaphor COLIN IS A CHILD in Forceville [1999a]), to avoid tedious cataloguing we will not systematically list the various types. However, in the concluding section we will make some tentative general observations about this issue.

3. The modern House of Horror The year is 1957 and in the small deer hunting community of Plainfield, Wisconsin, something truly horrifying is waiting to be discovered. Investigators will enter the house of one Edward Theodore Gein to find the remains of numerous female bodies, including the woman they were looking for, freshly killed Bernice Worden. The house will quickly be dubbed the Gein House of Horrors by the press as reports of strange artistic experiments and disturbing uses of the human body become apparent. Bowls made of human skulls, human skin suits, furniture upholstered by body parts and a necklace made of nipples are a few examples of the horrid imagery presented to those willing to know about such things. The case of Ed Gein becomes infamous in its time and a subject of infinite conversation and shock among the public. Atrocious crimes are no longer the subject of far-away places or past realities. The realization that danger can lurk in the neighborhood and that not everyone can be trusted becomes the new reality. Ed Gein constitutes the prototypical serial-killer-next-door and goes on to directly inspire such horror film legends as Norman Bates, Leatherface, and Buffalo Bill. In fact, the story becomes a source and influence for at least three generations of horror cinema, ranging most notably from Psycho (Hitchcock, USA 1960) to The

432

Gunnar Theodr Eggertsson and Charles Forceville

Texas Chain Saw Massacre (Hooper, USA 1974) and The Silence of the Lambs (Demme, USA 1991). Other horror films influenced by the Gein story include Three on a Meathook (Girdler, USA 1972), Deranged (Gillen and Ormsby, Canada/USA 1974), Maniac (Lustig, USA 1980), Ed Gein (Parello, USA 2000), and House of 1000 Corpses (Zombie, USA 2003). Horror scholars generally see the release of Hitchcocks Psycho as marking the shift from classic to modern horror film. The monsters populating classic horror films were always foreign in a way mummies, vampires, aliens but after checking in at Psychos Bates Motel the home became the new territory for horror. Reynald Humphries claims about modern horror that the element that needs to be stressed is that we are dealing with the modern, everyday world, not with one situated in the past. Gone are the mad scientists, the remote islands and settings, the dangers of invasion and radiation (Humphries 2002: 85). This element can be directly traced to the films that sprang from the obsession with the case of Ed Gein, where horror found a new home right next door in Anytown. But there is more to the Gein influence than meets the eye. By the time of his death in 1984, Gein had already become a legend in horror folklore, an inspiration to decades of horror cinema and revered by horror buffs as the prototype of every knife-, axe-, and cleaver-wielding maniac who has stalked Americas movie screens for the past thirty years (Internet Zombie 2006, http://www.houseofhorrors.com/gein.htm last accessed 11.05.07). On the surface, this seems very obvious. Not only did he bring terror into suburban reality, and cast a shadow of doubt and paranoia over even the most lovable neighbor; his violent crimes and methods have been recreated and imitated in countless cheap horror flicks since the 1960s. But there is a more deeprooted argument to be made regarding the extent of Geins crimes one that will be inspected more closely in this chapter. Sure enough, the shock of such atrocious and horrific crimes taking place in the peaceful rural community had an extreme and long-lasting effect not only on the evolution of horror art but on the American psyche in general. This shock derives from a very particular aspect of his violent crimes something that provided the base for the publics extremely horrific reactions to his actions, and that was responsible for a crucial element in horror cinema ever since: Ed Gein treated human beings in precisely the same way human beings treat their non-human counterparts. For the purpose of this chapter, this statement can be rephrased as: Ed Gein created a metaphor that has been embraced and recycled by horror cinema ever since investigators came upon the gutted, decapitated body of Mrs. Worden hanging upside down in Geins House of Horrors back in 1957. This metaphor went unnoticed at the time but after

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

433

fifty years of roaming wild and pervading the genre it is time to finally analyze it: HUMAN BEING IS ANIMAL,2 or to be more descriptive: HUMAN VICTIM IN HORROR FILM IS ANIMAL USED AS RESOURCE.

4. Metaphorical vulgarity The idea of human victims presented as animals is not necessarily anything new in the history of horror. After all, horror is about fear, and fear derives from something that is out of ones control. A very natural thing for us to fear is something that has power over us such as a being that holds a higher place in the food chain. It used to be the case that the human race had to fear other predators, but those days are long since gone (at least in contemporary urban societies) and we have placed ourselves, self-assuredly, on top of the food chain. No one feeds on us anymore, so ultimately a great deal of fear comes from the idea that something unexpected can jump in and eat us. Werewolves, vampires, atomic mutations and maggot-infested living dead corpses immediately spring to mind, but these monstrosities are not relevant to the subject of this chapter, as there is no metaphor in how they are presented. If a monster is hunting a human being for food, then the human is quite simply a non-metaphorical prey an animal subjugated to a higher species that fits into the natural chain of events. But if a human being is hunting another human being, then something different happens. The victim becomes a metaphorical prey and the killer a metaphorical predator. Therefore this chapter will concentrate on that special group of serial killers and madmen in horror films which does not draw any distinctive line between killing an animal and killing a human being. The focus will be on three films where human beings abuse other animate creatures, treating them as animals. The attacker considers his victim purely in terms of utilitarian value; he is after the victim for its flesh or bones or because he wishes purely to indulge his sadism or desire to kill. In other words, the human victims have their status as social human beings reduced to the status of non-human animals in Western society an expendable resource. If monster movies were to be included, obvious justifications would be present for most of the violence in cases where the attackers reasons can be explained away simply as acting on impulse or by natural instinct in terms of, for instance, selfdefense, territorial interest, or nourishment. Even in cases of monsters that appear to be very human in nature, such as humanoid vampires, the justification can be a purely biological one they are creatures that can not sustain themselves except on blood. The same goes for instance for the crea-

434

Gunnar Theodr Eggertsson and Charles Forceville

tures from the Alien (Scott, UK 1979) cycle, who need to kill other life forms in order to procreate. When human beings attack other human beings, the actions are more difficult to justify. In many cases hardly even an attempt is made in the films themselves to provide proper justification. The genre of extreme horror is one famous for poorly constructed narratives, usually existing solely in order to string together different scenes of gratuitous violence where the pleasure of the killer is shared by the audience rooting for progressively elaborate effect sequences of blood and gore. This has led to the claim that it is in fact the special effects crew that ranks first, above director, screenwriter, producer, or actors, in such films. The subgenre of extreme horror is furthermore an interesting field for metaphor studies, as it is generally considered to be of lesser aesthetic quality than the more conservative horror movie that style of horror which actually aims at scaring the audience, keeping up a reasonable storyline, and generally leaning towards some sort of intellectual process in the viewing experience. The extreme horror movie, on the other hand, focuses purely on the bodily experience: the more body-harassing details are provided, the more clearly does the film belong to the genre. This constitutes a great part of why these movies are generally frowned upon by the advocates of proper cinema which tends not to show everything. It is always more in the style of fine arts to metaphorize instead of vulgarly presenting the subject. But in the genre of extreme horror, metaphors abound be they intentionally created or not. 4.1 The mapping of the hunt: Wolf Creek In Wolf Creek (McLean, Australia 2005), three backpackers are driving through the Australian outback. After spending an afternoon by the Wolf Creek crater they return to find their car broken. A man comes by on a truck and offers to tow their car back to his place where he can fix it for them. The group is hesitant, but ultimately decides to accept his offer. The man, Mick, drives them in the dark of night to his secluded home, where they sit together and chat for a while before going to sleep. It is through this evenings discussion that the group discovers he used to be a professional shooter getting rid of vermin for clients, such as kangaroos, horses, pigs, buffalo, you name it, he proudly exclaims. But this veteran hunter has been forced out of his job because people use poison nowadays instead of hiring shooters. Therefore he has taken it upon himself to hunt people for his own pleasure the backpackers do not know that yet, but they will soon find out. Dur-

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

435

ing their conversation Mick describes different ways of hunting and killing animals and when one of the girls asks him if he really kills kangaroos he replies that he is in fact doing people a favor, making the following explicit analogy: theyre everywhere out here now like tourists! The HUMAN IS ANIMAL metaphor in Wolf Creek comes from different verbal references to kangaroos, Micks evening speech about hunting, and from visual cues in the latter half of the film, which mainly focuses on Mick hunting down two of the escaped backpackers.

Figure 1. The victims of Wolf Creek are shown driving, through a bullet hole in a kangaroo warning sign, as part of a running reference to the characters as kangaroos to be hunted.

The most obvious way the metaphor HUMAN IS ANIMAL surfaces is through the contextual information of Micks hunting background, combined with his weapons of choice (including a hunting knife and rifle) and his method of trapping and caging his prey: he tampers with peoples cars and then offers them help, thereby luring them to his home where he subsequently drugs his victims. In a telling scene, one of the victim girls stumbles upon a room full of hunting memorabilia, including video footage of other people having fallen prey to Mick in precisely the same way. After trapping and caging the backpackers, he proceeds to torture and kill them as he sees fit to appease his sadistic desires. And even though he does not intend for his victims to escape, he is well prepared for that possibility, thus turning the game into a proper hunt. When the two girls manage to flee he goes on the road to pursue and kill them. In a similar manner to most cinematic serial killers of his genre, Mick gets his first victim up close and personal (Clover 1992: 3132) when he cuts her up with a big hunting knife, mutilating her hand and severing her spinal chord in order to disable her. But he steps away from his genre colleagues when he pulls out his hunting rifle a weapon made to use at a distance. The second girl manages to flee out to the highway, where she is

436

Gunnar Theodr Eggertsson and Charles Forceville

picked up by an old man whom Mick shoots from far away. The girl takes the car and a chase ensues. After having been thrown off the road, Mick shoots her tires with the rifle. Even as he arrives at the crash site and finds her still alive, he does not kill her up close but ultimately shoots her from a few feet away.

Figures 2a and 2b. Kangaroos jump around the outback. The last survivor is cross-cut with the kangaroos to further emphasize his role as a hunted, wounded animal (Wolf Creek).

There are a few other hints to bring out the animal metaphor, given throughout the film as part of a running reference to the characters as kangaroos to be hunted. Very early on, the group of victims is filmed through a bullet-hole in a kangaroo warning sign (figure 1). A close-up of the road sign fills the screen, with the focus on the victims car in the background, seen through the hole, driving down the road. The focus then quickly shifts to the foreground, showing viewers not only that the sign portrays a shaded image of a kangaroo, but that the hole in question has been made by a bullet. Through a juxtaposition of the car, the bullet hole, and the kangaroo image, the specific metaphor HUMAN VICTIMS ARE KANGAROOS is first introduced. In addition

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

437

to Micks explicit reference to tourists (i.e., the backpackers) as kangaroo vermin, there is another juxtaposition involving the kangaroo domain later in the film. After being trapped by Mick, the third protagonist victim is not shown again until after the two girls have been killed. He is nailed to a cross next to a locked pen of starved dogs, sharing the room with the corpse of a previous torture victim (eaten, presumably alive, from the waist down). He manages to free himself and escape into the wild, wandering around until he is picked up by travelers. It is in these final scenes of the film that for the first time real-life kangaroos are finally shown roaming the outback, juxtaposed with the last survivor to further emphasize his role as a hunted, wounded, and lost animal (figures 2a and 2b). The metaphor HUMAN VICTIMS ARE KANGAROOS surfaces in two ways monomodally through the use of language (during Micks speech) and monomodally through visual juxtaposition (the road sign and the kangaroo herd). Furthermore, the more general metaphor HUMAN IS ANIMAL is created through the context of Micks hunting background and the unabashed nature of his murders, in which the victims could easily be substituted by an animal and the serial killer by a sports hunter. From this short description it should be clear that Mick employs explicit methods of animal game hunting, using terms of trapping, chasing and killing his victims with popular hunting weapons. Nevertheless he gains no utility from their bodies other than the pleasure and thrill of the kill he is shown either burning the victims or leaving them to hang and rot in his home. It is possible that he uses the material possessions stolen from the victims as a source of income, but from what is shown on-screen it seems that their sole purpose is to substitute for hunting trophies. Furthermore, stealing would hardly justify the torture and killing he could easily do that without resorting to his violent methods. Thus he becomes reminiscent of animal game hunters, where the object of desire is not the victim itself, but the pleasure gained from the sport of the hunt. And if Mick the killer constitutes part of the metaphor SERIAL KILLER IS SPORTS HUNTER, then his victims surely fall into that category of ultimate game hunting: HUMAN VICTIM IS ANIMAL RESOURCE. 4.2 The mapping of caging: Calvaire/The Ordeal Calvaire/The Ordeal (Du Welz, Belgium/France/Luxembourg, 2004) provides a very vivid example of a human being whose rights are violated to satisfy the most whimsical desires of a fellow human being. The HUMAN IS ANIMAL metaphor surfaces strongly through the more specific metaphor

438

Gunnar Theodr Eggertsson and Charles Forceville

PERSON IS CAPTIVE ANIMAL,

where the character of Marc is so presented in order to further emphasize his defenselessness towards his captor. In other words, by understanding Marc in terms of a trapped animal, the viewer can become even more sympathetic towards the character it adds to the terror of Marcs atrocious ordeal and taps into the idea that horror films systematically explore an inherent fear in humans to be treated in the same way as we treat animals. Therein lies the true terror of Calvaire. Cabaret singer Marc Stevens is traveling alone when his van breaks down in the middle of rural nowhere. He meets Boris, who is out looking for his dog in fact, he has been looking for his dog for a very long time and seems quite insane and brings Marc to innkeeper Paul Bartel, who offers him a place to stay and a helping hand. Bartel is especially interested in Marcs role as a singer because his wife Gloria, who has left him long ago, was also a singer. In a hauntingly beautiful scene Marc agrees to sing a song for Bartel, hoping to get on his way the following morning. But Bartel becomes convinced that Marc is his lost wife returned, destroys Marcs car, and takes the singer captive in his home, all the while pretending that Marc is his wife. He shaves Marcs head (figure 4c), dresses him up in her old clothes, and keeps him in his bed at night. As the days go by a group of villagers start showing interest in the affairs at the old inn. They also become convinced that Marc is Gloria returned and subsequently attack Bartels inn, killing both Boris and Bartel and attempting to gang-rape Marc, who manages to escape into the woods. In the final scenes the camera leaves Marc alone in the woods and takes off to wander aimlessly until the end credits roll perhaps implying that there is no turning back for our protagonist. The first example of the animal metaphor is found in the character of Boris, who is constantly seeking out his dog, Bella. There is an explicit analogy between Boris and his long lost dog on the one hand, and Bartel and his long lost wife on the other. The two characters are regularly juxtaposed visually and both are clearly out of their minds. This analogy is further emphasized later in the film, when Marc has become stuck in a woodland trap after a failed escape attempt. With his hands tied together, he is completely unable to move and lies there, crying, trapped until nightfall. Boris stumbles upon him, but instead of helping out, he simply sits down and starts stroking his head and talking about his dog, treating Marc as if he were Bella incarnate. Marc pleads with Boris to untie him but gets no answer and as if their positioning is not reminiscent enough of a man and his dog, Marc bites Boris in the thigh in a desperate act of frustration. Here the specific metaphor MARC IS BELLA or HUMAN IS DOG surfaces as part of the more inclusive HUMAN IS ANIMAL metaphor (figures 3a-c).

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

439

Figures 3a-c. From top to bottom: Marc ensnared in a trap in the woods; Boris treats the trapped Marc as if he were his lost dog; Marc is hunted in the final chase through the woods (Calvaire).

Another manifestation surfaces relatively early in the film, with the added sexual implication of Marc as an object of desire, when Marc takes a walk in the woods and comes across a farming area where he hears animal noises coming from a nearby stable. Silently he sneaks towards the sounds and spies on a family of male villagers the same ones that will raid the inn later on and attempt to rape Marc standing around one family member who is getting his genitals licked by a calf. There is also a pig present and a cow looking on. A few small dead animals hang from the stable rafts behind the men, who are all enthusiastically watching their relative lying on the stable floor getting a bestial blowjob to the sounds of their squealing pet pig. It is a clear case of an innocent animal being used for sexual pleasure in the same way as Bartel and these same villagers will use Marc later on. In a way, as Marc peeps through the wooden boards of the stable, he is looking into his own future near-fate: to be sexually exploited by a bunch of crazy men (figures 4a, b). The calf is the source domain of a premonitory metaphor MARC IS CALF derived from the larger animal metaphor and created from

440

Gunnar Theodr Eggertsson and Charles Forceville

the contextual information of what lies ahead for Marc (Bartel taking him to bed and dressing him up, the villagers rape attempt), the visual juxtaposition of Marc and the calf at the stable, as well as the sounds of the farm animals during the stable scene, most notably the pig squealing.

Figures 4a-c. From top to bottom: Marc peeks into the barn and into his own future; the villagers within take sexual pleasure from one of the farm animals; Marc is shorn by Bartel.

The final noteworthy sub-metaphor is that of the trap and the hunt. During his failed escape attempt, Marc is caught in an animal trap, as previously mentioned, and subsequently brought back to his captor, Bartel, and returned to his cage, where he is locked up, constantly watched over, and subjected to cruel and degrading treatment. When Boris drives Marc back to Bartel after freeing him from the trap in the woods (with Marc lying on the back of the tractor like a hunted animal), they pass two villagers who are in the process of gutting a boar they have hunted, creating a graphic hunting juxtaposition. Furthermore, during the second escape attempt after the villagers have killed both Boris and Bartel the viewer is again reminded of a hunt, where the villagers bear hunting rifles and use their pigs as a tracker or bloodhound. In the end, Marc is left alone in the woods, completely lost and

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

441

out of place in the world, much like the last survivor of Wolf Creek. These scenes all hint at the metaphor MARC IS HUNTED ANIMAL. In each manifestation of these sub-metaphors Marc is presented in terms of animals that are in human captivity thus creating the metaphor MARC IS CAPTIVE ANIMAL. By presenting the protagonist in such a way, certain elements become highlighted for his character portrayal. He is defenseless, suffering, subjugated, and stripped of freedom, but most importantly his life and well-being depend solely on the will and desire of his captor. All of these features belong to the social status of nonhuman animals in contemporary society but they do not usually apply to human beings at least not in a society that bases its legislation on universally declared human rights. 4.3 The mapping of slaughter: The Texas Chain Saw Massacre Few horror films deal as explicitly with slaughter or are as full of references to meat as the original The Texas Chain Saw Massacre (Hooper, USA 1974). Fictionally represented as being based on a true story, it is famously inspired, at least in part, by the Gein murders. The theme of slaughtering human beings like animals and putting their bodies to use (e.g., as food, clothing, and decoration) resonates throughout the film as the metaphor HUMAN VICTIM IS ANIMAL RESOURCE comes to life in numerous scenes. The storyline has become horror movie stock material. Five friends head into the countryside to visit an old family house that is currently uninhabited. Upon arrival, one by one they fall prey to the cannibalistic family next door, until the last surviving girl, Sally, manages to escape into the open road and gets picked up by a truck driver and wheeled away to safety. The cannibal family consists of two brothers, their father, and their grandfather, whose business used to be slaughtering cattle. However, similar to Wolf Creeks killer Mick, they have lost their jobs due to new farming technologies. The father works as a gas station attendant and the two boys seem to be employed only to rob graves and hunt people and animals for the family unit. Grampa is long retired, but retains the claim to fame as one of the greatest butchers this rural community ever had. The first shot to greet viewers after the opening credits is that of an armadillo road kill in the street. Driving past the dead armadillo the youngsters are filmed heading to their imminent doom on the open road. The juxtaposition clearly mirrors that of the kangaroo warning scene in Wolf Creek and serves as part of the running HUMAN VICTIM IS ANIMAL RESOURCE metaphor or in this case, HUMAN VICTIMS IN CAR ARE DEAD ARMADILLO IN STREET.

442

Gunnar Theodr Eggertsson and Charles Forceville

As they head down the road they pass the old slaughterhouse, at which point Franklin, whose relatives used to own the house they are driving towards, reminisces about how his grandfather used to sell his cattle to that place. He describes old-fashioned cattle slaughter methods in vivid detail, involving bashing in heads with a large sledgehammer. Although the audience does not yet know it at this stage unless they are alert to extreme horrors genre conventions here the abused-animal story helps build up the metaphors source domain, since the head-bashing will be acted out on some of the human victims as the film progresses. This lends support to the specific metaphor HUMAN VICTIMS ARE CATTLE, which is emphasized by crosscutting the youngsters discussion with images of cows in small stalls, huddled together on their way to slaughter, as the car drives past (figure 5a-f). Thus the metaphor is monomodally presented through the use of dialogue and visual juxtaposition, respectively, and multimodally through verbal and visual references to killings with sledgehammers. The scene at the films end, when Sally is held captive with her head over a basin, and the legendary butcher grandfather makes vain attempts to bash her head in, is particularly reminiscent of old-fashioned cattle slaughter, precisely reproducing common procedures as described by Franklin in the car.

Figures 5a-f. Discussions in the car about slaughtering, cross-cut with cattle (The Texas Chain Saw Massacre).

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

443

Another gruesome scene involves the first two murders of the friends, shortly after they arrive at the country house. A couple Kirk and Pam go knocking on the cannibal house next door looking for gasoline. Pam stays outside in a swing as Kirk enters the house, after getting no answers from their knocking. Inside, a wall is decorated with animal remains and on the soundtrack the constant noise of a squealing pig can be heard. When Kirk enters the opposite room, Leatherface, one of the brothers, pounds his head with a sledgehammer. Kirks body twitches and writhes like a freshly slaughtered animal as Leatherface hits him repeatedly and drags him into the room, which closes with a heavy steel door, reminiscent of an industrial slaughterhouse. When Pam follows to investigate she goes into a different room where she falls into a pile of animal bones. The whole room is decorated with animal remains (human and nonhuman alike, presumably) and to further emphasize the metaphor there is a live chicken hanging in a small cage, clucking as it anxiously tries to move around its confined space. The room is filled with macabre artworks, such as a couch made of bones and fetishized human skulls, and shots of animal remains are crosscut with the nervous chicken and the shocked Pam, who subsequently becomes nauseous. As she stumbles back out of the room Leatherface emerges to pull her into the slaughter room, where he sticks her up on a large meat hook hanging from the ceiling. He lets her hang there, screaming and convulsing, as he begins to cut up her boyfriends body with a chain saw. Finally, when their friend Jerry comes looking for them he follows a strange sound into the slaughter room, resonating from within a big icebox. Upon opening it he finds Pam still twitching, her body being kept in cool storage. Suddenly Leatherface reappears and kills Jerry with the sledgehammer. The way the violence is presented from the telling descriptions of slaughter by sledgehammer to the storing of meat in an icebox is all done in terms of animal treatment. Furthermore, it remains obvious through crosscutting (the dead armadillo, the pent-up cattle, and the caged chicken) that the victims are symbolized to be at the same level as subjugated animals, powerless to act against their impending doom as weak prey for a stronger attacker. The metaphor resounds mostly through the contextual nature of cattle slaughter as well as repeated hints of juxtaposing. The cannibal family hunt their victims and subsequently kill them. Even though in the case described above the victims all wandered in the house of their own free will, it is clearly put forth in the film that the brothers job is to catch people out on the road. The father sells barbecue equipment at his local gas station and it can be read between the lines that the meat he is selling is most likely human meat (possibly mixed with that of animals). The family mem-

444

Gunnar Theodr Eggertsson and Charles Forceville

bers eat other types of meat as well, but make no distinction between eating humans or animals. Furthermore, they put their human victims to more use than only for food. Leatherface makes clothes out of human skin, symbolized by his trademark mask, and the house is filled with monstrous artworks made from human bones and/or skin, similar to the objects found in the Gein house of horrors. Unlike the sports hunter of Wolf Creek these human hunters are interested in more than just the thrill of the kill, and unlike the loveseeking innkeeper of Calvaire their prey is not merely an object of sexual desire. To them, it is one of their main sources of income and sustenance, much like for the people of Plainfield. The symbolism of people-as-meat is a well known interpretation for The Texas Chain Saw Massacre, but the traditional interpretation does not include references to animal treatment; rather it is volunteered in the context of the Vietnam War as being the great machine that, at the time, was eating Americas youth (an interpretation presented in Ursula MacFarlane and David Richardsons Clive Barkers A-Z of Horror, UK 1997). The animal metaphor around the cannibal house is vividly based on the work of Ed Gein and serves as a constant reminder of the role our victims play throughout the film. In the end Sally, the sole survivor of the unfortunate five, has come to realize what it is like to be treated as an animal for the slaughter and may bring that realization with her back into society as she rides away on the truck in the finale scene.

5. Discussion and conclusions Although we investigated in detail three horror films only, we propose that the HUMAN VICTIM IS ANIMAL metaphor dominates many more such films. Other pertinent examples include The Hills Have Eyes (Craven, USA 1977), exploring the slaughter/meat metaphor; Hostel (Roth, USA 2005), mining the hunting/trapping metaphor; and Les Yeux Sans Visage/Eyes Without a Face (Franju, France 1960), revealing the caging metaphor but with the added dimension of torture and experimentation (see Eggertsson 2006 for more discussion of these films). We want to end by presenting our findings in the form of some tentative conclusions that can guide further theoretical and experimental research. We have provided further support for the insight that the medium film can coherently exploit verbal, visual, and sonic modalities to cue a specific conceptual metaphor (see Forceville 1999, 2006b; Forceville and Jeulink 2007). Sometimes manifestations of HUMAN VICTIM IS ANIMAL take the form

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

445

of monomodal metaphors (verbal or pictorial), sometimes they are multimodal in the narrow sense, sometimes multimodal in the broad sense (Forceville 2006a). The conceptual nature of the metaphor transpires largely through its constant repetition. While we might be in doubt whether to construe the metaphor on the basis of single occurrences, its reiteration, in many different stylistic guises, relentlessly drives the insight home. The metaphor is a highly embodied one (Lakoff and Johnson 1980; Gibbs 2006; Johnson 2007) in exploiting the mapping of very physical actions (shaving, hunting, trapping, caging, slaughtering, killing) directed against animals, but also has cultural dimensions (Gibbs and Steen 1999; Kvecses 2005) inasmuch as it highlights behaviors toward animals that are sanctioned or even encouraged by mainstream society; after all, the scenario-based presuppositions are needed to understand the inferences suggested in the texts (Musolff 2006: 36). The HUMAN VICTIM IS ANIMAL metaphor manifests itself structurally, but in various dimensions in the modern horror film. While in each film the victims are a resource for desire, the desires are of different kinds. The serial killer of Wolf Creek quenches his desires through the sadistic joy of killing. The mentally deranged innkeeper of Calvaire satisfies his sexual desires by using his victim as an object of pleasure and entertainment. The cannibalistic family members of The Texas Chain Saw Massacre meet their desires mostly through their love of meat. All of the victims are presented as animals to emphasize their subjugated position towards their attackers. The pertinent mappings in each case thus depend on the nature of the victimizers desires. Here is an (incomplete) list of these mappings:
Target domain
HUMAN VICTIM IS

Source domain
ANIMAL

No inherent social right to life Stripped of freedom and liberty Subjugated within a hierarchy Subject to imprisonment Subject to excessive pain, suffering Defenseless towards captors Voiceless towards society Object of sexual desire Object of entertainment Resource for food Resource for material goods Life and well-being depends on the will and desire of captor

446

Gunnar Theodr Eggertsson and Charles Forceville

Although by and large the mappable features apply across the range of animal species, in a given instantiation the species may play a role: Wolf Creek mostly discusses kangaroos, which are typically hunted in Australia, but makes references to horses, pigs and buffalos as well. Calvaire presents farmyard animals (pig, cow, calf) and includes a trap that is obviously made to catch the leg of an animal (probably a rabbit or a fox). The Texas Chain Saw Massacre mainly presents cattle as the metaphorical source, although it also refers to a pig squealing as well as to a caged chicken. It is to be noted that the precise way in which an animal can be abused varies per species depending on such factors as whether it is wild or domestic, big or small, edible or non-edible, furred or feathered. Such considerations are important, because they help counterbalance classic CMTs sometimes one-sided emphasis on the conceptual level. Yes, the central metaphor in these films is HUMAN VICTIM IS ANIMAL but the precise mappings vary in many small and not-so-small ways depending on the precise narrative context in which the metaphor surfaces. This view ties in with more recent work in CMT. Thus Kvecses observation that the mappings of the same metaphor may be different across any two languages or [language] varieties (2005: 123) needs to be even further refined to accommodate the specificities of ad hoc context. The recurrence of the metaphor HUMAN VICTIM IS ANIMAL can be considered an important indication to characterize a film as a horror film. Hence the films discussed illuminate that structural metaphors can contribute to genre theory (Altman 1999; Neale 2000, 2002). Note that this claim only holds if the metaphor is verbalized in this particular way. The more allencompassing HUMAN BEING IS ANIMAL may contribute to the characterization of other genres such as, arguably, the fable (although here the thorny question remains whether the metaphor is HUMAN BEING IS ANIMAL or ANIMAL IS HUMAN BEING, perhaps calling for a blending theory approach, see Fauconnier and Turner 2002). The centrality of the HUMAN VICTIM IS ANIMAL metaphor in the horror genre could be further investigated by focusing on promotion materials such as trailers and posters. Finally, the consistency with which the metaphor under scrutiny is deployed in horror film inevitably focuses attention on issues of morality, specifically those pertaining to animal rights. As Lakoff and Johnson point out, the most fundamental values in a culture will be coherent with the metaphorical structure of the most fundamental concepts in the culture (1980: 22). In the cases discussed in this chapter, the horrific mistreatment of humans that is central to the horror genres conventions can apparently be no more hauntingly presented than through the metaphor HUMAN VICTIM IS ANIMAL, in which

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

447

behaviors considered more or less conventional and acceptable when displayed toward non-human creatures become a source of fear, disgust, and repellence when mapped onto human beings (for more discussion on the role of metaphors in the constitution of cultural values, see Johnson 1993). The structural nature of the metaphor under discussion thus reveals the moral double standard that surfaces when it becomes apparent that it is not the violence itself that creates horror for the viewer it is rather the type of animal the violence is administered to (Singer 2001). Acknowledgment
The authors are indebted to Mats Rohdin for his thoughtful comments on an earlier draft of this chapter, but of course remain entirely responsible for all views expressed.

Notes
1. Musolff characterizes a metaphor scenario as a set of assumptions made by competent members of a discourse community about typical aspects of a source-situation, for example, its participants and their roles, the dramatic storylines and outcomes, and conventional evaluations of whether they count as successful or unsuccessful, normal or abnormal, permissible or illegitimate, etc. (Musolff 2006: 28). Arguably, the metaphor should read HUMAN ANIMAL IS NON-HUMAN ANIMAL, but since the English language differentiates between humans and animals the more typical anthropocentric phrase will be used.

2.

References
Altman, Rick 1999 Film/Genre. London: British Film Institute. Black Max 1979 More about metaphor. In Metaphor and Thought, Andrew Ortony (ed.), 1943. Cambridge: Cambridge University Press. Carroll, Noel 1996 A note on film metaphor. In Theorizing the Moving Image, 212 223. Cambridge: Cambridge University Press.

448

Gunnar Theodr Eggertsson and Charles Forceville

Clover, Carol J. 1992 Men, Women and Chain Saws: Gender in the Modern Horror Film. London: British Film Institute. Eggertsson, Gunnar Theodr 2006 Animal Horror: An Investigation into Animal Rights, Horror Cinema and the Double Standards of Violent Human Behaviour. Research MA diss., Media Studies Department, Universiteit van Amsterdam. Fauconnier, Gilles, and Mark Turner 2002 The Way we Think: Conceptual Blending and the Minds Hidden Complexities. New York: Basic Books. Forceville, Charles 1996 Pictorial Metaphor in Advertising. London/New York: Routledge. 1999a The metaphor COLIN IS A CHILD in Ian McEwans, Harold Pinters, and Paul Schraders The Comfort of Strangers. Metaphor and Symbol 14: 17998. 1999b Art or ad? The effect of genre-attribution on the interpretation of images. SPIEL 18: 279300. 2003 Bildliche und multimodale Metaphern in Werbespots [Trans. from English by Dagmar Schmauks] Zeitschrift fr Semiotik 25: 3960. 2005 Cognitive linguistics and multimodal metaphor. In Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach (ed.), 264284. Cologne: Von Halem. 2006a/this vol. Non-verbal and multimodal metaphor in a cognitivist framework: agendas for research. In Cognitive Linguistics: Current Applications and Future Perspectives, Gitte Kristiansen, Michel Achard, Ren Dirven, and Francisco Ruiz de Mendoza Ibez (eds.), 379402. Berlin/New York: Mouton de Gruyter. 2006b The Source-Path-Goal schema in the autobiographical journey documentary: McElwee, Van der Keuken, Cole. New Review of Film and Television Studies 4: 241261. 2007a Multimodal metaphor in ten Dutch TV commercials. Public Journal of Semiotics 1: 1951. http://semiotics.ca/ this vol. The role of non-verbal sound and music in multimodal metaphor. Forceville, Charles, and Marloes Jeulink 2007 The Source-Path-Goal schema in animation film. Paper presented at 10th International Cognitivist Linguistics conference (ICLC, July) at Jagiellonian University, Krakow (Poland). Grant, Barry Keith (ed.) 1996 The Dread of Difference: Gender and the Horror Film. Austin, TX: University of Texas Press.

Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor

449

Humphries, Reynald 2002 The American Horror Film: An Introduction. Edinburgh: Edinburgh University Press. Lakoff, George, and Mark Johnson 1980 Metaphors We Live By. Chicago: University of Chicago Press. Johnson, Mark 1993 Moral Imagination: Implications of Cognitive Science for Ethics. Chicago: University of Chicago Press. 2007 The Meaning of the Body: Aesthetics of Human Understanding. Chicago/ London: University of Chicago Press. Kvecses, Zoltn 2005 Metaphor in Culture: Universality and Variation. Cambridge: Cambridge University Press. Mittelberg, Irene, and Linda Waugh this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. Mller, Cornelia, and Alan Cienki this vol. Words, gestures and beyond: Forms of multimodal metaphor in the use of spoken language. Musolff, Andreas 2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21: 2338. Neale, Steve 2000 Genre and Hollywood. London/New York: Routledge. Neale, Steve (ed.) 2002 Genre and Contemporary Hollywood. London: British Film Institute. Rohdin, Mats 2003 Summary in English. In: Vildsvinet I Filmens Trdgrd: Metaforbegreppet inom Filmteorin [The wild boar in the garden of film: The concept of metaphor in film theory], 318329. Ph.D. diss., Stockholm: Edita Norstedts Tryckeri AB. this vol. Multimodal metaphor in classical film theory from the 1920s to the 1950s. Singer, Peter 2001 Animal Liberation. New York: Harper Perennial. Whittock, Trevor 1990 Metaphor and Film. Cambridge: Cambridge University Press.

Subject index

ad hoc concepts, 8, 12, 147153, 159161, 163166, 168 ad hoc pointer, 149, 155158, 162, 165, 168 advertising, 5, 67, 9, 21, 24, 28, 33, 34, 49, 57, 60, 73, 74, 75, 81, 8392, 9598, 100112, 119, 120, 126, 132, 148, 149, 155, 165, 175, 176, 214, 215, 216, 218, 228, 229, 231, 348, 383, 384, 387, 388, 390, 396 wine ~, 7, 28, 7376, 8184, 86 89, 9092 anchoring, 47, 53, 62, 89, 102, 110, 157, 166, 181, 202, 204, 235, 332, 349, 368 anger, 9, 23, 27, 29, 99, 101, 243 262, 265267, 271285, 290, 291 anthomorphism, 78, 82, 85, 86, 91, 112, 388, 447, See also personification Asterix [Alberto Uderzo and Ren Goscinny], 9, 243, 244255, 257259, 261 audience (readers), 3, 8, 26, 31, 32, 49, 60, 62, 63, 74, 8185, 9093, 96, 97, 106108, 110 112, 123, 124, 131, 148, 149, 152169, 173176, 178182, 184, 187189, 191, 192, 199 201, 204, 207, 213, 214, 217, 222, 223, 232, 233, 251, 273, 274, 276, 284, 290, 320, 333, 347, 348, 375, 384, 387, 393, 394, 397, 421, 430, 434, 442

background of ~, 7, 90, 93, 173, 174, 184, 187, 190, 191, 270 interpretive strategies by ~, 8, 147149, 153, 163, 180, 213, 421 Battleship Potemkin [Sergei Eisenstein], 410 Berlin, Symphony of a Great City [Walter Ruttman], 415 blending theory, 10, 20, 46, 122, 149, 169, 207, 235, 236, 359, 370, 375, 376 ~ involving music and language, 10, 359, 370, 375, 376, 446 mental spaces in ~, 20, 46, 119, 122, 137, 169, 235, 236, 370 373, 375, 376, ~ compared with conceptual domains, 122, 149, 236, 375, 376, 378 brand, corporate, 6, 28, 33, 4547, 4965, 81, 83, 89, 90, 91, 97, 102, 104, 106, 107, 110, 384, 388, 389, See also logo and layout The Cabinet of Dr. Caligari [Robert Wiene], 408 Calvaire [Fabrice Du Welz], 437439, 444446 cartoons, 3, 69, 12, 23, 34, 148 149, 155, 157, 173193, 197 199, 201, 202, 205207, 213, 214236, 244, 314, 348, 390, 410, 423

452

Subject index Chinese ~, 7, 119, 120, 123, 126, 131136, 140, 291, Chinese vs. Western ~, 14, 119, 131, 135139, 248 Dutch ~, 8, 217, 218, 222, 226, 227, 232, 384, 385 English ~, 21, 27, 7678, 90, 267, 284, 291 French ~, 7, 75, 77, 84, 85, 92, 271, 283, 388 Japanese ~, 9, 51, 248, 259, 265 275, 277281, 283, 284, 289, 290, 291, See also manga Spanish ~, 7, 7578, 82, 86, 89, 90, 108 ~ specific, 9, 12, 92, 121, 181, 260, 265, 271, 273, 284, 289, 290 critical discourse analysis, 49 decoding, 147, 149, 153, 169 deictic categories, 334 design strategy, 197, 199, 206, 207, 209 diagram, 320, 345, 346 ~ tree, 342, 343, 346, 347, 348 domain(s), 7, 9, 1214, 20, 24, 25, 2730, 3335, 4548, 5362, 76, 77, 7982, 87, 88, 9193, 95, 97100, 102, 107, 110 112, 121, 122, 127129, 131 134, 152, 153, 157, 169, 170, 181, 189, 207, 208, 213215, 218221, 223233, 235, 236, 248, 249, 258, 265267, 273, 297, 300, 307, 308, 310, 313, 316, 319321, 330, 331, 336, 337, 339, 340, 341, 344349, 359362, 364366, 368370, 375, 376, 383389, 393397, 404, 406410, 412, 413, 415 419, 422, 423, 430, 431, 437,

political ~, 69, 34, 148149, 157, 159165, 173193, 207, 423 ~ as genre, 79, 34, 207 editorial ~, 8, 213236 cartoonist, 2, 148, 167, 173, 174, 176179, 182, 184, 189, 186, 191, 192, 199, 215, 216, 230, 244, 245 intentions of ~, 148, 167, 174, 176, 179, 182, 184, 186, 191 cognitive grammar, 208 Cognitive/Conceptual Metaphor Theory (CMT), 4, 8, 9, 11, 12, 14, 19, 20, 21, 27, 28, 30, 46, 98, 121123, 137, 149, 173, 174, 245, 265267, 272, 279, 313, 316, 318, 319, 321, 331, 422, 429, 446 comics (see also animated film, manga, and runes), 6, 9, 13, 23, 29, 59, 215, 245, 247 255, 257261, 265269, 271, 273, 283 ~ and anger, 9, 29, 245, 247 255, 257260, 275, 277, 283 ~ and other emotions, 9, 261, 265269, 271, 283287 conceptual upload, 147, 148, 152, 158162, 165, 166 constructional schema, 208 contiguity, 13, 29, 97, 135, 226, 229, 334, 335, 340, 342, 347 corpus analysis 7, 27, 51, 52, 53, 75, 218, 223, 260, 333, 336, 397, 423 culture, 9, 14, 22, 26, 28, 54, 63, 73, 75, 87, 88, 90, 92, 93, 121, 122, 126, 131, 134137, 139, 153, 157, 161, 179, 181, 182, 200, 248, 257, 260, 261, 268, 291, 298, 304, 312, 313, 332, 363, 376, 389, 446

Subject index 439, 442, See also target and source conceptual ~, 7, 9, 121, 127, 131, 134, 225, 235, 236, 265, 266 schematic source ~, 27, 224226, 230232 categorical source ~, 213, 215, 221, 228232, 236 embodied knowledge, 9, 27, 28 embodiment (embodied meaning), 4, 6, 7, 12, 20, 21, 27, 28, 64, 92, 121, 135, 149, 179, 181, 185, 248, 258260, 272, 315, 321, 330, 333, 336, 340, 342, 364, 377, 423, 445 The End of St. Petersburg [Vsevolod Pudovkin], 416, 417, 422 features, 12, 13, 14, 20, 24, 29, 30, 33, 46, 89, 98, 100, 102105, 107, 110, 111, 113, 157161, 163, 166, 213, 225, 384, 385, 389, 391, 392, 396, 404, 407 409, 413, 415, 416, 422, 423, 441, 446 emergent ~ (properties), 8, 152, 160, 161, 166, 169 encyclopedic ~, 155, 159, 163, 164 mappable ~, 14, 225, 385, 389, 392, 396, 407, 409, 413, 415, 422, 446 film, 3, 5, 6, 1012, 14, 21, 24, 28, 3234, 140, 313, 314, 383, 384, 390392, 396, 397, 403 424, 429, 430, 436, 437, 446 animated ~ (also ~ animation), 9, 25, 233, 243, 244, 248, 250261, 272, 386, 394, 395 classical ~ theory, 403405, 421, 422, 424

453

horror ~, 11, 397, 430446 superimposition in ~, 254, 255, 405, 406408, 412, 415, 421, 423 ~ montage, 100, 102, 112, 395, 405, 408, 410, 411, 421 silent ~ (cinema), 11, 403, 405, 408, 410, 420 Fury [Fritz Lang], 413, 421, 422 genre, 511, 19, 3234, 45, 46, 51, 52, 55, 6164, 7376, 80, 85, 87, 91, 95, 96, 112, 175, 183, 184, 189, 191, 192, 198, 207, 218, 219, 232, 234, 333, 383, 384, 389, 390, 397, 408, 422, 423, 429, 430, 433435, 442, 446 gesture(s), 35, 9, 10, 13, 19, 22, 23, 25, 34, 47, 80, 97, 98, 214, 251, 261, 297322, 323, 329349, 364, 365, 376 co-speech ~, 10, 321, 330, 336 deictic ~, 332, 343 functions of~, 309, 318, 332, 333 manual ~, 299, 313, 322, 330, 336, 339, 340, 341, 342, 344, 346, 347, 348 metaphoric ~, 297, 300302, 304306, 309, 310, 316, 329, 336, 337 metonymic ~, 346, 347 spontaneous ~, 298, 307, 332, 336, 342, 348 referential ~, 298, 301, 306, 307, 333, 336, 337 synecdochic ~, 305, 337, 343, 344, 348 ~ and words, 316319 grammar, 22, 321, 330, 336 generative ~, 342, 344, 348 musical ~, 359

454

Subject index manga, 9, 13, 259, 261, 265, 266, 271274, 276281, 284, 285, 289, 290 manifestation, 8, 9, 29, 126, 321, 334, 430, 431, 445 nonverbal and multimodal ~ of metaphor, 5, 7, 13, 34, 92, 119, 120, 123, 125, 129, 130, 136, 137, 139, 140, 175, 235, 243, 244, 246, 250, 252, 254, 255, 267, 272, 333, 348, 422, 429, 431, 439 verbal ~ of metaphor, 4, 12, 13, 21, 22, 34, 123, 248, 283, 431 mapping(s), 10, 13, 20, 24, 2830, 46, 61, 79, 81, 95, 96, 100, 120, 122, 127, 128, 133, 135, 138, 139, 169, 181, 213, 214, 217219, 226, 228, 265, 266, 270, 271, 273, 283, 284, 289, 290, 336, 340, 341, 348, 359, 361, 364367, 369, 370, 373, 375, 376, 384, 391, 393, 407, 408, 414, 417, 422, 434, 437, 441, 445, 446 double ~, 129, 346, 347 metaphoric(al) ~, 10, 13, 20, 24, 31, 97, 102, 104, 107, 110, 111, 112, 132, 149, 155158, 173, 190, 207, 208, 272, 313, 317, 339, 341, 375 metonymic(al) ~, 7, 98, 99, 110, 112, 134, 329, 341, 347 partial ~, 12, 55 medium, 9, 21, 32, 157, 163, 166, 214, 218, 243, 244, 250, 253, 255, 258261, 266, 283, 403, 410, 412, 418, 420, 424, 430, 444 metaphor articulatory forms of ~, 299300, 307

Great Chain of Being, 13, 27, 28, 106, 112 happiness, 23, 265, 266, 283, 285 287, 289 indirect reference, 329, 331, 339 image alignment, 8, 61, 197209 image schema, 8, 30, 139, 167, 179, 187, 207, 208, 224, 317, 320, 330, 331, 340, 342, 349 SOURCE-PATH-GOAL ~, 8, 29, 127, 140, 179, 187 interpretation, 6, 810, 19, 20, 26 28, 32, 33, 4749, 53, 55, 58, 59, 62, 64, 87, 9093, 110 112, 147150, 152158, 160 162, 165, 167169, 173175, 180184, 187, 190192, 201 207, 213216, 219221, 224, 228232, 234, 248, 250, 317, 318, 331, 336, 337, 339, 341, 343, 346, 348, 372, 384, 385, 389, 395, 397, 403, 404, 408, 411, 418421, 423, 444 inference(s), 29, 47, 97, 128, 149, 151, 169, 230, 387, 445 integration, conceptual, 122, 236, 373, 374, See also blending intertextuality, 11, 47, 56, 415 invariance principle, 207, 208, 373 layout, 6, 45, 46, 52, 53, 6061, 64, 201 logo (see also brand), 6, 30, 4547, 5153, 5765, 97, 98, 103, 105, 113, 385 (audiologo), 388, 396 love, 98, 249, 254, 258, 265, 266, 285, 287, 289, 300, 302, 307, 367, 372, 373, 396, 444

Subject index artistic (also metaphor in artis tic texts / discourses, metaphor, creative) ~, 396 complex ~, 119, 120122, 127 131, 133135, 137, 180, 181, 191, 235, 256260 conceptual ~, 4, 710, 1214, 19, 21, 25, 26, 30, 47, 48, 65, 80, 96, 112, 119123, 125130, 134, 136, 137, 139, 149, 161, 173, 213216, 218, 219, 234, 235, 246, 248250, 252, 256 261, 265267, 271, 273, 280, 283, 284, 290, 300, 301, 307, 313, 316, 318, 319, 321, 331, 336, 345, 361, 391, 403, 419, 422, 429, 430, 444, 445, See also Cognitive / Conceptual Metaphor Theory construal of ~, 6, 10, 12, 14, 19, 20, 31, 33, 64, 336, 384, 385, 391, 397 conventional (conventionalized) ~, 26, 27, 29, 34, 53, 108, 113, 166, 167, 170, 179, 308, 342, 406, 407, 410, 422 creative ~, 8, 9, 12, 19, 25, 28, 34, 175, 244, 422, 429, 430 diagrammatic ~, 331, 342 diegetic/nondiegetic ~, 393, 396, 410, 411, 413, 415 directionality in ~, 10, 309, 369, 370, 375, See also invariance principle dual encoding in ~, 45, 47, 49, 54, 62 dynamic ~, 11, 95, 297, 300, 313, 316, 319, 320, 321 elaboration of ~, 104, 122, 170, 182, 268, 298, 313316, 321 emotion ~, 243, 244, 265268, 270272, 283, 284, 290

455

gestural ~, 65, 298, 300, 303, 304, 306308, 310, 311, 316, 347, See also metaphor in gesture ~ in advertising, 6, 7, 9, 28, 33, 34, 48, 74, 81, 82, 86, 87, 9092, 96, 111, 112, 175, 176, 214, 215, 218, 383, 390 ~ in animation, 6, 9, 13, 25, 233, 243, 258, 259, 260, 394, 395 ~ in artistic texts/discourses, 14, 25, 33, 396 ~ in film, 6, 1012, 14, 21, 24, 28, 33, 34, 140, 383, 384, 390392, 396, 397, 403406, 410413, 415, 418420, 429 431, 436, 438, 441, 442, 444 446 ~ in gesture(s), 9, 10, 299, 302 309, 312, 316, 318, 321, 323, 329, 336, 337, 346, See also gestural metaphor ~ in intonation, 9, 13, 297, 299, 302, 316318 ~ in speech, 10, 297, 303, 304, 317, 321, 330332, 336, 340, 341 interaction theory of ~, 12, 24 interaction of ~ with metonymy, 7, 13, 95100, 104, 109112, 334, 344 monomodal ~, 9, 19, 2225, 28, 31, 33, 74, 79, 81, 177, 188, 190, 297, 300, 302, 303, 304, 306, 313, 321, 334, 339, 385, 403, 410414, 420, 429431, 437, 442, 445 multimodal ~, 314, 19, 2122, 24, 25, 2831, 3334, 4549, 51, 53, 5557, 59, 60, 6264, 7375, 7983, 88, 91, 92, 96 99, 105, 111, 112, 119, 120, 122, 123, 125, 130, 137, 139,

456

Subject index 178, 180, 181, 187, 188, 189, 191, 199, 200, 202, 205, 207, 208, 213215, 217222, 224 234, 235, 236, 244, 248, 249, 258, 266, 267, 271, 297, 300, 302, 304, 305, 307310, 312, 313, 315, 316, 319321, 329, 331, 336, 337, 339341, 344, 346349, 359, 365, 369, 370, 375, 383389, 392, 393397, 404, 406423, 430, 431, 439, 442, 444, 446 spatial ~, 128, 311, 312, 344 346, 348 systems of ~, 62, 121, 128, 129, 131, 132, 176, 266, transparent ~, 308 verbal ~, 4, 68, 1014, 19, 21 24, 26, 29, 31, 34, 46, 47, 49, 73, 74, 81, 82, 84, 87, 88, 90, 123, 137, 140, 147, 148, 150 153, 156, 158161, 166168, 218, 243, 245, 248250, 256, 257, 260, 265267, 271, 273, 278280, 282284, 288290, 297, 300304, 308310, 313 317, 383, 396, 397, 405, 408, 409, 422, 430, 431, 435, 445 verbalization of conceptual ~, 13, 19, 30, 31, 47, 48, 74, 192, 207, 306, 404 verbo-gestural ~, 297, 300, 308, 310, 312, 315, 321 verbo-pictorial ~, 11, 86, 91, 199, 310, 383, 420 verbo-visual ~, 173 visual ~, 6, 8, 23, 24, 31, 46, 49, 54, 107, 131, 135, 147150, 152, 153, 155157, 159, 161 163, 165168, 174, 175, 178, 192, 213215, 218, 223, 224, 228, 230, 234236, 244, 245,

140, 147, 148, 150, 152, 158, 161, 173175, 180, 181, 183 185, 190192, 197200, 202, 203, 205, 206, 208, 215, 233 235, 244, 251, 252, 260, 266, 271, 272, 283, 290, 297300, 302, 304, 307, 308, 310, 312 314, 316, 319, 321, 329331, 334, 336, 339, 341, 342, 347, 348, 359, 360365, 369, 370, 376, 383, 384, 389393, 395 397, 403405, 407, 412, 413, 416, 419424, 429431, 442, 445 naturalization of ~, 63, 396 ontological ~, 106, 163, 166, 175, 181, 182, 313 orientational ~, 175, 176, 180 182, 185, 187, 190 pictorial ~, 3, 6, 8, 9, 19, 2225, 28, 33, 46, 49, 55, 81, 82, 92, 148, 177, 187, 197, 199, 202, 243246, 250, 259, 265267, 271273, 276, 283, 285, 288 290, 330, 385, 389, 390, 395, 403, 416, 430, 431, 445 poly-interpretability of ~, 390 primary ~, 119, 121, 122, 128 130, 133, 137, 140, 181, 197, 208, 209, 256, 257, 260 products and processes in ~, 297, 300, 319, 321 projective ~, 393 prosody in ~, 316, 321 sleeping ~ (versus waking ~), 316 source and target in ~, 4, 7, 9 14, 1921, 2325, 2731, 33 35, 4548, 5357, 5962, 74, 79, 8183, 86, 88, 92, 97 102, 104, 107, 108, 110, 111, 120, 122, 127, 128, 132, 147, 148, 155166, 169, 174, 177,

Subject index 248, 250, 271, 290, 330, 390, 413 metaphoricity, 297, 300, 303, 308, 316, 319, 323, activation of ~, 307, 308, 321 ~ and attention, 308 ~ as cognitive process, 300, 316, 319 creation of ~, 297, 300 foregrounding of ~, 316 modality independent ~, 300, 319, 321 metonymy, 7, 10, 12, 13, 2325, 34, 61, 90, 95112, 119120, 128, 130, 134137, 139, 158, 282, 306, 329343, 346348, 349 ~ and metaphor, 7, 10, 12, 13, 24, 25, 95100, 104, 107, 109111, 120, 139, 306, 329 332, 334337, 339341, 347 348 external ~, 329, 334, 338, 340, 346, 347 functions of ~, 98, 111, 139, 347 highlighting in ~, 96, 98, 100, 102, 103, 106, 110, 111 internal ~, 329, 334, 335, 343, 347, See also synecdoche ~ mapping, 7, 96, 98, 100, 110, 112, 120, 129, 135, 329, 340, 341, 346, 347 multimodal ~, 119, 120, 134 137, 139, 334, 348 visual ~ (pictorial ~), 90, 105, 134137 Metropolis [Fritz Lang], 407, 415, 421, 422 The Million [Ren Clair], 411, 412, 413, 421 mode/modality, 2, 411, 2125, 29, 31, 33, 34, 4547, 5153, 62, 7375, 80, 81, 91, 95, 96,

457

100, 102, 103, 106, 108, 110 112, 119, 120, 137, 139, 147, 148, 168, 169, 174, 175, 177, 180, 181, 188, 190192, 199, 207, 213, 233, 234, 251253, 257261, 265, 266, 283, 290, 297, 299, 300, 302, 307, 310, 312, 313, 316, 318, 319, 321, 322, 329, 330, 331, 333337, 339, 341, 342, 344346, 348, 349, 361, 363, 375, 376, 383, 385, 387, 395, 404, 405, 413, 420422, 430, 431, 444 sub ~, 95, 100, 103, 104, 106, 110, 112 types of ~, 5, 6, 22, 23, 4648, 53, 80, 97, 300, 404, 405 models mental ~, 46, 48, 60, 112 abstract ~, 50 Idealized Conceptual Models (ICM, folk models), 225, 243, 244, 246, 248250, 256, 258, 260, 261 Modern Times [Charlie Chaplin], 413415, 421, 422 module, 100, 148, 149, 153, 167 language ~, 149, 153, 169 perceptual ~, 154, 162 specialized ~, 148, 167 motivation, 13, 95, 102, 134, 151, 170, 265, 266, 272, 278, 279, 288, 289, 396, 397, 419 experiential ~, 127, 130, 208, 283, 289 physiological ~, 271, 289 music, 4, 5, 6, 10, 13, 19, 2123, 35, 47, 51, 53, 57, 123, 126, 131, 133, 136, 139, 261, 300, 331, 359378, 383385, 389 392, 395398, 403405, 411 413, 415, 421, 430, 431 text painting in ~, 359, 364370

458

Subject index representation, 23, 24, 29, 54, 100, 102, 149, 178, 199, 204, 207, 248, 256, 259, 301, 346, 360, 376, 377, 397, 405, 412, 430 artistic ~, 33 cinematic / film ~, 404, 411, 415 dynamic ~, 341, 347, 373 gestural ~, 330, 336 medium specific ~, 260 metaphorical ~, 9, 111, 243, 245, 260, 261, 339 multimodal ~, 6, 21, 29, 34, 88, 244, 283, 329, 330, 331, 341, 342, 347, 348 surface ~, 97 visual ~ (pictorial ~), 31, 54, 56, 105, 106, 109, 154, 198, 207, 245, 250, 251, 256, 258, 259, 261, 266, 285 resemblance, perceptual, 31 rhythm, 100, 310, 332, 367, 372, 373, 378, 386, 393, 395 runes, pictorial, 23, 59, 245, 246, 247, 250, 254, 255, 272, 275, 276, 277, 280, 281, 283, 289 scenario, 11, 48, 154, 176, 187, 202, 215, 223228, 234, 235, 254, 276, 280, 283, 297, 305, 429, 430, 445, 447 Semiotic(s), 5, 10, 329, 334 Jakobsonian ~, 331, 332, 334, 335, 348 Peircean ~, 283, 331, 348 ~ process, 304, 306 sensory ~ domain, 80 ~ experience, 74, 79, 80, 89, 91 ~ motor abilities, 342 ~ perception, 73, 80 ~ schema, 167 ~ information, 167

diegetic ~, 393, 396, 397 nondiegetic ~, 393, 396, 413 multimodal metaphor definition of ~, 4, 5, 46, 74, 82, 97, 120, 148, 174, 191, 198 199, 233, 235, 266, 297, 299, 336, 339, 383, 404, 429, 430, See also metaphor, multimodal explicitness of ~, 13, 55, 56, 100, 397 types of ~, 6, 9, 25, 28, 33, 47, 53, 91, 404, 405, 430 ~ versus monomodal metaphor, 9, 2225, 300, 302, 313, 413, 431 narrative, 11, 33, 111, 178, 185, 281, 290, 394, 397, 405, 415, 430, 434, 446 ~ meaning, 178 ~ and action/events, 11, 178 ~ and time/chronology, 185, 190, 36 ~ image, 187 ~ styles, 198 ~ context, 290, 446 film ~, 409 October [Sergei Eisenstein], 411, 422 Old and New [Sergei Eisenstein], 411, 416, 417, 422 personification (see also anthropomorphism), 13, 28, 51, 62, 80, 83, 84, 100, 106, 112, 113, 120, 224, 386 pragmatic adjustment, 151 Relevance Theory, 8, 24, 147, 148 166, 168, 199

Subject index simile, 24, 25, 55, 74, 88, 186, 197, 198, 214, 411 sign iconic ~, 162, 256, 337 indexical ~, 59, 245, 246, 251, 255, 256, 260, 265, 272, 274, 276, 281, 283, 291 sonic analogs, 10, 363367, 369, 370, 376, 377 song, 123, 136, 359, 367, 369375, 384, 385, 390, 396, 398 sound, 36, 10, 19, 21, 22, 23, 25, 31, 47, 53, 57, 80, 96, 97, 103106, 214, 233, 234, 250, 251, 255, 261, 298, 299, 321, 359368, 377, 383397, 404, 405, 412, 413, 419421, 430 diegetic versus nondiegetic ~, 393, 396, 397, 405, 412, 413 Strike [Sergei Eisenstein], 406, 407, 409, 411, 422 Surrealist art, 24 The Swindle [Federico Fellini], 418, 419, 421, 422 synecdoche, 24, 331, 335, 336, 342, 347, See also internal metonymy syntactic structure, visual, 333, 342 The Texas Chain Saw Massacre [Tobe Hooper], 432, 441, 442, 444446 thinking for speaking, 297, 320, 321 touch, 5, 22, 23, 47, 75, 80, 86, 261, 300 trope, 8, 12, 23, 34, 334, 405 television commercials, 7, 9597, 110, 111, 119 upload, conceptual, 148, 152, 158 160, 162, 165, 166 innovative ~, 148, 152, 160, 165 stable ~, 152, 159, 162

459

universality vs. individuality, 174, 181, 190, 289 verbal image, 137, 405, 408410, 416, 417 visual-conceptual interface, 147, 156158, 162, 163, 165 winespeak, 6, 7374, 7980, 85, 86, 92, 93 wine tasting note, 7, 74, 75, 91 Wolf Creek [Greg McLean], 434 436, 441, 444446

Author index
Agawu, Victor Kofi, 364 Ajzen, Icek, 232 Alexandrov, Grigori, 411 Allanbrook, Wye Jamison, 364 Altman, Rick, 32, 446 Amoraritei, Loredana, 74, 93, Arnheim, Rudolf, 404, 405, 412, 413 Askegaard, Sren, 50, 63 Balzs, Bla, 405, 406, 410, 411, 416, 420 Baldry, Anthony, 5, 176 Balmer, John T.M., 50, 52 Barcelona, Antonio, 12, 46, 95, 98, 120, 139, 330, 340, 341, 349 Bartsch, Renate, 26, 29, 30, 394 Baruch, Yehuda, 60 Batra, Rajeev, 96, 214, Bazin, Andr, 405, 406, 418, 419, 420 Becker, Jonathan A., 216 Bell, Steve, 176 Bengtsson, Anders, 50, 57, 60, 63, Black, Max 12, 13, 24, 28, 244, 405, 429 Blair, Preston, 248, 249, 258, Bohle, Ulrike, 323 Boiles, Charles L., 363 Bolter, Jay David, 21 Bordwell, David, 256, 391, 397, 405, 406, 409, 415, 420, Bouvet, Danielle, 299, 331, 336, 348 Bressem, Jana, 304, Brooke-Rose, Christine, 31 Bruce, Nigel, 74 Brugman, Claudia, 207 Brunn, Stanley D., 216 Buckland, Warren, 404 Caballero, Rosario, 6, 12, 14, 79, 93, 112, 397 Calbris, Genevive, 299, 320, 331, 332, 336 Cameron, Lynne, 112, 321 Cappetta, Rossella, 51 Carroll, Noel, 24, 33, 46, 177, 244, 266, 330, 390, 405, 406, 408, 423, 429 Carston, Robyn, 150, 152 Cavanagh, Patrick, 169 Chafe, Wallace, 302, 322 Charteris-Black, Jonathan, 32, 397 Cheney, George, 58, 64 Christensen, Lars Thger, 50, 58, 64 Chun, Rosa, 65 Ciccolella, Francesco, 50, 51, 64, Cienki, Alan, 9, 13, 21, 25, 34, 214, 297, 299, 302, 304, 311, 317, 318, 319, 320, 321, 323, 330, 331, 333, 336, 348, 364, 430 Clair, Ren, 410, 411, 412 Clark, Herbert, 298 Cohen, Ted, 405, 423 Collins, James C., 51 Colston, Herbert, 21 Cook, Guy, 96 Cook, Nicholas, 370 Coulson, Seana, 20, 122, 149 Crisp, Peter, 27 Croft, William, 98, 330, 340, 363 Csaba, Fabian, 50, 57, 60, 63 Culpeper, Jonathan, 12 Cumming, Naomi, 377 Cunliffe, Ann L., 51 Cupchik, Gerry D., 34 Czerniawska, Fiona, 52 Damasio, Antonio R., 363, Danaher, David, 331

462 Author index

Danto, Arthur C., 403, 420, 423 Dastjerdi, H. Vahid, 181 Davies, Gray, 65 Day, Sean, 80 de Chernatony, Leslie, 46 Deacon, Terrence W., 377 Deutscher, Guy, 214 Dez Velasco, Olga Isabel, 7, 95, 97, 99, 110, 158 Dines-Levy, Gail, 176 Dirven, Ren 12, 24, 95, 111, 112, 120, 139, 330, 334 Duncan, Susan, 316, 320 Durand, Jacques, 23 Edwards, Janis L., 173, 174, 176, 179, 216 Eerden, Bart, 9, 13, 29, 244, 249, 254, 257, 260, 272 Efron, David, 298, 320 Eggertsson, Gunnar, 11, 14, 395, 444 Eisenstein, Sergei, 405411, 416 418 Ejxenbaum, Boris, 405, 408, 410 El Refaie, Elizabeth, 8, 13, 34, 110, 149, 150, 155, 176, 177, 192, 193, 207, 216, 219, 233, 348, 390, 405, 410 Elliot, Richard, 51 Entzenberg, Claes, 423 Epstein, Jean, 424 Eves, Anmarie, 214 Fauconnier, Gilles, 7, 8, 10, 20, 46, 122, 169, 235, 330, 370, 378, 446 Fein, Ofer, 244 Feldman, Ofer, 216 Fischer, Roger A., 182 Fishbein, Martin, 232 Fodor, Jerry, 8, 149, 153, 167 Forceville, Charles, 411, 13, 14, 20, 24, 25, 28, 29, 3235, 46, 47, 49, 51, 55, 60, 61, 74, 81, 82, 86, 88, 91, 92, 9598, 112, 120, 122, 123, 137, 140, 147 149, 155, 174, 175, 177, 180, 181, 191, 197199, 202, 207, 214, 216, 217, 220, 233235, 243, 244249, 251254, 257, 259, 260, 261, 262, 265, 266, 271, 272, 274277, 279, 280, 283, 288, 291, 299, 300, 310, 313, 314, 330, 336, 339, 348, 349, 378, 383, 384, 387, 389, 390, 394, 395, 397, 404, 413, 422, 423, 429, 430, 431, 444 Forns, Johan, 363 Fox, John, 52 Fox, Renata, 52 Franke, George R., 214 Frick, Wolfgang, 65 Fricke, Ellen, 331, 333, 343 Friedman, Vanessa, 56 Furuyama, Nobuhiro, 349 Geeraerts, Dirk, 291, 341 Gentner, Dedre, 13, 33, 214, 225, 226, 236 Gevaert, Caroline, 27, 291 Gibbs, Raymond, 14, 20, 21, 28, 34, 121, 139, 149, 151, 170, 214, 235, 248, 261, 321, 330, 331, 336, 348, 395, 421, 445 Gilmartin, Patricia, 216 Gilot, Franoise, 313 Gineste, Marie-Dominique, 8, 24, 152 Gioia, Dennis, 51 Gluck, Malcolm, 74 Glucksberg, Sam, 214, 228 Goatly, Andrew, 31 Goldin-Meadow, Susan, 332, 364 Gombrich, Ernst H., 193 Goodwin, Charles, 332 Goossens, Louis, 98, 330, 341 Goscinny, Ren, 245, 247, 250, 261 Grady, Joseph, 27, 65, 121, 122, 208, 256, 265 Green, Christopher D., 59, 194 Groarke, Leo, 216, 220

Author index Grondelaers, Stefan, 291 Grusin, Richard, 21 Haferland, Harald, 304 Hake, Sabine, 403, 408 Harquail, Celia V., 51 Haser, Verena, 6, 14 Hatch, Mary Jo, 50, 51, 64, 65 Hayward, Malcolm, 32 Heckler, Susan E., 57, 58, 65 Heinke, Dietmar, 169 Herold, Debora, 317 Hiraga, Masako, 331 Hirschberg, Julia, 316 Hoeken, Hans, 214 Holm, Lisbeth S., 53 Hols, Edith, 23 Holt, Douglas B., 46 Holyoak, Keith J., 369 Horn, Robert E., 178 Hrschelmann, Kathrin, 183, 184, 192 Hubbard, Edward, 80 Huhmann, Bruce A., 214 Humphreys, Glyn, 169 Humphries, Reynald, 432 Indurkhya, Bipin, 24, 152, 393, 421, 422 Jakobson, Roman, 97, 111, 329 332, 334, 335, 341, 348, 349, 415 Jeulink, Marloes, 29, 444 Johansson, Ulla, 53 Johnson, Christopher, 208 Johnson, Mark, 3, 4, 1921, 35, 46, 64, 7980, 98, 106, 121, 132, 139, 163, 166, 175176, 179, 181, 198, 207208, 213214, 225, 232, 243, 244, 250, 260, 265266, 267, 313, 317, 319, 330, 331, 336338, 340, 344, 345, 349, 422423, 429, 430, 445447 Johnson, Michael, 421 Johnston, Ollie, 248, 249, 258

463

Kapferer, Jean-Nol, 49, 50 Kaplan, Stuart J., 24, 175 Kasher, Asha, 244 Keller, Hans, 370 Kendon, Adam, 298, 299, 302, 304, 309, 318, 322, 332, 364 Kennedy, John M., 23, 24, 34, 59, 175, 244, 245, 246, 250, 272, 276, 330 Khordoc, Catherine, 244 Killgren, Lucy, 57 Kittay, Eva, Feder, 423 Klein, Naomi, 63 Knop, Sabine de, 23 Koller, Veronika, 6, 14, 49, 52, 60, 105 Kller, Wilhelm, 337 Kotze, Zacharias, 291 Kvecses, Zoltn, 9, 11, 20, 28, 29, 99, 104, 121, 122, 139, 174, 175, 181, 243, 244, 247, 248, 249, 250, 254, 256, 257, 258, 259, 260, 261, 265, 267, 268, 269, 280, 291, 330, 422, 445, 446 Kress, Gunther, 5, 34, 61, 65, 96, 177, 178, 348, 349 Kriegel, Uriah, 154 Kristiansen, Gitte, 12, 50 Kunda, Ziva, 50 Ladewig, Silva, 304, Lake, Carlton, 313 Lakoff, George, 34, 13, 1921, 25, 26, 27, 28, 35, 46, 61, 64, 79 80, 98, 106, 121, 128129, 132, 139, 163, 166167,175176, 179, 181, 198, 207208, 213 214, 225, 232, 235, 243244, 250, 260, 265267, 276, 280, 313, 317, 319, 330331, 336 338, 340, 344345, 361, 373, 377, 407, 410, 419, 422423, 429, 430, 445447

464 Author index

Langacker, Ronald W., 208, 340 Larson, Steve, 35 Lassen, Inger, 49 Laudenbach, Peter, 62 Lehrer, Adrienne, 74, 77 Lidov, David, 365 Lindsay, Greg, 59 Lindsay, Vachel, 420, 424 Lodge, David, 331, 341 Loewenstein, Jerry, 13, 33 Low, Graham, 321 Maalej, Zouhair, 29 Maes, Alfons, 8, 14, 149, 231, 348, 390 Maestri, George, 248, 249, 258 Mandler, Jean, 340 Matsuki, Keiko, 268 Matsunaka, Yoshihiro, 9, 13, 29, 248, 259, 261, 268270, 273, 284, 291 Mautner, Gerlinde, 65 McCarthy, Michael S., 214 McCloud, Scott, 245, 291 McLuhan, Marshall, 21 McMahon, Jennifer A., 154 McNeill, David, 9, 25, 34, 299, 302, 316, 320, 322, 330332, 336, 337, 364 McQuarrie, Edward, 34, 49, 96, 97, 113, 156, 165, 214, 220 Messaris, Paul, 64, 100, 175, 177 Mick, David, Glen, 34, 214 Miller, Christine M., 34 Mnguez, Norberto, 154 Mittelberg, Irene, 10, 13, 95, 98, 299, 304, 313, 319, 330, 331, 333, 336, 346, 348, 349, 430 Moingeon, Bertrand, 50 Morford, Jill, 331 Morris, Desmond, 304 Morrison, Susan, 173, 216 Mothersbaugh, David L., 214 Mller, Cornelia, 9, 13, 25, 34, 214, 299, 301, 302, 304306, 308, 310, 311, 313, 314, 316, 318 323, 330333, 336, 339, 348, 364, 430 Mumford, Alan, 182 Muniz, Albert M., 63 Musolff, Andreas, 11, 48, 429, 430, 447 Neale, Steve, 446 Neumann, Ragnhild, 304 Nusbaum, Howard C., 365 OGuinn, Thomas C., 63 OHalloran, Kay, 5 Oakley, Todd, 122, 235 Ochs, Eleanor, 332, 349 Okrent, Arika, 365 Ortony, Andrew, 3, 19, 228 Pankhurst, Anne, 111 Panther, Klaus-Uwe, 95, 98, 120, 139, 330, 339, 342 Papasolomou, Ioanna, 174 Pascual, Esther, 20 Peirce, Charles Sanders, 256, 262, 272, 283, 289, 331, 332, 334, 337, 343, 346, 348, 377 Peamarn, Cristina, 149 Perlman, Marcus, 14 Petrakis, Stefanos, 20 Peynaud, mile, 74, 77 Philippe, Robert, 173 Phillips, Barbara, 34, 49, 96, 97, 113, 156, 165, 175, 214 Pierrehumbert, Janet, 316 Pilkington, Adrian, 150 Pimentel, Ronald W., 57, 58, 65 Plumb, Steve, 215, 216, Polzenhagen, Frank, 112 Prings, Ralf, 12, 24, 95, 120, 330, 334 Porras, Jerry I., 51 Pragglejaz Group, 14, 301 Pratt, Michael G., 53 Proctor, Stella, 174 Proctor, Tony, 174 Pudovkin, Vsevolod, 411, 416, 417

Author index Radden, Gnther, 95, 98, 99, 111, 120, 139, 330, 340, 341 Rafaeli, Anat, 53 Ramachandran, Vilayanur, 80 Ratner, Leonard G., 364 Rayson, Paul, 52 Richards, Ivor Armstrong, 23, 423 Ricoeur, Paul, 11 Ritchie, David, 174, 181, 182, 187, 191, 192 Rogers, Priscilla, 52 Rohdin, Mats, 10, 11, 13, 14, 34, 404, 411, 423, 429 Rompay, Thomas, van, 34 Rothbart, Myron, 63 Rozik, Eli, 24 Ruiz de Mendoza, Francisco, Jos, 7, 95, 97, 99, 110, 149, 151, 158, 163, 170, 208 Salt, Barry, 405 Scart, Vronique, 152 Schilperoord, Joost, 8, 14, 149, 231, 348, 390 Schirato, Tony, 178 Schultz, Majken, 50, 51, 64, 65 Schwartz, Jonathan, 48 Scott, Linda, 96, 175, 214 Seitz, Jay A., 175 Semino, Elena, 12 Shapiro, Michael, 334 Shen, Yeshayahu, 33, 213, 225, 226, 228, 235, 236 Shibles, Warren, 23 Shinohara, Kazuko, 9, 13, 29, 248, 259, 261, 268270, 273, 284, 291 Shintel, Hadas, 365, 377 Shore, Bradd, 28, 319 Shotter, John, 51 Simons, Jan, 29 Singer, Peter, 447 Slobin, Dan I., 320 Smith, Gregory, W. H., 176 Smith, Ken, 23 Soenen, Guillaume, 50

465

Speckmann, Gerald, 304, 305, 306 Sperber, Dan, 8, 24, 97, 147, 148 Spitzer, Michael, 35 Stanford, W(illiam) B(edell), 404 Steen, Gerard J., 28, 32, 139, 248, 261, 321, 421, 445 Stckl, Hartmut, 100, 103 Stockwell, Peter, 27 Streeck, Jrgen, 304, 318, 332 Strunck, Jeanne, 49 Sun, Sewen, 34, 61, 96, 197, 214, 220, 222 Swales, John, 52 Sweetser, Eve E., 20, 129, 139, 299, 330, 331, 336, 337, 338, 345 Talebinejad, M. Reza, 181 Tanaka, Keiko, 96, 148 Taub, Sarah, 322, 331, 336 Taylor, John, 98 Taylor, Marjorie, 63 Templin, Charlotte, 173 Tendahl, Markus, 149, 151, 170 Teng, Norman, 8, 13, 34, 61, 96, 197, 198, 214, 220, 222 Terkourafi, Marina, 20 Teendorf, Sedinha, 304, 305, 306, 323 Thagard, Paul, 369 Thibault, Paul J., 5, 176 Thomas, Frank, 248, 249, 258 Thompson, Kristin, 256, 391, 397, 405, 406, 415, 420 Thorau, Christian, 35 Thornburg, Linda L., 95, 98, 120, 139, 330, 339, 342 Tillyard, E.M.W., 13, 27 Tom, Gail, 214 Tomasello, Michael, 363 Turner, Mark, 7, 8, 10, 13, 20, 25 28, 35, 46, 61, 98, 106, 122, 139, 169, 181, 207, 235, 248, 330, 373, 377, 378, 407, 410, 419, 446 Tversky, Barbara, 23 Tynjanov, Jurij, 406, 408

466 Author index

Uderzo, Albert, 245, 247, 250, 261 Ungerer, Fiedrich, 55 Urios-Aparisi, Eduardo, 7, 13, 14, 112 Van Dijk, Teun A., 49 Van Enschot-van, Dijk, Renske, 214 van Leeuwen, Theo, 5, 34, 47, 48, 61, 65, 96, 178, 348 Van Mulken, Margot, 214 Van Noppen, Jean-Pierre, 23 Vega-Moreno, Rosa, 150, 151, 152, 161, 168 Ventola, Eija, 5, 34 Versante, Laura, 309 Vervaeke, John, 59, 175 Vestergaard, Torben, 49 Villafae, Justo, 154 Wales, Katie, 335 Ward, Janet, 408 Warren, Beatrice, 98 Wattanasuwan, Kritsadarat, 51 Waugh, Linda, 10, 13, 95, 98, 299, 304, 319, 331, 332, 335, 346, 365, 430 Webb, Jen, 178 Webb, Rebecca, 299, 336 Wee, Thomas Tan Tsu, 50 Whittock, Trevor, 24, 28, 34, 244, 334, 348, 422, 429 Wiggin, Amy A., 34 Wilcox, Phyllis P., 299, 330, 331, 339 Wilcox, Sherman, 331 Willenbrock, Harald, 62 Williams, Richard, 248, 249, 258 Wilson, Deirdre, 8, 24, 97, 147, 148, 152 Wittgenstein, Ludwig, 359, 363, 376 Wodak, Ruth, 48 Wolf, HansGeorg, 112 Wolff, Philip, 214 Worth, Sol, 175, 404 Yamanaka, Keiichi, 270 Yu, Ning, 7, 8, 13, 28, 80, 96, 98, 126, 129, 132, 139, 248, 291 Yus, Francisco, 8, 12, 13, 100, 152, 155, 169, 348, 390 Zakia, Richard D., 177 Zbikowski, Lawrence, 10, 13, 35, 214, 365, 370, 376, 377 Zwaan, Rolf, 32

Metaphor and metonymy index

This index contains the verbalizations used by authors to designate conceptual metaphors and *metonyms

A PURPOSEFUL LIFE IS A JOURNEY,

BAD IS DOWN,

317 163, 164

121, 122, 140

ABORTING IS UNKITTING A BABY BOOTIE,

THE BALLOT BOX IS A DICE, DOWN,

BEING SUBJECT TO CONTROL IS

107, 109

176, 345 31

ABSTRACT A IS CONCRETE B, 422 ABSTRACT AS CONCRETE, 313

BICYCLE IS DRESSAGE HORSE, BOOK IS MEDICINE,

*ACTION FOR OBJECT INVOLVED IN ACTION, 339 ACTIONS ARE (SELFPROPELLED) MOTIONS, 122, 128, 130, 131 AFFECTION IS WARMTH, 104, 110 AMERICAN NEWS IS LIKE HORROR NOVEL, 198 ANGER IS A BURDEN, 259 ANGER IS A CAPTIVE ANIMAL, 244, 259, 238, 441 ANGER IS A DANGEROUS ANIMAL, 256, 258
ANGER IS A HOT FLUID IN A

225 THE BOOK IS A BALL, 161

BRANDS ARE LIVING ORGANISMS,

6, 46, 47, 53, 56, 6062, 64 BRANDS ARE PEOPLE, 5153, 57, 61, 65 BUILDINGS ARE RUBBISH, 157, 159
CAR IS A (LONGJUMP) ATHLETE,

100,

110 106, 107, 110 386, 388 CATEGORIES ARE CONTAINERS, 338 COFFEE MACHINE IS BUTLER, 30 COFFEE MACHINE IS SERVANT , 30 COFFEE MAKER IS MOTORCYCLE, 389 COLIN IS A CHILD, 431 COMPANIES ARE LIVING ORGANISMS, 48
CARS ARE FISH, CONCEPTUAL STRUCTURE IS GEOMETRIC PHYSICAL CAR IS PERSON, 102,

(PRESSURIZED) CONTAINER, 247, 248, 256, 258, 259, 265, 267, 271, 273, 277, 278, 280, 283, 290, 291 ANGER IS AN OPPONENT (IN A STRUGGLE), 258, 259 ANGER IS FIRE, 243, 279
ANGER IS GASTRIC CONTENTS IN THE DIGESTIVE SYSTEM, ANGER IS HEAT,

248 181

27 27

ANGER IS OBJECTS IN THE HEART, ANGER IS SWELLING, CONTAINER, 280 ANIMAL IS HUMAN BEING,

338, 345 422 CONCRETE IS CONCRETE, 27 CONTENTS ARE CONTAINERS ,338

CONCRETE A IS CONCRETE B, COOKIE IS PERSON DOING FITNESS EXERCISES,

STRUCTURE, 337,

ANGER IS THE HEAT OF A FLUID IN A

385, 386, 396

CORNCOB AND FRENCH BEAN ARE

446

AUTHORITIES HIDING BEHIND BUREAUCRACY IS CLERK BEHIND BOOK,

417, 422

388, 396 223, COUNTRY IS LITTLE BAD BOY, 223 COUNTRY IS PATIENT, 225, 229
COUNTRY IS ANGRY TEACHER,

BRIDE AND GROOM,

468

Metaphor and metonymy index 157

HAVING CONTROL IS UP, DOWN,

CULTURE IS A PAPER SERVIETTE, DEATH IS A DEVOURER,

176, 345

HIGH STATUS IS UP/LOW STATUS IS

407 *DEGREE OF MOTIVATION FOR

SUCCESS IS SIZE OF HEART, AGAINST TERRORISM IS AN INFINITY DEPLOYMENT IN A VAST GEOGRAPHICAL AREA,

176, 345

HUMAN (BEING) IS ANIMAL,

126

THE DEPLOYMENT OF ARMED FORCES

203 167

THE DIFFERENCE BETWEEN NORTH AND SOUTH IS AN ABYSS, THE DIRECTION OF THE ECONOMIC TREND IS THE DIRECTION OF THE ARROW AS IT IS VIEWED FROM EITHER THE BUSINESS PERSONS POSITION OR THE LABORERS POSITION IN THE UNCONVENTIONAL SPATIAL FRAME,

433, 437, 441, 446 HUMAN (BEING) IS DOG, 394 HUMAN VICTIM IS ANIMAL, 11, 429 431, 433, 444446 HUMAN VICTIM IS ANIMAL RESOURCE, 437, 441 HUMAN VICTIMS ARE CATTLE, 442 HUMAN VICTIMS ARE KANGAROOS, 436, 437
HUMAN VICTIMS IN CAR ARE DEAD ARMADILLO IN STREET ,

441 235,

IDEAS ARE (PHYSICAL) OBJECTS,

205

162, 163, *EFFECT FOR CAUSE [Relaxation for Drinking Tea], 102, 103 *EMBLEM FOR PRODUCT [Tea Tag for Tea], 102, 103
EMOTION IS EXTERNAL METEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF,

THE EARTH IS A SAUCEPAN,

337, 338, 340, 342 (THE INSTITUTION OF) CINEMA IS MOLOCH, 408, 421 INTENSITY IS HEAT, 181, 256, INTENSITY IS QUANTITY, 257 INTENSITY IS SPEED, 257
INTRODUCING AN IDEA IS PRESENTING AN OBJECT,

318 317

KNOWN IS DOWN, LIFE IS A BURDEN,

270, 271, 273, 283, 290 268 EMOTIONS ARE FORCES, 9

EMOTION IS FORCE, EYES ARE CONTAINERS FOR THE EMOTIONS,

250, 258

FINANCIAL SUCCESS IS NATURAL GROWTH, FOOTBALL IS SUN,

54, 55 220, 221

*GARMENT FOR PERSON [Baby bootie for baby], 107, 109, 110 GOOD IS UP, 65, 313, 317 GREAT CHAIN OF BEING, 13, 27 HAPPY IS UP, 289

26, 7, 8, 20, 26, 29, 35, 119, 120, 122, 125, 127 129, 136, 138, 139, 179, 419 LIFE IS A PLAY, 26, LIFE IS A STAGE, 7, 26, 126, 130, 131, 133, 134, 136, 139 LIFE IS BONDAGE, 26, *LOCATION FOR OBJECT, 339 LOUDER IS BIGGER, 279 LOVE IS A JOURNEY, 98, 300 LOW STATUS IS DOWN, 176, 345
LIFE IS A JOURNEY, MACHINE IS MONSTER, MACHINE IS MOLOCH,

406 406, 408

Metaphor and metonymy index

MACHINES ARE PEOPLE, THREAD,

469

410, 422,

PITCH RELATIONSHIPS ARE RELATIONSHIPS IN PHYSICAL

MAKING A DECISION IS PULLING A

107110 MAN IS AN ANIMAL, 406, 422, See HUMAN IS ANIMAL and PERSONS
ARE ANIMALS MASSACRE IS RELIGIOUS RITUAL,

392

MEASURE IS COOKING EQUIPMENT,

226
MEASURE IS INGREDIENT,

226

MEASURE/POLICY IS COOKING POT,

234
MEN CHASING COAT ARE MEN PLAYING RUGBY,

412 392,

MENTAL STATE IS FAST TRAIN,

393
THE MIND IS A MACHINE, MIND IS BODY,

161

20 226

361, 362 226, POLITICAL DOMAIN/PUBLIC AREA IS HOSPITAL ROOM, 225 POLITICAL MOVEMENT IS MEDICINE, 225 POLITICIAN IS TODDLER, 177, 180, 188, 189, 191 POLITICIAN IS DOCTOR, 225, 229, 230 POSITIVE IS UP, 317 POWER/HIGH STATUS IS UP, 344, 345 *PROTOTYPICAL ITEMS OF A CULTURE STAND FOR THAT CULTURE, 137 PUBLIC SCHOOL IS ARMY, 391 PURPOSES ARE DESTINATIONS, 122, 128
POLICY IS DISH, REAL CANNON IS CIRCUS CANNON,

SPACE,

MINISTER IS COOK,

MISGUIDED BELIEF IS AN OPPOSITE

201, 202 MOBILE PHONE IS PIANO, 384, 385

MOTHER RUSSIA IS HEADLESS PERSON,

BODY ORIENTATION,

390, 391 *REPRESENTATION FOR REPRESENTED, 339 *SALIENT FEATURES OF A THING STAND FOR THAT THING, 137 SIMILARITY IS ALIGNMENT, 8, 197, 208, 209 SOFT COLOR IS WARMTH, 103, 104, 110 SPACE-IN-FRONT IS FUTURE, 180
STATE OF BEING IS ORIENTATION IN VERTICAL SPACE, STATES ARE LOCATIONS,

416, 422
MULTINATIONALS ARE DRIP, NEGATIVE IS DOWN, POT,

233

317

NEW VOTING SYSTEM IS COOKING

233235 28 216, 217

OBJECT A IS OBJECT B,

OIL IS INTRAVENOUS DRIP, ONES HEART,

ONES MENTAL CAPACITY IS SIZE OF

127

362 128130,

377 *PART FOR WHOLE [Car front for Car], 99, 105, 107, 109, 128, 130, 228, 341 *PLACE FOR INSTITUTION, 99
PEACE IN THE MIDDLE EAST IS AN STREETS RUNNING WITH BLOOD ARE

200, 201 438, 441 PERSONS ARE ANIMALS, 106

PERSON IS CAPTIVE ANIMAL,

ENDANGERED SPECIES,

409 133, 134, 140 SUCCESSFUL IS UP, 7, 129, 130, 140 SUMMER TV PROGRAMMING IS (LIKE) LEFT BEHIND PET, 222
SUCCESSFUL IS BIG, SWIRLING WINE IS SWIRLING BALLET DANCER,

MAP SPILLED WITH INK,

86

470

Metaphor and metonymy index

UNKNOWN IS UP, WINE IS A BABY,

SYNTACTIC STRUCTURES ARE GEOMETRIC PHYSICAL STRUCTURE,

317 84 87

345

WINE IS A BALLERINA, TEA TAG IS A HYPNOTIZERS WATCH, WINE IS A TEXTILE

102104
TENT IS TRAIN, JAW,

WINE IS A THREEDIMENSIONAL

394, 495

ARTEFACT,

77 77

THE TENNIS RACKET IS A SHARKS

WINES ARE BUILDINGS,

165 26 179 26

WINES ARE GEOMETRICAL ENTITIES,

TIME IS A DEVOURER, TIME IS A MOVER, TIME IS A THIEF,

77
WINES ARE JEANS,

88 77 83

TIME IS A MOVING OBJECT,

WINE IS LIVING ORGANISMS, WINES ARE PEOPLE,

26

TIME IS MOTION THROUGH SPACE,

WINES ARE PIECES OF CLOTHING, WOMEN ARE ANGELS,

185
TIME PASSING IS MOTION, TSUNAMI IS SHARK,

179 199

464

TV PROGRAMS ARE GARBAGE,

76 418, 419, 421 WORKERS ARE SHEEP, 414 WORKERS ARE SLAUGHTERED OXEN, 411