Semiotics and Intertextuality of Videogames

CONTENTS
INTRODUCTION .................................................................3 A SHORT PRIMER .................................................................6

PLAYING WITH SEMIOTICS
................................................................11
CODES IN VIDEOGAMES ................................................................19
INTERTEXTUALITY ................................................................23
CONTINUE/CONCLUSION ................................................................27
BIBLIOGRAPHY
................................................................29
Semiotics and Intertextuality of Videogames
There is terrible irony present in the scholarly study of videogames: though the
medium is an effective communicator, it requires a tremendous amount of communication to convince those that hold it in derision, and think of it only as a childs play-thing. The videogame is a medium that speaks to its generation, its culture. It possesses a language all its own, and yet due to its progenitors, it can speak to the same discourses of traditional media. Videogames are a model of expression that utilizes several modes of communication within itself. The visual: text, words and narratives borrowed and derived from a great wealth of literature and invention; computer-generated imagery, evocative of animation and film techniques and styles. The aural: sounds, both in the realm of diegesis and beyond, used to manipulate ambience, depth, and function; more recently, human voice acting, further blurring the line between the videogame medium and one of its antecessors, the film. The haptic: the tactile sensations between a player the mediums actor the physical topology of the controller, and the medium itself. These modes coalesce to provide the user feedback at prescribed moments when meaning can be inferred. The videogame is a profoundly sophisticated means of communication, even
more so than aforementioned. Games are models of experiences rather than textual descriptions or visual depictions of them. When we play games, we operate those models, our actions constrained by their rules...Videogames are a medium that lets us play a role within the constraints of a model world, (Bogost, How to do things 4). While I would be cautious to downplay the significance of the textual and visual elements of the videogame, it is impossible to ignore the intellectual significance of the mediums gestalt, the encapsulation of distilled experience. And yet we do. Or at least, we have. The academic inquiry of new media, and consequently, videogames, is dwarfed by the stalwart canons of liberal studies. Due in part to its relative adolescence, with the earliest non-commercial games dating back to the 1960s1, and perhaps as well, its general perception among the masses. (Videogames) are a part of the entertainment software industry, and
There are examples of computer games (we cannot really call them digital because of the analog computers used) being created from the early 1950s, like the tennis simulator, Tennis for Two. This inspection of the medium examines more commercialized titles, beginning with Pong, released for the Magnavox Odyssey in 1972.
1
they are generally considered a leisure practice by players and the general public alike. Videogame play is considered an unproductive expenditure of time, time that fills the breaks between works, (Bogost, Rhetoric of Videogames 120). Regardless of the validity of the medium, videogames are almost exclusively seen as a form of entertainment, predominantly skewed towards children not something to be studied by academics. However, I, along with others, do not hold that to be true. If McLuhan was cor-
rect in his magnanimous affirmation that the medium is the message (McLuhan 7), and that the delicious, meaty morsels of content are communicatively superfluous, then it does not matter that the medium is routinely used to slay dragons and play war; if anything, the distractive tendencies of its content plays perfectly with McLuhans model, as really any gamer would tell you, it is easy to become lost in those model worlds. As I have expressed, I feel the videogames penultimate function is as a mode of
expression, of communication. But what is it to communicate with one another? How are ideas formed, framed, and rendered from one mind to another? Because of the videogames close relationship to the differing media and modes that compose it, I propose that we can often use the same points of analysis to better understand its parts. Umberto Eco wrote that every act of communication to or between human beings or any other intelligent biological or mechanical apparatus presupposes a signification system as its necessary condition, (Eco 9). This, I believe, is the essence of communication, a system of significations, called semiotics, which decode abstractions that we have previously given meaning to, and encode further interpretations. At the heart of semiotics is the realization that the whole of human experience, without exception, is an interpretive structure mediated and sustained by signs, (Deely 6). The study of semiotics, a contentious field of interdisciplinary research that spans across the social sciences moreover linguistics, psychology, and philosophy attempts to explain the action of signs (Deely 22). Early 20th century linguist, Ferdinand de Saussure wrote of and
created the term, semiology2, a synonym for the field to describe what he foresaw as a field of academic interest. A science that studies the life of signs within society is conceivable; it would be a part of social psychology and consequently of general psychology; I shall call it semiology. Semiology would show what constitutes signs, what laws govern them (Leeds-Hurwitz 4). Saussure went on to write of critical fixtures of semiotic studies that will be later described within a videogame context. Additionally, Saussures contemporaries were also breaching the field. Charles Sanders Peirce, an American philosopher, wrote of nearly identical relationships3, albeit more explicitly, and declared his study semiotic. From these two scholars alone comes a tree of academics interpretively defining communication. There are several different families of semiotic research. The one investigated in
this paper is anthropsemiosis, a system used to describe all of the sign processes that human beings are directly involved in, and, looked at another way, names those sign processes which are species, specifically human, and as Deely puts it, the one closest to us, (28). Can a relatively young academic field be used to explain an even younger medium like the videogame? Absolutely, almost shockingly so. The videogame, as I said, is a valuable method of communication, and as such, is a veritable trove of signs some obvious and familiar, and others foreign, but all work together through their different modes to signify meaning. Much like the gestalic models that Bogost earlier mentioned, meanings in multimedia are not fixed and additive...but multiplicative...making a whole far greater than the simple sum of its parts, (Lemke 72). And the agency of communication in videogames is not derived from a disorderly mess of signs, but rather a hierarchy. When Michel Foucault spoke the relationship of the verbal (referring to text he observed) and the visual, he said, what is essential is that verbal signs and visual representation are never given at once. An order always
2 3
From the Greek semeion:, a sign, mark. Without knowledge or collaboration with Saussures work.
hierarchizes them, running from the figure to discourse or from discourse to the figure, (33). As I will describe, signs work in the videogame in much the same way Foucault described paintings: in order to establish discursive, rhetorical structures, meaningful signs must combine to form systems within videogames these systems combining to make larger systems, which in turn beget even larger systems. I will demonstrate, starting with the mediums basic semiotics units, that this organization creates a grid of intertextuality, encompassing the videogame medium, the sub-culture surrounding the medium, and the culture enclosing that all of which is a part of the paramount semiotic structure, human experience.
A Short Primer
In order to analyze the role of semiotics in videogames, we first must have a solid
understanding of the subjects fundamentals. As mentioned earlier the most succinct definition of semiotics is a field in which examines the actions of signs. Interesting, you may say, but what are signs? What sort of actions can they exhibit, and why should we care? The two aforesaid scholars, Saussure and Peirce, developed two distinct but
similar methods for analyzing semiotic structures. A sign is something that represents something else. I say something not out of a lack of words or an inability to find an appropriate one, but rather because what I am presenting is more an equation; a proposition that states that any variable material or immaterial concept or object is representative, significant, of another. Saussure in his book Course in General Linguistics offered not only what a sign could be, but more specifically what constitutes a sign. Saussure interpreted signs as a dichotomy, comprised of two parts, the signified and the signifier. The signified is the visible part, or rather, the most obvious part, as it is the the explicit aspect of a sign, present during the interaction, a material presence
of some sort, (Leeds-Hurwitz 9). The signified is what you literally see when you gaze upon the sign; literally, meaning not at all figuratively, as it is the figurative which is the signified (even if there is a great similarity between the two at the time). The signified is the invisible part, or the tacit element of a sign, what might be termed an immaterial presence, something literally absent yet functionally present because it has been invoked, (Leeds-Hurwitz 9). I feel that it is difficult to understand the concept without the use of a visual aid,
so let us briefly consult a sign that we are all familiar with. The object in Figure 1A is a cartoonish depiction of a tree, made with simple lines, curves, and angles, but when it is looked upon and registered, the individual components that comprise it go unnoticed, and instead you think, tree oak, evergreen, sumac leaves green with rough, driedout bark, earthy smells. As the viewer of this sign, you play the role of the social actor, one whom uses their knowledge of contextual evidence to extrapolate the action of the sign. When you see the tree sign in Figure 1A, what you are viewing is the active signifier: representing what you can gleam out of your cultural knowledge of what the object should mean, the signified. Leeds-Hurwitz suggests perhaps a
better example in her description of Saussures relationship: instead of a tree, let us imagine a white wedding dress an actual dress, and not just a cartoon depiction as I presented before, because a sign is a function, and not married to the mode of the image. The wedding dress signifies a wedding (that is, the dress meaning the signifier, while the concept and practice of the wedding is the signified). But theres more to
Figure 1A
this particular sign that could be said. Depending on the cultural situation of the social actor that is interpreting the sign, a wedding dress could be representative of many things in some cultures the white color of the dress is indicative of purity which extends into an idea of a virgin bride, while in other circumstances the wedding dress is symbolic of the marriage itself and not just the wedding ceremony, and furthermore by others interpretations, the dress could be representative of an industry geared to take monetary advantage of lovelorn brides who want a perfect wedding (22-27). There are many semiotic principles at work within this simple, conjured image. First, the sign is dependent on context, whether it is contextually from the
location of its introduction, or the awareness of the interpreter. The sign first of all depends on something other than itself. It is representative but only in a derivative way, in a subordinate capacity. The moment a sign slips out from under this subordination, as frequently happens, at just that moment does it cease for a while to be a sign... Thus on its own, it is a mere object or thing become object, waiting to become a sign, (Deely 35). Deely continues to say that without this content, or in my own preferred terms, the context, the sign ceases to be a sign. He explains that it is still something, often an object, as it cannot become inexistent (or if it can, that is a stasis more suited towards philosophers like Deely to contend with), but instead it must wait to be re-contextualized, to be given further meaning. What is important to realize here is that signs cannot develop meaning on their own, that objects, images and patterns of behavior can signify, and do so on a large scale, but never autonomously. (Barthes, Elements of Semiology 10). Signs require human intervention and invention in order to function and to be understood. Second, it is necessary to note how many different interpretations of the wedding
dress that I alone was able to produce. As I have shown, signs are not created from a void. They are reliant upon the context of their environments and intentions, and allow
for further definition by semioticians. Because the wedding dress serves as a signifier to several different signifieds4, we can declare the wedding dress a polysemic sign. Alternatively, if a signified is tied to multiple signifiers, it is polysemic as well, and signs can often be both. There are other terms that dictate how a sign is interpreted. Referring to Figure 1B, notice
Figure 1B
how the same sign trying to evoke the tree is now displayed in color, with a green top and a brown trunk. Motivation...refers to the degree in which the signified determines the signifier, (Lee-Hurwitz 26). Because the sign more closely illustrates what an actual tree looks like, the signifier is more closely constrained to the signified, and is described as being highly
motivated or highly constrained. We can continue to use the tree example to describe a signs attribute of convention. Convention is used to refer to a signs sense of tradition, or how it is often used in a particular system. If one views Figure 1, they are likely to see tree because that is what they are conditioned to do. Lee-Hurwitz explains that highly conventionalized signs are often ignored due to their ubiquity, and include other popular signs such as the male and female symbols. Also, when analyzing signs, it is imperative to be able to distinguish detonation from connotation. Denotation refers to the explicit, obvious, straightforward, first meaning of a sign; the related term connotation refers to the implicit, conventional, second meaning of a sign, imposed by a specific culture, (LeeHurwitz 27). If we reuse the example of the wedding dress from earlier, it is simple to see that the dress denotes clothing, but it connotes wedding. Or at least, it connotes wedding for the culture of the intended audience of this paper. Denotation often crosses cultural boundaries; connotation almost never does, (Lee-Hurwitz 27).
4
Which will differ depending on the type of reading occurring: dominant, sub-ordinate, oppositional, etc.
10
Of these semiotic principles that have been laid out, the most pertinent one to grasp and isolate is the signs dependence upon human direction an interpreter must be present in order for a sign to be interpreted, but even more so is the signs dependence upon other signs (which were earlier signified by people). Umberto Eco describes this relationship as unlimited semiosis. In order to establish what the interpretant of a sign is, it is necessary to name it by means of another sign which in turn has another interpretant to be named by another sign and so on. At this point there begins as process of unlimited semiosis, which, paradoxical as it may be, is the only guarantee for the foundation of a semiotic system capable of checking itself entirely by its own means...the very definition of sign implies a process of unlimited semiosis...semiosis explains itself by itself, (Eco 69-71). In other words, every sign relies upon another in order to be interpreted by the social actors within its realm. Eco, you may have noticed, uses unfamiliar terminology within his description
of the semiotic process, which he actually borrows from the aforementioned forerunner of semiotics, Peirce. I chose to introduce Saussure first because understanding his methodology allows us in turn understand Peirces viewpoints5. Peirce believed the sign to instead be a trichotomy: the sign or the representatum, the object, and the interpretant, (Leeds-Hurwitz 23). The representatum corresponds directly to Saussures signifier and acts in the same way. However, instead of presenting the signified as a complete unit of expression, Peirce splits it into two the object, the idea or figure that the representatum reflects, and the interpretant, over and above the unique essential structure that makes signification possible in the first place, (Deely 25), the invisible component that represents the meaning generated by the representatum in reference to the object. The process that a social actor uses to
5
A phenomenon not unlike that which was discussed with the interconnectivity of signs.
11
interpret, to go from the representatum to gather the interpretant about the object, Peirce calls semiosis a term now commonly accepted as a synonym for signification. As Leeds-Hurwitz explains, Saussure and Peirces differing ideas are not grounds for contradiction, Peirce was simply slightly more explicit than Saussure, (23). It is because of the validity of both of these methods or maybe as Deely would argue, points of view that I choose to examine the semiotic structure of videogames in accordance with Saussures more direct system, but with the accoutrements of Peirces further inspection of sign variations.
Playing with Semiotics

With this primer of semiotic theory under our belts, we finally have a working knowledge of fundamentals that can be used to analyze the semiology in videogames. But again we should address: can we, and if so, should we? I feel that the disconnect between new and old media is not a drastic one, and that all media (media in the McLuhan sense, that is, almost everything) have a common lineage. Rhetorics inevitably vary by their substance (here articulated sound, there image, gesture, or whatever) but not necessarily by their form; it is even probable that there exists a single rhetorical form, common for instance to dream, literature, and image, (Barthes, Rhetoric of the Image 161-162). That rhetorical singularity Barthes references is what I am referring to myself (because what is a rhetoric if not a chain of signifiers of an ideology, as he would put it). These commonalities allow human communication to be analyzed with generally the same methods just as we analyzed a tree or a wedding dress, we could analyze the latest Call of Duty. So we are able to apply elder methods to new analyzes, but is there a point to it?
Do we stand to learn anything from close readings that we could not have gathered from extraneous study of the medium? I believe there is still much to learn from individual fields like semiotics through the inspection of unfamiliar environments.
12
As Bogost would retort, The content and context of a media artifact is not as inessential as McLuhan would have it. The medium is the message, but the message is the message, too, (How to do things 5). For the sake of simplicity along with establishing a temporal continuity of sign
play in videogames, it is interesting to begin discussing semiotics with a game from early in the creation of the videogame industry. Older videogames and those created by noncommercial developers have a long history of using representational visuals to exhibit meaning. Adventure (1979), a game created for the Atari 2600, allows the player a term specific to videogame culture and conveniently coincides with our semiotic idea of the social actor to control a small, colored square through a digital environment. This square whom you control starts out in front of a similarly colored structure featuring a forward-facing black grid and it all is surrounded by a semi-permeable perimeter of rectangles (Figure 2). Thats how the game begins. There is no foreword in-game text that provides context to your objectives, surroundings, or even who you are, and yet without direct effort you are able to collect this context through semiosis. As you direct your avatar throughout this game environment, you notice things; you realize that when you go through openings between the perimeters boundaries the environment changes its appearance either through its layout or the color (Figure 3). This signifies that your
Figure 2: Starting point of Adventure.
Figure 3: Adventure - Environmental differences
13
character, this square, is able to traverse this area and you are able to differentiate between various locations. But then upon your wandering you discover objects familiar to you within the negative space that you walk in: door keys, arrows (resembling directional arrows, like on a keyboard) and goblets. In this realm of abstraction where all you can infer are shapes, you finally come
upon something of substance, a key. As a social actor from a system that understands what that shape confers, you know what a key is and what it can do. Adding to the significance of this object is a relationship unique to the videogame, the ability to interact, manipulate the object. Your character picks up the key and carries it along with them (Figure 4) while exploring the environment, but you decide instead to equip yourself with the arrow to further investigate its uses, wherein you note that it points towards you when you hold it. Along the way you encounter an object resembling a serpentine dragon, moving aggressively unlike anything that you have seen before. Just as you knew what the key was, you are additionally able to decipher this sign to be a dragon a beast not known for its benevolency and this suspicion is confirmed when it opens its jaws to attack, followed with a harsh, deep rumble. You accidentally run forward directly into the path of the dragon forcing it to produce a defeated noise signifying its demise. It takes a new position symbolizing its condition (Figure 5).
Figure 5: Adventure - Found objects interacting with others through semiosis. Figure 4: Adventure - Interacting with found artifacts.
14
Using your innate semiotic skills you are able to gather many things about this videogame from the encounter: first, you are in a world where dragons exist and the tool you possess seemed to enable you to slay them. This jumpstarts a series of significations much like the unlimited semiosis effect Eco described earlier. Using your knowledge as the intended social actor for this game, you realize the arrow was a sword the whole time, because what other than a sword to slay a dragon in popular culture? This secondly, in turn, compels you to examine the sword more closely. The arrow that we earlier saw is the signifier to the swords signified the reason we did not realize it sooner was because the motivation of the sign was not closely constrained. Regardless, due to the convention of the sword being a symbol of dragon slaying and knighthood, we are able to identify the sign accurately within the game without being given further subtext. Lastly, by knowing this we are better able to signify the rest of the signs that we have come into contact with thus far, we allow chains of semiosis to take effect. The structure we started out in front of in the games beginning now more obviously is a signifier for a castle thereby connotating a medieval aesthetic the keys we discover open the corresponding colors castle gates, and the character in which you play the square-shaped congregation of yellow pixels now resembles a knight. All of this can be inferred within a minuscule period of time due to semiotics at play. I mentioned earlier that there was no in-game text provided for context for
Adventure, but what I did not mention was that was that there was accompanying literature with the game that did so. Often instruction manuals and booklets would provide a deeper backstory and explain game mechanics in games that did not have the time or graphical (or even narrative) resources to do so for instance, there was little indication that Super Mario Brothers 2s (1988) narrative was set within a dream, without having read the manual where it was much more obvious. But that system, of reading supplementary literature, does not break the semiotic structures that we
15
have supposed in Adventure, nor any other game that we could analyze, but rather it makes for a more descriptive experience one would understand signs more readily having have read all the available context, just as a social actor would better understand particular signs having understood the cultural system that the signs came from. Luckily Adventures designers understood the mechanics of semiotics enough to imbibe the game with enough meaning to allow players over 30 years in the future to comprehend its gameplay. Modern games appear to have moved away from the model where an instruction
manual is necessary, instead relying on other forms of media to provide backstory, like books, websites, and animations that may or may not be canonical, and rely furthermore on semiosis within the game itself. In contemporary videogames these moments can be juxtaposed between playable moments like in loading screens (Figure 6), or outside of those times (Figure 7). Notice in Figure 7 that I am intentionally describing the words under the images as signs. More specifically, words are a part of a semiotic and linguistic system in which language is composed of groups of signs we call words, and words are composed of units of signs called letters. Semiosis still occurs when meaning is derived the idea of a sign simply being a picture is not accurate as we can see with this display as well as with the sound of the dragons roar from before. The representational
Figure 6: Braid - Instructional loading screens.
Figure 7: Braid - Diegetic, active in-game instructions.
16
style shown in Figure 7 is especially interesting because the player is able to actively test out what the signifier, the instruction suggesting to jump with the spacebar, is supposed to signify even without knowledge of the higher systems at work here, a player could interpret the signs meaning along with consecutive tutorial signs. When we discuss the role of semiotics in more current games like Braid in Figure
6 and 7, we are invoking more complicated relationships than what was observed in Adventure. Videogames, just like signs, grow exponentially in complexity and sophistication, and necessitate the use of more specific types of signs. In his scholarly research of semiotics, Peirce identified 66 potential varieties of signs, 3 of which have gained wide acceptance: the concepts of icon, index, and symbol, (Lee-Hurwitz 23). Because we are able to treat the videogame as a serious medium worthy of semiotic study, it is possible to locate these types of signs within games, and often due to the graphical fidelity, its easier to do so in slightly more modern games than Adventure where we first identified Saussures principles. To further prove the validity of the assertion that classical semiotic thought is active in videogames, as well as these specific varieties identified by Peirce, let us look briefly at a slightly older title, The Legend of Zelda for the Nintendo Entertainment System (1986) a spiritual successor to Adventure. The Legend of Zelda plays out much like how Adventure started: you begin the game with little narrative context of what you are supposed to do or who you are, and you are tasked with completing your objectives by manipulating objects that interact with one another. But when compared to Adventure this game is relatively more realistic (figure 8).
Figure 8: The Legend of Zelda
17
The Nintendo Entertainment System was capable of producing more robust 8-bit graphics as opposed to the Atari 2600 because of expanded memory, which allowed it to produce detailed and vibrant virtual environments. This also meant that signs intended for use on games on this console could also be more detailed. Peirces icon has a relationship of similarity or resemblance... (and displays) a similarity between the present and the absent components (of the sign), (Lee-Hurwitz 23). An icon is a sign that closely resembles what it is intended to represent; it is a highly-motivated sign. Refer
Figure 9: The Legend of Zelda - Comparison of ingame representation v. extratextual representation.
to Figure 9 for example: the green and brown sprite that you control serves as an icon to what to what you are supposed to see as according to the games literature (situated to the right), because of the close proximity of its representation. With improvements of graphical integrity it has become increasingly easy to denote icons in videogames, and extragame content often has a 1:1 signification with in-game content (Figure 10: Assassins Creed
Figure 10: Assassins Creed 2 - Comparison of in-game representation v. extratextual representation (of a modern game).
2 comparison). That is not to say that games of higher graphical quality are intrinsically better at at communicating through visual semiosis, however. Describing cartooning, Scott McCloud sees simplification as more a way of amplification. When we abstract an
18
image through cartooning, were not so much eliminating in details as we are focusing on specific details, (McCloud 201). Additionally, realism, through its emphasis on denotation as opposed to connotation, makes it more difficult for the artist to make his point symbolically, (Arnheim 141). Iconography viewed through the lens of semiotics can make these distinctions. Indices use a part of something to stand for the whole (Lee-Hurwitz 23). A common trope in The Legend of Zelda series is the characters representation of vitality. As depicted in Figure 11, the characters health is represented by an orderly group of hearts. Upon injury, the player is alerted by a disastrous noise and the
Figure 11: The Legend of Zelda
visual of the heart being halved or quartered, signifying that they have taken damage. The index present here is subtle and more complex than anything we would have seen in Adventure; the heart works as an index by representing a vital part of the body not the whole body, as it is an index, but enough to make it obvious to what it represents the body, as an idea, then acts as signifier to the signifed of the concept of health, a small intricate subset of semiosis. Perhaps the most important sign that Peirce identified was the symbol, a
structure so prevalent it has nearly become synonymous with the entirety of semiotics, and is often the one of most interest in the communication field. A symbol has the relationship of arbitrariness...any sign using an arbitrary connection between the present and absent components is a symbol... (and can often include) objects, behaviors, texts, ideas, and people, (Lee-Hurwitz 23 and 30). It is this arbitrary aspect, despite its pervasiveness, that makes the symbol difficult to isolate and signify. The arbitrariness of a symbol speaks to its situation in the culture in which it was created, or was intended to
19
be used in without being privy to that system it is initially difficult to interpret these particular signs. I feel the best example of symbols in the videogame is the health bar and its multitudinous variations. The health bar, as I will touch on when speaking of medium intertextuality, is present in several games of varying scopes and time periods. The health bar is in a way represented by the heart containers of The Legend of Zelda series earlier discussed, but it appears in more generic terms such as in game series like Mega Man (1987), Mortal Kombat (1992), and the even the first Halo (2001) game6. There is no health bar for real life, nor in any other variety of media only
videogames. Symbols are a form of shorthand; encapsulating cultural knowledge in particular ways, they serve a valuable role in deliberate passing on of traditions... People use symbols as a way of conveying considerable amounts of information in a small space or short time, (Lee-Hurwitz 31-34). The health bar is shorthand unique to the videogame, and can only be picked up through osmosis as Lee-Hurwitz would say, through contact in that medium. I will soon suggest, however, that because of the transcendal properties of both semiotics and videogames, other social actors could become of aware of videogame symbols and other signs without ever playing them.
Codes in Videogames
The word code in videogames connotes several different signifieds: primarily cheat codes, like the strings of characters and commands used to in order to unlock special privileges allowed by the games designer, and the videogames coding, the machine language that controls and shapes the mediums visuals, sounds, and mechanics. But in semiotics, codes are something else altogether. Like with the language analogy presented earlier, signs congregate into increasingly larger systems. A group or system of signs is called a code by semioticians. Placement of signs into appropriate grouping stresses that meaning arises not solely, not even primarily, from
Subsequent titles in that series did employ a similar mechanic, but the first most closely identifies with the aesthetic I am speaking towards).
6
20
the relationship of signifier to signified but from relations between signs, (Lee-Hurwitz 51). Semiotic codes work the same way in videogames just as they would in other forms of media. While we could easily reanalyze the videogames examined before, it would be
more prudent to introduce an even more recent game to illustrate coding systems. Mass Effect 3 (2012) is a third-person action/role-playing game developed to function in a much more fast-paced manner than its predecessors, and substantially retooled the series game interfaces to reflect this change. These alterations required that the player be able to receive a wealth of information in a short period of time just what signs were made to do. To meet these ends, groups of signs are often used in conjunction with one another in what is called a heads-up display. Not unlike the dashboard of your car, a heads-up display provides the user a quick synopsis of necessary information to them; in Mass Effect 3, this often occurs during combat (figure 12).
Figure 12: Mass Effect 3 - Gameplay Heads-up Display
21
The image in Figure 12 reflects a singular moment7 of this code system, where the focal point in the middle of the image features a segmented circle with several different signs on it. Some of these signs can innately go through semiosis; they are icons and obvious, some color coded to signify futility or blank to reflect unavailability, but still others are symbols and require an intimate knowledge of this particular videogame to understand. This circle within Mass Effect 3s mechanics is called the Power Wheel, that is, a code developed by the games designers for users to be able to quickly identify what actions they can take. This type of constructed system is called a digital code because the signs that make the code up are clearly identifiable and distinguished by the social actor, the player. In the lower right hand corner, however, we see a compass. This, we may not
notice, is a code as well. The individual signs consist of red dots to signify hostile nonplayable characters, blue ones representing friendly ones (including the central most one that represents your own avatar), and the arrow pointing due to the next objective and its distance in meters consolidating and signifying that this is indeed a radar within the realm of this videogame. But the average player, or even the above average player, would not see these parts they would only see the radar. Analogic codes, of which this is, contain signs that run together, being separated only by the analyst (or determined semiotician) for the purpose of interpretation, (Lee-Hurwitz 52-53). While the radar is an analogic code, its not the only apt determiner we can assign to it. It is also a paradigm. A paradigm, as defined by Saussure, is a code in which social actors only choose one central sign to display, or focus upon Lee-Hurwitz compares it to us wearing different pieces of clothing, but calling the ensemble one outfit. The power wheel is a paradigm, the health and shield bars below it is a paradigm, as well as the enemy health bar above it8.
7 8
This type of analysis is called synchronic analysis; if we observed the codes evolution through time it would be called a diachronic analysis (Lee-Hurwitz 64). As I elaborated before, the health bar motif is as prevalent as the use of arbitrary, numerical scoring in this medium.
22

There are more codes here, though, and larger ones at that. The largest one
shown in Figure 12 is the aforementioned heads-up display, another common element in games. We saw the beginnings of this code in games like The Legend of Zelda, with all the pertinent signs relaying information at the top of the screen, and this is the natural evolution of that idea; signs, and codes as a result, are always growing. The HUD in Mass Effect 3 is a special type of code, also identified by Saussure, called a syntagm. A syntagm is that new set relating from the combination of elements drawn from different paradigms...meaning is primarily located at the level of the syntagm, (Lee-Hurwitz 55). Meaning is located here, but it is still necessary to break it down to the paradigmatic unit in order to be situationally comprehended, as Barthes presents the syntagm as a fundamental chain...which must be carved up, (Elements of Semiology 65). Mass Effect 3s HUD is a syntagm composed of several active paradigms, but as Barthes suggests as well as practice dictates, the paradigms must be interpreted individually so the syntagms meaning can be deciphered. It is at the syntagmatic level where the breadth of the videogames meaning is
derived, often displayed through the same long, overarching methods: systems of rules delineated by Bogost in his analysis of the videogames operative mechanics; perceptions of space, whether hyper-realized in sports or racing games, or highly abstract in Super Mario Bros (Wade 78); genre, used not unlike its application in traditional media, as well as narrative; and perspective, whether it be top-down or isometric, first-person or third, the perspective is perhaps the most complex system of signifieds of them all. Perspective is a syntagm that influences how an individual game will function: its play mechanics and interface, visual style, and even influence how the social actor the player perceives themselves, their avatar, or both as one. All videogames communicate via a cocktail of syntagmatic clusters, a language,
which varies in paradigmatic composition, like dialects, from game-to-game, but they all
23
must singularly communicate from these previously introduced high-level syntagms. All videogames are kin to one another.
Intertextuality
As we know, high functioning syntagms and lesser paradigms form larger systems, which beget even larger structures until they encompass a language completely their own. But these languages are bound to context and proximity. Even at the highest levels of semiotic processes within a domain, be it videogames or anything else, a foreign social actor will not be able to interpret every sign or relationship, while at the same time they could in another domain, or culture, from which they are a part of. A culture is essentially a grouping of symbolic codes (Leeds-Hurwitz 17), and is the highest term given to semiotic structures. The whole of videogames the medium, its games, its social actors and all that surrounds them is a culture. Videogame culture viewed from the outside stereotypes its users as those who forgo all other cultural, social or even hygienic activities in favor of videogames, (Bogost, Unit Operations 52). But as the consumer base expands and record financials are set each sales cycle, more and more uninitiated actors become acquainted with the culture, disregard or even reshape those stereotypes, and thus they become more familiar with the medium and its peculiarities and its specificities they learn to interpret the signs and structures that were once unintelligible to them. Every time we make meaning by reading a text or interpreting a graph or picture we do so by connecting the symbols at hand to other texts and other images read, heard, seen, or imagined on other occasions...which connections we make (are) characteristic of our society and our place in it...(Lemke 73). Participants in this culture are able to understand individual videogames because they have played other videogames, representatives of the genre.
24

Semiotics and Intertextuality of Videogames It is Hodge and Kress metasign,
markers of social allegiance which permeate the majority of texts (Leeds-Hurwitz 27), that allows social actors to interpret other signs with signs. In our discussion of other sign types we have already touched upon examples of metasigns in videogames, most notably and
Figure 13: Mortal Kombat
ubiquitous, I believe, the health bar (reiterated in Figure 13), a sort of volumetric percentage representation of an abstract quantity. As a metasign, which are mainly comprised of important codes and some individual structures, the health bar is dispersed evenly throughout the medium, and even though it can be represented different visually, its meaning is universally understood. The most important semiotic aspect in the videogame medium involves the use
of metasigns and their ability to speak across discrete units. The breadth of videogames ability to communicate comes from its intertextual qualities. Intertextuality is a concept derived from Mikhail Bakhtin, describing the ability of a text to make reference to another or to several others (Leeds-Hurwitz 41). Intertextuality allows games to speak amongst themselves through semiotic means, transcending time, bureaucracy, and physical media. Several intertextual elements have already been introduced: of course, any
metasign like the recently mentioned health bar, but any other shared trope in videogames as well numerical scoring systems, narrative clichs9, HUD elements, and environment depiction standards. Additionally, there are videogames that include less than subtle homages to previous titles. Software development itself has a rich history of its developers inserting messages into their coding, and videogames are no exception to this. Many videogames contain Easter eggs, included materials hidden by developers and often meant to be humorous to or lightly editorialize the
9
Many already shared with classic literature, such as Joseph Campbells monomyth.
25
game (Bogost, How to do things 37). They are often obscurely located within the game environment and difficult to view, but often make meta-jokes, about the game itself, other games, or even people or events (Figure 14). Intertextual elements are not only within and around videogames themselves, but also manifest themselves as interlopers exploring other semiotic domains, who then retrieve new elements. As the videogame industry continues to grow, it begins to adopt business practices increasingly similar
Figure 14: Grand Theft Auto : San Andreas - In-game Easter egg
to that of traditional media and that of the
the entertainment complex. To increase awareness of the product, videogame icons and symbols now saturate various markets clothing, food, ringtones, toys, bumper stickers, nearly any sort of merchandise you can think of has had a Call of Duty or Halo emblem on it. You see trailers for videogames on television just as you would films before, and print advertisements everywhere else. This is intertextual and extratextual because it illustrates the videogames ability to make reference to itself even while being outside of its original text. At the same time, videogames show the capacity of recursiveness, mirroring its
ability of impacting surrounding cultures by displaying the action of cultures in itself. In-game real world advertisements exist in videogames where it makes diegetic sense, ones that emulate a sort of factual reality like in sports or racing games. These games can display advertisements from actual companies, updated dynamically through constant internet connection, and adjusted to suit the advertisers target demographic. For example, in 2008, the Barack Obama presidential campaign paid for advertisements
26
Semiotics and Intertextuality of Videogames to be displayed in games like NBA Live 0810 (Figure 15). Advertisements especially legitimatize the videogame as an intertextual medium, because as Barthes would point out, the semiotic properties of the advertisement are purely intentional, formed with a view
Figure 15: NBA Live 08 - In-game advertisments
to the optimum reading, (Rhetoric of the
Image 152). This (along with the millions of dollars spent to produce such exchanges) signifies that the videogame medium is mature enough to clearly accommodate its own semiotic processes, an individual game simultaneously referring to itself and games before it, in addition to incorporating outside significations. However, not all intertextual elements are as obvious nor as integral as others.
The most subtle element in videogames, gameplay, is its most influential and also unique to the medium. The process of gameplay that is, the gestalic culmination of elements that compose the play mechanics of a videogame is made up of interactions of codes; codes in the semiotic-sense, and the coding of machine languages. Videogames, just as other completely digital media, are created from a myriad of computer languages, complicated structures complete with their own systems of signifiers and signifieds. To create a videogame from basal codes is complicated, time consuming, and fiscally unadvisable. In order to circumvent a portion of the menial labor in the creation of videogames, many developers use pre-configured software frameworks called game engines. Game engines are provided to developers by other videogame developers thus establishing semiotic sub-culture systems amongst themselves to aid in the production of their medium. One engine can create several different videogames. The engines are often presented as proprietary or open source software suites containing development tools to render graphics, alter in-game physics or artificial intelligence, to script events, sounds, or any number of manipulable
10
Source
27
elements. Game engines move far beyond literary devices and genres. Unlike cultural categories like the modern novel or film noir, game engines regulate individual videogames artistic, cultural and narrative expression, (Bogost, Unit Operations 57). Bogost seems to suggest that while game engines take the drudgery out of game development, they are also partly responsible for the massive growth of the game industry, allowing developers to focus more on discursive opportunities rather than the mechanical agency of a game. Because of the intrinsically similar language, and thereby semiotic structures, that game developers use for particular titles, we can observe clear, intertextual commonalities in gameplay. Figures 16 depicts different videogames
Figure 16b: Lineage 2 Figure 16a: Star Wars Republic Commando
that have been produced from the same game engine, the Unreal Engine 2.While these games do not belong to the same genres, or share a great deal of high level syntagmatic codes, we can observe basal similarities in these images, namely perspective.
Figure 16c: Deus Ex: InvisibleWar
28

Semiotics and Intertextuality of Videogames While game engines, to a degree, reign in artistic expression within the
constraints of the software (in the same way iMovie does film for novices), they are still an efficient method of videogame production, of progenerating videogame symbols, to sustain and disseminate those signs, and display the most lucid influence of semiotic structures in videogames, along with the intertextual relationships they evoke. Videogame developers have their own rhetorical purposes, outside of their
individual games, when creating a game engine. Videogames have great communicative properties, but to the average social actor, signification is only for their own reception. Some game engines address this by using graphical user interfaces to allow one to manipulate game mechanics without working knowledge of a programming language, and some engines simplify this even more by being geared toward beginners.
Click Here to Continue
29
Bibliography
McLuhan, Marshall. Understanding Media The Extensions of Man. Cambridge: MIT Press, 1964. Print. Bogost, Ian. How to do things with videogames. Minneapolis: University of Minnesota Press, 2011. Print. Bogost, Ian. Unit Operations. Cambridge: MIT Press, 2008. Print. Eco, Umberto. Theory of Semiotics. Bloomington: Indiana University Press, 1976. Print. Lemke, J.L. Metamedia Literacy: Transforming Meanings and Media. Visual Rhetoric in a DigitalWorld. Ed. Carolyn Handa. Boston: Bedford/St. Martins, 2004. 71-93. Print. Kress, Gunther. Multimodality, Multimedia, and Genre. Visual Rhetoric in a DigitalWorld. Ed. Carolyn Handa. Boston: Bedford/St. Martins, 2004. 38-54. Print. Arnheim, Rudolph. Pictures, Symbols and Signs. Visual Rhetoric in a DigitalWorld. Ed. Carolyn Handa. Boston: Bedford/St. Martins, 2004. 137-151. Print. Barthes, Roland. Rhetoric of the Image. Visual Rhetoric in a DigitalWorld. Ed. Carolyn Handa. Boston: Bedford/St. Martins, 2004. 152-163. Print. Barthes, Roland. Elements of Semiology. New York: Hill and Wang, 1964. Print McCloud, Scott. Understanding Comics: The Invisible Arts. Visual Rhetoric in a DigitalWorld. Ed. Carolyn Handa. Boston: Bedford/St. Martins, 2004. 195-208. Print. Wade, Alex. Spatial Typologies of Games. Associazione Italiana di Studi Semiotici,ed.5 2009, Print. Foucault, Michel. This is not a pipe. Berkeley: University of California Press, 1983. Print. Leeds-Hurwitz, Wendy. Semiotics and Communication: Signs, Codes, Cultures. Hillsdale: Hove and London, 1993. Print. Deely, John. Basics of Semiotics. Bloomington: Indiana University Press, 1990. Print.
The digital work produced in this paper was created with the Construct 2 engine by Scirra Inc. (2012), operating under the free license.
http://jarrodt.me/thesisgame
Games
Robinett, Warren. Adventure. Atari Inc., 1979. Atari 2600. Ubisoft Montreal. Assassins Creed 2. Ubisoft, 2009. Multiplatform Number None, Inc. Braid. Number None, Inc.. 2009. PC. Rockstar North. Grand Theft Auto: San Andreas. Rockstar Games, 2004. Multiplatform. Bungie. Halo: Combat Evolved. Microsoft Game Studios, 2001. Xbox. Nintendo. The Legend of Zelda. Nintendo, 1986. Nintendo Entertainment System. NCsoft. Lineage 2. NCsoft, 2004. Windows/PC. Bioware. Mass Effect 3. Electronic Arts, 2012. Multiplatform. Capcom. Mega Man. Capcom, 1987. Nintendo Entertainment System. Midway. Mortal Kombat. Midway, 1992. Arcade. EA Canada. NBA Live 2008. EA Sports, 2007. Multiplatform.

Semiotics and Intertextuality of Videogames

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Semiotics and Intertextuality of Videogames

Uploaded by

Copyright:

Available Formats

CONTENTS

INTRODUCTION .................................................................3 A SHORT PRIMER .................................................................6

CODES IN VIDEOGAMES ................................................................19