Musical Time in Visual Space

Brian Evans Department of Art, University of Alabama brian.evans@ua.edu

The laws of harmony are the same for painting and music. Maurice Ravel

Abstract
There is a renewed interest in composing visual music. A time-based, visual art, using abstract material, can look to fundamentals from traditional Western music practice as a place to start. These fundamentals map into the visual art foundations of design and color. Visual equivalents of the musical ideas of consonance, dissonance, tension, and release, can be easily understood and applied to the composition of abstract animation. With the current state of digital technology the medium of time-based sonic/visual art is now available to any composer interested in expanding their artistic expression into the visual domain.

be made using non-narrative visual space? Are there visual counterparts for the traditional building blocks of musical time? Focusing on motion and the materials of the traditional art foundations, design, pictorial composition, and color harmony, a time-based grammar of visual music can be developed. The construction of this grammar starts by defining visual consonance and dissonance. As in musical harmony, controlling movement from dissonance to consonance supplies a means of moving through time musically. With a grammar established a visual music theory can be developed. Visual music can be composed and abstract animation, in truth any time-based visual expression, can be seen as musical.

Music Foundations

Introduction

Music composers create coherent temporal structures using the materials of soundpitch, timbre, rhythm, etc. They organize sonic events. The hope is that the unfolding of these events has a quality that would be described as musical. There is no broad agreement on what musical means, but there is a wide array of sonic art that most call music and so consider musical, from plainsong to Mississippi blues, North Indian ragas to the extended drones of the Aboriginal digeridoo. Musical motion (at least in most Western music) is created through tension and resolution of tension through controlled dissonance resolving to consonance. This tension and release is expressed through harmonic pitch relationships and rhythmic patterns that together develop as chords, motives, phrases and cadences. These are the basis of a musical syntax and provide a framework upon which expressive, musical ideas are articulated. Like music, the fundamental dimension of abstract animation is time. Many animators of non-representational images seek a visual unfolding of events that they describe as musical. Examples are many, from the work of Viking Eggling (1923) and his visual counterpoint to the digital harmony of John Whitney (1980), to recent time-based visual works by formally trained music composers. (Evans, 2003, Miller 2002) Hence many experimental animators call what they do visual music. Can a truly musical expression

Stravinskys statement, Music means nothing outside itself, illustrates a common mindset, held by music composers for centuries. (Stravinsky, 1956) Music composition was a formalist activity. This was also a modernist view. Modernist ideas have been somewhat tattered over the past few decades, but they can still present a viable basis for discussing musical time. We can start with a simple definition of music as the structuring of time with the materials of sound patterns. (Lets exclude literary forms such as song, opera and theatre.) The development of musical instruments has a history measured in millennia. These instruments were designed as generators of abstract sounds, without a referent in the real world beyond the instruments themselves. These abstract sounds were used to create temporal structures such as sonatas, fugues and symphoniesabsolute music. In general Stravinsky is talking about art for arts sake. From this we can define musical as an aesthetic response to the perception of sonic pattern t h e appreciation of significant form, a primary focus of modernism and its formalist leanings. (Bell 1914) In music these patterns are built on foundations of repetition, contrast and variation. There is no structure, no pattern, without repetition. Repetition by itself can of course become boring, so contrast is useful in keeping the listener engaged. As any utterance is multi-dimensional, it is possible to repeat in one dimension while contrasting

Proceedings ICMC 2004

another. For example playing the same notes on a different instrument repeats the pitches while contrasting the timbre. This is variation, a more subtle but effective technique for creating and developing musical pattern. The fundamental dimension of music is time. To make music is to move coherently through time. This is evident in Western tonal music where the music moves the listener through time by first establishing a tonal center (balance and harmonic stability). From this stability the listener is moved to a sonic dissonance that builds tension. Tension is resolved through resolution back to tonal consonance and stability. The music returns to the comfort of familiarity. Any activity that moves us through time, including dance, poetry, theatre, etc., uses the idea of tension/release. Boy gets girl Boy loses girl. Boy gets girl. From here we can construct a syntax, a consensus on the componentspatterns that repeat, vary and contrast, and the combining and ordering of these patterns to create tension and resolution. Combinations of sound materials become musical statements. The motif, the minimal material needed to expresses a musical idea, becomes the building block of the phrase, a musical line. Phrases are combined and articulated in time by cadences or punctuation points.

Figure 1. Illustration of visual rightness from the book Composition by Arthur Wesley Dow, 1899. Dow defines the image on the left as a motif, and the other four as variations. The impulse was quickly lost (but never abandoned) in the Hollywood deluge of character animation and big screen storytelling. With developments in technology over the past decade there is a renewed and growing interest in visual music composition. Art theorist Rudolph Arnheim says, One of the basic visual experiences is that of right and wrong. This idea is the basis of design foundations as taught in most art schools. (Arnheim, 1966) Visual rightness is visual consonance. Any introductory design book talks about how to achieve visual balance and harmony and good visual composition. (from Dow, 1899, see Figure 1, to Lauer and Pentak, 1999) From this premise composing visual music is a simple process. If rightness is codified and understood, wrongness is easily created by not being right. Movement from visually wrong to visually right is a construction of tension/release. Film and video artists have developed their own grammars over the past century, and movement from tension to release is a part of those grammars. The practice of storyboarding is an example. (Begleiter, 2001) A director plans a scene as a series of moments, composed as a series of drawings that guide the eventual motion, framing, and composition of the picture plane as it changes in time. A scene is built on phrases that are delineated and cadenced with moments of visual rightness. The documentary films of Ken Burns are good examples of this. (Burns 1990) The camera, under strict motion control and compositional planning, pans over still photographs. Motion and picture composition, structured in phrases, punctuate time with cadences of visual rightness. The viewer moves through time musically. Color is another visual dimension available to the visual music composer. Ideas of color harmony have developed over the past centuries based on the premise that there is an

A Visual Music

Gauguin in describing his painting Manao Tupapau said, The musical part: undulating horizontal lines; harmonies of orange and blue, united by yellows and purples (their derivatives) lit by greenish sparks. The literary part; the spirit of a living person linked to the spirit of the dead. Night and Day. (Chipp, 1968) Here we get an insight into a visual artists separation of form and content. Gauguin used the word musical to describe the formal aspects of the work. When visual artists talk about composition they are focused on the design, the organization of materials in 2D or 3D space. Composition is traditionally only one aspect of a visual artwork. The formalist explorations of the 20th century illustrate the desire by visual artists to achieve a purely musical expression, to create a visual music by bringing form and content together. Kandinsky, considered by many to be the father of modernist abstraction, wrote, A painter, who finds no satisfaction in mere representation, however artistic, in his longing to express his inner life, cannot but envy the ease with which music, the most non-material of arts today achieves this end. He naturally seeks to apply the methods of music to his own art. And from this results that modern desire for rhythm in painting, for mathematical, abstract construction, for repeated notes of color, for setting color in motion. (Kandinsky, 1914) This became truly possible with film. To follow the path established by traditional music practice, visual music was formal, abstract and temporal . In the early years of film, abstract animation was developed by visual artists who desired to compose visual music. (Russet and Starr, 1976)

Proceedings ICMC 2004

experience of balance when color combinations reduce to an experience of neutral. (Cheuvral, 1854 and Albers, 1963) For example combining a primary and its opposing secondary color is considered harmonious. When visually mixed these colors reduce to grey. Grey is the most neutral, balanced color experience. Moving from weighted to balanced color spaces can also be effective in moving us through time in visual space. (Evans 1990) Consider the 1939 film of The Wizard of Oz. When Dorothy is safe in the stability of home the film is in black and white. When she is off on her adventures in Oz the film is in color. The film returns to neutral black and white at the end, when Dorothy returns to the comfort and safety of home. (Baum, 1939) Codified art foundations in design and color theory are useful as entry points for those interested in composing visual music. These foundations were developed empirically over centuries by practicing visual artists. Over the past decade there has been significant research in the neurobiology of vision. This research validates and expands on these art fundamentals. Our knowledge of how the eyes and the brain process and understand visual information is growing rapidly. This research offers a fertile field upon which to continue exploring time-based visual art, and indicates promising directions for future work. (Zeki, 2000, and Livingstone, 2002)

are all time-based. Western music tradition is a wellcodified and understood practice and can provide guidance in the creation of visual music work. Whether formalism can succeed as an end in itself continues to be debated. It does continue to be of value as a technique, a device of construction, and a basis for exploring visual music composition. Technology has opened visual space to composers interested in expanding their musical ideas into new media. Fundamentals of music composition can be easily mapped into time-based visual design and new avenues of musical expression are possible. As Morton Feldman once commented to Cage, on realizing the new possibilities that had opened up for composers in the early post WWII years, Now that things are so simple, theres so much to do.

References
Arnheim, R. (1966). A review of proportion, Module, Albers, J. (1963) Interaction of Color. New Haven, CT: Yale Baum, L. F. (1939). The Wizard of Oz. Warner Studios. (Motion Picture) Begleiter, M. (2001). From Word to Image, Storyboarding and the Filmmaking Process. Studio City, California: Michael Wiese Productions. Bell, C. (1914). Art. London. Burns, K. (1990). A Civil WarA Film by Ken Burns. PBS Home Video. Cage, J. (1980) Silence. Wesleyan, CT: Wesleyan University Press. Cheuvral, M. E. (1854). The Principles of Harmony and Contrast of Colors and their Applications to the Artst. West Chester, Pennsylvania: Schiffer Publishing Ltd. Chipp Herschel B. (1968). Theories of Modern Art. Berkely and Los Angeles, California: University of California Press. Dow, A. W. (1899) Composition. Garden City, NY: Doubleday. Eggling, V. (1923) Diagonal Symphony: Rhythm 21. Germany. (Motion Picture). Evans, B. (1990). Temporal Coherence with Digital Color. Digital ImageDigital Cinema, SIGGRAPH 90, Art Show Catalog, LEONARDO, Supplemental Issue. London: Pergamon Press Evans, B. (2002). limosa. Tuscaloosa, AL. (Music Animation). Kandinsky, W. (1914). The Art of Spiritual Harmony. London: Constable and Company, Ltd. Livingstone, M. (2002). Vision and Art: The Biology of Seeing. New York: Henry N. Abrams. Lauer, D. and Pentak S. (1999). Design Basics. Orlando: Harcourt Brace and Company. Miller, D. (2002). Vis a Vis. Boston, MA. (Music Animation). Russett, R. and Starr C. (1976). Experimental Animation. New York: Van Nostrand Reinhold Company. Stravinsky, I. (1956). The Poetics of Music in the Form of Six Lessons. New York: Vintage Books. Whitney, J. (1980) Digital Harmony. Peterborough, NH: McGrawHill. Zeki, S. (2000) Inner Vision, An Exploration of Art and the Brain, Oxford: Oxford Press.

After Formalism

John Cage had a different view of musical time. In the book Silence he wrote, Music means nothing as a thing. (Cage, 1958) With his music composition 4' 33" he removes himself from the modernist viewpoint and loosens up the idea of musical. He simply frames time by specifying duration, claiming that all sounds heard in the specified duration. whatever they may be, are the musical materials of the piece. Again, the fundamental dimension of music (sonic or visual) is time. In truth, the most fundamental dimension of experience is time. (It is odd that some art schools are introducing time into their foundation courses, calling it 4D, as an afterthought to two and three dimensional work. Of course for human experience, time is the first dimension as none of the other dimensions exist for us without it.) We understand the world through signals received through the senses. We are able to make sense of the world because the signals are structured. Perceptions of sound and light are built from the reception of waves (patterns) of changing air pressure and electro-magnetic energy. Hence pattern is axiomatic for experience, for life. For us, time passes as the experience of patterns. Experienced time is by definition musical. The set of possibilities is infinitely large for time-based art. Formalist ideas of music composition give us a basis for entering visual space with musical intent. We can learn much from the traditions of all art forms, as in the end they

Proceedings ICMC 2004