PRESENTED AT: ISEA (International Symposium on Electronic Art), Helsinki, 1994. -------------------------------------------------------- Lev Manovich THE LABOR OF PERCEPTION: ELECTRONIC ART IN POST- INDUSTRIAL SOCIETY I. -------------------------------------------------------- Just as it would be futile to consider video art in isolation from television, it would be equally unproductive to theorize new emerging forms of computer art without considering their uneasy connections to contemporary image industries, such as the computer graphics industry. Computer artists need this industry to provide them with the latest technological toys which will set them apart from their colleagues still working in the traditional, pre-industrial mediums. The industry uses the artists as beta-testers for new software and hardware. More importantly, the industry uses the mythology of art -- our Romantic-modernist belief that the artist is a unique person, a visionary who transcends the everyday reality and pushes the boundaries, etc. -- as the most effective sales tool. What better way to market a piece of software than to have an endorsement from the artist? (Thus, paradoxically, computer artist is somebody who transcends the here and now in the act of creation, but can do so only with the help of the very latest tools, the tools of here and now). If computer art does not exist in isolation from computer graphics industry, let us examine the history and the direction of the industry. Why did computer graphics -- the industry concerned with finding more effective ways to produce, store, distribute and present images -- achieve such importance? Why is it that today new disciplines which study images and vision continue to expand: image processing, computer vision, research on human-computer interfaces, vision science, and so on? What are the reasons these currently prominent image industries and image sciences have acquired such prominence? Let us begin with three images (figures 1, 2, 3). The first image: a portrait of Tatlin by a fellow Soviet designer El Lissitsky (figure 1). Time: early 1920s. A compass, extending straight from Tatlin's eye, a metaphor of vision for work. The second image: SAGE (the "Semi-Automatic Ground Environment") -- the first human-machine interactive display system (figure 2). Time: mid 1950s. The third image: virtual reality interface designed at NASA/Ames Human Factors Research Center (figure 3). Time: now. Instead of the metaphor of the eye-compass, a reality: video monitors strapped to the eyes. The notion of vision as work is now fully realized: the operator wearing the gear works by mentally processing visually presented information. The gear is designed using all the available knowledge accumulated by experimental psychology about human vision. In the photograph we see the last leftover from the age of manual labor -- an arm in a DataGlove. It will soon disappear since through gaze tracking the operator can control the system by merely looking at different points in virtual space. II. -------------------------------------------------------- Modernization brought with it a special discipline concerned with efficiency -- engineering. The job of an engineer was to ensure maximum performance with a minimum investment of energy, materials, and time, be it the performance of machines (mechanical engineering), communication systems (communication engineering) or human bodies (scientific management, time and motion studies). Inspired by modern engineering, the avant- garde of the 1920s tried to systematically apply its principles to vision. To engineer vision meant to eliminate waste, to use minimal material resources. Thus, constructivist graphic design streamlined typography, eliminating complicated typefaces in favor of block letters consisting of straight lines; it also eliminated illustrations and "wasteful" decorations by making type itself the main element of design. The goal: maximum impact with minimum use of ink (figure 4). To engineer vision also meant to minimize the psycho-physical resources required of the viewer. Dziga Vertov writes in his famous 1923 manifesto: "The least advantageous, the least economical communication of a scene is theatrical communication."  In contrast, montage forces the eye to see the right thing at the right time, thus eliminating the visual waste of theater, ballet, painting, and other traditional forms. In montage, "camera drags the eyes of a film viewer from hands to legs, from legs to eyes and the rest in the most advantageous order..."  To engineer vision also meant to ensure perception in the shortest possible time. Here as well, the avant- garde promoted montage as an example of possible economy, in this case economy of time. Maud Lavin describes the 1930 manifesto of the group of leading German designers headed by Kurt Schwitters: "Walter Dexel writes that modern man has the right to expect communications in the shortest possible time. Willi Baumeister points out that photomontage is efficient, allowing for the quick grasp of several images at once."  Finally, to engineer vision also meant to be able to measure its efficiency, or, to use the language of a communication engineer, to measure "system performance." Eisenstein, fresh from engineering school, invented his first theory of artistic communication, the famous "montage of attractions": "Let us search for the unit which will measure the influence exerted by art! Science has its 'ions,' its 'electrons,' its 'neutrons.' Art will have -- attractions!"  To summarize: The job of the avant-garde artist was to engineer vision, and to engineer vision meant to affect the viewer with engineering precision, predictability, and effectiveness. III. -------------------------------------------------------- In its desire to engineer vision, the avant-garde was ahead of its time. The systematic engineering of vision took place only after World War II with the shift to post-industrial society. For post-industrial society, mental labor of information processing is more important than manual labor. In contrast to a manual worker of the industrial age (figure 5), an operator in a human-machine system (figure 6) is primarily engaged in the observation of displays which present information in real time about the changing status of a system or an environment, real or virtual: a radar screen tracking a surrounding space; a computer screen updating the prices of stocks; a video screen of a computer game presenting an imaginary battlefield; a control panel of an automobile showing its speed, etc. In short, vision becomes the major instrument of labor, the most productive organ of a worker in a human-machine system. And this is why following World War II we witness unprecedented amount of research into imaging and vision. The figure which stands at the gates to this post- industrial society of perceptual labor is a radar operator of World War II. 1. First of all, in order to ensure the maximum performance of such human-machine system as radar, it became necessary to engineer it around the capacities and the limitations of human vision. At the end of the World War II, a new field emerges -- human engineering. Let me quote from the description of its history found in an 1965 overview of the field: "The primary emphasis in time-and-motion engineering has been on man as a worker; that is, as a source of mechanical power. It was not until World War II that a new category of machines appeared -- machines that made demands not upon the operator's muscular power, but upon his sensory, perceptual, judgmental, and decision-making abilities. The job of a radar operator, for example, requires virtually no muscular effort, but makes severe demands on sensory capacity, vigilance, and decision-making ability. This new class of machines raised some intricate and unusual questions about human abilities: How much information can a man absorb from a radar screen?"  Already before the war, experimental psychologists assisted in selecting military personnel for such jobs as pilot or airplane observer by administering special aptitude tests. During the war, a much greater number of pilots, radar operators and other similar personnel became needed. The emphasis was shifted, therefore, from selecting personnel with particularly good perceptual and motor skills to designing the equipment (controls, radar screens, dials, warning lights) to match the sensory capacities of an average person. And it was the field of experimental psychology that possessed the knowledge about the sensory capacities of an average, statistical person: how visibility and acuity vary between day and night; how the ability to distinguish colors and brightness vary with illumination or distance; what the smallest amount of light is which can be reliably noticed; and so on. All this data was now utilized for designing better displays and controls of the first modern human-machine systems such as radar installations or high-speed aircrafts. The term "human engineering" was eventually replaced by another term standard today -- "human factors." The radar operator who in the 1940s and 1950s was the prototypical example of a human-machine system, was replaced by the 1980s by a new prototypical figure, the computer user. Thus, references to "human-machine systems" became references to "human-computer systems." The same amount of intellectual energy and research which in the middle of the century went into theorizing the performance of a radar operator and adapting him and radar display to each other, today goes into the work on new computer interfaces, such as NASA/Ames VR system (figure 3). 2. The work on radar also directly leads to the development of interactive computer graphics. Next to photography, radar provided a superior way to gather information about enemy locations. In fact, it provided too much information, more information than one person could deal with. Was there a way to process and display information gathered by radars more effectively? The key principles and technologies of computer graphics -- CRT (cathode-ray tube) display, bit-mapped graphics, interactive control, were developed as a way of solving this problem. The research took place at MIT. After the end of the War, Air Force created a secret Lincoln Laboratory. The job of Lincoln Laboratory was to work on human factors and new display technologies for SAGE -- the "Semi-Automatic Ground Environment," a command center to control the U.S. air defenses established in the mid-1950s. The earlier version of the center, called Cape Cod network, was operating right out of the Barta Building at MIT. Each of 82 Air Force officers was monitoring his own computer display which showed the outlines of New England Coast and locations of key radars (figures 6, 7). Whenever an officer would notice a dot indicating a moving plane, he would use a light gun to tell the computer to track this dot. This was the first human-machine interactive computer graphic display system, developed to alleviate the mental labor of information processing. Vision, enchanced by computer graphics technology, became the only means to deal with information overflow. IV. -------------------------------------------------------- Computer graphics helped to process radar information more efficiently, but was there a way to take the human, who was too slow to keep up with the computers, completely out of the loop ? This is the third crucial development in engineering of vision -- the work on computer vision. In 1961, the National Photographic Interpretation Center (NPIC) was created to produce photoanalysis for the rest of the U.S. intelligence community and, as Manual De Landa points out, by the end of the next decade computers "were routinely used to correct for distortions made by satellite's imaging sensors and by atmospheric effects, sharpen out-of-focus images, extract particular features..." Computer analysis of photographic imagery also became the only way to deal with the pure volume of intelligence being gathered. The techniques of image processing, which can automatically increase an image's contrast, remove the effects of blur, extract edges, record differences between two images, and so on, greatly eased the job of human photoanalysts. But was it possible to completely replace them by computers? Roberts' 1965 paper "Machine Recognition of Three- dimensional Solids" is considered to be the first attempt at solving the general task of automatically recognizing three-dimensional objects. His program was designed to understand the artificial world composed solely of polyhedral blocks (figures 8, 9). Using image processing techniques, a photograph of a scene was first converted into a line drawing. Next, the techniques of 3-D computer graphics were used, also developed by Roberts. Thus, the two fields were born simultaneously: 3-D computer graphics and computer vision, automation of imaging and of sight. -------------------------------------------------------- In summary, the rise of modern image industries and image sciences, such as computer graphics, human-factors research or computer vision, can be seen as a part of the shift to the post-industrial society of perceptual labor. This shift involves two processes -- two stages of automation. First stage of automation: human and machine are integrated in new human-machine systems which increasingly came to dominate both the battlefield and the workplace after World War II (radar screen, aircraft controls, computer terminals of the automated factory). Human vision became the key instrument of post- industrial labor as the channel of communication between human and machine. This leads into research into more efficient human-machine interfaces -- from Ivan Sutherland's Sketchpad to today's VR. Second stage of automation: the complete replacement of human cognitive functions by a computer, such as the substitution of human vision by computer vision. What does it mean to teach a computer how to see? In the field of computer vision, "vision" refers to two goals. First, it means the identification of various objects represented in an image. Second, it means reconstruction of three-dimensional space from the image. For instance, a missile not only has to identify a target but also to determine the position of this target in three-dimensional space. Here, vision is not meant for the contemplation of a sunset or appreciation of art; instead, it is reduced to the common denominator shared by humans and low level organisms: to detect an obstacle, a predator, a prey. I believe that most of the new research into vision and imaging after World War II can be understood as following these two directions: on the one hand, making human vision in its new role of human-machine interface as efficient, as productive as possible; on the other hand, transferring vision and other human cognitive capacities from human to a computer. V. -------------------------------------------------------- What does this analysis entails for forming aesthetic criteria by which we can judge computer art? Let us look at the two paradigms in turn. First, as I pointed out, in a post-industrial society vision acquires a new role of human-machine interface -- from radar screens of World War II to such contemporary developments as VR. The industry aims to make human vision as productive, as efficient as possible. If we still believe that art is something which is anti-productive, anti-utilitarian, the computer artist can be defined as designer of bad interfaces: interfaces which are inefficient, wasteful, confusing. One example of such "bad" interfaces is a display where, instead of usual modernist clarity, or "good form," the viewer encounters formlessness, chaos, "the madness of vision" (figure 10). Another example can be a pseudo-interactive work: a screen with a menu where every choice gets you to the same place. Second, since we are also witnessing a movement towards the complete automation, including the replacement of human vision by computer vision, we need to completely reevaluate the very term "computer art." The term presently refers to the making of art with the help of a computer, the art to be enjoyed by human observers. The artist is the one who makes the creative choices. This Romantic paradigm reaches its extreme in the recent trend of artificial life art, where the computer is programmed to simulate the laws of evolution, mutating images to create endless new combinations; while the artist assumes the role of God, selecting which of these images will survive. I suggest to redefine "computer art" to mean "art for computers," art to be enjoyed not by humans but by computers. Moreover, using the tools of expert systems, artificial life and neural networks, we can evolve not only computer artists -- the programs to create images -- but also computer critics, the programs to evaluate them. What kind of images will be pleasurable for a computer? It is hard to make predictions, but I can guess that following its human master, the computer will adopt efficiency as the main aesthetic criteria. Thus, the computer may prefer images which are efficient in terms of storage -- images which compress well. Rewriting art history from this perspective, the computer critic will prefer minimalist abstraction to Jackson Pollock, and will champion Malevich, as the most important artist of the twentieth century -- the artist who anticipated the aesthetics of compression, and thus was already ahead of today's computer artists who still try to resist the poetics of the productive, functional, industrial (figures 11, 12). As Dziga Vertov wrote in 1923, "I am a mechanical eye."  VI. -------------------------------------------------------- The preceding examples, of course, should be taken only half seriously. My main point is to urge computer artists to examine their relationship to the computer graphics industry, and to address the impact of this and other contemporary image industries not just on art practice but on society at large. The notion that the artist functions outside of society, history and industry is a modernist myth. Modernist artists were not only the pioneers of the utilitarian aesthetics of modern industrial design or the pioneers of the techniques of modern advertisement and political propaganda; as I suggested in this essay, they have also pioneered post-modern engineering of vision, the integration of human and machine in human- machine systems and the replacement of human by computer vision. Today computer graphics industry is one of sites of this engineering. Whether computer artists acknowledge or ignore their relationship to this industry, it exists. Acknowledging rather than ignoring this is the first step toward a critical computer art practice. -------------------------------------------------------- NOTES 1. Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 139. 2. Ibid., 139. Emphasis in the original -- L.M. 3. Maud Lavin, "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle of New Advertising Designers," in MONTAGE AND MODERN LIFE: 1919-1942, ed. Matthew Teitelbaum (Cambridge: The MIT Press, 1992), 54. 4. Qtd. in Jacques Aumont, MONTAGE EISENSTEIN (London and Bloomington: BFI Publishing and Indiana University Press, 1987), 41. Emphasis mine -- L.M. 5. Alphonse Chapanis, MAN-MACHINE ENGINEERING (Bemont, CA: Wadsworth Publishing Company, Inc., 1965), 9-10. 6. Ibid., 8. 7. William Estes, "Experimental Psychology: an Overview," in THE FIRST CENTURY OF EXPERIMENTAL PSYCHOLOGY, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979), 630. 8. See Paul Edwards, "The Closed World. Systems discourse, military policy and post-World War II US historical consciousness," in CYBORG WORLDS: THE MILITARY INFORMATION SOCIETY, ed. Les Levidow and Kevin Robins (London: Free Association Books, 1989); Howard Rheingold, Virtual Reality (New York: 1991). 9. Panel proceedings of SIGGRAPH '89 (Boston, Mass., July 31-August 4, 1989), in COMPUTER GRAPHICS 23, 5 (ACM SIGGRAPH: New York, 1989), 22-24. 10. L.G. Roberts, "Machine perception of three- dimensional solids," in OPTICAL AND ELECTO OPTICAL INFORMATION PROCESSING, ed. J.T. Tippett (Cambridge: The MIT Press, 1965) . 11. The notion of "the madness of vision" is explored by the French philosopher Cristine Buci-Glucksmann. Describing her work, Martin Jay writes: "Resistant to any totalizing vision from above, the baroque explored what Buci-Glucksmann calls 'the madness of vision,' the overloading of the visual apparatus with a surplus of images in a plurality of spatial planes. As a result, it dazzles and distorts rather than presents a clear and tranquil perspective on the truth of the external world." Martin Jay, DOWNCAST EYES: THE DENIGRATION OF VISION IN TWENTIETH-CENTURY FRENCH THOUGHT (Berkeley: University of California Press, 1933), 47-48. 12. Vertov, "Kinoki," 141.