Lev Manovich THE LABOR OF PERCEPTION Work or Play Walter Benjamin's writings kept coming back to the prototypical perceptual spaces of modernity: the factory, the movie theater, the shopping arcade. Scrutinizing these new spaces, Benjamin insisted on the contiguity between the perceptual experiences in the workplace and outside of it: Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless, today's pedestrians are obliged to do so in order to keep abreast of traffic signals. Thus technology has subjected the human sensorium to a complex kind of training. There came a day when a new and urgent need for stimuli was met by the film. In a film, perception in the form of shocks was established as a formal principle. That which determines the rhythm of production on a conveyer belt is the basis of the rhythm of reception in the film. [1] For Benjamin, the modern regime of perceptual labor, where the eye is constantly asked to process stimuli, equally manifests itself in work and leisure. The eye is trained to keep pace with the rhythm of industrial production at the factory and to navigate through the complex visual semiosphere beyond the factory gates. What would be the equivalents of film and conveyer belt for the perceptual experience of post-modernity? The most direct equivalents are an arcade type computer game and a military training simulator. But now, not only the two experiences provide the same stimuli but they also share the same technology. In fact, since the early 1990s, many companies which before supplied very expensive simulators to the military are busy converting them into entertainment arcade-based systems. One of the first such systems already commercially operating in a number of major cities, including Chicago and Tokyo -- Battletech Center from Virtual World Entertainment, Inc. -- is directly modeled on SIMNET (Simulation Network) developed by DARPA (Defense Advanced Research Projects Agency). SIMNET can be thought of as the first model of cyberspace, the very first collaborative VR environment. SIMNET consists of a number of individual simulators, networked together, each containing a copy of the world database and the virtual representation of all other participants in the conflict such as the Kuwaiti theater of operations. Similarly, a Battletech Center comprises a networked collection of futuristic cockpit models with VR gear. For seven dollars each, seven players can fight each other in a simulated environment. In another example, in 1992 Lucas Arts has teamed up with Hughes Aircraft, combining the expertise in computer games of the former with the expertise in building actual flight simulators of the latter, in a joint venture aimed at theme-park type rides. [2] A computer game and a flight simulator (or an actual cockpit) are only the most obvious examples of how contemporary visual culture is increasingly permeated by interactive computer graphic information displays. Their presence points to an essential feature of the post-industrial society in which the human, both at work and at play, functions as a part of human-machine systems where vision acts is a main interface between the human and the machine. This article will consider some historical aspects of this phenomenon. Human-machine system is defined as "an equipment system, in which at least one of the components is a human being who interacts with or intervenes in the operation of the machine components of the system from time to time." [3] In contrast to a manual worker of the industrial age, an operator in a human-machine system is primarily engaged in the observation of displays which present information in real time about the changing status of a system or an environment, real or virtual: a radar screen tracking a surrounding space; a computer screen updating the prices of stocks; a video screen of a computer game presenting an imaginary battlefield; a control panel of an automobile showing its speed, etc. [4] From time to time, some information causes an operator to make a decision and to intervene in the system's operation: tell the computer to track an enemy bomber noticed on the radar screen; buy or sell a stock; press a joystick; change gears. In some situations these interventions may be required every second (a pilot engaged with an enemy, a computer game player, a financial analysis monitoring stock prices), while in others they are needed very rarely (a technician monitoring an automated plant, power station, a nuclear reactor; a radar operator monitoring a radar screen, waiting for potential enemy planes). The first kind of situation can be seen as a direct continuation of the experience described by Benjamin. In the quoted passage Benjamin characterized modern experience as a constant periodic rhythm of perceptual shocks; the experience shared by an assembly line worker, by a pedestrian, and by a film viewer. This experience is also characteristic of the cybernetic workplace: the constant overwhelming amount of information; the constant cascade of cognitive shocks which require immediate interventions (a pilot engaged with an enemy, a player of a computer game). [5] The second kind of situation, however, points to another work experience, new to post-industrial society: work as waiting for something to happen. A radar operator waiting for a tiny dot to appear on the screen; a technician monitoring an automated plant, power station, or nuclear reactor, knowing that a software bug will eventually manifest itself, making a pointer on one of numerous dials shoot into the red... From Taylorism to Cognitive Science Industrial society was characterized by the centrality of the concepts of manual labor, production of goods, and fatigue. Between 1940 and 1960, these were gradually replaced by new concepts of cognitive labor, information processing, and noise. Taylorism, Gilberts' motion studies, and behaviorism gave way to engineering psychology, "human information processing," and cognitive science. In short, with the transformation of industrial society into post-industrial society, the disciplines of the efficiency of the body were replaced by the disciplines concerned with the efficiency of the new instrument of labor -- the mind. In _The Human Motor: Energy, Fatigue and the Origins of Modernity_ Anson Rabinbach demonstrated how the scientific ideas of thermodynamics, formulated in the middle of the nineteenth century, became central for the conception of work in modernity. Helmholtz, who discovered the law of the conservation of energy, promoted this law as the universal principle which equally applies to nature, machines, and humans. Helmholtz "portrayed the movements of the planets, the forces of nature, the productive force of machines, and of course, human labor power as examples of the principle of conservation of energy." [6] All work was understood as the expenditure of energy, with a crucial consequence of redefining human labor as labor power, the expenditure of the energy of a body. Thus a worker was redefined as a "human motor." This, in turn, lead to the emergence, towards the end of the century, of the movement which Rabinbach calls the European science of work, "the search for the precise laws of muscles, nerves, and the efficient expenditure of energy centered on the physiology of labor." [7] In manual labor, the energy stored in the body where it was accumulated through the intake of food, sleep, and rest is transferred into muscular force -- hammerer striking a blow, filer filing a machine part, and so on. Therefore, psychologists, physiologists and industrial experts searched for methods to maximize both the accumulation of a worker's energy (through proper nutrition, shorter working hours, appropriate breaks) and its expenditure in labor. Just as an engineer designing an engine was concerned with the most efficient transfer of fuel energy into movement, European work experts aimed to maximize worker efficiency and to eliminate possible waste. Central to the quest for the efficiency of the human motor was the struggle against fatigue, understood as the equivalent of entropy. "As entropy revealed the loss of energy involved in any transfer of force, so fatigue revealed the loss of energy in the conservation of Kraft to socially useful production. As energy was the transcendental, 'objective' force in nature, fatigue became the objective nemesis of a society founded on labor power." [8] The European science of work may appear to be very similar to the American scientific management movement pioneered by Frederick Winslow Taylor, a former engineer turned management consultant. As a part of his program, Taylor aimed to minimize and standardize the time required by a worker to perform each operation. He employed the method of time studies whereby the best workers were timed and the results became the norm to be followed by the rest. [9] Later, Frank and Lilian Gilberts (he -- an engineer, she -- a psychologist) popularized another method of motion study. [10] They argued that maximizing worker productivity is best achieved by the elimination of unnecessary movements and making the necessary more efficient. Although both time and motion studies and the European science of work were concerned with the efficiency of manual work, there was a fundamental difference between the two approaches. [11] Taylorism aimed for maximum productivity, and had no concern for the exhaustion and deterioration of the human motor. In contrast, European scientists aimed for optimum productivity, and therefore were concerned not only with the rationalization of the workplace, but also with the workers' health, nutrition, safety, and the optimal length of a workday. In short, Taylorism had no reservations about replacing one exhausted human motor with another -- the philosophy which in the U.S. seems to go hand in hand with the emerging ethics of the consumer society and with immigration policies which assured the constant supply of a cheap labor force. Europeans, on the other hand, were committed to caring for and servicing the human motor. The two paradigms converged after World War I, when European industrialists partly adopted the more brutal, but ultimately more effective Taylorist methods, while U.S. managment experts became more sensitive to workers' physiology and psychology. Taylorism reduced the worker's body to a mechanical machine and had no concern for her or his mind. Indeed, as Marta Braun points out, Taylorism aimed to systematically rob the worker of any degree of independence or even understanding of the overall work process by "separating responsibility for the execution of work from its planning or conception." [12] This disdain for the mind was shared by behaviorism, which matured at the same time as the European science of work and Taylorism, and which equally well characterizes the imaginary of hard-edged social engineering of the first half of the twentieth century. In 1913, J.B. Watson, the founder of behaviorism, explicitly defined it as the science of social control: "Psychology as the behaviorist views it is a purely objective experimental branch of natural science. Its theoretical goal is the prediction and control of behavior." [13] Behaviorism approached the human subject as an input-output system of stimulus and response to be controlled through conditioning. Concerned with controlling the body, it almost completely suppressed any studies of perceptual or mental processes between 1920 and 1950 in the U.S. It was a psychology well suited for controlling the subject already reduced to the brainless human motor. In the 1950s cognitive psychology begins to displace then dominant behaviorism. Since then, what comes under scrutiny of psychologists are mental functions: perception, attention, text comprehension, memory, and problem solving. I read this is as one of the most important signs of the shift from industrial to post-industrial society. The point is not whether corporeal labor was indeed universally displaced by mental labor: this is different from country to country, from industry to industry. What is important is that the obsession with the rationalization of corporeal work (Taylorism, European science of work, psychotechnics) disappeared, displaced by new obsession with the rationalization of the mind (cognitive psychology, artificial intelligence, cognitive engineering). Regardless of the percentage of the work force that still may be engaged in manual labor, society is no longer concerned with spending more intellectual resources to perfect workers' movements. What Taylor's scientific management was for the age of industrialization, cognitive sciences became for the age of automation. In the 1940s, Herbert Simon worked on theories of management, the field of research originated by Taylor. Having recognized the increasing importance of mental skills in the corporate workplace, Simon became one of the pioneers of cognitive science with his work on automatic reasoning by computer. In 1964 he wrote that "the bulk of productive wealth consists of programs...stored in human minds." [14] Another pioneer of cognitive science was Jerome Bruner. Reflecting back on his work in the 1950s, he noted in 1983: "It seems plain to me now that the 'cognitive revolution'...was a response to the technological demands of the 'post-industrial revolution.' You cannot properly conceive of managing a complex world of information without a workable concept of mind." [15] The replacement of manual work by cognitive work is directly related to automation. Already in 1961, in an influential study of automation in French industry, Pierre Naville and his fellow sociologists had described the transition from the "work of the laborer to the work of communication," work which became primarily "cognitive or semiotic." [16] In his summary of this study Rabinbach writes, "The appearance of the cerebral worker whose material and product is 'information' is emblematic of the vast distance traversed between the worker who surveys complex technologies of communication and the 'man-beef' of Taylor." [17] It is important to note that automation does not lead to the replacement of human by machine. Rather, the worker's role becomes one of monitoring and regulation: watching displays, analyzing incoming information, making decisions, and operating controls. And it is the corresponding human functions of perception, attention, memory, and problem solving which become the subject of research by new cognitive sciences. The rise of cognitive sciences is one aspect of the larger shift from industrial to post-industrial society and the corresponding new image of work and play: visual and mental processing of information rather than corporeal activity. A complimentary development is the emergence, during World War II, of the new discipline of applied experimental psychology, or, as it was also called, "human engineering." Human Engineering The gradual expansion of the practical applications of experimental psychology provides a precise map of the new occupations and new conditions of modern experience which call for perceptual skills. During World War I, England, Germany, and France utilized experimental psychologists to design and administer tests for aviation pilots, aeronautical, and airplane observers, hydrophone operators, and submarine "listeners-in." [18] During peacetime, a number of psychologists published papers on the readability of written text and of highway signs and on the visibility of lights at sea. [19] However, in the industrial world which conceived of the worker as a human motor and was largely concerned with the productivity of manual rather than perceptual labor, these studies were an exception rather than the mainstream rule. It was World War II which finally put to use the expertise of experimental psychologists. Why did this happen? The first textbook on applied experimental psychology (1949) opens by describing the recent origins of the field: For years experimental psychologists have worked diligently in academic laboratories studying man's capacities to perceive, to work, and to learn. Only very slowly, however, have the facts and methods which they have assembled been put to use in everyday life. A particularly glaring gap in modern technology, both industrial and military, is the lack of human engineering -- engineering of machines for human use and engineering of human tasks for operating machines. Motion-and-time engineers have been at work on many of these problems, but the experimental psychologist is also needed for his fundamental knowledge of human capacities and his methods of measuring human performance. The recent war put the spotlight on this gap. The war needed, and produced, many complex machines, and it taxed the resources of both the designer and operator in making them practical for human use. The war also brought together psychologists, physiologists, physicists, design engineers, and motion-and-time engineers to solve some of these problems. Though much of their work began too late to do any real good, it has continued on a rather large scale into the piece. Today, there are many groups busy with research on man-machine problems. They use different names to describe the work in its various aspects: biotechnology, biomechanics, psychoacoustics, human engineering, and systems research. Other names may be appropriate and may appear in the future. In casting about for a title for this book, we tried to select one that would describe the subject matter without the restrictive connotations attaching to some of the names mentioned above. Applied Experimental Psychology seems best to fill these requirements, because the traditional data and subject of experimental psychology are fundamental to this field. [20] Already before the war, experimental psychologists assisted in selecting military personnel for such jobs as pilot or airplane observer by administering special aptitude tests. During the war, a much greater number of pilots, radar operators and other similar personnel became needed. The emphasis was shifted, therefore, from selecting personnel with particularly good perceptual and motor skills to designing the equipment (controls, radar screens, dials, warning lights) to match the sensory capacities of an average person. [21] And it was the field of experimental psychology that possessed the knowledge about the sensory capacities of an average, statistical person: how visibility and acuity vary between day and night; how the ability to distinguish colors and brightness vary with illumination or distance; what the smallest amount of light is which can be reliably noticed; and so on. [23] All this data was now utilized for designing better displays and controls of the first modern human-machine systems such as high-speed aircrafts or radar installations. The development of these new human-machine systems during the war pushed human perceptual and mental performance to the limit and this was the second reason why experimental psychologists were called in. The performance of a human-machine system was limited by human information capacity to process information. In the words of the authors of _Applied Experimental Psychology_, We can make a machine that will do almost anything, given enough time and enough engineers. But man has limits to his developments, at least as far as we can see it. When we think how much a single radar can do in a small fraction of a second, and then realize by comparison that even the simplest form of reaction for a human being requires about a fifth of a second, we realize what we are up against... The full potential of radar, for example, lagged far behind physical developments because human operators could not master the complex operation of this machine system. We had to worry about such things as a new kind of visual signal -- very small and not very bright. [24] Considering that the authors described the work of time-and-motion engineers as directly leading to applied experimental psychology, this rhetoric can be expected. Taylor was impatient with the limitations of the body; now there was a similar impatience with the limitations of human information processing. With Taylor, it was the question of the speed of muscular movements; now, it became the question of reaction time: the minimum time in milliseconds required for an operator to detect a signal, to identify it, to press a control. In order to measure normal human sensory capacities, experimental psychologists have always put subjects in, so to speak, boundary conditions. They measured sensory thresholds, such as the least amount of light which can be detected. They also measured just noticeable differences (j.n.d.), the smallest difference between two stimuli which can be detected. Finally, they measured reaction times, the measure which became the main tool to deduce the time taken by different mental processes. In order to measure these characteristics, a number of standard experiments were designed, and they remained largely unchanged from the times of Weber, Fechner, and Wundt. In a detection experiment, the task of an observer is to detect the presence of barely visible stimuli, for instance a tiny light briefly flashed in the dark (did I see something?). In an identification experiment, the task is to identify which of possible stimuli was presented, for instance which of two colors (which one did I see?). In a recognition experiment, the task is to not only detect something, but to recognize what it is, for instance: what was the shape that briefly appeared (what did I see?) During World War II, the radar operator, the anti-aircraft gunner, the aircraft pilot found themselves in the same situations in which nineteenth century psychologists put their experimental subjects. The setups of psychophysical experiments became, in all details, the conditions of military work; the tasks devised by psychologists to study human vision became the actual tasks faced by the operators of human-machine systems. Like the subject of a detection experiment, a radar operator scans the radar screen for a barely noticeable dot of light. [25] Like the subject of an identification experiment, he has to try to guess whether this dot is the same or different from another dot which from his previous experience he knows to correspond to a friendly airplane. An anti-aircraft gunner is subjected to a recognition experiment, trying to identify a plane by its shape. And all of them, especially the pilot, are engaged in a sort of reaction time experiment. Thus, nineteenth century psychophysical setups became the military, and soon, civilian workplaces of post-industrial society; from there, they traveled back into laboratories, leading to such close interrelations between basic research in experimental psychology and its practical applications that they were no longer separable. For example, a 1947 article in American Psychologist describes the work of Naval Research Laboratory as following these three directions: "the design of gun fire control and missile control instruments from the point of view of ease and efficiency of operation; the design and evaluation of synthetic gunnery and missile control trainers; and basic psychological research." But what is meant here by "basic research"? We read that "at present, all basic research studies are aimed at the eye- hand coordination problem involved in target tracking." "Target tracking" is just one example of a military task which traveled into a psychological laboratory, and gradually become a standard psychophysical experiment. [26] The terms "applied experimental psychology," "human engineering" and "man- machine engineering" were replaced by another term standard today -- "human factors." The radar operator who in the 1940s and 1950s was the prototypical example of a human-machine system, was replaced by the 1980s by a new prototypical figure, the computer user. Thus, references to "human-machine systems" became references to "human-computer systems." The same amount of intellectual energy and research which in the middle of the century went into theorizing the performance of a radar operator and adapting him and radar display to each other, today goes into the work on computer interfaces. In retrospect then, we should recognize the radar operator as the central figure standing at the origins of post- industrial society, the figure which put directly into motion the new disciplines of the efficiency of the mind: engineering psychology, human information processing, and cognitive science. If radar screen of the 1940s was the first modern visual human-machine interface, VR gear is the most recent. While VR is commonly associated with the notions of escape from reality, unrestricted play and fantasy, in fact it is yet another development in the history of "human engineering." As an example, consider a popular photograph from the late 1980s which showcased virtual reality interface designed at NASA/Ames Human Factors Research Center. [27] The gear was constructed by human factors specialists, the direct descendants of the "human engineers" of the 1940s. The specialists utilised all the knowledge accumulated by psychology about the human vision in order to employ most efficiently. In the photograph we see the last leftover from the age of manual labor -- an arm in a DataGlove. It will soon disappear since through gaze tracking the operator can control the system by merely looking at different points in virtual space. Perceptual labor became the foundation of both work and play. NOTES 1. Walter Benjamin, "On Some Motives in Baudelaire," in Illuminations, ed. Hannah Arendt (New York: Schochen Books, 1969), 175. 2. On the connection between SIMNET and Battletech Centers, see Tony Reveaux, "Virtual Reality Gets Real," New Media (January 1993): 36-41. On VR entertainment systems in the context of location-based entertainment -- arcades and theme parks -- see Richard Cook, "Serious Entertainment," Computer Graphics World (May 1992): 40-48. 3. Alphonse Chapanis, Man-Machine Engineering (Bemont, CA: Wadsworth Publishing Company, Inc., 1965), 16. 4. A 1965 textbook on human-machine systems calls an automobile "a first rate example of a true man-machine system...a highly complex system in which the operator plays a commanding role or actively intervenes in the system from time to time." Chapanis, Man-Machine Engineering, 16. 5. Now, however, these shocks arrive exclusively through the visual channel (dials, computer screen, head-mounted display). Thus of the roles mentioned by Benjamin, it is the film viewer rather than the assembly line worker who directly anticipates the experience of an operator in this type of human- machine situation. 6. Anson Rabinbach, The Human Motor: Energy, Fatigue, and the Origins of Modernity (Basic Books, Inc., 1990), 3. 7. Ibid., 10. 8. Ibid., 68. 9. Frederick Winslow Taylor, The Principles of Scientific Managment (New York, 1967). 10. William R. Spriegel and Clark E. Myers, eds., The Writings of the Gilbreths (Homewood, IL., 1953). 11. Rabinbach, The Human Motor, 117, 277. 12. Marta Braun, Picturing Time: the Work of Etienne-Jules Marey (1830-1904) (Chicago: The University of Chicago Press, 1992), 337. 13. Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century of Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979), 27. 14. Qtd. in Douglas Noble, "Mental Materiel: The Militarization of Learning and Intelligence in U.S. Education," in Cyborg Worlds: the Military Information Society, ed. Les Levidov and Kevin Robins (London: Free Association Books, 1989), 34. 15. Qtd. in ibid., 34-35. 16. Qtd. in Rabinbach, The Human Motor, 298. 17. Ibid., 298. 18. Morris Viteles, Industrial Psychology (New York: W.W. Norton & Company, Inc., 1932), 43. 19. Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook of Experimental Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1287-1340. 20. Alphonse Chapanis, Wendell R. Garner, and Clifford T. Morgan, Applied Experimental Psychology (New York: John Wiley & Sons, Inc., 1949), v. Ibid., 8. 21. William Estes, "Experimental Psychology: an Overview," in The First Century of Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 1979), 630. 22. Chapanis, Applied Experimental Psychology, 7-8. 23. As Paul Fitts notes in his 1951 overview of engineering psychology, "radar operators are often forced to search for weak signals at near-threshold levels." Fitts, "Engineering Psychology and Equipment Design," 1290. 24. Franklin Taylor, "Psychology at the Naval Research Laboratory," American Psychologist 2, no. 3 (1947): 87, 91. 25. On NASA/Ames virtual reality research in the 1980s, see Scott S. Fisher, "Virtual Interface Environments," in The Art of Human-Computer Interface Design, ed. Brenda Laurel (Reading, Mass.: Addison-Wesley Publishing Company, 1990): 423-438.