Lev Manovich

Tools for the Analysis and Visualization of Large Image and Video Collections for the Humanities

Project team:

PI: Dr. Lev Manovich, Professor of Visual Arts, University of California, San Diego (UCSD); 
Director, Software Studies Initiative, California Institute for Telecommunications and Information Technology (Calit2.

Almila Akdag, Postdoctoral Researcher, e-Humanities Group, The Royal Netherlands Academy of Arts and Sciences; Visiting Scholar, Visual Arts and Communication Design, Sabanci University, Istanbul, Turkey.

Loretta Auvil, Senior Project Coordinator at Illinois Informatics Institute, University of Illinois; SEASR co-PI. 

Jeremy Douglass, Technical Director, Software Studies Initiative, UCSD.

Elizabeth Losh, Director of Academic Programs, Sixth College, Program in Culture, Art, and Technology, UCSD.

Project summary:

Since 2008, Software Studies Initiative at California Institute for Telecommunications and Information Technology (Calit2) and University of California, San Diego (UCSD) has been developing a comprehensive set of software tools for the quantitative analysis and visualization of large collections of images and video. The tools were designed for academic researchers in the humanities, and have already been used by scholars in a number of disciplines including art history, archeology, film and media studies, dance studies, and game studies. We have also been working with a number of prominent cultural institutions and collections including the Library of Congress, Getty Research Institute, the Austrian Film Museum, and the Netherlands Institute for Sound and Image, in using our techniques with their collections and data sets. The software development and its applications has received support from the National Endowment for the Humanities (NEH), the National Science Foundation (NSF), the University of California Humanities Research Institute (UCHRI), UC San Diego, and the California Institute for Telecommunications & Information Technologies (Calit2).

In our new three year project funded by $477,000 grant from The Andrew W. Mellon Foundation, we will work to fully integrate our techniques and tools into the SEASR/Meandre environment, a major platform for digital humanities research developed with key support from the Andrew W. Mellon Foundation. The integrated tools will come with comprehensive documentation and a set of examples covering a number of fields in the humanities and humanistic social sciences. This integration will address a current goal of SEASR to “continue to evolve to include processing of images and other multimedia data formats.” We anticipate these tools being used by an ever-expanding range of people, including academics and students in the humanities and humanistic social sciences, museum curators and visitors, and cultural creators who want to better understand how their work fits within a larger context

In addition to making available to others software tools, accessible user interfaces, documentation, and examples, Software Studies Initiative will also collaborate with other researchers to carry out large-scale case studies. Each case study will demonstrate how, within a particular field, quantitative analysis and visualization of images and/or video can open new research possibilities for that field. Each study will include documentation of the appropriate SEASR workflows, a paper describing the data, the methods used, the findings, and high-resolution still and animated visualizations: 

Almila Akdag will lead the case study which will combine network analysis and image processing to explore a few million images and user data from deviantArt (the most popular social network for user-generated art).

Jeremy Douglass will lead the analysis of our one million manga images dataset. 

Elizabeth Losh will lead the case study which applies our methods to thousands of hours of political video on the web and TV news. 

Over 200 undergraduate and graduate UCSD students will participate in the project over its three year period, exploring selected data sets as part of their classes in visualization and computational art history, and digital humanities.

Contact:

Lev Manovich, Director, Software Studies Initiative [manovich@ucsd.edu]

More information:

Our methods for the analysis and visualization of large visual data sets

Our projects (analysis of image sets covering video games, visual art, graphic design, maagzines, newspapers, comic books, TV, films, animation, motion graphics.) 

http://www.flickr.com/photos/culturevis/collections/ (Over 900 visualizations and sketches from our lab)

Our open source software tools (digital image processing and visualization of image sets of any size.)

Case study: One million manga pages

Pilot project: Digging Into Global News

The World’s Technological Capacity to Store, Communicate, and Compute Information.

Martin Hilbert1 and Priscila López.

Science, February 10, 2011.

Abstract:

We estimate the world’s technological capacity to store, communicate, and compute information, tracking 60 analog and digital technologies during the period from 1986 to 2007. In 2007, humankind was able to store 2.9 × 1020 optimally compressed bytes, communicate almost 2 × 1021 bytes, and carry out 6.4 × 1018 instructions per second on general-purpose computers. General-purpose computing capacity grew at an annual rate of 58%. The world’s capacity for bidirectional telecommunication grew at 28% per year, closely followed by the increase in globally stored information (23%). Humankind’s capacity for unidirectional information diffusion through broadcasting channels has experienced comparatively modest annual growth (6%). Telecommunication has been dominated by digital technologies since 1990 (99.9% in digital format in 2007), and the majority of our technological memory has been in digital format since the early 2000s (94% digital in 2007).

Illustration from the article in Washington Post about this research:

Data Visualization and Computational Art History

Course syllabus

UCSD
Spring 2012
Visual Arts Department,UCSD

undergraduate course: VIS 149 / ICAM 130: Special Topics
graduate course: VIS 219: Special Topics

Comparing van Gogh paintings done in Paris and Arles.
X-axis = median brightness. Y-axis=median saturation.
Software: ImagePlot (developed by Software Studies Initiative directed by Lev Manovich).

DOWNLOAD:

Lev Manovich. How to Follow Software Users? (Digital Humanites, Software Studies, Big Data).

Abstract:

Big data is the new media of 2010s. Like previous waves of computer technologies, it changes what it means to know something and how we can generate this knowledge. So far, all big data projects in digital humanities that I am aware of used digitized cultural artifacts from the past. If we want to apply the big data paradigm to the study of contemporary interactive software-driven media, we are facing fascinating theoretical questions and challenges. What exactly is “big data” in the case of interactive media? How do we study the interactive temporal experiences of the users, as opposed to only analyzing the code of software programs and contents of media files? This article provides possible answers to these questions and proposes a methodology for the study of interactive media as “big data.”

Reference:  This new article is not published anywhere yet. If you want to reference it, use the URL of this post.

SOFTWARE:

ImageMontage (released 3/30/2012)

ImageSlice (released 3/30/2012)

ImagePlot (released 9/2011)

EXAMPLES:

28-page PDF illustrates three key techniques used in our lab (softwarestudies.com) to explore large image and video sets: montage, slice, and image plot.:

Visualizing Image and Video Collections: Examples

TUTORIALS:

To learn how to use montage and slice visualization techniques with your datasets, consult:

Visualizing image and video collections: tutorials

THEORY:

Lev Manovich. Media Visualization: Visual Techniques for Exploring Large Media Collections. (released 3/30/2012)

Article about the theory and methods of media visualization, with the analysis of the examples.

More information about our digital humanities projects:
softwarestudies.com

Members of Software Studies Initiative exploring a visualization of a collection of 113 video clips.

DOWNLOAD:

Lev Manovich. Media Visualization: Visual Techniques for Exploring Large Media Collections.

This new text presents the theory and the techniques of media visualization used in our lab, with the analysis of the examples.

Exploring visualization of  4525 Time maagzine covers on a super high resolution display.

Mobility Shifts NYC conference full schedule

Friday, October 14

10:00 am - 12:30 pm
Wollman Hall, Eugene Lang Building, 65 West 11th St., 5th floor

Lev Manovich (Software Studies Initiative, UC San Diego)

Data Literacy and Cultural Analytics

The joint availability of numerous large data sets on the web and free tools for data scraping, cleaning, analyzing and visualizing enable potentially anybody to become a citizen data miner. But how do we enable this in practice? What are the necessary elements of “data literacy”? How do we inspire students in traditionally non-quantitative fields (art history, film and media studies, literary studies, etc.) to start playing with big data?

One the limitations of the existing popular data analysis and visualization tools is that they are designed to work with numbers and texts – but not images and video. To close this gap, In 2007 we have established Software Studies Initiative (softwarestudies.com) at University of California, San Diego. The lab’s focus in on development of new visualization methods particularly suited for media teaching and research. In my presentation I will show a sample of our projects including visualization of art, film, animation, video games,magazines, comics, manga, and graphic design. Our image sets range from 4535 covers of Time magazine to 320,000 Flickr images from “ArtNow” and “Graphic Design” groups, and one million manga pages.

In September 2011 we released ImagePlot - free software tool that visualizes collections of images and video of any size. I will discuss how we use Image Plot in classes with both undergraduate and graduate students to create collaborative projects which reveal unexpected cultural trends and also make us question our existing concepts for understanding visual culture and media.

Saturday, October 15

1:30-4:30 pm: Progressive Digital Pedagogy: Remix, Collaboration, Crowdsourcing
Wollman Hall, Eugene Lang Building 65 West 11th St., 5th floor

Elizabeth Losh (Sixth College, UC San Diego)

In recent years progressive digital pedagogy has borrowed from five major aspects of the popular culture developing around computational media: 1) remix practice, 2) multimodality, 3) accelerated response, 4) crowd sourcing, and 5) narrowcasting. Yet for many years the conventional classroom pedagogy around teaching “current events” has remained unchanged: it still generally focuses on having learners mechanically cut out recent news stories produced by traditional print journalists with little attention to how the news is made, how it remixes sources, how it appeals to particular audiences, or how particular patterns of visual imagery and verbal rhetoric could be analyzed critically. This talk focuses on recent work by the Software Studies initiative at U.C. San Diego by the Cultural Analytics group and shows how media visualization and crowd sourcing could be used in educational contexts with large publically accessible libraries of digitized news and smaller archives of government public information videos.

ImagePlot is a free software tool for visualizing collections of images and video of any size. (The largest set we tried so was: 1,074,790 one megabyte images).

DOWNLOAD IMAGEPLOT 0.9

ImagePlot was developed by the Software Studies Initiative (softwarestudies.com) with support from the National Endowment for Humanities (NEH), the California Institute for Telecommunications and Information Technology (Calit2), and the Center for Research in Computing and the Arts (CRCA).

Along with the program, we also distribute a number of articles by Lev Manovich, Jeremy Douglass and Tara Zepel that address methodologies for exploring large visual cultural data sets, and discuss our digital humanities projects which use ImagePlot. (The articles can be also downloaded directly from softwarestudies.com.)

Visualizations created with ImagePlot have been shown in science centers, art and design museums, and art galleries, including Graphic Design Museum (Breda, Netherlands), Gwangju Design Biennale (Korea), and The San Diego Museum of Contemporary Art.

ImagePlot software was developed as part of our Cultural Analytics research program.

ImagePlot works on Mac, Windows, and Lunix.

Max visualization resolution: 2.5 GB (2,684,354,560 grayscale pixels, or 671,088,640 RGB pixels).

Lev Manovich. Style Space: How to compare image sets and follow their evolution (part 2).

(part 1 is here)

[august 4-14, 2011]

selected points (see complete text for details)

Many social and natural processes follow a familiar Bell curve (normal
distribution). What are the shapes of distributions of large cultural
data sets? Because humanists only recently started to work
with big data sets, it is too early to make any generalizations. However,
it would not be surprising if the distributions of features of
very large cultural sets do follow the Bell curve pattern:
a dense cluster containing most of the data, gradually falling off to the side,
and a large very sparse area.

If we want to visually compare two or more image sets to each other in
relation to two visual properties, we can project them into a 2D space
defined by these visual properties as we did with Piet Mondrian’s and
Mark Rothko’s paintings in part 1. Using min and max values of the measured
properties of all images in out sets combined as the boundaries of the
visualization will allow us to use the visualization area most
efficiently. However, if we want to understand the footprint of each image set in
relation to the absolute mix and max - i.e. lowest and highest
possible values of visual features of all possible images - we need to
map our images differently.

A related idea is to render parts of an image set over the background showing
the complete set. This allows us to see the footprint of the these parts
in relation to the larger footprint of all images. For example, we can compare
pages of two manga titles from our complete set
of 883 titles comprising 1,074,790 pages.

new article:

Lev Manovich.
Style Space: How to compare image sets and follow their evolution (part 1)

[august 4-6, 2011]

selected points (see complete text for details)

A style space is a projection of quantified properties of a
set of cultural artifacts (or their parts) into a 2D place. X and Y
represent the properties (or their combinations). The position of
each artifact is determined by its values for these properties.

We are not claiming that such representations can capture all
aspects of a visual styke. A “style space” representation is
a tool for exploring image sets. It is particularly effective for
large sets.) It allows us compare all images in a set (or sets).
according to their visual values.

Separating a “style” into distinct visual dimensions and
organizing images according to their values on these dimensions
allows us to see more clearly how differences between the images
in a set. Visual differences are translated into spatial distances.
Images which are visually similar will be close; images which
are different will be further away.