Time and Bits: Managing Digital Continuity, Getty Center, Feb. 1998.
L to R: Peter Lyman, Alexander Rose, Ben Davis, Doug Carleston, Brian Eno, Jaron Lanier, Howard Besser, Margaret MacLean, Brewster Kaele, Kevin Kelly, Stewart Brand, Danny Hillis.
Mapping the Project, Grasping the Consequences
By Ben Howell Davis and Margaret MacLean
This was a very unhappy interface. And small wonder. No doubt this entire virtual environment was being encrypted, decrypted, reencrypted, anonymously routed through satellites and cables, emulated on alien machinery through ill-fitting, out-of-date protocols, then displayed through long-dead graphics standards. Dismembered, piped, compressed, packeted, unpacketed, decompressed, unpiped and re-membered. Worse yet, the place was old. Virtual buildings didn’t age like physical ones, but they aged in subtle pathways of arcane decline, in much the way that their owners did. A little bijou table in the corner had a pronounced case of bit-rot: from a certain angle it lost all surface tint.
Bruce Sterling, Holy Fire
Bruce Sterling is a journalist and science fiction author who founded and manages the Dead Media Project, an Internet mailing list about extinct forms of human communication. He was to attend the conference at the Getty Center that is described in this report, but could not, and he was missed.
In this vivid scenario from his most recent book, he comments darkly on the heroine’s attempt to interact with a virtual environment that had been rendered unintelligible by obsolete technology. The power of this novel resonates with the situation in which we find ourselves right now: the end of the most materially obsessed century in history is characterized by a ubiquitous technology that produces ephemera.
The fact is, computers solve many problems, but they create new ones. Around the world, decisions are being made to commit cultural memory to digital technology—and marking time with bits—under the assumption that someone, somewhere, is taking care of the details. What we are discovering is that "someone, somewhere" is not there. Not yet. Representing the Getty, through its Conservation and Information Institutes, we [Davis, MacLean] saw not just an opportunity, but a responsibility to make this fact known to our constituencies so that they could make informed decisions about their use of digital technology.
The mission of the Getty Conservation Institute is to further the appreciation and preservation of the world’s cultural heritage. As part of its commitment to enhance the philosophy and practice of conservation, the Institute undertakes research and applied projects, shares its knowledge base worldwide, and collaborates with partners to promote an informed awareness of and involvement in safeguarding the world’s cultural heritage. Within this context, we recognize the importance of providing sound guidance—when asked—about the means by which governments, organizations, and other authorities might best protect their own heritage.
The mission of the Getty Information Institute is to strengthen the presence, quality, and accessibility of cultural heritage information through global networks. The Communications Program of the Information Institute is focused on digital publication, digital design, and digital distribution. Inherent in those activities is the concern about how long what is produced digitally will last. And if we are committing cultural memory to digital technology, what risks are we taking?
Many communities are concerned with the longevity of digital materials, as you will see in this summation of the "Time and Bits: Managing Digital Continuity" project, that culminated in a meeting held at the Getty Center in February 1998. Among them, the library and archives constituency in particular has been making considerable progress in publicizing the issues. For this reason, we commissioned two of its luminaries—Howard Besser and Peter Lyman—to summarize where we are and how we got here, included as part of this report.
We also recognized that ensuring the survival of digitally recorded information—especially cultural information—would require the involvement of many groups, professions, businesses, and non-profit organizations. Indeed, it will take real cooperation and collaboration among these communities, a first for some of them.
It might be helpful to identify the focus of the discussions within the context of the larger array of information management issues that are such a challenge for so many of us. The revolution of digital information technologies has made it possible to ask and answer two questions that until now have seemed ludicrous: Is it possible to quantify the aggregate of all human information? And is it possible to save it digitally? Michael Lesk contributed a paper to the Time and Bits Web site before the meeting [since then noted physicist Philip Morrison has written a compelling article for Scientific American based on the paper (July 1998)] that asks and answers these very questions in digital terms. Lesk’s conclusion is that "soon after the year 2000 the production of disks and tapes will outrun human production of information to put on them." In other words, technically speaking, you could save everything. But how long would it last? And who or what decides to save it all? Given what we all know of political reality, both the possibility and the wisdom of having a central authority in charge of deciding what gets saved are inconceivable.
In their landmark report, the Task Force on Archiving of Digital Information provided an analysis of the digital landscape, focusing on features, including stakeholder interests, that affect the integrity of digital information objects and which determine the ability of digital archives to preserve such objects over the long term. The Task Force then introduces the principle that responsibility for archiving rests initially with the creator or owner of the information and that digital archives may invoke a fail-safe mechanism to protect culturally valuable information.
During the panel discussion, participant Brian Eno spoke of the possibility that without some general standards and rules, society might end up saving that which is easiest to save. Communities will need to decide what is valuable to them. What they need are methods to do so, with likely attention to three main kinds of documents [text, images, whatever] in ascending order of complexity...
Simple documents whose original form could be in bits or atoms;
Interactive documents that started out in digital form, whose purpose includes their connectedness to other documents, media, or server locations;
Documents created in atoms whose original version is precious; and/or that have, over time, accrued annotations, additions, smudges, strike-throughs, and other evidence of use and significance.
So, this project centers largely on mapping the problem. In approaching any problem, one can begin by dividing it in three parts: causes, consequences, and solutions. In this situation, the consequences are the clearest of the three. Discussions of causes and solutions are found later on in the report.
Consequences of the Problem
Deanna Marcum, President of the Council on Library and Information Resources, has observed that when a culture loses its memory, it loses its identity. In a letter to the editor of The Washington Post in January 1998, she explained "The Greater Digital Crisis" :
We run the risk that digital information will disappear. Indeed, portions of it already have become inaccessible. Either the media on which the information is stored are disintegrating, or the computer hardware and software needed to retrieve it from obsolete digital formats no longer exist. The extent of the problem will emerge as more and more records are requested for retrieval and cannot be read. There are already documented examples of this ....
Military files, including POW and MIA data from the Vietnam War, were nearly lost forever because of errors and omissions contained in the original digital records. Ten to twenty percent of vital data tapes from the Viking Mars mission have significant errors, because magnetic tape is too susceptible to degradation to serve as an archival storage medium.
Research conducted by the National Media Lab ... has shown that magnetic tapes, disks and optical CD-ROMs have relatively short lives and, therefore, questionable value as preservation media. The findings reveal that, at room temperature, top-quality data VHS tape becomes unreliable after 10 years, and average-quality CD-ROMs are unreliable after only five years. Compare those figures with a life of more than 100 years for archival-quality microfilm and paper. Current digital media are plainly unacceptable for long-term preservation.
Finding a late-model computer to read a 51/4-inch floppy disk—a format common only a few years ago—or the software to translate WordPerfect 4.0 is practically impossible. On government and industry levels, the problem is magnified: old DECtape and UNIVAC drives, which recorded vast amounts of government data, are long retired, and programs like FORTRAN II are historical curiosities.
The data stored by these machines in now-obsolete formats are virtually inaccessible. The year-2000 problem concerns only obsolete formats for storing dates. It is merely a snapshot of the greater digital crisis that puts future access to important government, business, and cultural data in such jeopardy.
Further consequences of the problem are shown clearly and directly in the powerful film Into the Future: Preservation of Information in the Electronic Age, by Terry Sanders.3
The film, about the hidden crisis of the digital information, shows some deeply troubling consequences of the problem, such as the badly deteriorated Navajo community archive of videotaped interviews with tribal elders. Intended as a legacy for the younger generations, these authentic and emotional conversations will soon be gibberish, if the tapes are not converted to some archival material and standard.
Filmmaker Sanders, who was kind enough to join the February meeting to present and take questions on his film, also included interviews with the inventors of the Information Age, including Peter Norton (Norton Utilities) and MacArthur Fellow and Internet creator Tim Berners-Lee. Even handled as it is in a low-key fashion, it is a sobering experience to witness prophets such as these acknowledging the enormity and seriousness of the problem—the lack of agreement, tools, or standards for ensuring the survival of cultural heritage in digital form.
Even with the consequences of this problem seeming so clear and so final, its solutions are not. They will need to accommodate the complexities of technology, the unforeseeability of the future, and the power of the profit motive. In fact, to get closer to solutions, we needed to delimit the subject so that some progress could take place. Needing to jump in somewhere, we determined that a structured, substantive conversation would be in order, among people who live comfortably on the brink of the millennium—both fascinated and troubled by the stage of technology in which we are currently stuck.
Gathering the Group
In order to get at the scope of the problem, we felt that it was necessary to do some "out of the academy" thinking. Our co-conspirator in this project is Stewart Brand, who founded the Whole Earth Catalog, and wrote The Media Lab: Inventing the Future at MIT. Brand brought in most of the Board of Directors of his Long Now Foundation, established in 1996 to foster long-term responsibility. A 10,000-year clock and library for the deep future are its founding projects. The directors are—among other things—all "generalists with a passion for detail."
Douglas G. Carlston is an attorney, and co-founder and CEO of Broderbund Software. As a businessman, Doug brought practical, economic insights to the table. "There’s a subtext here which is purely economic. Which is, if you don’t have a constituency representing that body of information going forward, you don’t necessarily have people with the economic motive or incentive to maintain it continuously." He wrote "Storing Knowledge," as part of the preparatory material for the conference discussion.
The British musician, producer, and artist Brian Eno continues to break new ground in fusing the worlds of art and technology in his compositions and performances. He contributed his observations and philosophies through tales of his creative and technological successes and frustrations. "One of the problems is that when digital material corrupts, it corrupts absolutely. We’ve been used to analog materials which deteriorate rather gracefully....[A] tape played over and over again does lose high frequencies. But you can still tell what the music is, you can still hear the song. What has happened with some of my earliest digital recordings is that they have become absolutely silent. There’s nothing there at all. A lot of people are finding that in all sorts of ways they prefer the type of deterioration that characterizes atoms to the type of deterioration that characterizes bits."
Danny Hillis is Vice President of Research and Development at Walt Disney Imagineering, and a Disney Fellow. He was deeply involved in the invention and development of parallel processing, and was co-founder and chief scientist of Thinking Machines Corporation. Danny brought to the conversation his unique perspective on the big picture. "I have a different view of technology than Alan Kay’s, which is, technology is the stuff that doesn’t work yet. There was a time when the violin was technology. And in fact, what we’re dealing with is that digital technology has proved so useful that we’ve started using it before it really works."
Kevin Kelly is Executive Editor of Wired and the author of Out of Control: The New Biology of Machines, Social Systems and the Economic World.6 Among other things, Kevin contributed several conceptualizations of possible approaches to managing the abundance of information we have. "The Internet is basically the largest Xerox machine in the world. If something can be copied it will be copied. On the Internet, it goes everywhere. But what it doesn’t do is it doesn’t go forward in time very well. So that’s the problem that we have."
Paul Saffo is the Director of the Institute for the Future, a 29-year-old research and forecasting foundation located in Menlo Park, California. Unexpectedly, El Niño made Paul’s participation in the conference impossible.
Jaron Lanier is a leading computer scientist and early pioneer in the development of virtual reality technology. He created the interactive video program "Moon Dust," among others. He is also an accomplished musician and composer. Jaron believes strongly that digital information must be kept in perpetual use in order for it to survive. Said simply, "unused data dies."
Howard Besser, Adjunct Associate Professor at UC Berkeley’s School of Information Management and Systems, specializes in digital archiving. His multidisciplinary approach led him to identify several points of inquiry: the viewing problem; the scrambling problem; the inter-relation problem; the custodial problem, and the translation problem. These categories provide a useful structure for examination of the issues. Together with his Berkeley colleague Peter Lyman, he co-authored the conference background paper.
Peter Lyman is the University Librarian, and Professor in the School of Information Management and Systems at UC Berkeley. Peter shared his insights concerning digital preservation with participants as not only a librarian, but as an ethnographer as well. Several of his observations related to social and cultural organization. "So I think the question is: What kind of organizational matrix or cultural orientation can make people want to preserve these things?"
Brewster Kahle is the inventor and founder of Wide Area Information Servers, Inc. [WAIS]—recently acquired by America Online—the Internet Archive, and an on-screen search service called Alexa. Kahle brought with him Alan Rath’s sculpture The World Wide Web, 1997: Two Terabytes in 63 Inches. The ingenious sculpture captures a moment in the life of the Web and crystallizes in physical form many of the challenges presented by the expanding body of digital information and objects. In his words, "I think the issue that we are grappling with here is now that our cultural artifacts are in digital form, what happens?"
The Internet Archive raises the possibility of saving everything. It downloads the entire Internet each day (about 3 terabytes) onto digital tape and saves it. The Archive will provide historians, researchers, scholars, and others access to the vast collection of data on the Web, and ensure the longevity of this information.
Defining the Problem Space
This group was charged with defining the "problem space." In engineering parlance, a problem space is composed of near-term problems that could be solved with tools available today, a middle-term problem set that will take some innovation in today’s tools to get through, and long-term troubles whose solutions will require whole new ways of thinking.
The most dramatic images to come out of the two days were the problem-space diagrams introduced by Kevin Kelly and Danny Hillis, and amended by everyone else.
Figure 1 is the simple version: At opposite ends of one axis are bits and artifacts: that is, information in digital form as opposed to actual things. Intersecting that axis are the polar opposites of digital information that is precious (curated?) and all digital information. Add to that a third axis of digital information that is interactive (applications) and digital information that is static (text or digital still pictures). In this simplified problem space the preservation of artifacts, interactive media, saving everything is very problematic. Obviously this leaves us with static, precious bits as a workable problem. For instance, putting the pages of the 100 most important books on a CD-ROM.
A more expanded version of the problem space (Figure 2) is a multi-axis system composed of human-scale pieces (a text file for instance) vs. huge pieces (all of NASA’s data), information that is used a lot vs. information that will be used rarely, a longevity time span of 50 years vs. 10,000 years, static vs. interactive data, and finally digital information where the creator is cooperative in being responsible about using long-term media and the situation where they don’t care at all how long it lasts.
In this problem space (Figure 2.) the long-term problems that will require very different ways of thinking are the situations where information is rarely used, will need to last 10,000 years, are huge in scale, are interactive, and where the creator has been uncooperative. The near-term problem that is more solvable is a data set that is used a lot, is static, is human scale, will only need to last for 50 years, and the creator has been cooperative.
Meeting at the Getty Center
The participants arrived in Los Angeles on Sunday, 8 February, and worked through to Tuesday evening. Together with Stewart Brand, we organized the time on Monday and Tuesday as a series of meetings. All day Monday was a closed session of free-ranging discussion, starting with summary position statements by each participant, and continuing with defining the problem. A summary of the conversations constitutes "Setting the Stage," herein. On Tuesday morning, the meeting participants viewed—again—the Sanders film, revisited the progress made since the end of the previous day, and organized themselves for the afternoon session. On Tuesday afternoon, after a luncheon with a dozen or so members of the press, the group convened for a public discussion of the subject and the progress made in the closed meetings. Present were many journalists; specialists in information management; and library and archives professionals from regional universities, corporations, entertainment companies, foundations, and research laboratories. We also welcomed interested staff, and a few fortunate visitors to the Getty Center. A transcript of the afternoon event comprises "Public Session: Panel Discussion."
The project and the meeting, and the unexpectedly strong response since then, support the idea of the three partners that there is a growing recognition that this interesting, troubling, large-scale problem ahead will affect virtually everyone who uses computers. It is also becoming clear that neither the problems nor the solutions are simply technical. As Michèle Cloonan, Chair of the Department of Library and Information Science at UCLA, noted in her essay "The Preservation of Knowledge":
...the emergence of electronic information will result in a fundamentally different way of approaching ... preservation ... which has been object-based (books, broadsides, maps, etc.) and time-oriented (permanent durable paper should last for at least for at least 250 years). Thus the notion of saving object X for Y years may become obsolete. We will need to secure the longevity of information so that the information itself does not disappear. And it must be done in concert with librarians, publishers, manufacturers and anyone else involved in the handling of information.
The future plan for Time and Bits at the Getty Center is to continue the conversation, make the signal brighter.
1. Bruce Sterling, Holy Fire (New York: Bantam Books, 1997): 31
2. Donald Waters and John Garrett, Preserving Digital Information, Report of the Task Force on Archiving of Digital Information [www.clir.org/pubs/abstract/pub63.html].
3. For more information regarding either of Sanders’ films on this subject—Into the Future: On the Preservation of Knowledge in the Electronic Age; or Slow Fires: On the Preservation of the Human Record—please contact the American Film Foundation at (310) 459-2116.
4. Stewart Brand, The Media Lab: Inventing the Future at MIT, (New York: Penguin Books, 1988).
6. Kevin Kelly, Out of Control: The New Biology of Machines, Social Systems and the Economic World (New York: Addison-Wesley Publishing Co., 1995).
Copyright, J. Paul Getty Trust, 1998.