Time
and Bits: Managing Digital Continuity, Getty Center, Feb. 1998.
L to R: Peter Lyman, Alexander
Rose, Ben Davis, Doug Carleston, Brian Eno, Jaron Lanier, Howard Besser,
Margaret MacLean, Brewster Kaele, Kevin Kelly, Stewart Brand, Danny Hillis.
See also: Wagner letter,
2-9-2000
Introduction:
Mapping the Project, Grasping the
Consequences
By Ben Howell Davis and Margaret
MacLean
This was a
very unhappy interface. And small wonder. No doubt this entire virtual
environment was being encrypted, decrypted, reencrypted, anonymously routed
through satellites and cables, emulated on alien machinery through ill-fitting,
out-of-date protocols, then displayed through long-dead graphics standards.
Dismembered, piped, compressed, packeted, unpacketed, decompressed, unpiped and
re-membered. Worse yet, the place was old. Virtual buildings didn’t age like
physical ones, but they aged in subtle pathways of arcane decline, in much the
way that their owners did. A little bijou table in the corner had a pronounced
case
of bit-rot: from a certain angle it lost all surface tint.
Bruce
Sterling, Holy Fire
Bruce Sterling
is a journalist and science fiction author who founded and manages the Dead
Media Project, an Internet mailing list about extinct forms of human
communication. He was to attend the conference at the Getty Center that is
described in this report, but could not, and he was missed.
In this vivid
scenario from his most recent book, he comments darkly on the heroine’s attempt
to interact with a virtual environment that had been rendered unintelligible by
obsolete technology. The power of this novel resonates with the situation in
which we find ourselves right now: the end of the most materially obsessed century
in history is characterized by a ubiquitous technology that produces ephemera.
The fact is,
computers solve many problems, but they create new ones. Around the world,
decisions are being made to commit cultural memory to digital technology—and
marking time with bits—under the assumption that someone, somewhere, is taking
care of the details. What we are discovering is that "someone,
somewhere" is not there. Not yet. Representing the Getty, through its
Conservation and Information Institutes, we [Davis, MacLean] saw not just an
opportunity, but a responsibility to make this fact known to our constituencies
so that they could make informed decisions about their use of digital
technology.
The mission of
the Getty Conservation Institute is to further the appreciation and
preservation of the world’s cultural heritage. As part of its commitment to
enhance the philosophy and practice of conservation, the Institute undertakes
research and applied projects, shares its knowledge base worldwide, and
collaborates with partners to promote an informed awareness of and involvement
in safeguarding the world’s cultural heritage. Within this context, we
recognize the importance of providing sound guidance—when asked—about the means
by which governments, organizations, and other authorities might best protect
their own heritage.
The mission of
the Getty Information Institute is to strengthen the presence, quality, and
accessibility of cultural heritage information through global networks. The
Communications Program of the Information Institute is focused on digital
publication, digital design, and digital distribution. Inherent in those
activities is the concern about how long what is produced digitally will last.
And if we are committing cultural memory to digital technology, what risks are
we taking?
Many
communities are concerned with the longevity of digital materials, as you will
see in this summation of the "Time and Bits: Managing Digital
Continuity" project, that culminated in a meeting held at the Getty Center
in February 1998. Among them, the library and archives constituency in
particular has been making considerable progress in publicizing the issues. For
this reason, we commissioned two of its luminaries—Howard Besser and Peter
Lyman—to summarize where we are and how we got here, included as part of this
report.
We also
recognized that ensuring the survival of digitally recorded
information—especially cultural information—would require the involvement of
many groups, professions, businesses, and non-profit organizations. Indeed, it
will take real cooperation and collaboration among these communities, a first
for some of them.
It might be
helpful to identify the focus of the discussions within the context of the
larger array of information management issues that are such a challenge for so
many of us. The revolution of digital information technologies has made it
possible to ask and answer two questions that until now have seemed ludicrous:
Is it possible to quantify the aggregate of all human information? And is it possible
to save it digitally? Michael Lesk contributed a paper to the Time and Bits Web
site before the meeting [since then noted physicist Philip Morrison has written
a compelling article for Scientific American based on the paper (July 1998)]
that asks and answers these very questions in digital terms. Lesk’s conclusion
is that "soon after the year 2000 the production of disks and tapes will
outrun human production of information to put on them." In other words,
technically speaking, you could save everything. But how long would it last?
And who or what decides to save it all? Given what we all know of political
reality, both the possibility and the wisdom of having a central authority in
charge of deciding what gets saved are inconceivable.
In their landmark
report, the Task Force on Archiving of Digital Information provided an analysis
of the digital landscape, focusing on features, including stakeholder
interests, that affect the integrity of digital information objects and which
determine the ability of digital archives to preserve such objects over the
long term. The Task Force then introduces the principle that responsibility for
archiving rests initially with the creator or owner of the information and that
digital archives may invoke a fail-safe mechanism to protect culturally
valuable information.
During the
panel discussion, participant Brian Eno spoke of the possibility that without
some general standards and rules, society might end up saving that which is
easiest to save. Communities will need to decide what is valuable to them. What
they need are methods to do so, with likely attention to three main kinds of
documents [text, images, whatever] in ascending order of complexity...
Simple
documents whose original form could be in bits or atoms;
Interactive
documents that started out in digital form, whose purpose includes their
connectedness to other documents, media, or server locations;
Documents
created in atoms whose original version is precious; and/or that have, over
time, accrued annotations, additions, smudges, strike-throughs, and other
evidence of use and significance.
So, this
project centers largely on mapping the problem. In approaching any problem, one
can begin by dividing it in three parts: causes, consequences, and solutions.
In this situation, the consequences are the clearest of the three. Discussions
of causes and solutions are found later on in the report.
Consequences
of the Problem
Deanna Marcum,
President of the Council on Library and Information Resources, has observed that
when a culture loses its memory, it loses its identity. In a letter to the
editor of The Washington Post in January 1998, she explained "The Greater
Digital Crisis" :
We run the risk
that digital information will disappear. Indeed, portions of it already have
become inaccessible. Either the media on which the information is stored are
disintegrating, or the computer hardware and software needed to retrieve it
from obsolete digital formats no longer exist. The extent of the problem will
emerge as more and more records are requested for retrieval and cannot be read.
There are already documented examples of this ....
Military files,
including POW and MIA data from the Vietnam War, were nearly lost forever
because of errors and omissions contained in the original digital records. Ten
to twenty percent of vital data tapes from the Viking Mars mission have
significant errors, because magnetic tape is too susceptible to degradation to
serve as an archival storage medium.
Research
conducted by the National Media Lab ... has shown that magnetic tapes, disks
and optical CD-ROMs have relatively short lives and, therefore, questionable
value as preservation media. The findings reveal that, at room temperature,
top-quality data VHS tape becomes unreliable after 10 years, and
average-quality CD-ROMs are unreliable after only five years. Compare those
figures with a life of more than 100 years for archival-quality microfilm and
paper. Current digital media are plainly unacceptable for long-term
preservation.
Finding a late-model
computer to read a 51/4-inch floppy disk—a format common only a few years
ago—or the software to translate WordPerfect 4.0 is practically impossible. On
government and industry levels, the problem is magnified: old DECtape and
UNIVAC drives, which recorded vast amounts of government data, are long
retired, and programs like FORTRAN II are historical curiosities.
The data stored
by these machines in now-obsolete formats are virtually inaccessible. The
year-2000 problem concerns only obsolete formats for storing dates. It is
merely a snapshot of the greater digital crisis that puts future access to
important government, business, and cultural data in such jeopardy.
Further
consequences of the problem are shown clearly and directly in the powerful film
Into the Future: Preservation of Information in the Electronic Age, by Terry
Sanders.3
The film, about
the hidden crisis of the digital information, shows some deeply troubling
consequences of the problem, such as the badly deteriorated Navajo community
archive of videotaped interviews with tribal elders. Intended as a legacy for
the younger generations, these authentic and emotional conversations will soon
be gibberish, if the tapes are not converted to some archival material and
standard.
Filmmaker
Sanders, who was kind enough to join the February meeting to present and take
questions on his film, also included interviews with the inventors of the
Information Age, including Peter Norton (Norton Utilities) and MacArthur Fellow
and Internet creator Tim Berners-Lee. Even handled as it is in a low-key
fashion, it is a sobering experience to witness prophets such as these
acknowledging the enormity and seriousness of the problem—the lack of
agreement, tools, or standards for ensuring the survival of cultural heritage
in digital form.
Even with the
consequences of this problem seeming so clear and so final, its solutions are
not. They will need to accommodate the complexities of technology, the
unforeseeability of the future, and the power of the profit motive. In fact, to
get closer to solutions, we needed to delimit the subject so that some progress
could take place. Needing to jump in somewhere, we determined that a
structured, substantive conversation would be in order, among people who live
comfortably on the brink of the millennium—both fascinated and troubled by the
stage of technology in which we are currently stuck.
Gathering
the Group
In order to get
at the scope of the problem, we felt that it was necessary to do some "out
of the academy" thinking. Our co-conspirator in this project is Stewart
Brand, who founded the Whole Earth Catalog, and wrote The Media Lab: Inventing
the Future at MIT. Brand brought in most of the Board of Directors of his Long
Now Foundation, established in 1996 to foster long-term responsibility. A
10,000-year clock and library for the deep future are its founding projects.
The directors are—among other things—all "generalists with a passion for
detail."
Douglas G.
Carlston
is an attorney, and co-founder and CEO of Broderbund Software. As a
businessman, Doug brought practical, economic insights to the table.
"There’s a subtext here which is purely economic. Which is, if you don’t
have a constituency representing that body of information going forward, you
don’t necessarily have people with the economic motive or incentive to maintain
it continuously." He wrote "Storing Knowledge," as part of the
preparatory material for the conference discussion.
The British
musician, producer, and artist Brian Eno continues to break new ground in fusing
the worlds of art and technology in his compositions and performances. He
contributed his observations and philosophies through tales of his creative and
technological successes and frustrations. "One of the problems is that
when digital material corrupts, it corrupts absolutely. We’ve been used to
analog materials which deteriorate rather gracefully....[A] tape played over
and over again does lose high frequencies. But you can still tell what the
music is, you can still hear the song. What has happened with some of my
earliest digital recordings is that they have become absolutely silent. There’s
nothing there at all. A lot of people are finding that in all sorts of ways
they prefer the type of deterioration that characterizes atoms to the type of
deterioration that characterizes bits."
Danny Hillis is Vice President of
Research and Development at Walt Disney Imagineering, and a Disney Fellow. He
was deeply involved in the invention and development of parallel processing,
and was co-founder and chief scientist of Thinking Machines Corporation. Danny
brought to the conversation his unique perspective on the big picture. "I
have a different view of technology than Alan Kay’s, which is, technology is
the stuff that doesn’t work yet. There was a time when the violin was
technology. And in fact, what we’re dealing with is that digital technology has
proved so useful that we’ve started using it before it really works."
Kevin Kelly is Executive Editor of Wired
and the author of Out of Control: The New Biology of Machines, Social Systems
and the Economic World.6 Among other things, Kevin contributed several
conceptualizations of possible approaches to managing the abundance of
information we have. "The Internet is basically the largest Xerox machine
in the world. If something can be copied it will be copied. On the Internet, it
goes everywhere. But what it doesn’t do is it doesn’t go forward in time very
well. So that’s the problem that we have."
Paul Saffo is the Director of the
Institute for the Future, a 29-year-old research and forecasting foundation
located in Menlo Park, California. Unexpectedly, El Niño made Paul’s
participation in the conference impossible.
Jaron Lanier is a leading computer
scientist and early pioneer in the development of virtual reality technology.
He created the interactive video program "Moon Dust," among others.
He is also an accomplished musician and composer. Jaron believes strongly that
digital information must be kept in perpetual use in order for it to survive.
Said simply, "unused data dies."
Howard
Besser,
Adjunct Associate Professor at UC Berkeley’s School of Information Management
and Systems, specializes in digital archiving. His multidisciplinary approach
led him to identify several points of inquiry: the viewing problem; the
scrambling problem; the inter-relation problem; the custodial problem, and the
translation problem. These categories provide a useful structure for
examination of the issues. Together with his Berkeley colleague Peter Lyman, he
co-authored the conference background paper.
Peter Lyman is the University
Librarian, and Professor in the School of Information Management and Systems at
UC Berkeley. Peter shared his insights concerning digital preservation with
participants as not only a librarian, but as an ethnographer as well. Several
of his observations related to social and cultural organization. "So I
think the question is: What kind of organizational matrix or cultural
orientation can make people want to preserve these things?"
Brewster
Kahle
is the inventor and founder of Wide Area Information Servers, Inc.
[WAIS]—recently acquired by America Online—the Internet Archive, and an
on-screen search service called Alexa. Kahle brought with him Alan Rath’s sculpture
The World Wide Web, 1997: Two Terabytes in 63 Inches. The ingenious sculpture
captures a moment in the life of the Web and crystallizes in physical form many
of the challenges presented by the expanding body of digital information and
objects. In his words, "I think the issue that we are grappling with here
is now that our cultural artifacts are in digital form, what happens?"
The Internet
Archive raises the possibility of saving everything. It downloads the entire
Internet each day (about 3 terabytes) onto digital tape and saves it. The
Archive will provide historians, researchers, scholars, and others access to
the vast collection of data on the Web, and ensure the longevity of this
information.
Defining the
Problem Space
This group was
charged with defining the "problem space." In engineering parlance, a
problem space is composed of near-term problems that could be solved with tools
available today, a middle-term problem set that will take some innovation in
today’s tools to get through, and long-term troubles whose solutions will
require whole new ways of thinking.
The most
dramatic images to come out of the two days were the problem-space diagrams
introduced by Kevin Kelly and Danny Hillis, and amended by everyone else.
Figure 1 is the
simple version: At opposite ends of one axis are bits and artifacts: that is,
information in digital form as opposed to actual things. Intersecting that axis
are the polar opposites of digital information that is precious (curated?) and
all digital information. Add to that a third axis of digital information that
is interactive (applications) and digital information that is static (text or
digital still pictures). In this simplified problem space the preservation of
artifacts, interactive media, saving everything is very problematic. Obviously
this leaves us with static, precious bits as a workable problem. For instance,
putting the pages of the 100 most important books on a CD-ROM.
Figure 1.
A more expanded
version of the problem space (Figure 2) is a multi-axis system composed of
human-scale pieces (a text file for instance) vs. huge pieces (all of NASA’s
data), information that is used a lot vs. information that will be used rarely,
a longevity time span of 50 years vs. 10,000 years, static vs. interactive
data, and finally digital information where the creator is cooperative in being
responsible about using long-term media and the situation where they don’t care
at all how long it lasts.
Figure 2.
In this problem
space (Figure 2.) the long-term problems that will require very different ways
of thinking are the situations where information is rarely used, will need to
last 10,000 years, are huge in scale, are interactive, and where the creator
has been uncooperative. The near-term problem that is more solvable is a data
set that is used a lot, is static, is human scale, will only need to last for
50 years, and the creator has been cooperative.
Meeting at
the Getty Center
The
participants arrived in Los Angeles on Sunday, 8 February, and worked through
to Tuesday evening. Together with Stewart Brand, we organized the time on
Monday and Tuesday as a series of meetings. All day Monday was a closed session
of free-ranging discussion, starting with summary position statements by each
participant, and continuing with defining the problem. A summary of the
conversations constitutes "Setting the Stage," herein. On Tuesday
morning, the meeting participants viewed—again—the Sanders film, revisited the
progress made since the end of the previous day, and organized themselves for
the afternoon session. On Tuesday afternoon, after a luncheon with a dozen or
so members of the press, the group convened for a public discussion of the
subject and the progress made in the closed meetings. Present were many
journalists; specialists in information management; and library and archives
professionals from regional universities, corporations, entertainment
companies, foundations, and research laboratories. We also welcomed interested
staff, and a few fortunate visitors to the Getty Center. A transcript of the
afternoon event comprises "Public Session: Panel Discussion."
The project and
the meeting, and the unexpectedly strong response since then, support the idea
of the three partners that there is a growing recognition that this
interesting, troubling, large-scale problem ahead will affect virtually
everyone who uses computers. It is also becoming clear that neither the
problems nor the solutions are simply technical. As Michèle Cloonan, Chair of
the Department of Library and Information Science at UCLA, noted in her essay
"The Preservation of Knowledge":
...the emergence
of electronic information will result in a fundamentally different way of
approaching ... preservation ... which has been object-based (books,
broadsides, maps, etc.) and time-oriented (permanent durable paper should last
for at least for at least 250 years). Thus the notion of saving object X for Y
years may become obsolete. We will need to secure the longevity of information
so that the information itself does not disappear. And it must be done in
concert with librarians, publishers, manufacturers and anyone else involved in
the handling of information.
The future plan
for Time and Bits at the Getty Center is to continue the conversation, make the
signal brighter.
Notes
1. Bruce
Sterling, Holy Fire (New York: Bantam Books, 1997): 31
2. Donald
Waters and John Garrett, Preserving Digital Information, Report of the Task
Force on Archiving of Digital Information
[www.clir.org/pubs/abstract/pub63.html].
3. For
more information regarding either of Sanders’ films on this subject—Into the
Future: On the Preservation of Knowledge in the Electronic Age; or Slow Fires:
On the Preservation of the Human Record—please contact the American Film
Foundation at (310) 459-2116.
4. Stewart
Brand, The Media Lab: Inventing the Future at MIT, (New York: Penguin Books,
1988).
5. http://www.longnow.org
6. Kevin
Kelly, Out of Control: The New Biology of Machines, Social Systems and the
Economic World (New York: Addison-Wesley Publishing Co., 1995).
Copyright, J. Paul Getty Trust, 1998.