This document was last modified in Fall of 1994, but still has substantial relevence. Back then, people said to me "Well, it may be growing exponentially now, but it can't keep that up."

Uh, yeah.

More details on recent growth are available at The Growth of the Web Report. It's still exponential though. :-)

Wow, The Web is BIG

Updated November 10, 1994! Really big. Ok, let's look at it this way. My wanderer takes forever to run now and I don't run it very frequently partially because of this. At last check, it had found 870 sites, and that's mostly only because I didn't let it finish. Let's look at the traffic over the NSF Backbone. Specifically, let's look at three specific quantities. Guess which one is biggest. Guess which one is biggest by a lot. If you guessed the first or the second, you're wrong.

The total for 1992 is about 500M. That's not a small amount of data, and remember, all data that goes over the web within a certain area never touches the backbone, so doesn't get counted in this figure. Actually, that's a lot of data. About 500,000 pages of text or 1000 fairly large images. Though, on physical disk, I can easily carry that much disk in one hand.

The total for Jan-Mar of 1993 is about 5G. That's 10 times as much as all of 1992. That's a lot of data. If that were disk space it would actually cost quite a bit, (about $5,000, I think) for me at least. It's a lot, really. On physical disk, it's a few disks, but once again, carryable, though one arm may be pushing it.

The total for one day in May is approximately 10G. Wow. One day. I don't want to think about it. The amount that went over the backbone in one month, that is, February 1994 is 347G. 347 Gigabytes of data! That's a hell of a lot. Ok, so 11.4 Terabytes go over the whole net, and that's more than a hell of a lot. That's so much that if you really try and conceive of how big it is, your brain will coredump and you'll be fsck'ing childhood memories for hours. Don't think about it. And physical disk. Heh. I've seen 40 or 50 gig of disk in one place and that's a lot. As much as 269 gig in one place is just, well, a lot.

The total for the last 6 hours (for September 1994) is about 13G. Aaaaaah! Let's do some math. How much data went over the web in 1992? How much data went over the web in the last 15 minutes? About the same amount. Actually, more probably went over in the last 15 minutes. Not kidding here.

This all corresponds to more than 200,000% annual growth. Ok, so that's exaggerating to include data that far back. Let's just look at the past few months or so. Ooh, that lowers the value a lot. Down to a diminutive 3000% percent annually. That's a 1% growth per day. In case it's not apparent, I'm impressed. The SIPB Web Server , the one I maintain, has seen lots of usage. Extrapolating our data for the past year. By Christmas of '94, we should be getting 1000 connections per second during peak usage. We may have to get a faster machine. Isn't extrapolation fun boys and girls :-) Ok, so that's an exaggeration. A little. At the end of October, we were getting about 1 connection per second, average. Six months prior, we were getting about 1 connection every 4 or 5 seconds. Peak usage is about 2.5 times average, so we have another couple years before we're at 1000 connections per second. Oh well. It was a fun extrapolation while it lasted. In any case, we can still predict that there will be one web server for every man, woman and child on the planet before the turn of the millenium. (actually, sometime in 1998 :-)

Really though, clearly a lot of this is the initial growth of the web, but it really is growing really really really fast. Ok. Enough of me. :-)

This page and presentation of information is Copyrighted by Matthew Gray (mkgray@mit.edu). Permission is granted to distribute and use this information freely, so long as this notice is kept with it. Art is copyright 1994 by Eri Izawa <rei@mit.edu>. Thanks.
mkgray@mit.edu