Big company, small teams

committed 04 Nov 2009

Werner Vogels spoke about Amazon’s Web Services at LISA09’s keynote. It was half good, half meh. Half of it was a sell job which really didn’t sit well. Obviously, I’m a bit biased on the topic, but the real issue is that the keynote is supposed to advance the trade, not pitch a vendor which is what part of it felt like.

What was interesting was the first part which talked about how AWS came about. Amazon spent some time in the early 2000s changing their monolithic application to a series of services. According to Werner, at this point, any given page can call up to 600-800 individual services.

My first thought is that this is just trading build complexity for operational complexity, and that just doesn’t seem very good. I’ve thought about it before (shiv) and should write about it at some point.

The interesting part is that they’ve changed their organization to match the services. Each team is a combination of development and operations and is focused on that service. They extend it as well as make sure it stays working.

The side effect of this was that they foudn that the engineering teams were spending quite a bit of time (70%) interacting with the networking (infrastructure?) team. They were spending a lot of time building servers and scaling them out.

And then came the idea that if they could make the infrastructure a service, it would make it easier for the engineering teams to spin the infrastructure work faster. So, this is what gave birth to AWS.

So – what started as a way to reduce the build time led to small teams led to a major IT service. The drive for efficiency is universal.

Postscript: Talking with Alva, he brought up the point that the cloud only allows for people to move their data/services, but it hasn’t even begun to address the inertia that many existing companies have. There’s inertia in capital (already have servers that are paid for), procedure (already have rules for data ownership), and process (already have automation that needs to be retooled). This is why many of the AWS customers are the small startups that don’t have any of these already. A bit of the obvious, but it was well articulated (at least, it was better than what I have here).