Urban Planning in the Datacenter?

Jan 31, 2007 by Matt Ammerman

It was written only a month ago, but Robert Mitchell’s article ‘Virtual machines: the new ghetto in the data center’ is a classic in my book. Mitchell’s article draws up the similarities between data center engineering and urban planning. Like city planners, service providers must be careful that the efficiencies they introduce to their load-bearing technologies don’t harm the quality of service to their end users. It seems like a no-brainer… but immediately driving down that cost-to-service pain is so very tempting nonetheless.

When you scale an infrastructure to handle the load of more tenants, you can scale out – add more machines (a la Google) and harness the power of clustered computing, or scale up – add more power to existing hardware, or replace it with more powerful hardware to shoulder the load. Similar urban planning scenarios dictate that to accommodate more tenants you either build more houses (scale out) which results in urban sprawl, or you build high rise apartment structures (scale up) and end up with super-expensive maintenance costs. Either tactic has a beneficial short term outcome, but problems arise because with success come more tenants moving in. You could scale up or out forever, the problem is that both lose efficiency over time.

 

 

But then there’s virtualization. Mitchell’s metaphor is that virtualization is the ghetto of datacenters – that is, virtualization is the high density housing of more tenants in the same physical structure. Don’t scale up, don’t scale out – instead, build partitioning elements and use the same space for more tenants. The metaphor is dead on. Mitchell ‘s case, though, is that improper use of virtualization leads to terrible inefficiencies when it comes to per tenant load. One of the more interesting comparisons that Mitchell makes is the way that virtualization causes tenant processes to ‘fight’ for computing time.

“Unfortunately, some organizations may be stepping back into a world more familiar to mainframe users in the ’60s and ’70s. In that era of timesharing, applications got a slice of the mainframe.”

The use of virtualization technologies may lead to increased overall efficiencies for the provider, but slow response times and lowered service levels for the end user due to sharing the same computing resources with other tenants. Mitchell’s metaphor calls virtualization a compromise – a way to cram more tenants into the same space.

As SaaS becomes more prevalent and the number of SaaS providers grows, more companies will be exploring the various techniques for balancing cost-to-service efficiency with load-bearing capability. Scaling becomes a major concern, and any of the above mentioned approaches to scaling costs a good deal of money – hence the added costs of starting a SaaS company and imminent concern of provider buckle risk.

To complete the metaphor… new SaaS providers will need to build cities, when traditional software packages are dealt with more like prefab homes.

I’m curious to hear some stories about scaling in data centers.  I think we all know of Google’s approach, but what has your experience been?  Have you had better luck taking up more real estate with commodity boxes, or paying for high-end raw processing power?  If you are using virtualization technologies, what have the net effects been on the end user experience?   

(Note: I purposely left ‘computing in the cloud’, like Amazon’s EC2, out of this conversation because I want to discuss the burdens of scaling as they pertain to providers who build, instead of buy, their hosting solutions)