Jun 26, 2010 by Sinclair Schuller
I was on a panel entitled “HYBRID CLOUDS — THE BEST OF BOTH WORLDS?” at the Structure 2010 conference this past week. (Check out a recording of the panel) For those readers who are unfamiliar with the concept of the “hybrid” cloud, essentially, it’s the idea that one logical infrastructure cloud can be composed from two or more clouds of different deployment locale (on-premises vs. elsewhere) or visibility levels (public vs. private)
I was lucky enough to be on this panel with a number of very intelligent people who had some amazing insight into the topic. One of the questions posed to the panel really struck a chord. Essentially, the question was something along the lines of “Are standards necessary for the hybrid cloud to become a reality?” (Roughly minute 29:07 in the recording) The standards being referred to are standards that would abstract the command and control of virtual infrastructure. Each cloud provider (e.g., Amazon through EC2, GoGrid, etc.) typically provides an API by which one can control the deployment and ongoing operations of a virtual resource. Since each cloud provider has a different API, there is a form of lock-in that rears its head since, although each cloud might provide a similar container (say, a Windows 2008 Server), your code for managing and leveraging that cloud is non-portable.
Mårten Mickos, CEO of Eucalyptus, fielded the question stating that standards are necessary for hybrid cloud to really take off and that a standard would accelerate adoption significantly. I agree with the spirit behind Mårten’s view, but not with the idea of formal standards (and as you’ll read, I think Eucalyptus is poised to do some amazing things). Now when we discuss “standards” we mean an industry standard sponsored by a standards body or some sort of consortium. Others on the panel respectfully disagreed with Mårten’s position. Joe Weinman responded that standards aren’t necessary, and that different providers need to be free to evolve solutions that best fit the problem domain and customer need. My position was somewhat in alignment with Joe’s; standards, by definition, must cater to lowest common denominator subsystems which can bias efforts towards standardization rather than on new techniques to solve new problems in the cloud. It’s important to understand why this is the case, and also why it doesn’t mean that although a formal standard shouldn’t be pursued in the near term, generalization should absolutely be pursued.
In some instances, “formal” standards can stifle innovation or distract focus from solving specific problems by becoming abstract enough to be standards compliant, but not specific enough to solve a problem as well as possible. For example, if some cloud provider comes up with a novel way to address something new, they can still conform to the standard, but any uniqueness they bring to the table will be lost behind the abstraction. Just think of the SQL-92 standard. Most every DB vendor has some level of conformity, but various RDBMS’ do something amazingly useful things outside of the SQL-92 standard. If you leverage those proprietary pieces, you can no longer benefit (at least not by much) from the standard itself since you’ve broken conformity. Furthermore, any standards that have evolved in the past have typically fragmented into many sub-standards, dialects, etc. (just like the SQL case)
Now, does this mean that we shouldn’t pursue standardization? Well, if we’re talking about a “formal” standard, we probably don’t need to rush it. Technological generalization, however, is a different story. We’re at a point in the cloud’s evolution that requires organizations like Eucalyptus to focus on providing a coherent means to manage multiple specific cloud implementations. This is amazingly powerful and valuable and doesn’t require a formal standard. Eucalyptus has focused generalizing the Amazon EC2 API as a standard API that could be applied against any other cloud system. Overall, EC2 is a wise choice since it has the broadest market penetration and the most generally applicable API when looking from an abstraction point of view. Eucalyptus is doing some amazing work to make sure that you can leverage API expectations to work with a variety of virtualization clouds. Their efforts will undoubtedly boost adoptability, reduce adoption risk, and increase the viability of private and hybrid cloud deployments. Given Amazon’s API penetration, a market “gold standard” has been created just by shear adoption, and I think people will naturally flock to a proper abstraction like Eucalyptus because of sheer necessity to ensure compatibility with the gold standard while preserving the ability to experiment with new solutions. By choosing EC2, Eucalyptus’ approach provides a standards-like advantage without the baggage since the standard is chasing the market leader, rather than creating a standard and trying to enforce it. Kudos to Mårten and Eucalyptus for helping catalyze adoption of the cloud!
Do you feel that formal API standardization will be absolutely required in order for cloud, private cloud, or hybrid cloud to work, or will things organically tend to a relatively standard oligopoly of well known abstractions without a formal standard in place? Do standards create stifle and slow down innovation by always focusing on the lowest common denominator, or does it promote innovation by guaranteeing a baseline?
read more
