The Up-stack Scramble – Cloud Nine for Developers will be Trench Warfare for Vendors

Mar 23, 2011 by Devon Watson

The Cloud Computing industry has been in a state of technologic and rhetoric driven flux ever since the term “cloud computing” was coined. Coming from both a software and venture capital background, I enjoy paying close attention to the often incongruent evolution of both our industry’s capabilities and its marketing claims. This disparity isn’t unique to the Cloud industry. Most new markets experience this while the vernacular jargon gets socialized and standardized on. Case in point, the term Nanotechnology was actually coined to refer to the concept of self-replicating autonomous bots (ala Michael Crichton’s Prey novel), what the world got was a much broader and less grandiose set of mainly micro-scale innovations all loped under that industry name.

Similarly, Cloud Computing was initially taken to mean perfectly abstracted infinitely elastic computing scale in an on-demand basis. We don’t have this yet. What we have, and Im referring specifically to IaaS since it is our basic building block, is easy to configure, easy to provision, on-demand, utility priced virtual machines. This is a great progression for the software and IT professions, but it falls short of fulfilling the fanciful capabilities drawn in people’s minds. Cloud IaaS makes it easier and cheaper to do what we’ve been doing for years, but hasn’t profoundly changed the actual requirements, workflow or operational hassles that developers and IT administrators face. As a Senior Enterprise Architect at a large enterprise remarked to me last year – “we’ve run the course with virtualization, we did that years ago.”

Making compute power available has been solved – the current challenge is to put that instant-on horsepower it to work. To be clear – this is being done fairly well in some use cases: big data crunching (CG rendering, map reduce) and consumer facing websites come to mind because they have scale mechanics that rely on simple replication and balancing.

Business applications, however, are still beholden to the underlying topology of servers that supports them. The application itself and the IT system must be intrinsically wired together in order to function properly. For example – business applications often store large amounts of data and will store data across multiple servers. If user number 523 logs in and her data is on DB server number 5, how do you fetch that data when she sends a request through UI server 3? Answer – the application has to know who she is, where her data is stored and how to retrieve it. This enmeshing is necessary to solve many similar complexities, but makes it extremely difficult to take advantage of today’s current Cloud infrastructures – without completely re-architecting and rebuilding applications from the ground up and adding multiple highly complex new systems.

In order for this current world of instant-on cloud servers to be truly impactful and revolutionary for developers and IT operators we must continue to elevate them further away from the underlying mechanics and break the bonds between application and infrastructure. In order to do this, we need some middle tier that abstracts the application away from the underlying server topology and surrounds that application with the management, authentication, user-routing and scaling mechanics needed to seamlessly take advantage of newly provisioned resources. This is why we are seeing a universal up-stack migration by the industry leaders like Amazon, Microsoft and SalesForce.com into the still-evolving realm of PaaS (platform as a service) where the original vision of throwing code into an autonomous external cloud of elastic computing power is starting to coalesce in a couple offerings.

SalesForce.com’s “Force.com” platform for instance holds true to the “deploy and forget” ideal, but ties the developer to a restricted programming environment based on their own proprietary programming language in return. To counteract this, SalesForce.com has made bold moves by first partnering with the leading provider of virtualization technology (the underpinnings of the cloud movement) VMware to create VMforce for Java and then acquiring Heroku for the Ruby programming language. It wont be long before these two additional platforms are able to take advantage of the value added capabilities built into Force.com. The last few months have also seen the acquisition of Makara (another PHP PaaS) by Redhat, giving this enterprise heavyweight a compelling Private PaaS story for their client base. Amazon, the standard in IaaS, has been building new up-stack capabilities in-house and recently released their Beanstalk service which simplifies scaling Java applications. Not to be outdone, VMware last week announced the acquisition of WaveMaker a Rapid Application Development tool that compliments their SpringSource acquisition. Its not hard to imagine a new RAD-PaaS offering with these capabilities.

Scrolling this stream of movement, it can be hard to divine where this is all heading and where the competitors will bump into one another. It all comes down to the application developers and what up-stack capabilities you can offer them that improve one or more of: their build-time productivity/speed; the business value they can incorporate into their applications or the ease/quality of the ongoing application delivery. The following graphic gives a basic snap shot of where we are and how I see this evolving.