Analyst Report: Cloudify from GigaSpaces - Delivering Mission-Critical Applications to the Cloud at the Speed of Business
A Robert Frances Group Publication
Published March 2014 | Principal Author: Gary MacFadden, Principal Research Analyst, Robert Frances Group, Inc.
The pace at which information technology innovation and adoption has transformed virtually every aspect of business and industry is breathtaking. In less than a generation, the advent of the Internet and intelligent mobile computing devices along with secure access to Big Data and vastly improved analytics capabilities have tipped the computing paradigm on its head.
Today, companies of all sizes and every ilk struggle with how best to deliver products, services and their brand messages to business partners or consumers through the Internet, or what are commonly referred to today as Cloud services. For larger, multinational organizations highly dependent on data and information, such as banking/financial services, healthcare/life sciences, online gaming and retailers, telecom or transportation, the problem is even more acute.
Fierce competition to maintain customer loyalty while dealing with cost and regulatory pressures is forcing both CIOs and line of business application owners to rethink their strategies for bringing applications to the Cloud. In some cases, new applications are being built to support Cloud-based business initiatives such as real-time customer analytics or e-commerce. In other cases, legacy online transaction processing (OLTP) applications such as electronic banking or trading applications may need to be modified before migrating them to the Cloud.
The greatest challenge in bringing existing or new applications to market via the Cloud is keeping up with the speed of business. Today, business requirements change rapidly, sometimes within a matter of days or hours. Quickly automating, updating and supporting mission-critical applications and making them easily accessible to clients, partners and employees has become the lifeblood of all data and information-driven businesses.
Meanwhile, IT operations and infrastructure teams are struggling to keep pace with the rapid change – which is why many lines of business owners pursue solutions from outside providers who offer turnkey Cloud infrastructures – or Infrastructure as a Service (IaaS) – that include servers, networks and storage. These “public” Cloud offerings have quickly become popular, while companies with highly sensitive applications where security and compliance is a major concern are “standing up” their own private Cloud infrastructures.
A recent InformationWeek commentary by author Chris Murphy, quoting Bank of America’s global technology infrastructure executive David Reilly, writes, “Getting IT pros to give up old habits is one of the hardest things about building a new, private Cloud architecture. Why do we need different boxes for servers, storage, and network switches in the datacenter? They’re all just computers. Why can’t companies fill their datacenters with white-box computers stuffed with x86 chips and a ton of memory, controlled by software that can make that box an in-memory storage device today, a software-defined switch tomorrow, and a server next week?”
Perhaps the short answer to Mr. Reilly’s question is: IT needs more tools to automate the process of dynamically deploying and managing the underlying infrastructure or IaaS layer to support the rapid changes to applications and policies required in today’s business climate.
However, seamlessly linking public, private or a hybrid combination of both Cloud infrastructures to enterprise-grade applications and policies is no easy task. The need to rapidly deploy, manage and scale these Cloud dimensions or layers while simultaneously creating a common platform that delivers a business/application-centric approach to Cloud deployments – with virtually no software code changes – is why GigaSpaces created Cloudify.
The appeal of Cloud computing for larger organizations is undeniable, whether to support application development teams, augment backup and recovery, provide additional compute for peak demand periods or to extend product reach to additional markets and consumers. One homogeneous Cloud offering is unlikely to fit all these use cases. Indeed, a single organization may want to deploy several types of Cloud solutions simultaneously, including the use of private Clouds, public Clouds or a hybrid approach that includes a combination of both.
To quote another senior IT executive responsible for Cloud implementation at a major U.S. investment bank, “The goal is to consolidate a lot of the workloads on the same infrastructure, boost speed to market, allow elastic AWS-type scaling and better resilience and availability.”
Cloud Services Adoption and Growth
In just the last few years, the market for Cloud-related services and solutions has exploded. The largest public Cloud service provider (CSP), Amazon Web Services (AWS), alone was anticipated to bring in $3.8 billion in 2013 after generating less than $1 billion in 2010 – with the bulk of that revenue assumed to be coming from small- to medium-sized businesses (SMBs). Revenue projections for 2015 are in the $8-billion range and perhaps as high as $20 billion by 2020.
Murphy also references an InformationWeek Private Cloud report published last November that found “17% of the companies we surveyed said they use private Cloud for all apps, 30% for some apps, and 30% are testing or developing a private Cloud. Only 23% said their companies had no interest.”
Due to AWS’s one-infrastructure-fits-all approach, industry pundits question whether larger firms with special needs or any organization and business that relies almost exclusively on Big Data and the Cloud will embrace AWS standards and restrictions en masse. Research by Wikibon founder Dave Vellante suggests: “Amazon is the pioneer and as such takes many arrows. Nonetheless, its aggressive move into enterprise spaces warrants consideration and caution by practitioners to use AWS properly and for the right strategic fit.”
Based on interviews with dozens of medium and large enterprise IT executives, Vellante concludes, “[Business executives] must understand the right strategic fit for Amazon, which today is essentially test and dev apps and corporate skunk works programs. At the same time, IT executives should forge relationships with service providers that can mimic many AWS benefits within specific verticals or domains while providing vastly improved partnership models around security, governance, risk management and strategy.”
Infrastructure standardization allows CSPs to massively scale their platforms and keep expenses low to attract more cost-conscience customers. Conversely, this single infrastructure approach poses challenges for larger organizations looking to migrate hundreds of existing applications to the Cloud while also maintaining availability, scaling, security and compliance requirements.
Cloud Building Blocks
Application owners eager to leverage the Cloud in order to realize faster time to business value are challenging their technology partners and IT organizations to be more agile, leaner and quicker to market. The time and resources needed to support the infrastructure for this new Cloud-based application delivery paradigm is why AWS and other so-called Infrastructure as a Service (IaaS) or Platform as a Service (PaaS) providers have become so popular and profitable.
The concept of Software as a Service (SaaS) has been around for 50 years or more. IBM’s first computers were leased to users who bought access to time-shared applications delivered through phone lines or private networks. Today, popular applications such as CRM from SalesForce.com or email and collaborative services from Google or Yahoo! all leverage the Cloud. Enterprises deploy applications in the Cloud so employees, partners and customers, can easily access them.
For those organizations responsible for supporting their own mission-critical applications and quickly bringing them to market, whether hosted privately, publicly or via a hybrid approach, a new more innovative approach is necessary. This need for speed to business value while maintaining control of a stable, repeatable deployment environment is the essence of Cloudify.
Meeting Cloud Automation and Orchestration Challenges
A recent Gartner study projected that through 2015, “80 percent of outages impacting mission-critical services will be caused by people and process issues, and more than 50 percent of those outages will be caused by change, configuration, release integration and handoff issues.”
In addition, a recent survey conducted by the Ponemon Institute determined that “the average cost of data center downtime across industries was approximately $7,900 per minute (a 41-percent increase from the $5,600 in 2010).” VentureBeat editor-in-chief Dylan Tweney calculated Google lost $545,000 when an outage affecting all of their online services went down for 5 minutes in August 2013.
With an increasingly larger portion of business being transacted online, the impact of downtime on a company’s bottom line can be significant. With the move to Cloud and SaaS delivery models, both customer-facing applications and an organization’s entire IT infrastructure are at risk. Moreover, a worst-case scenario would be different teams running different tools for each layer – a sort of “anti-DevOps” approach. No doubt, there are still siloed IT organizations where multiple, redundant and incompatible tools are being deployed.
Chris Wolf, a research vice president at Gartner, in a recent posting states, “One of the great myths today is that there is all of this centralized hybrid Cloud management happening – for the most part, it doesn’t exist in terms of what folks are actually doing. In nearly every case where we see a hybrid Cloud environment, the customer is using separate sets of tools to manage its public and private Cloud environments. There just truly isn’t a ‘single pane of glass’ today; that’s the problem.”
While Wolf’s analysis might well reflect the current state of enterprise Cloud, there is no doubt that organizations building their own Cloud environments will surely look to standardize on automation and orchestration tools and processes that offer the most flexibility and speed of deployment capabilities while maintaining alignment with business goals. GigaSpaces developed Cloudify specifically to address these critical end-user requirements.
Cloudify enables customers to onboard and scale any app, on any Cloud, with no code changes, while maintaining full visibility and control.
GigaSpaces CTO Nati Shalom, in a recent blog post entitled “Eight Cloud and Big Data Predictions for 2014”, suggests, “Orchestration and automation will be the next big thing in 2014. Having said all that, the remaining challenge of enterprises is to break the IT bottleneck. This bottleneck is created by IT-centric decision-making processes, a.k.a. ‘IaaS First Approach,’ in which IT is focused on building a private Cloud infrastructure – a process that takes much longer than anticipated when compared with a more business/application-centric approach.”
In addition, Shalom states, “One of the ways to overcome that challenge is to abstract the infrastructure and allow other departments within the organization to take a parallel path towards the Cloud, while ensuring future compatibility with new development in the IT-led infrastructure. Configuration management, orchestration and workflow automation become key enablers in enterprise transition to Cloud, and will gain much attention in 2014.”
The Emergence of DevOps
Shalom also points to the emergence of DevOps, a combination of development and IT operations, as a “key example for a business-led initiative that determines the speed of innovation and, thus, competitiveness of many organizations. The move to DevOps forces many organizations to go through both cultural and technology changes in making the business and application more closely aligned, not just in goals, but in processes and tools as well.”
Cloudify Unifies the Cloud Stack
Cloudify is a Cloud orchestration platform developed by GigaSpaces that allows any application to run on any Cloud, public or private, with no code changes. More than three years ago, GigaSpaces anticipated the need for higher level tools to accelerate the onboarding and migration of mission-critical applications to the Cloud.
Since its release, Cloudify has been adopted by many large organizations – including multi-national financial institutions – as a de facto standard for bridging the gap between the IaaS layer and the application layer. With Cloudify, it is now possible to adopt a Cloud automation and orchestration framework that provides IT organizations, systems integrators, software developers and application owners with the ability to quickly deploy applications securely with standard interfaces (APIs).
Cloudify is designed to bring any app to any Cloud, enabling enterprises, ISVs, and managed service providers alike to quickly benefit from the Cloud automation and elasticity that organizations need today. Cloudify helps users maximize application onboarding and automation by externally orchestrating the application deployment and runtime. Cloudify’s DevOps approach treats infrastructure as code, enabling users to describe deployment and post-deployment steps for any application through an external blueprint, which users can then take from Cloud to Cloud, unchanged.
Cloudify is now available as an open source solution under the Apache license agreement or through GigaSpaces directly for those organizations looking for a premium services package. Cloudify and GigaSpaces work with many other open source Cloud automation and orchestration tools such as OpenStack’s Heat as well as Chef and Puppet. Cloudify also enables applications migrating to and from OpenStack, HP’s Cloud Services, Rackspace, AWS, CloudStack, Microsoft Azure and VMWare.
TOSCA(Topology and Orchestration Specification for Cloud Applications) from OASIS is an open source specification that works to “enhance the portability of Cloud applications and services.” The goal of TOSCA is to enable cross-Cloud, cross-tools orchestration of applications on the Cloud. In the 3.0 version of Cloudify, GigaSpaces is working on putting TOSCA into the mix and using its concepts as a canonical application model.
Cloudify uses an orchestration plan, or blueprint, that is inspired by TOSCA. The blueprint contains an application topology model: IaaS components, middleware components and application components. For each of these elements, Cloudify describes the component lifecycle and dependencies with other components (dubbed relationships). In addition, each node defines a set of policies that allow Cloudify to enforce application availability and health.
Cloudify translates these topologies into real, managed installations by running automation processes described in the blueprint workflows. These workflows trigger the lifecycle operations implemented by the Cloudify plugin, which uses different Cloud APIs as well as tools such as Chef, Puppet and others.
OpenStack: The Open Source Alternative to AWS
“Deploy, manage and scale” is a mantra for rapid application delivery in the Cloud. As previously mentioned, AWS and other CSPs accomplish this through standardization on a single IaaS platform. Three years ago, AWS competitor Rackspace, along with NASA, introduced the OpenStack initiative – essentially as an open source alternative to AWS – which now has more than 50 IT vendors and CSPs actively supporting and participating in the community, including AT&T, Cisco, Dell, HP, IBM, Intel, NetApp, Red Hat, Suse, VMware and Yahoo!.
OpenStack is a Cloud operating system acting as an IaaS platform to control large pools of compute, storage and networking resources throughout a data center. Released in September 2012 as open source software under the Apache license, OpenStack has become the most deployed IaaS platform in the world, embraced by thousands of service providers, government agencies, non-profit organizations and multinational corporations. The OpenStack community just announced the 8th release of its Havana software for building and supporting public, private and hybrid Cloud application infrastructure.
A recent survey conducted for Red Hat by IDG Connect shows 84 percent of enterprise IT decision makers surveyed say that OpenStack is part of future private Cloud plans. “Sixty percent of survey respondents indicated they are in the early stages of their OpenStack deployments, and have not yet either completed the implementation stage or are early in the process. Survey respondents cited management visibility (73 %); deployment speed (72 %); platform flexibility (69 %); better agility (69 %); and competitive advantage (67 %) as the unique benefits offered by OpenStack over private Cloud alternatives.”
In early 2013, to keep pace with the innovation coming out of the open source Cloud community, Amazon introduced OpsWorks to provide AWS customers with a “powerful end-to-end platform that gives you an easy way to manage applications of nearly any scale and complexity without sacrificing control.” While OpsWorks is not an open source product, so far Amazon does not charge extra for it. OpsWorks does however support a few open source solutions such as Chef version 11 and open source development languages such as Java, PHP and Ruby on Rails along with open source databases MySQL and Memcached.
It remains to be seen whether a wide swath of customers will embrace OpsWorks – even within the AWS framework – because a dedicated, proprietary AWS solution that locks companies and their applications into the Amazon Cloud will likely have limited appeal for many organizations. Large enterprises and some SMBs not only need a solution that manages Cloud infrastructure resources and interactions with users, they also need portability across private and hybrid (private/public) Cloud platforms.
Cloudify Special Sauce
In the words of Yaron Parasol, VP of product management at GigaSpaces, the Cloudify model “is simple, yet it has the profound understanding of a unified process that helps us drive our vision forward. Our assumption was quite different from what we saw as best practice adopted by others. Coming from an application perspective, we never drew a line between IaaS, Middleware (PaaS) and Application. For us, all of these are part of the application production fabric. They are all interdependent on one another in configurations, metadata and certainly in the flow of setting them up or fixing the application when it breaks or needs to change.”
As Parasol is keen to point out, “Ten years ago, nobody even dreamed about this level of automation. Today it’s not just a buzzword – it’s a requirement. The siloed IT structure doesn’t fit the new reality of IT.”
Cloudify possesses several attributes and capabilities that set it apart from most Cloud enablement tools. Cloudify’s unified, application-centric approach obviates the need for redundant Cloud management tools at each layer (SaaS, PaaS, IaaS) and works seamlessly with complementary open source tools such as Heat, Chef and Puppet, as well as the TOSCA specification, to deliver Cloud automation and orchestration through a “single pane of glass” or unified tool.
Here are several reasons why Cloudify is deployed by many large organizations across a variety of industries challenged to deliver multiple, diverse Cloud solutions:
- Cloudify is an open source framework for deploying, managing and scaling applications on a Cloud environment. Using “application blueprints,” Cloudify automates the setup, deploy, monitor, auto-repair and scale processes of mission-critical and Big Data applications on public and private Clouds.
- Cloudify’s DevOps approach treats infrastructure as code, enabling users to describe deployment and post-deployment steps for any application through an external blueprint, which users can then take from Cloud to Cloud, unchanged.
- Any app, any stack: Existing applications can easily move to the Cloud without any code or architecture changes. Cloudify supports any kind of application, using any application stack, with a relational or non-relational data store, or for use with other middleware components.
- Any Cloud: Run your app on any environment – public, private or hybrid Cloud; plain virtualization, or a combination of environments (for Cloud bursting, e.g.). Cloudify completely isolates the application from the underlying Cloud runtime. Therefore, users get the same flexibility they are accustomed to, no matter which environment they choose.
- Full control: Losing control and visibility when moving to the Cloud means nothing gained in terms of agility and productivity. Cloudify offers built-in monitoring and management services and facilitates using your existing tools on the Cloud.
- Cloudify allows users to “future-proof” their apps to run on OpenStack at a later stage and move to OpenStack when ready without any additional investment.
- Cloudify provides consistent management and deployment for applications between different OpenStack and non-OpenStack providers such as IBM/Softlayer and Rackspace.
- Cloudify gives users a unified framework to build their own Cloud management tool set on any Cloud and thus enables users to retain control over every aspect of their underlying infrastructure.
- With Cloudify, enterprises can gradually move their applications onto their new OpenStack environments with increased agility and reduced risk.
- Cloudify is external to your application stack, meaning no code changes are required.
- Cloudify is not just for development; it deploys, manages and monitors your application, automating and orchestrating the application throughout its entire lifecycle – taking you all the way from development to production.
- Cloudify helps organizations become automated, elastic and agile.
“Enabling businesses to build and move applications to OpenStack Clouds is critical, and open source solutions like Cloudify make it easier for developers to deploy and manage those applications across OpenStack environments,” said Mark Collier, COO, OpenStack Foundation.
Cloudify User Insights
A large multinational financial services company that has implemented Cloudify offers the following insights.
- Setup usually takes quite a bit of time when there are no best practices and no structure to follow. Cloudify allowed us to model this process in a very well defined way using application blueprints. When dealing with hundreds or even thousands of applications that we might want to migrate to the Cloud, this can be quite a timesaver.
- Regarding ongoing operations, when an application or component fails, there is the cost of downtime and the cost of bringing the application back up again. Cloudify does a few things for us to avoid such situations.
- Cloudify supports proactively scaling out an application when load increases, thus avoiding downtimes caused by overload.
- Cloudify enables auto-healing application components upon failure, thus minimizing the Mean Time to Recovery (MTTR) when an application fails.
- For enterprises like ours, with private Clouds, the ability for each application to consume only the amount of resources that it needs at any given point in time means significant improvement in our data center utilization and cost savings.
- Development and testing in many cases requires replicating complete environments, which is very costly without a good automation framework. Cloudify allowed us to set up an entire application with all of its supporting resources at the click of a button. In fact, we have a few users that are using Cloudify only for that.
GigaSpaces will soon release Cloudify 3.0, taking orchestration capabilities to the next level.
Based on this breakthrough technology, a commercial version of the product will be released called Cloudify Premium. In addition to the enhancements that we will see with Cloudify 3.0, the Premium edition will come with a variety of features – from an advanced web UI, to tailor-made blueprints to built-in plugins.
The TOSCA team at OASIS is updating its data serialization format from XML to YAML as the latter is both human readable and improves TOSCA’s ability to model business relationships, “nodes” (e.g., host or web servers) and interfaces, all of which are integral to the TOSCA topology and key for cross-Cloud, cross-tools orchestration of applications on the Cloud.
The TOSCA 2.0 draft discusses ways to improve the specification’s ability to elaborate on policies, such as allowing for SLAs of a node (topology component). Policies are still described in XML, not yet in YAML, and are tool dependent.
Many within the OpenStack community believe “Stackers” must embrace AWS as Amazon controls the innovation curve for public Cloud development and deployment and OpenStack’s future is predicated on driving hybrid Cloud compatibility with the major public Clouds. This view dovetails with Cloudify’s capabilities and strategic direction: Any app, on any Cloud, with no code changes, while maintaining full visibility and control.
Conclusion: Cloudify Unifies Cloud Automation and Orchestration
There is no doubt that enterprises are looking for additional deployment support when it comes to implementing private and public Cloud solutions. The benefits of freeing up internal infrastructure and enabling DevOps to bring applications to market more quickly are unquestionably real. What is not so clear is which path to the Cloud will provide the least friction and the fastest time to value.
Cloudify is winning converts in the banking, finance, retail, hospitality and telecom industries as well as gaining credibility with partners such as IBM, HP and Alcatel-Lucent when customers require native OpenStack and multi-Cloud support. A framework and approach that touches all the Cloud layers unifying the Cloud stack while simultaneously simplifying DevOps is a compelling combination of capabilities.
Enterprises looking to “Cloudify” their mission-critical applications will be hard pressed to find a more comprehensive, innovative, intuitive, open-source, community-supported solution than GigaSpaces has created in Cloudify.
 http://datasecuritycompliance.blogspot.com/2013/05/configuration-management-for-virtual.html Gartner Group quote (Ronni J. Colville and George Spafford).
 http://searchcio.techtarget.com/quiz/Quiz-Managing-a-data-center-in-a-Cloud-centric-world?q0=3&q1=2&q2=2&q3=3&q4=0&q5=2&q6=3&q7=1&x=74&y=5 Quote by Gartner analyst Chris Wolf.
GigaSpaces Technologies provides software middleware for deployment, management and scaling of mission-critical applications on Cloud environments through two main product lines, XAP In-Memory Computing and Cloudify. Hundreds of Tier-1 organizations worldwide are leveraging GigaSpaces’ technology to enhance IT efficiency and performance, from top financial firms, e-commerce companies, online gaming providers, healthcare organizations and telecom carriers.
About the Author
Gary MacFadden is a 20-year veteran of the IT research and advisory industry, a co-founder of RFG, a founder of the Parity Research blog and a frequent contributor to the Wikibon Research Community. He can be reached at firstname.lastname@example.org, at email@example.com or via cell at 203-313-0443.
© 2014 Robert Frances Group, Inc. All rights reserved. http://www.rfgonline.com/
No part of this document may be reproduced in any form by any means, nor may it be distributed without the permission of Robert Frances Group, Inc., nor may it be resold or distributed by any entity other than Robert Frances Group, Inc., without prior written authorization of Robert Frances Group, Inc.