When Big-IT meets Big-Web

Last week I had the honor to join an interesting event held by Battery Venture in Stanford titled “BiG IT meets Big Web” which brought together people from the financial industry (JP Morgan, NYSE, BOFA, Barclays, for example) and the Web facing world (Twitter, Netflix, Nimbula, DataStax, Bing Mobile, and others.)

As someone who happened to cover those two worlds for a while through my work at GigaSpaces, this was a unique opportunity meet the two camps under the same roof for the first time.

The two opening presentations by Brad Spiers (SVP President at Bank of America) and Adrian Cockcroft (Cloud Architect at Netflix) were great examples of the different in challenges and culture between the worlds.

Brad talked about Risk Management and Monte Carlo simulations as the core of their IT and DevOps as automation that is being done through scripting by the operation guys to ensure consistent deployment of their apps.

Adrian talked about their cloud strategy: running their entire data center on the cloud, how their devs handle also operations, as well as how they achieved continuous deployment to reduce their release cycles.

Adrian Kunzle, MD., Head of Engineering & Architecture at JPMorgan Chase, was somewhat in the middle between the two camps and provided a good example on how Big IT can adopt some of the lessons from Big Web despite the unique challenges that the finance industry faces – i.e. being highly regulated with lots of legacy apps that are hard to change, etc.

I thought that the main lessons from these talks could provide an interesting view on how Big IT can adopt some of the ideas and lessons from the Big Web world.

My Take: Bringing Big IT and Big Web closer

The move to private cloud and a PaaS model could be a great opportunity to address two of the main challenges that are often thought of as conflicting: control and agility.  A common Cloud/PaaS gives the IT better governance and control of the stack that runs their application as well as a way to measure and control their application’s SLA in a consistent way. On the other hand, it enables better agility as the cycle to deploy new application and setup a new environment can be reduced significantly.

The list below summarizes the main points that makes it possible to bring the worlds closer together .

• Monte Carlo simulations and risk management applications are not the only applications that a typical bank runs. There are still many applications that can take advantage of the agility that comes with the Cloud/PaaS model and they could serve as the early adopters within the bank.
• Applications should be abstracted from the notion of public vs private cloud – i.e. they should be able to run in both environments in the same way. The choice of where to run should be determined based on business justification – costs, security, etc.
• Continuous deployment can be done by integrating the development and QA systems directly into the PaaS. As many of the banks are highly regulated the actual rollout to production will still involve more control but the cycle for rolling out up to staging environment could be fully automated and the time to release a new feature could be shortened significantly.
• Many of the Big IT organizations have consolidated their use of language to Java and .Net. Hence by supporting Java and .Net, it is possible to cover 90% of the total applications.

Final notes

At the end of the day, and hearing the different views, it was clear that the difference in challenges and culture between the two industries is still fairly large. In many cases, and for good reasons, Big IT is highly regulated, many of the applications are highly transactional, and don’t have the same level of flexibility in terms of consistency,  latency and even the types of applications that they are running. Those differences have been the excuse for people in the two camps to  dismiss each other’s arguments and live in separate worlds in the way they operate and runs their applications.

We’re reaching the point where it becomes clear that there is a direct link between the way you run your applications and the way you run your business.

The Big Web world had the luxury to start with a clean slate and quickly iterate and develop efficient models for running their applications (and their businesses). For different reasons (regulation, legacy, applications) Big IT couldn’t move at the same speed but there no reason why it couldn’t adopt those same models even if it wouldn’t apply to their entire IT organization.

On the other hand, Big IT learned how to deal with scaling under extreme latency and throughput and without compromising on consistency. Big Web could benefit from some of the patterns that address this sort of extreme requirements without giving up on consistency or latency as I already noted in An interesting note on Google Megastore & CAP.

Anyway, I came out of this day encouraged that the two worlds can actually converge despite the challenges and differences. This sort of meeting is definitely a good and important start in that direction.

In the next post I’ll cover the second part of that day – specifically the DevOps pannel.