For an in-depth discussion of SBA and scalability, see our White Paper: The Scalability Revolution – From Dead End to Open Road.

Overview

In many application domains today, especially in financial services, the number of clients, the depth of services provided, and the data volumes are all growing simultaneously; in parallel, middle-office analytics applications are moving towards near-real-time processing. As a result, application workload is growing exponentially. One GigaSpaces customer is expecting to grow from 100K trades to 80 million trades – in only two years!

In order to understand the scalability problem, we must first define scalability: scalability is the ability to grow an application to meet growing demand, without changing the code, and without sacrificing the data affinity and service levels demanded by your users.

We identify two situations in which scalability is interrupted:

• A scalability crash barrier – occurs if your application, as it is today, cannot scale up without reducing data affinity or increasing latency to unacceptable levels.
• A marginal cost barrier – occurs when the cost of scaling your application progressively increases, until scaling firther is not economically justifiable.

For most contemporary applications, particularly transactional applications in a low-latency environment, these barriers are inevitable. But this is not the only possible case. Theoretically, an application can achieve linear scalability – the ability to grow as much as needed, at a fixed price per capacity unit – in which case it would never face scalability barriers.

Tier-Based Business-Critical Applications – Scalability Interrupted

Consider two typical business-critical applications – front-office applications and back/middle office analytics applications. It is clear that both types of applications do not have linear scalability, because scaling becomes progressively more difficult and expensive as the application grows.

Interestingly, these two very different business-critical applications have striking similarities: they are both stateful, and both use a messaging tier for coordination, a data tier for storage of state information, and a business tier for the actual processing – in other words, they are both founded on the tier-based architecture.

More interesting still, both types of applications encounter similar scalability problems: cluster nodes get overloaded by inefficient clustering; different clustering models for each tier cause unnecessary ping-pong inside the tiers; unknown scaling ratios between system components cause unexpected bottlenecks when the system scales; growing messaging volumes might overload the processing components; the network becomes the bottleneck; inability to virtualize the tiers causes coordination problems; and different H/A models for each tier makes it difficult to guarantee recovery from partial failure.

Each of these problems can cause a scalability crash barrier. To avoid hitting a crash barrier, application administrators are forced to apply temporary fixes-complex, resource-consuming coordination and clustering mechanisms – and scale up each tier just to accommodate the additional overhead. As the system scales, these fixes need to be applied again and again, making scalability progressively more expensive.

The application's administrators find themselves caught between a rock and a hard place: if they apply the scalability fixes, the application grows more and more complex, until it hits a marginal cost barrier; but if they don't apply these fixes, the application quickly hits a scalability crash barrier, and must be replaced.

This dilemma is inherent in tier-based applications: because the system is divided into separate tiers, it increases in complexity as it scales, requires more and more overhead just to manage this complexity, and makes it more and more costly to increase capacity.

Therefore, tier-based applications cannot be linearly scalable. This is proven by the well-known Amdahl's Law, which states that if all processors in a system spend some of their time on overhead – as is the case in all tier-based systems – the speed improvement yielded by additional processors quickly hits an upper boundary (for 10% overhead, maximal improvement is 10X).

If scalability is a road, and applications are cars driving on the road, non-linear scalability is a dead end. To avoid ending up as a wreck, tier-based business-critical applications must become linearly scalable – and the only way to do this is a change of architecture.

Towards a Linearly-Scalable Architecture

The key to a linearly-scalable architecture is to expand the application by adding self-sufficient units. This way, as the application grows, there is no complex coordination that consumes resources and leads to scalability barriers.

This is a clear departure from the tier-based architecture: instead of running each application tier in a separate cluster of computing resources, all the tiers are compressed into a single unit, which is duplicated when the application is scaled. The middleware problem simply evaporates when each component becomes self-sufficient.

But in a stateful environment, how can you manage a workflow when each machine is completely self-sufficient? How can machines share state information between them? The secret is to collocate all steps of the business process – putting them on the same machine, in the same JVM. This requires developing a processing unit – a mini-application which can perform the entire business process on its own and return a result.

The processing unit manages its own workflow, providing messaging and data storage facilities to its collocated service instances. This saves the need to contact external resources, and means that all process information can be stored in local memory, reducing latency to a minimum.

To ensure reliability, state data and partial results can be persisted to a database, or replicated to another, identical processing unit. Thus, the processing unit can be made as reliable as needed: transient data can be stored in memory only; very sensitive data can be persisted to a remote database. This improves on the tier-based model, which forces all processing components to pay the high price of persistency – even if this type of reliability is not really required.

In the processing unit model, user requests that enter the system are distributed between the processing units using content-based routing – if the request has value A it goes to Processing Unit 1; if it has value B it goes to Processing Unit 2. This type of routing is very simple and inexpensive, but most importantly, it does not increase in complexity as the application scales.

The concept of a processing unit makes linear scalability possible, but this is not enough. A critical requirement is that as the distributed application scales, it should behave and look as one server, whether viewed by designers, developers, administrators, or clients. Everyone involved should see the system as one coherent unit: scaling should be simplified to the point of transparency.

Space Based Architecture – Scalability as Open Road

Space-Based Architecture (SBA) is a way to implement processing units in your application, transforming scalability from dead end to open road.

At the core of an SBA processing unit is the space – a middleware infrastructure, collocated with the business process services, which provides in-memory messaging and data storage facilities for the processing unit. Like a conveyor belt in a production line, the space helps coordinate the business process, and allows business services to deposit their partial results on a shared work area, which can be accessed by other services in turn.

The space's messaging facility allows direct in-memory access to specific messages – this saves each business process service the overhead of connecting to an external messaging server and screening out unneeded messages from the queue.

When the space acts as data store, the services can save results at in-memory speeds, because they are collocated with the space on the same physical machine.

The space has a built-in clustering model, which allows it to share data and messages with other spaces on the network; this allows processing units to guarantee reliability by replicating state data between them. The space can also persist its data to a database. All this occurs in the background, without affecting the latency of the processing units.

This unified clustering model allows clients of the application to execute requests against a generic cluster proxy – the cluster matches the request at the object level, and routes it to the appropriate space. This routing is done in a distributed manner, without requiring a central server.

A central part of SBA is a deployment framework that takes care of running the processing unit on physical machines. The framework consists of SLA-driven containers that run on all the computers in the deployment scope.

Via the user interface of the deployment framework, it is easy to deploy as many processing units as needed – with one click. A general deployment descriptor describes a generic scaling policy for all possible deployment sizes, so deployment doesn't become even a bit more complex when scaling up. What's more, the framework dynamically responds to application workload.

Because of the flexibility of the deployment framework, SBA permits a combination of scaling-up (within a multi-core machine) and scaling-out, without breaking the SOA model.

SBA guarantees linear scalability, while ensuring simplicity – the application's scale becomes transparent to all involved, as if it was a single server:

• Scalability ceases to be a design consideration.
• The application presents a unified API and clustering model, so developers can write code as if for a single server – physical distribution is completely abstracted.
• The entire application is deployed through a single deployment command.
• Clients can perform operations on the entire cluster in a single operation.

The SBA Value Proposition

SBA removes the scalability dead-end facing today's tier-based applications, and guarantees improved scalability in four dimensions:

1. Fixed, minimal latency;
2. Low, predictable hardware/software costs;
3. Reduced development costs;
4. Consistent data affinity.

An SBA Example

For an example of an SBA application, see the eXtreme Application Platform Tutorial.