For businesses to gain a competitive advantage requires the most current and in-depth information available for informed decision-making. Many organizations have implemented an Operational Data Store (ODS) to integrate data from a variety of Systems of Record (SoRs). This centralized view stores a snapshot of an organization’s most current operational data, improving the general quality of the information, making it easier for users to diagnose problems before searching through component systems.
What is an Operational Data Store and How is it Used?
Since an ODS is perceived differently by different people in various roles,ย let’s try to clarify the common definition. An ODS collects, interprets and distributes data from different information systems, and is refreshed on a daily or even an hourly basis. Traditionally the ODS acts as an optimized staging areaย for operational reporting, and BI applications. By streamlining data analysis and utilizing more current data, decision-making becomes more efficient, enhancing the organizationโs ability to make more timely, data-driven decisions.ย
Why do Organizations Deploy an Operational Data Store?
A main benefit of an ODS is its ability to aggregate data from multiple sources, offering a current snapshot of relevant business metrics. While many organizations use different SoRs to manage various aspects of their data, reporting on each data source separately provides a siloed view of the data. The ODS allows for reporting across multiple SoRs for a more complete view, handling processed and transformed data. In addition, some SoRs offer limited reporting capabilities, so an ODS is a way for users to gain more comprehensive reporting. Since access to SoRs is usually restricted to a select few users, the centralized ODS opens up reporting capabilities to a broader audience within the organization.
The Shortcomings of a Traditional Operational Data Store
Traditional Operational Data Stores pose challenges for enterprise, especially as they embark on digital transformation initiatives:
Designed for operational reporting, not for real-time API serving: while a traditional ODS supports operational reporting use cases, it does not offer real-time API services to access the SoRs. Thus it’s unsuitable to meet the requirements of new digital applications.
High latency: the traditional ODS is based on a relational database, or sometimes on a disk-based NoSQL database. These database systems can not provide high performance when handling large amounts of data, and thus canโt support demanding low-latency applications.
Low concurrency: as traditional databases offer limited scalability, they pose a challenge when it comes to user concurrency. Once multiple concurrent users are accessing the data store, the performance takes a hit and as a result, the ODS can not support a high level of concurrent users beyond a certain threshold.
Stale data reporting: a traditional ODS is not a real-time replication of the SoRs because data is refreshed only periodically – hourly or sometimes daily. While this refresh rate is acceptable for end-of-day reporting scenarios, it is not suitable for digital applications that require real-time data such as in trade risk analysis and reporting, ecommerce, fraud, dynamic pricing and more.
The Evolution of Operational Data Stores – A Paradigm Shift
Digital transformation has driven a paradigm shift, signified by the many organizations that are introducing new real-time digital applications to replace offline services. A contemporary breed of technology companies in areas such as fintech and insurtech are introducing new business models and services that require more power and capabilities than what a traditional ODS can offer. These digital banks are differentiating themselves with continuous innovation and expanded online services. Digital insurance companies are leaving โofflineโ behind, issuing insurance policies in 90 seconds, and paying claims in just three minutes.
These disruptors are not chained to legacy infrastructure and processes and can rapidly develop and release new digital apps to meet customer demand for innovative services. Traditional financial services organizations including banks and insurance companies as well as other industries such as retail, transportation, healthcare, and even higher education, are under pressure. Most are still using mainframe and other legacy data platforms for many of their SoRs and databases, and they must now compete with the new players, more demanding customers, new regulations, and a constantly changing landscape. This means they need to make the necessary changes to modernize their infrastructure. How can this be done without ripping and replacing mission-critical services of record?
The data transformation paradigm shift has spurred the emergence of a new generation of operational data stores and architectural designs. Gartner has coined the modern operational data store as a Digital Integration Hub (DIH). GigaSpaces calls it Smart DIH.ย
No Need to Rip-And-Replace
When planning to implement a next-generation ODS in an organization, there are two ways to go about it. If there is no ODS in place, you can deploy an โout-of-the-boxโ solution that includes all the relevant components. These will typically include a high-performance operational store and compute engine, database integration or CDC, smart caching, analytics, microservices API, and event-driven architecture.
On the other hand, if you already have a traditional ODS in place, it does not necessarily need to be replaced. Instead, you can choose to augment it with the missing layers, by adding the cache, and utilizing event-driven, microservices architecture and agile technologies that enable fast development and deployment cycles
The Benefits of a Next-Generation Operational Data Store
Implementing a next-generation ODS can support digital transformation by eliminating the limitations of the traditional ODS, thus providing an organization the speed, scale and agility required to enable new digital applications.ย
Benefits of the next generation Operational Data Store include:
Super fast performance. With a distributed, high performance in-memory computing and storage engine, a next generation ODS offers the speed and scale to power the most demanding digital applications. Colocation of applications with data in the same memory space takes performance to the next level, as data does not need to travel over the network. The distributed in-memory core also allows for high concurrency of users, without impacting performance. And for planned and unplanned peaks, autonomous scaling guarantees the expected performance, without the need to overprovision.
A next generation ODS also allows you to run analytics on real-time data, while enriching it with historical data so your predictive modeling is as robust and as accurate as you need it to be.
GigaSpaces benchmarks
Always on. When digital applications read directly from a system of record, they will be impacted when the SoR is down. And when you manage multiple systems of record, the chance that one of them will become unavailable increases. ‘A next generation ODS decouples the API layer from the SoRs, thus allowing the organizationโs applications to keep working even if SoRs are down.ย
Autonomous scaling is triggered automatically based on system workload with no downtime
You can also read how PriceRunner, a leading eCommerce site in Scandinavia scales at 20X normal load on Black Friday while retaining millisecond performance levels.
Tiered storage capabilities can also move data between hot, warm and cold storage based on business rules to balance cost and performance. This way, your most important data is always in RAM, providing the fastest access possible while less important data can reside on less expensive storage types.
A multi-region data fabric. Global organizations such as investment banks, retail, or manufacturing companies often operate data centers in separate remote sites for reasons such as high availability, data locality, and regulation adherence. These data centers need to be synchronized in real-time. In addition, hybrid cloud deployments are becoming popular, and data in many cases needs to be replicated between on-premise and cloud and sometimes multiple cloud deployments in different regions. With cross-site and cross-region replication, a next generation ODS replicates your data in real-time, with low network overhead and without impacting your production performance.
Multi Region Replication
Quicker time-to-market. Connecting to systems of record and databases is often a time-consuming task. It may involve reviewing hundreds of data tables to understand the schema. With a unified API layer and automated schema discovery and blueprint creation, a next generation ODS reduces weeks of manual schema mapping to literally one-click. The microservices architecture allows you to quickly bring new services to market.
Find out more about a Next Generation ODS in this short video.ย
Select Use Cases of Next Generation ODS
- Sociรฉtรฉ Gรฉnรฉrale, a multinational investment bank and financial service company gains 99.999% availability with a next generation ODS powered mission critical data distribution system. It replicates trading data in real-time between Paris, London, New-York and Hong Kong. Its next generation ODS enabled the bank to rapidly grow the number of applications from 200 to over 400 apps.
- Avanza Bank, the largest stockbroker and brokerage firm in Sweden, can take microservices from an idea to production within a day. And it benefits from a mean response time of 1.7 milliseconds per microservice.
- PriceRunner, a multinational ecommerce price comparison company, retrieves 200 million merchant offers every day, maintaining a database of over 16TB of data in-memory for a superfast performance time of less than 8 milliseconds. During the Covid-19 crisis they launched two new innovative services to production within 2 days.
Conclusion
Whether you like to call it a Digital Integration Hub or you prefer to stick with the good old Operational Data Store terminology, you can gain a dramatic improvement in throughput and scalability, bulletproof high availability and innovation-driving agility by modernizing your architecture with a Next Generation ODS.ย
