Skip to content
GigaSpaces Logo GigaSpaces Logo
  • Products & Solutions
    • Products
      • Smart DIH
      • Smart Cache
    • Solutions
      • Industry
        • Financial Services
        • Insurance
        • Retail and eCommerce
        • Telecommunications
        • Transportations
      • Technical
        • Operational BI
        • In Memory Data Grid
        • Hybrid Cloud Data Fabric
        • Multi-Tiered Storage
        • Kubernetes Deployment
        • Streaming Analytics for Stateful Apps
    • Performance Benefits of HPE Superdome Flex Server with GigaSpaces
      vid-icon Whitepaper

      Learn about the combination of HPE Superdome Flex server and GigaSpaces in-memory data processing delivers extreme performance; in more than 99% of cases, latency was less than one millisecond for a data query.

      DOWNLOAD
    • Contact us
    • Try Free
  • Resources
    • Resource Hub
      • Webinars
      • Demos
      • Solution Briefs & Whitepapers
      • Customer Case Studies
      • Use Cases
      • Benchmarks
      • Analyst Reports
    • col3
      • Blog
      • Technical Documentation
    • Digital Integration Hub: The Architecture of Digital Transformation
      vid-icon Blog

      Learn how a Digital Integration Hub (DIH) connects to multiple systems of record and data stores regardless of whether they reside and aggregates operational data into a low-latency data fabric, supporting modernization initiatives by offloading from legacy architecture and providing a decoupled API layer to power modern on-line applications.

      LEARN MORE
    • Contact Us
    • Try Free
  • Company
    • Col1
      • About
      • Customers
      • Management
      • Board Members
      • Investors
      • Events
      • News
      • Careers
    • col2
      • Partners
      • OEM Partners
      • System Integrators
      • Technology Partners
      • Value Added Resellers
    • col3
      • Support & Services
      • University
      • Services
      • Support
    • Getting Digital Transformation Right with Avanza Bank
      webinar-icon On-Demand Webinar

      Watch the webinar for a compelling and insightful talk with Joakim Sahlström, CTO at Avanza Bank, who shares his vision for banks in the digital age and how you can simultenously overcome architectural challenges while successfully leveraging new technologies to develop and deploy innovative digital services.

      WATCH NOW
    • Contact Us
    • Try Free
  • Contact Us
  • Free Trial
  • Products & Solutions
    • Products
      • Smart DIH
      • Smart Cache
    • Solutions
      • Industry
        • Financial Services
        • Insurance
        • Retail and eCommerce
        • Telecommunications
        • Transportations
      • Technical
        • Operational BI
        • Mainframe & AS/400 Modernization
        • In Memory Data Grid
        • Transactional and Analytical Processing (HTAP)
        • Hybrid Cloud Data Fabric
        • Multi-Tiered Storage
        • Kubernetes Deployment
        • Streaming Analytics for Stateful Apps
    • Roles
      • Architects
      • CXOs
      • Product Teams
    • Contact Us
    • Try Free
  • Resources
    • Resource Hub
      • Webinars
      • Demos
      • Solution Briefs & Whitepapers
      • Customer Case Studies
      • Benchmarks
      • Cost Reduction Calculators
      • Analyst Reports
    • Blog
    • Technical Documentation
    • Contact Us
    • Try Free
  • Company
    • About
    • Management
    • Customers
    • Board Members
    • Investors
    • Events
    • News
    • Careers
    • Partners
      • OEM Partners
      • System Integrators
      • Technology Partners
      • Value Added Resellers
    • Support & Services
      • University
      • Services
      • Support
  • Contact Us
  • Try Free

Optimizing the Balance Between Performance and Cost with GigaSpaces v15.8

Subscribe to our blog!

Subscribe for Updates
Close
Back

Optimizing the Balance Between Performance and Cost with GigaSpaces v15.8

6min. read
Lee Blum January 31, 2021

Latest release significantly reduces memory footprint and infrastructure costs, and boosts digital application performance. 

2020 has been a tough year all around. With tight budgets and limited resources, enterprises are looking to optimize infrastructure TCO. At the same time, they are looking to accelerate digital transformation. According to the 2021 Gartner Board of Directors Survey, 69% of boards of directors accelerated their digital business initiatives following COVID-19 disruption. Such digital business initiatives require the speed, scale and agility to handle the ever growing amount of business data. 

With this in mind, I am excited to announce today the release of GigaSpaces v15.8 which offers advanced functionality to significantly reduce memory footprint, reduce infrastructure costs, and boost digital application performance. The main three pillars of InsightEdge v15.8 include:

  1. Reducing RAM footprint: optimizing in-memory data store RAM to reduce hardware costs by up to 70% while retaining blazing performance
  2. Boosting SQL query performance with smart data locality: 10x faster response time on reporting and BI compared to previous releases
  3. Cloud native lifecycle management: enable agile deployments of new versions of data services with no system downtime 

Smart RAM Footprint Reduction to Save up to 70% on Infrastructure Costs 

With v15.8, you can now optimize any object by simply marking it as “Storage Optimized”. This will automatically reduce the RAM storage footprint the object requires. The degree of optimization depends on the ratio of indexed properties to unindexed properties (fields). The more unindexed properties you have, the bigger the reduction will be. 

Figure 1: Screenshot showing how to optimize a property with a single-click

Consider a BI dashboard that displays data records with 100 properties, when only 20 properties out of the 100 are indexed. Selecting “Storage Optimized” for all unindexed properties will reduce RAM utilization by up to 70%.

Why is it important? It means significant cost savings. Here’s an example. Let’s assume 1TB of RAM costs $10 an hour (for real life pricing options, see for example AWS EC2 on-demand pricing). This amounts to $7K monthly, or $86K annually. Backup partitions would double the cost to $173K, and if you have a remote disaster recovery data center for high availability, this will set the cost at $346K. Assuming RAM footprint reduction of 50%, you will save $173K annually for every 1TB of data. This cost savings can add up fast if you utilize more than 1TB of data, or if you add additional clusters, such as a cluster in NY and a cluster in London. 

Figure 2: Table showing expected cost savings with optimized RAM storage and 50% RAM footprint reduction

Below are benchmark results that present the expected footprint reduction in various ratios of indexed properties. This benchmark is based on 100k objects, with 100 fields of type string, length 10.

Figure 3: Benchmark showing footprint reduction for 100k objects, 100 fields of type string, length 10

When optimizing the RAM footprint, the impact on performance is as seen below. The difference in latency is between 1-2 milliseconds on remote operations:

For optimal tuning, the user can select which objects to optimize and which not, trading storage and performance as required. Early Access to the storage optimization feature is available today.

Figure 4: Benchmark showing performance of optimized objects vs query of non-optimized objects

Boosting Query Performance by More than 10X

InsightEdge now allows you to boost query performance with smart data locality using Broadcast Objects. An Object can now be designated as a “Broadcast Object” with a single click. 

Figure 5: Screenshot showing how to designate an object as a Broadcast Object with a single click

This enables server-side JOIN performance by automatically replicating selected small tables of data to all the nodes in the cluster. In other words, it gives you the flexibility to balance storage footprint and performance, and can improve your reporting and BI performance by 10x. The scenarios that utilize the Broadcast Objects feature in the most optimized way are cases where you JOIN two tables, when one table is a large dynamic table of transactions, while the other table is a small static table that does not change frequently, such as daily exchange rates. The small static table will be replicated to all nodes, but being small it will have a minor impact on RAM footprint. This will allow local JOIN operations, significantly reducing network overhead and resource utilization, leading to low latency and higher concurrency. 

Let’s take a real life example. A hedge fund was querying a large table of live stock quote records, JOINed with three other static tables with additional information about the data source and the equity. When the four tables were independent sharded tables, the response time was too slow for their needs.

Figure 6: Diagram showing a query JOINing 4 tables

They then designated the three reference tables as Broadcast tables, leaving only the Quote table as an independent partitioned table. The performance improved dramatically, and the queries ran 12x faster. The standard deviation also dropped, leading to more predictable performance. You can see the results in the following table, when running 50 concurrent users:

Figure 7: A table comparing query performance of 4 sharded tables vs. 1 sharded table + 3 broadcasted tables with 50 concurrent users

Cloud Native Lifecycle Management 

With v15.8, GigaSpaces adds support for Kubernetes Operator to provide full lifecycle management for your data applications. This allows organizations to use Kubernetes Helm for day-1 deployment in a cluster, then use Kubernetes Operator for day-2 management tasks. It allows deployments of new data services or versions with business logic to production without any downtime. It also allows for auto scaling up or out to support unexpected workloads.

Figure 8: One-click service deployment

Additional Enhancements

The confluent Hub now includes a Kafka Connect sink connector. This allows easier mapping and integration of data streams, allowing consolidation of multiple heterogeneous data sources to a unified in-memory Data Space.

Figure 9: Kafka Connect GigaSpaces on the Confluent Hub

Summary

With InsightEdge Version 15.8, we continue to innovate with new tools to help you balance cost and performance. You can significantly lower your infrastructure costs by reducing RAM footprint, while improving query performance with object broadcasting and streamline your deployment using the Kubernetes operator. I invite you to experience it for yourself today. You can download v15.8 here for a free trial.

Additional Links

  • Read more about What’s New in Release 15.8 on our documentation website
  • Download Release 15.8 from the GigaSpaces download center

SHARE:

Share this Article

Tags:

Hybrid Cloud Cloud Machine Learning Insightedge Spark Business Continuity

Lee is the Director of Product Management at GigaSpaces. He has over 15 years of experience in R&D, software architecture and product management positions in the domains of big data, analytics, high-performance storage and cyber solutions. Follow Lee on Twitter: @theLeeBlum.

Subscribe to Our Blog



PRODUCTS & SOLUTIONS

  • Products
    • Smart DIH
    • Smart Cache
  • Solutions
  • Industry
    • Financial Services
    • Digital Banks
    • Insurance
    • Retail and eCommerce
    • Telecommunications
    • Transportation
  • Technical
    • Operational BI
    • Mainframe & AS/400 Modernization
    • In Memory Data Grid
    • Hybrid Cloud Data Fabric
    • Multi-Tiered Storage
    • Kubernetes Deployment
    • Streaming Analytics for Stateful Apps

RESOURCES

  • Resource Hub
  • Webinars
  • Blogs
  • Demos</