Are you Really using the Fastest and Most Scalable Cache with your Application?

We are hearing lately reports from several accounts “discovering” that Terracotta ehcache is not the fastest and most scalable cache out there. GigaSpaces local cache actually performs much better.
With the benchmark results below we see GigaSpaces Cache is about 8 times faster than echache when having multiple threads accessing the cache.

This benchmark test EHCache most popular use case against GigaSpaces cache. This benchmark is not about demonstrating GigaSpaces entire spectrum of functionality (elasticity , eventing , SQL query , Remoting , Map-Reduce, noSQL support, ESB support, CEP support, Cloud enabling , PaaS…) . The focus here is on a simple caching scenario many users test when evaluating caching products.

You can download the source code and run this benchmark yourself. With a small effort you can use any other cache implementation and compare it with GigaSpaces Cache. The benchmark is self-contained. It includes the source code, compiled code as well the ehCache 2.5 libraries.

What the Benchmark is Doing?

1. The benchmark will first write 100 objects into the cache.
2. Then, it will to read these repeatedly for 10 seconds.
3. After 10 seconds an additional thread will be started performing the read activity for 10 seconds until there will be 8 threads running. The total read benchmark time is 80 seconds.
4. Each thread calculating its average read time and a final result is displayed after each iteration with the total average read performance for all threads.

How can I run the Benchmark myself?

1. Download GigaSpaces XAP , extract the zip file and apply the license file provided as part of the email sent to you after the download.

2. Download the benchmark package and extract the attached into an empty folder.

3. Move into the localcacheVsehCacheBench folder.

4. Rename *.at files to *.bat if you are running on windows.

5. Edit the setExampleEnv.bat/setExampleEnv.sh to have the correct location of the JSHOMEDIR variable as GigaSpaces root folder.

6. To run the ehCache benchmark run: runBenchmarkehCache.bat/sh.

7. To run the GigaSpaces benchmark run: runBenchmarkGigaSpaces.bat/sh.

Expected Output

The expected output for ehCache benchmark should look like this:

Terracotta ehCache benchmark started!
1 Client threads started
——— Average TP for all 1 threads:5556916 read/sec———————
2 Client threads started
——— Average TP for all 2 threads:1523256 read/sec———————
3 Client threads started
——— Average TP for all 3 threads:1144619 read/sec———————
4 Client threads started
——— Average TP for all 4 threads:1036346 read/sec———————
5 Client threads started
——— Average TP for all 5 threads:998566 read/sec———————
6 Client threads started
——— Average TP for all 6 threads:907827 read/sec———————
7 Client threads started
——— Average TP for all 7 threads:893828 read/sec———————
8 Client threads started
——— Average TP for all 8 threads:820463 read/sec———————
——— Exit Demo ————–
The expected output for GigaSpaces benchmark should look like this:
GigaSpaces cache benchmark started!
1 Client threads started
——— Average TP for all 1 threads:8312982 read/sec———————
2 Client threads started
——— Average TP for all 2 threads:7971583 read/sec———————
3 Client threads started
——— Average TP for all 3 threads:8069370 read/sec———————
4 Client threads started
——— Average TP for all 4 threads:8783607 read/sec———————
5 Client threads started
——— Average TP for all 5 threads:8414489 read/sec———————
6 Client threads started
——— Average TP for all 6 threads:8356409 read/sec———————
7 Client threads started
——— Average TP for all 7 threads:8451054 read/sec———————
8 Client threads started
——— Average TP for all 8 threads:8225495 read/sec———————
——— Exit Demo ————–

Benchmark was running on Linux OS, using Sun JDK 1.6 , 16 Cores Intel CPU , Cisco UCS Machine.

Disclaimer

The benchmark in this post tests Echcache in a local cache mode only (standalone) vs GigaSpaces Local Cache. It compares the scalability of get operation which is a common use case for client side cache. The setup is based on the default out of the box configuration of both products. This benchmark wasn’t designed to test and compare the full Terracotta and GigaSpaces product. I kept the results objective providing full source code and configuration for the benchmark. I encourage any of you to try out the benchmark yourself and see the result. Your Feedback is appreciated !

Shay Hassidim

Deputy CTO , GigaSpaces