I have been involved in a nice gaming project in the last couple of weeks.
In the first meeting, giving an overview, the prospect presented their challenge:
“We store game data in memory as we need very fast response time, and we are reaching our limits in terms of supporting our business. We are launching games which support many thousands of users and we are not able to store a game’s data in a single server.
We tried using a database and running our aggregations against it. The response time we got was not acceptable. We want to store game’s data across multiple JVM, but how will which JVM stores which pieces of the data?”
Well, aggregations and data partitioning go very well together if you have the right technology in hand. Nati Shalom wrote a good post on this subject.
The interesting thing is that GigaSpaces supports aggregation since way before the Map/Reduce term was even thought of… When using JavaSpaces’ readMultiple API against a clustered space, the read request is broadcasted to all the cluster members, who execute the request (locally) in parallel, the client side receives back an array of responses which it then needs to aggregate (Reduce). With OpenSpaces, we have Springified this and it is also supported through Spring Remoting with a pluggable reducer at the client side.
A couple of hours after the dev was kicked off, we were able to demo how a single game can span multiple JVMs, aggregate the game’s data in a simple fashion, and display the aggregated result. The system, which was limited to x10,000 concurrent users, can now scale to x100,000 and x1,000,000 linearly using GigaSpaces.