Schema Evolution

Added by Gilad David Maayan, last edited by Aleia Hutcherson on May 25, 2008  (view change)
  GigaSpaces 5.X

Documentation Home
Quick Start Guide
Release Notes

Previous release
  Search Here
Searching GigaSpaces Platform 5.X Documentation

                                               
Summary: The space schema can be evolved by using codebase downloading and Java extentions, or the explicit data migration process.

Overview

Developing software is a never ending process. There is never a final version or a last release. This means you need to deploy new versions of the application, while using a modified version of the database schema. In many cases, the deployment is done by adding space classes, or changing the structure of existing space classes – removing some field classes or changing the inheritance structure.

GigaSpaces supports addition of new space classes, as well as space data migration from one schema to another. New classes can be added without interrupting the space operation. Data migration can be done using the space underlying Entry representation – the ExternalEntry. A simple migration program can transform data stored in the space from one schema to a new schema, and create it in a new space that is used by the new version of the application.

The following sections describe both of these patterns.

Evolving the Space Schema using Inheritance

You can evolve the space schema using codebase downloading and inheritance, by adding a new Entry class type to your application. This class can extend an existing space Entry class. Clients are able to work with the existing Entries, as well as the newly added Entry types.

Example

  1. Class A is written to a space by client X.
  2. Class A is taken or read from the space by client Y, using a template based on A.
  3. At some point, an A' class is created, which extends A.
  4. Entries from type A' are written to the space by client Z.
  5. Class A' meta data is acquired by the space server via a codebase from client Z.
  6. Client X now receives Entries from type A', and A' meta data is acquired by client X from the client Z codebase.
  7. Client X continues to typecast A' to A, and to work with Entries of type A', as if they were of type A.

The system does not have to shut down during the process and clients are not interrupted.

Evolving the Space Schema using Data Migration

With this pattern, a simple program should be created in order to transform a space Entry from one schema to another.

The transformer application reads an Entry from the source space (the old class schema that was used), transforms it to its new schema, and writes it to a target space (the new class schema) that is used with the new version of the application.

As seen in the code example below, every migrated Entry preserves its Entry UID.

The Entries are read from the source space using the GSIterator, and are written into the target space using transactions and the writeMultiple operation.

Below is a transformer program, that transforms a class with two attributes into a class with one attribute.

To speed up the transformation process, you may implement the transformation process as IWroker, by running the transformation business logic in the source space or in the target space.

The migration process can also be optimized by running multiple threads that handle the migration of different space classes in parallel.

Example

package com.j_spaces.transformer;

import java.util.ArrayList;
import java.util.Collection;
import net.jini.core.entry.Entry;
import net.jini.core.lease.Lease;
import net.jini.core.transaction.Transaction;
import net.jini.core.transaction.TransactionFactory;
import com.j_spaces.core.IJSpace;
import com.j_spaces.core.client.ExternalEntry;
import com.j_spaces.core.client.GSIterator;
import com.j_spaces.core.client.LocalTransactionManager;
import com.j_spaces.core.client.SpaceFinder;

public class Transformer {
	public Transformer() {
	}

	public IJSpace sourceSpace;

	public IJSpace targetSpace;

	static String namesOld[] = { "attrib1", "attrib2" }, typesOld[] = {
			String.class.getName(), String.class.getName() };

	static String namesNew[] = { "attrib1" }, typesNew[] = { String.class
			.getName() };

	static String className = "com.j_spaces.test.MyClass";

	public static void main(String[] args) {
		Transformer transformer = new Transformer();

		try {
			transformer.sourceSpace = (IJSpace) SpaceFinder
					.find("rmi://localhost/./sp1");
			transformer.targetSpace = (IJSpace) SpaceFinder
					.find("rmi://localhost/./sp2");

			ExternalEntry template1 = new ExternalEntry(className, null,
					namesOld, typesOld);
			ExternalEntry template2 = new ExternalEntry(className, null,
					namesNew, typesNew);
			transformer.sourceSpace.snapshot(template1);
			transformer.targetSpace.snapshot(template2);

		} catch (Exception e) {
			e.printStackTrace();
			System.exit(0);
		}

		transformer.tranform();
		System.out.println("Transformation done!");
	}

	void tranform() {
		Collection templates = new ArrayList();
		try {
			ExternalEntry template = new ExternalEntry(className, null,
					namesOld, typesOld);
			templates.add(template);
			LocalTransactionManager transactionManager = (LocalTransactionManager) LocalTransactionManager
					.getInstance(targetSpace);
			GSIterator sourceIterator = new GSIterator(sourceSpace, templates,
					100, true, Lease.FOREVER);
			int count = 0;
			Transaction.Created tCreated = null;
			Transaction txn = null;
			ArrayList<Entry> entries = new ArrayList<Entry>();
			while (sourceIterator.hasNext()) {
				ExternalEntry entry = (ExternalEntry) sourceIterator.next();
				Object newValues[] = new Object[1];
				newValues[0] = entry.getFieldValue(0);
				ExternalEntry newentry = new ExternalEntry(className,
						newValues, namesNew, typesNew);
				newentry.m_UID = entry.m_UID;
				count++;
				entries.add(newentry);
				if (count == 100) {
					while (true)
						try {
							// startTX
							tCreated = TransactionFactory.create(
									transactionManager, 1000);
							txn = tCreated.transaction;
							Entry[] ents = new Entry[entries.size()];
							entries.toArray(ents);
							targetSpace.writeMultiple(ents, txn, Lease.FOREVER);
							txn.commit();
							System.out.println("Done writeMultiple!");
							count = 0;
							entries.clear();
							break;
						} catch (Exception e) {
							txn.abort();
							e.printStackTrace();
							System.out.println("try again....");
						}
				}
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

Wiki Content Tree

Your Feedback Needed!

We need your help to improve this wiki site. If you have any suggestions or corrections, write to us at techw@gigaspaces.com. Please provide a link to the wiki page you are referring to.

Labels

 
(None)