Jofti

Info

Jofti is designed to be completely modular so that different strategies for Indexing and parsing objects can be slotted into the framework. The index, parser and Cache adapters are all configurable at runtime enabling support for different index types, and adapters to be used without code changes. See the User Guide for details on usage.

 

Technical overview

Jofti is made up of a number of modules. The diagram below shows an instance of Jofti configured with two Cache instances. One configured as a ListenerAdapter where changes are applied to the Cache and the results are picked up as callbacks to the Index, and the second as a wrapper adapter where changes are applied to the Cache via the adapter.

The Index Manager

The Index Manager is responsible for providing access to the Caches and Indexes for the client code. All Jofti instances have to be retrieved or added using the Cache Manager.

This object is also responsible for managing the Index and Cache lifecycles for configured Caches as well as adding in Indexes to an already existing Cache instance.

The adapter

An adapter exists to provide an interface between the indexes and a particular Cache implementation through a well defined set of interfaces.

There are a number of adapters that ship with Jofti.
These are:

• MapCacheAdapter
• JBossCache Adapter
• EHCacheAdapter
• OSCacheAdapter
• CoherenceListenerAdapter
• EHCacheListenerAdapter
• OSCacheListenerAdapter
• ObjectGridListenerAdapter

The actions that act upon the cache are either mediated through an adapter or, if using one of the listener adapters driven via the Cache's own callback methods to a registered listener. The adapter is responsible for performing any actions that are required to keep the data in the cache and the index in synch. The adapter is also responsible for handling callback events (where appropriate) that are used to signal events like data expiry, addition or removal.
Support for other types of cache can be added by creating an appropriate adapter and configuring it in the index.

The Index

The implementation currently available in Jofti is a concurrent BTree* index based on the B-link Tree paper by Lehman and Yao on locking for concurrent operations on b-trees - 1981.

The use of an ordered indexing structure places some restrictions on the type of data that can be indexed. The main one is that the data types of Object attributes that you want to search on must implement Comparable. The classes in the JVM that already implement this interface and are supported by Jofti by default are:

• java.lang.String
• java.lang.Integer
• java.lang.Double
• java.lang.Float
• java.lang.Long
• java.lang.Short
• java.lang.Byte
• java.math.BigInteger
• java.math.BigDecimal
• java.util.Date
• java.sql.Timestamp
• java.net.URI

And the primitive classes:

• int
• long
• float
• double
• short
• char

All of Java's Primitive types (except boolean) also can be used as Jofti boxes the primitive in its Java wrapper Class. The boolean is provided with a Comparable wrapper class by Jofti and user-defined types that implement Comparable can also be used.

Although, the data in an index must be Comparable, it need not be homogenous in type. It is perfectly permissable for a single index to contain data of many different types. However, the types are not mingled and cannot be searched as if they were the same. E.g. if you put in two String "foo", "bah" into the index and an Integer 20 - the query "String = 20" will fail to find any matches.

This is termed a dimension in the index. Each simple class type has its own dimension in the index.

Similarly, data of the same type is partitioned into dimensions according to the field that it is in each particular class. So given a String type field called 'property' in class Foo in the index and a String type field called 'property' in class Bah will reside in separate dimensions.

The concurrent BTree approach was chosen as the initial index type because of a few reasons:

• While not being as theoretically optimal as a Binary Tree,or an AVL Tree (as the structure is in memory), there is a lot less overhead in the BTree to achieve reasonable concurrency (as there is no rotation step needed).
• Version 1.2 of Jofti enables a maximum number of nodes from the Index to be kept in memory and page the rest to disk - again easier to achieve with BTrees.
• Linear Hashing or other hash based indexing can provide a quicker lookup of individual values and is more suited to some types of indexing requirements, but they are unable to perform range searches and the BTree approach gives a better alround flexibilty as the default index type. (Linear hashes and other indexing techniques will be included as modules in a subsequent release of Jofti).

Object Introspector

The intropector is responsible for constructing a suitable representation of which attributes and classes are to be indexed and introspecting the objects passed in to the Cache based on this representation and retrieving the values to be placed into the Index.

The current implementation is based on the property naming conventions in the JavaBean format which can access properties via compliant getters.

Accordingly, specification of the attributes in the configuration is also provided by JavaBean property notation.

Query Parser

The query parser, which uses ANTLR for the String parsing step, is responsible for translating the syntax tree generated by the String parsing into a suitable internal query format. It is also responsible for interpreting the nested predicates (if any) in the query and providing hints to Jofti as to the type of join operations to perform on each predicate in the query.

The current implementation supports SQL or EJB3 QL format queries, although an XPath compliant module will also probably be included in a subsequent version.