8.2. Persist

To load data, we use RStore which has following features,

• enhance java classes for persistence

• create database schema

• delete database schema

• persist and retrieve objects from database

Prepare RStore project

Download RStore.zip from the book site. Create a regular Java project named rstore with default settings. Select the project in Project Explorer and choose File → Import. In Import dialog, expand general and choose Archive File and import RStore.zip into folder rstore.

Dependencies

RStore requires HSQLDB, Data Nucleus JDO, MyBatis and Hibernate libraries. Following table lists the dependencies.

Table 8.1. Dependencies

Table 8.2. Download URL

Download the packages and place the essential jars in projects lib directory. Also add the jars to project build path.

Automatic Dependency Management

Thats a handful of downloads. Rather than struggling with downloads, it makes a lot of sense to use dependency management softwares like Maven or Apache Ivy.

Maven is the leading contender in this space, but if you are in a hurry then go for Apache Ivy, which is equally powerful. Apache Ivy is quite easy to start with, zero configuration and comes with a Plugin for Eclipse.

With Eclipse Plugin, IvyDE, you may shift to automatic dependency management in just under ten minutes. All you have to do is, install IvyDE plugin and add the ivy.xml file, which is in the root folder of RStore project, as Ivy Library. Thats all, Ivy downloads all the dependencies and add them to project. It also updates project build path, links package’s API documentation and source. In case jars are deleted from project workspace, it restores them from its cache to keep the project in shipshape. Refer Apache Ivy for quick start.

Run RStore

Open HSQLDB view and start HSQLDB. Next, open Ant View using Windows → Show View. Drag and drop build.xml to Ant View, which shows following targets.

Figure 8.3. Ant View

Select Run target and run it. RStore compiles the project and enhances the domain classes for persistence using Data Nucleus Enhance. It then parses fins.xml file from data directory and creates four Symbol with DataGroup, Data and Facts as found in xml file. Using JDO, it creates the database schema and inserts four symbol objects to database tables. Once objects are persisted, RStore retrieves a Symbol with all related data first via JDO and then through alternate ORM, Hibernate and MyBatis and output the details to Console. At this point, you may browse the tables using Database Manager. Targets createschema creates empty tables and dropschema drops the schema.

XML file, data/fins-data.xml, contains data for four symbols from year 2007 onwards. These data are from real listed companies from India where financial year generally between 1st April to 31st March, hence you will find financial year end dates as 31st March instead of usual 31st Dec. Class in.rstore.processor.SymbolParser parse the XML file. Method startElement() creates a new empty Symbol and pushes it to a Stack and as parse progresses, populates the Symbol with data. On SAX endElement event, it pops Symbol object from the stack and passes it to ISymbolProcessor whose method ISymbolProcessor.processSymbol() persists the object using JdoData. Let’s see how JDO accomplish this.

ORM Configuration

ORM softwares use two types of setup files.

• Configuration file - ORM requires various configurations to deal with different databases and this file defines configuration properties that is essential to connect and interact with database.

• Mapping file - maps Java classes to tables and its fields to table columns.

For these files, each ORM has its own naming convention and path. Following table shows RStore’s configuration and mapping files.

Table 8.3. ORM Setup

In this section, we use JDO to persist Symbols. JDO is a specification to persist data. There are many implementations of JDO specification and we use implementation by DataNucleus known as DataNucleus Access Platform.

Configuration file

DataNucleus JDO uses datanucleus.properties to configure connection and other properties.

src/datanucleus.properties

javax.jdo.option.ConnectionDriverName=org.hsqldb.jdbcDriver
javax.jdo.option.ConnectionURL=jdbc:hsqldb:hsql://localhost/finsdb
javax.jdo.option.ConnectionUserName=sa
javax.jdo.option.ConnectionPassword=
javax.jdo.option.Mapping=hsql


datanucleus.metadata.validate=false
datanucleus.autoCreateSchema=true
datanucleus.validateTables=false
datanucleus.validateConstraints=false

Preferences set for HSQLDB is used to configure the JDBC connection.

Java Object to Table Mapping

JDO uses package.jdo as its mapping file, and allows to place it in various locations. We prefer to place it along with persistable classes.

in.fins.shared/package.jdo

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jdo PUBLIC
    "-//Sun Microsystems, Inc.//DTD Java Data Objects Metadata 2.0//EN"
    "http://java.sun.com/dtd/jdo_2_0.dtd">
<jdo>
    <package name="in.fins.shared">
        <class name="Symbol">
            <field name="name" primary-key="true" />
            <field name="label" />
            <field name="dataGroups">
                <element column="symbol_name"/>
            </field>
            <query name="selectSymbolNames" language="javax.jdo.query.JDOQL"><![CDATA[
            SELECT s.name FROM in.fins.shared.Symbol s
            ]]></query>
        </class>
        <class name="DataGroup">
            <field name="id" primary-key="true" value-strategy="identity" />
            <field name="category" />
            <field name="dataList" >
                <element column="datagroup_id"/>
            </field>
            <field name="positionDate" persistence-modifier="none" />
        </class>
        <class name="Data">
            <field name="id" primary-key="true" value-strategy="identity" />
            <field name="date" />
            <field name="facts" serialized="true" />
        </class>
    </package>
</jdo>

Package element names the package that contains persistable classes and Class element maps a class to a table and Field element map field to table column. In Class element, table attribute is used to name the table, and in its absence, class name is used as table name. For primary key field, primary-key attribute is used. Attribute persistence-modifier set to none indicates that field is not persistable as in field positionDate. We will explain the Query element in the next section when we retrieve the objects. JDO creates following schema from this mapping file.

Figure 8.4. Schema

In the absence of table attribute in Class elements, JDO uses class names to name tables. JDO maps the simple fields like String, Date and long to equivalent database column types but it is quite interesting to understand how JDO maps the List<T> items.

For example, take Symbol which has a field List<DataGroup> that holds a list of DataGroup which comes from certain rows of DATAGROUP table. In JDO, this 1-N unidirectional relationship may be handled using three strategies; Join Table, Primay Key - Foreign Key (PK-FK) and Serialized list. We use PK-FK and Serialized list strategies in RStore.

We use PK-FK strategy for List<DataGroup> field in Symbol, which is specified as

    <field name="dataGroups">
       <element column="symbol_name"/>
    </field>

This tells JDO to create a column SYMBOL_NAME in DATAGROUP table and use it as foreign key of primary key of SYMBOL. To construct the list of DataGroup, JDO retrieves rows from table DATAGROUP whose SYMBOL_NAME is same as SYMBOL.NAME. Likewise, between DATAGROUP and DATA tables, DATAGROUP.ID and DATA.DATAGROUP_ID are PK and FK respectively.

JDO also creates columns DATAGROUPS_INTEGER_IDX and DATALIST_INTEGER_IND and use them to sort the rows, if required.

Next, let’s understand Serialized list strategy. Field List<Fact> in Data class, uses this strategy as we have set serialize attribute to true. Usually, items from a collection are stored in individual rows, but in some cases it is optimal to store the whole list in a single row. There are roughly 800 facts in a Symbol, and FACT table ends up with 800 rows for each symbol if we use PK-FK strategy. But with the Serialization, JDO serializes the List<Fact> and inserts it into a column named Facts which occupies a single row of DATA table.

There is a couple of drawbacks with this approach. For starters, one will not be able to query the facts directly. But most importantly, it is not possible to deserialize facts into a Fact of another package. In other words, fully qualified name of the class should be same in both phases, persistence and retrieval. For this reason, we placed all domain classes in package in.fins.shared instead of say in.rstore.xyz, as we have to retrieve them in Fins. Classes and fields which are not serialized, are freely interchangeable between any package, but we must honor the java serialization rules for serialized field, else we get Class cast exception in runtime.

JDO and Hibernate also supports annotation of persistence classes and with annotation one may avoid mapping files altogether. But with this, persistence classes end up with ORM specific annotations. To avoid lock-in with a specific ORM, we go with mapping files for the sake of interchangeability.

Persist

Once we are through with the mapping, it is quite easy to persist an object in JDO. Following code block explains JDO persistence logic.

        PersistenceManagerFactory pmf =
                JDOHelper.getPersistenceManagerFactory("datanucleus.properties");    

        PersistenceManager pm = pmf.getPersistenceManager();                         
        Transaction tx = pm.currentTransaction();                                    try {
            tx.begin();
            pm.makePersistent(symbol);                                               
            tx.commit();                                                             
        } finally {
            if (tx.isActive()) {
                tx.rollback();
            }
            pm.close();
        }

The actual method that takes care of persistence in RStore is JdoDao.insert().

in.rstore.dao.jdo/JdoDao.java

        public void insert(Symbol symbol) {
                PersistenceManager pm = pmf.getPersistenceManager();
                Transaction tx = pm.currentTransaction();
                try {
                        tx.begin();
                        Symbol existingSymbol = pm.getObjectById(Symbol.class,
                                        symbol.getName());
                        SymbolHelper.updateSymbol(symbol, existingSymbol);
                        tx.commit();
                } catch (JDOObjectNotFoundException e) {
                        pm.makePersistent(symbol);
                        tx.commit();
                } finally {
                        if (tx.isActive()) {
                                tx.rollback();
                        }
                        pm.close();
                }
        }

JdoDao.insert(). initially tries to retrieve symbol object if similar one already exists in the database. If found, it uses SymbolHelper.updateSymbol() method to update the retrieved object’s fields with new values. JDO keeps track of any modification to retrieved object and auto updates the tables on any changes. In case object is not found in the database, then PM.makePersistent() method persists the new symbol.

Next section explains object retrieval via JDO, Hibernate or MyBatis.