Background

GWT generates JavaScript code that makes a call to server side components thru the Servlet Interface. The Service Implementation class extends the “RemoteServiceServlet” class. This allows GWT to respond to requests made from the GWT generated Javascript.

Requirements

• Maven
• Java 5 or above
• Successful completion of my previous post

Procedure

Before we begin we need to create directories.

mkdir -p src/main/java/com/test/server

Create the Interface

First you create a an interface that can be used by the client. Maven generates a client side Interface based this.

vi src/main/java/com/test/client/GreetingService.java

package com.test.client;

import com.google.gwt.user.client.rpc.RemoteService;
import com.google.gwt.user.client.rpc.RemoteServiceRelativePath;

@RemoteServiceRelativePath("greet")
public interface GreetingService extends RemoteService {
  String greetServer(String input) throws IllegalArgumentException;
}

Create the Implementation

vi src/main/java/com/test/server/GreetingServiceImpl.java

package com.test.server;

import com.test.client.GreetingService;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;

/**
 * The server side implementation of the RPC service.
 */
@SuppressWarnings("serial")
public class GreetingServiceImpl extends RemoteServiceServlet implements
    GreetingService {

  public String greetServer(String input) throws IllegalArgumentException {
    String serverInfo = getServletContext().getServerInfo();
    String userAgent = getThreadLocalRequest().getHeader("User-Agent");

    return "Hello, " + input + "! I am running " + serverInfo
        + ".It looks like you are using:" + userAgent;
  }
}

Define the Servlet

Add the servlet to the web.xml.

vi src/main/webapp/WEB-INF/web.xml

  <!-- Servlets -->
  <servlet>
    <servlet-name>greetServlet</servlet-name>
    <servlet-class>com.test.server.GreetingServiceImpl</servlet-class>
  </servlet>

  <!-- Servlet-Mapping -->
  <servlet-mapping>
    <servlet-name>greetServlet</servlet-name>
    <url-pattern>/matrix/greet</url-pattern>
  </servlet-mapping>

Change the EntryPoint

Make the changes described in the following patch file by hand to the existing file.

vi src/main/java/com/test/client/Matrix.java

--- src/main/java/com/test/client/Matrix.java
+++ src/main/java/com/test/client/Matrix.java	
@@ -5,15 +5,26 @@
 import com.google.gwt.event.dom.client.ClickEvent;
 import com.google.gwt.event.dom.client.ClickHandler;
 import com.google.gwt.user.client.Window;
+import com.google.gwt.user.client.rpc.AsyncCallback;
 import com.google.gwt.user.client.ui.Button;
 import com.google.gwt.user.client.ui.RootPanel;
 
 public class Matrix implements EntryPoint {
-
+	private final GreetingServiceAsync greetingService = GWT.create(GreetingService.class);
+	
 	public void onModuleLoad() {
 		Button button = new Button("Send", new ClickHandler() {
 			public void onClick(ClickEvent event) {
-				Window.alert("Hello World!");
+		        greetingService.greetServer("World", new AsyncCallback<String>() {
+		            public void onFailure(Throwable caught) {
+		              // Show the RPC error message to the user
+		              Window.alert("Remote Procedure Call - Failure");
+		            }
+
+		            public void onSuccess(String result) {
+		                Window.alert(result);
+		            }
+		          });
 			}
 		});
 		button.setStyleName("sendButton");

Test the application

mvn compile gwt:run

After clicking the button on the screen a JavaScript alert box will open returning text from the server side. You can use Firefox FireBug plugin to verify the server side communication.

Requirements

• Maven
• Java 5 or above
• Successful completion of my previous post

Project Configuration

In order to enable internationalization support you need to add the following “goal” and “i18nMessagesBundle” elements inside “gwt-maven-plugin”:

vi pom.xml

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>gwt-maven-plugin</artifactId>
        <version>2.4.0</version>
        <executions>
          <execution>
            <goals>
            ...
            <goal>i18n</goal>
            ...
            </goals>
...

insert the “i18nMessagesBundle” element here:

        <configuration>
          ...
          <i18nMessagesBundles>
                    <i18nMessagesBundle>com.test.client.Messages</i18nMessagesBundle>
          </i18nMessagesBundles>
          ...
        <configuration>

Externalize the language strings to a property file.

The following message bundle is converted to a Java Class by Maven. In order to specify additional bundles just insert an additional “i18nMessagesBundle” tag in the pom.xml file.

Properties File

Create the directory that will hold the property file if not done so already:

mkdir -p src/main/resources/com/test/client

vi src/main/resources/com/test/client/Messages.properties

sendButton = Send
hello = Hello {0}!

Java code Change

Modify the java code to reference the property values instead of hard-coded strings.

--- matrix-blank/src/main/java/com/test/client/Matrix.java	
+++ matrix/src/main/java/com/test/client/Matrix.java	
@@ -10,10 +10,11 @@
 
 public class Matrix implements EntryPoint {
 
+	private final Messages messages = GWT.create(Messages.class);
 	public void onModuleLoad() {
-		Button button = new Button("Send", new ClickHandler() {
+		Button button = new Button(messages.sendButton(), new ClickHandler() {
 			public void onClick(ClickEvent event) {
-				Window.alert("Hello World!");
+				Window.alert(messages.hello("World"));
 			}
 		});
 		button.setStyleName("sendButton");

Test the Change

mvn compile gwt:run

The French Version

Creating a French version of the application is simple!. Just Create a copy of the file and append an _fr or the locale you want to support.

cp src/main/resources/com/test/client/Messages.properties \
src/main/resources/com/test/client/Messages_fr.properties

Modify the file and update the English to French.

vi src/main/resources/com/test/client/Messages_fr.properties

sendButton = Envoyer
hello = bonjour {0}!

Configure

You can add the following line into the module element of your “gwt.xml” file for each locale you want to support:
vi src/main/resources/com/test/Matrix.gwt.xml

<extend-property name="locale" values="fr"/>

Start the application

mvn compile gwt:run

Test the Change

At this point you have 2 options to see the French version of the site:

1. You can put the following line in the host HTML page
   

   <meta name="gwt:property" content="locale=fr">
   

2. Append the client property value to the query string of the URL: http://127.0.0.1:8888/Matrix.html?gwt.codesvr=127.0.0.1:9997?locale=fr

Either way you will see the French version of the site.

Reference

http://code.google.com/webtoolkit/doc/latest/tutorial/i18n.html

Appendix

Creating Keys for place holder parameters, the lines of the property file should look like this:

myString = First parm is {0}, second parm is {1}, third parm is {2}.

The key “myString” can be used in static HTML by wrapping the content with a html tag with an assigned id.

    <h1 id="myString"></h1>

The id attribute can be used as a handle to replace the text in Java code like this:

    RootPanel.get("appTitle").add(new Label(constants.myString("one","two","three")));

To get the list of locale’s supported by your JVM just run the following code:

	public static void main(String args[]) {
		Locale locales[] = SimpleDateFormat.getAvailableLocales();
		List arrayList = Arrays.asList(locales);
		Collections.sort(arrayList, new Comparator() {
			public int compare(Locale o1, Locale o2) {
				return o1.getDisplayName().compareTo(o2.getDisplayName());
			}
		});
		for (Locale locale : arrayList) {
			System.out.println(locale.toString() + "\t" + locale.getDisplayName());
		}
	}

The application described on this page displays a Send Button on the page. It displays a JavaScript alert() message when the button is clicked.

Background

The GWT SDK allows you to generate an application using their generation tool. However I never liked using this tool because the application it generated is useless to me unless I understand how the application is working. The following page breaks down a simple Hello World GWT application step by step and allows the reader to follow along.

Requirements

• Maven
• Java 5 or above

This page covers GWT version 2.4.0.

The first step is to create directory that will hold your project. The example below uses “matrix” as the project name. You can replace each instance of “matrix” with whatever you want to call your project.

mkdir matrix
cd matrix
# create some additional directories
mkdir -p src/main/java/com/test/client
mkdir -p src/main/resources/com/test
mkdir -p src/main/webapp/WEB-INF

vi pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project
  xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <!-- POM file generated with GWT webAppCreator -->
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.test</groupId>
  <artifactId>matrix</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>GWT Maven Archetype</name>

  <properties>
    <!-- Convenience property to set the GWT version -->
    <gwtVersion>2.4.0</gwtVersion>
    <!-- GWT needs at least java 1.5 -->
    <webappDirectory>${project.build.directory}/${project.build.finalName}</webappDirectory>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.google.gwt</groupId>
      <artifactId>gwt-servlet</artifactId>
      <version>${gwtVersion}</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.google.gwt</groupId>
      <artifactId>gwt-user</artifactId>
      <version>${gwtVersion}</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.7</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <!-- Generate compiled stuff in the folder used for developing mode -->
    <outputDirectory>${webappDirectory}/WEB-INF/classes</outputDirectory>
    <plugins>
      <!-- GWT Maven Plugin -->
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>gwt-maven-plugin</artifactId>
        <version>2.4.0</version>
        <executions>
          <execution>
            <goals>
              <goal>compile</goal>
              <goal>test</goal>
              <goal>generateAsync</goal>
            </goals>
          </execution>
        </executions>
        <!-- Plugin configuration. There are many available options, see 
          gwt-maven-plugin documentation at codehaus.org -->
        <configuration>
          <runTarget>Matrix.html</runTarget>
          <hostedWebapp>${webappDirectory}</hostedWebapp>
        </configuration>
      </plugin>

      <!-- Copy static web files before executing gwt:run -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <executions>
          <execution>
            <phase>compile</phase>
            <goals>
              <goal>exploded</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <webappDirectory>${webappDirectory}</webappDirectory>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.3.2</version>
        <configuration>
          <source>1.5</source>
          <target>1.5</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Host HTML Page

The following is the Host HTML Page. The page imports the generated Javascript and starts the Javascript application. Similar to the “Entry Point” main() method of many other programming languages.

vi src/main/webapp/Matrix.html

<!doctype html>
<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
    <link type="text/css" rel="stylesheet" href="Matrix.css">
    <title>X</title>
    <script type="text/javascript" language="javascript" src="matrix/matrix.nocache.js"></script>
  </head>
  <body>
    <iframe src="javascript:''" id="__gwt_historyFrame" tabIndex='-1' style="position:absolute;width:0;height:0;border:0"></iframe>
    <noscript>
      <div style="width: 22em; position: absolute; left: 50%; margin-left: -11em; color: red; background-color: white; border: 1px solid red; padding: 4px; font-family: sans-serif">
        Your web browser must have JavaScript enabled
        in order for this application to display correctly.
      </div>
    </noscript>
    <div id="sendButtonContainer"></div>
  </body>
</html>

CSS Styles

GWT components are highly customizable. It makes sense to define sizes, colors, alighment, images, and other visual aspects of the component in CSS.

vi src/main/webapp/Matrix.css

.sendButton {
  display: block;
  font-size: 12pt;
}

Web Application Descriptor

The following is a basic web.xml file for the application.

vi src/main/webapp/WEB-INF/web.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
    "http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>

  <!-- Servlets -->
  <!-- Servlet-Mapping -->

  <!-- Default page to serve -->
  <welcome-file-list>
    <welcome-file>Matrix.html</welcome-file>
  </welcome-file-list>
</web-app>

vi src/main/resources/com/test/Matrix.gwt.xml

<?xml version="1.0" encoding="UTF-8"?>
<module rename-to='matrix'>
  <inherits name='com.google.gwt.user.User' />
  <inherits name='com.google.gwt.user.theme.standard.Standard' />
  <entry-point class='com.test.client.Matrix' />
  <source path='client' />
</module>

vi src/main/java/com/test/client/Matrix.java

package com.test.client;

import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.core.client.GWT;
import com.google.gwt.event.dom.client.ClickEvent;
import com.google.gwt.event.dom.client.ClickHandler;
import com.google.gwt.user.client.Window;
import com.google.gwt.user.client.ui.Button;
import com.google.gwt.user.client.ui.RootPanel;

public class Matrix implements EntryPoint {

	public void onModuleLoad() {
		Button button = new Button("Send", new ClickHandler() {
			public void onClick(ClickEvent event) {
				Window.alert("Hello World!");
			}
		});
		button.setStyleName("sendButton");
		RootPanel.get("sendButtonContainer").add(button);
	}
}

Run the project

mvn eclipse:clean eclipse:eclipse
mvn clean compile gwt:run

What’s Next?

• Internationalize your application
• Invoke Server Side Code Using GWT

Background

The Commons Validator Framework is composed of a core set of classes responsible for validating data. Since the framework is versatile and does not make assumptions about implementation, you can’t use it out of the box unless you write wrapper code to adapt it to your needs.

Years ago the popular Struts framework had provided a set of wrapper classes in the “org.apache.struts.validator” package. The spring MVC framework had made similar adaptations. I had found it strange that the makers of spring batch did not have a drop in replacement so that had lead me to write this page.

Requirements

• Java 5 or above with Maven 2
• Successful completion of the example application described here.

This tutorial continues from where the previous one left of so please complete the examples in those articles first before proceeding.

Procedure

Start by adding the following dependencies into your pom.xml file.

pom.xml

<!-- The commons Validator Framework -->
	<dependency>
	    <groupId>commons-validator</groupId>
	    <artifactId>commons-validator</artifactId>
	    <version>1.1.4</version>
	</dependency>
<!-- Spring Wrapper classes for the Validator Framework -->
        <dependency>
	    <groupId>org.springmodules</groupId>
	    <artifactId>spring-modules-validation</artifactId>
	    <version>0.8</version>
	</dependency>

The following is the core class that makes this all happen.

src/main/java/com/test/SpringCommonsValidator.java

package com.test;

import java.util.List;

import org.springframework.batch.item.validator.ValidationException;
import org.springframework.batch.item.validator.Validator;
import org.springframework.context.MessageSource;
import org.springframework.context.MessageSourceResolvable;
import org.springframework.validation.BeanPropertyBindingResult;
import org.springframework.validation.Errors;

/**
 * Allows for the commons validation framework to print human readable 
 * validation failure messages. Also its very good at adapting the 
 * Spring Validator Interface to Spring Batch Validator, thus allowing 
 * this class to be used as a Validator in the spring batch framework.
 * 
 * This class delegates most of its work to the spring validator.
 * 
 */
public class SpringCommonsValidator<T> implements Validator<T> {

	private org.springframework.validation.Validator validator;
	private MessageSource messageSource;

	public void validate(T item) throws ValidationException {

		if (!validator.supports(item.getClass())) {
			throw new ValidationException("Validation failed for " + item + ": " + item.getClass().getName()
					+ " class is not supported by validator.");
		}

		BeanPropertyBindingResult errors = new BeanPropertyBindingResult(item, "item");

		validator.validate(item, errors);

		if (errors.hasErrors()) {
			throw new ValidationException("Validation failed for " + item + ": " + errorsToString(errors));
		}
	}

	private String errorsToString(Errors errors) {
		@SuppressWarnings("rawtypes")
		List errorList = errors.getAllErrors();
		StringBuffer buffer = new StringBuffer();
		
		boolean first = true;
		for (Object object : errorList) {
			if(object instanceof MessageSourceResolvable) {
				MessageSourceResolvable r = (MessageSourceResolvable)object;
				if(!first) { buffer.append("; "); } 
				buffer.append(messageSource.getMessage(r, null));
				first = false;
			}
		}
		return buffer.toString();
	}


	public void setValidator(org.springframework.validation.Validator validator) {
		this.validator = validator;
	}

	public void setMessageSource(MessageSource messageSource) {
		this.messageSource = messageSource;
	}

	public MessageSource getMessageSource() {
		return messageSource;
	}
}

Resource Bundle

The following property file is used to display the error messages. I guess some additional work can be done to internationalize these messages but since batch programs are not end-client facing… what’s the point?

On the other hand, having error messages in a property files does have its advantages.

src/main/resources/messages.properties

errors.required={0} is required.
errors.minlength={0} can not be less than {1} characters.
errors.maxlength={0} can not be greater than {1} characters.
errors.invalid={0} is invalid.
 
errors.byte={0} must be a byte.
errors.short={0} must be a short.
errors.integer={0} must be an integer.
errors.long={0} must be a long.
errors.float={0} must be a float.
errors.double={0} must be a double.
 
errors.date={0} is not a date.
errors.range={0} is not in the range {1} through {2}.
errors.creditcard={0} is an invalid credit card number.
errors.email={0} is an invalid e-mail address.

Configuration

Modify the spring configuration file to add the validatorFactory, beanValidator and springValidator. These classes allow you to have the ValidatingItemProcessor validate each item that is passed to it.

src/main/resources/simpleJob.xml

Replace the following section

<!-- Processor -->
<beans:bean name="empProcessor" class="com.test.EmployeeProcessor">
</beans:bean>

with the following

<!-- Processor -->
<beans:bean name="empProcessor" class="org.springframework.batch.item.validator.ValidatingItemProcessor">
    <beans:property name="validator" ref="springValidator"/>    
</beans:bean>

<!-- Validator -->
    <beans:bean id="validatorFactory"
          class="org.springmodules.validation.commons.DefaultValidatorFactory">
    <beans:property name="validationConfigLocations">
        <beans:list>
          <beans:value>validation.xml</beans:value>
          <beans:value>validator-rules.xml</beans:value>
        </beans:list>
      </beans:property>
    </beans:bean>
     
    <beans:bean id="beanValidator" class="org.springmodules.validation.commons.DefaultBeanValidator">
        <beans:property name="validatorFactory" ref="validatorFactory"/>
        <beans:property name="useFullyQualifiedClassName" value="true"/>
    </beans:bean> 

    <beans:bean id="springValidator" class="com.test.SpringCommonsValidator">
        <beans:property name="validator" ref="beanValidator"/>
        <beans:property name="messageSource" ref="resourceBundleMessageSource"/>        
    </beans:bean> 

    <beans:bean id="resourceBundleMessageSource" class="org.springframework.context.support.ResourceBundleMessageSource">
        <beans:property name="basename" value="messages"/>    
    </beans:bean> 

Commons validator configuration

The following is the validation configuration file used by commons-validator. If you have used the struts framework before then this file may look familiar to you. This file contains some default “shrink-wrapped” validation checks that were originally included with struts. There is nothing to customize here for now so please just copy and paste this into your program and continue on.

src/main/resources/validator-rules.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE form-validation PUBLIC
          "-//Apache Software Foundation//DTD Commons Validator Rules Configuration 1.1.3//EN"
          "http://jakarta.apache.org/commons/dtds/validator_1_1_3.dtd">
 
<form-validation>
    <global>
        <validator name="required"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateRequired"
            methodParams="java.lang.Object,
                        org.apache.commons.validator.ValidatorAction,
                        org.apache.commons.validator.Field,
                        org.springframework.validation.Errors"
            msg="errors.required">
        </validator>
        <validator name="requiredif"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateRequiredIf"
            methodParams="java.lang.Object,
                               org.springframework.validation.ErrorsAction,
                               org.apache.commons.validator.Field,
                               org.springframework.validation.Errors"
            msg="errors.required" />
 
        <validator name="validwhen" msg="errors.required"
            classname="org.apache.struts.validator.validwhen.ValidWhen" method="validateValidWhen"
            methodParams="java.lang.Object,
                       org.springframework.validation.ErrorsAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors" />
 
        <validator name="minlength"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateMinLength"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.minlength"
            jsFunction="org.apache.commons.validator.javascript.validateMinLength" />
 
        <validator name="maxlength"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateMaxLength"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.maxlength"
            jsFunction="org.apache.commons.validator.javascript.validateMaxLength" />
 
        <validator name="mask"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateMask"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.invalid" />
 
        <validator name="byte"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateByte"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.byte" jsFunctionName="ByteValidations" />
 
        <validator name="short"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateShort"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.short" jsFunctionName="ShortValidations" />
 
        <validator name="integer"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateInteger"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.integer" jsFunctionName="IntegerValidations" />
 
        <validator name="long"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateLong"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.long" />
 
        <validator name="float"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateFloat"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.float" jsFunctionName="FloatValidations" />
 
        <validator name="double"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateDouble"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.double" />
 
        <validator name="date"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateDate"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.date" jsFunctionName="DateValidations" />
 
        <validator name="intRange"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateIntRange"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="integer" msg="errors.range" />
 
        <validator name="floatRange"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateFloatRange"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="float" msg="errors.range" />
 
        <validator name="doubleRange"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateDoubleRange"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="double" msg="errors.range" />
 
        <validator name="creditCard"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateCreditCard"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.creditcard" />
 
        <validator name="email"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateEmail"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.email" />
 
        <validator name="url"
            classname="org.springmodules.validation.commons.FieldChecks" method="validateUrl"
            methodParams="java.lang.Object,
                       org.apache.commons.validator.ValidatorAction,
                       org.apache.commons.validator.Field,
                       org.springframework.validation.Errors"
            depends="" msg="errors.url" />
    </global>
</form-validation>

Validation Rules

The next file contains the actual validations that need to be performed.

These validations are:

1. last Name is required
2. last Name must be between 20 and 35 characters in length
3. rank is required.

src/main/resources/validation.xml

<!DOCTYPE form-validation PUBLIC
          "-//Apache Software Foundation//DTD Commons Validator Rules Configuration 1.1.3//EN"
          "http://jakarta.apache.org/commons/dtds/validator_1_1_3.dtd">
<form-validation>
 
    <formset>
        <form name="com.test.Employee">
            <field property="lastName" depends="required,minlength,maxlength">
                <arg0 key="lastName" />
                <arg1 name="minlength" key="${var:minlength}" resource="false"/>
                <arg1 name="maxlength" key="${var:maxlength}" resource="false"/>
                <var>
                    <var-name>minlength</var-name>
                    <var-value>20</var-value>
                </var>
                <var>
                    <var-name>maxlength</var-name>
                    <var-value>35</var-value>
                </var>                
            </field>
            <field property="rank" depends="required">
                <arg key="rank" />
            </field>
        </form>
    </formset>
</form-validation>

Run the batch

Run the batch application by typing the following on the command line.

mvn clean compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Observe the Results

The following message should display on the console indicating a validation failure.

SEVERE: Encountered an error executing the step
org.springframework.batch.item.validator.ValidationException: Validation failed for Employee [empId=7876, lastName=ADAMS, title=CLERK, salary=1100, rank=null]: lastName can not be less than 20 characters.; rank is required.
	at com.test.SpringCommonsValidator.validate(SpringCommonsValidator.java:38)
	at org.springframework.batch.item.validator.ValidatingItemProcessor.process(ValidatingItemProcessor.java:77)
	at org.springframework.batch.core.step.item.SimpleChunkProcessor.doProcess(SimpleChunkProcessor.java:125)
	at org.springframework.batch.core.step.item.SimpleChunkProcessor.transform(SimpleChunkProcessor.java:288)

Next Steps

The application described here is a quick and dirty approach to illustrate batch validation. In a typical production grade application you will want to allow the job to continue processing the remaining records in a file and send the bad records to a error file. The error file can be reviewed manually and re-processed on the next run.

Comments and feedback are Welcome.

Thanks!

Background

Please review The Domain Language of Spring Batch prior to reading this further.

StepExecution

In simple terms each “job” is broken up into individual “steps”. The process of executing a step is called a “StepExecution”. Each execution of a step can either result in Success or Failure. If individual steps of a larger job fail it would be nice to know where the step left off. The Designers of this framework recognized this and came up with a way to allow steps to resume. The JobRepository allows Readers and Writers to store status, start and end times, exit codes, and count information for each Execution of a step within a Job. This information is stored using the ExecutionContext at each commit interval.

ExecutionContext

The “ExecutionContext” is a collection of name/value pairs that are persisted to the database. Each instance that a Step is executed by the spring batch framework has its own ExecutionContext. Many of the default Reader/Writer implementations publish information to the ExecutionContext by default.

Outline of this page

Here is a breakdown of what will be demonstrated:

1. Simulate a job failure by modifying the EmployeeProcessor.java and throw an exception towards the middle of the file.
2. Examine the output file and verify that all prior records were written out.
3. Examine the JobRepository database to verify the ExecutionContext record contains the recovery information.
4. Modify EmployeeProcessor.java to not throw the test exception.
5. Examine the output file and verify that all the remaining records were written out.
6. Note that the Footer of the file is incorrect.
7. Fix the EmployeeItemWriter.java to read the ExecutionContext and output the correct footer.

This page picks up where the last one left of. So if you have not done so already please implement the example application described here.

Requirements

• Java 5 or above and Maven 2
• Spring Batch 2.x
• Successful completion of xxx

Inserting An Error

The below highlighted code stops the process when the record for the “president” is processed.

package com.test;

import org.springframework.batch.item.ItemProcessor;

public class EmployeeProcessor implements ItemProcessor<Employee, Employee> {

	public Employee process(Employee emp) throws Exception {
		if("PRESIDENT".equals(emp.getTitle())) throw new Exception("test exception: found President");
		// if salary >= 2500 then set rank as "Director"		
		if(emp.getSalary() >= 2500 ) {
			emp.setRank("Director");
		} else {
			emp.setRank("N/A");
		}
		return emp;
	}

}

Run the command:

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Open the target/output_data.txt file and observe that only 7 records were processed.

Comment out line 8 from EmployeeProcessor.java above and re-run the batch.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Open target/output_data.txt one more time and observe that the remaining records were processed however note that the footer information is incorrect. It should say 14 records instead of 7.

Making Spring Batch Restartable

If you remember in the previous article we modified the EmployeeItemWriter class to print out a header and footer records. The footer record was generated using state variables (totalAmount, and recordCount). Since the EmployeeItemWriter maintains state, when the batch job restarts the state variables are re-initialized to their original values. Since the second invocation of the job starts from the middle of the input file the writer does not get an opportunity to properly initialize the state variables.

To fix this problem we will use the executionContext. The executionContext gets updated on each commit interval. It is the place where Spring batch keeps information that is important to the currently running job Execution.

We will modify the open() and update() methods of the EmployeeItemWriter to lookup the values from the ExecutionContext.

/src/main/java/com/test/EmployeeItemWriter.java

	public void open(ExecutionContext executionContext) throws ItemStreamException {
		if(executionContext.containsKey("current.recordCount")) {
			recordCount = new Long(executionContext.getLong("current.recordCount")).intValue();
		} else {
			recordCount = 0;
		}		
		
		if(executionContext.containsKey("current.totalAmount")) {
			totalAmount = new BigDecimal(executionContext.getDouble("current.totalAmount"));
		} else {
			totalAmount = BigDecimal.ZERO;
		}		
		this.delegate.open(executionContext);
	}

	public void update(ExecutionContext executionContext) throws ItemStreamException {
		executionContext.putLong("current.recordCount", recordCount);
		executionContext.putDouble("current.totalAmount", totalAmount.doubleValue());
		
		this.delegate.update(executionContext);
	}

We don’t need to modify “empReader” bean since it is backed by “FlatFileItemReader” and it’s implementation is restartable by default.

Clear the database

In order to start fresh again, Delete all files under src/main/resources/db. This is where HSQLDB keeps its database files. Once you delete this the DDLUtils framework will regenerate the database with all the schema information.

Re-run the job

Comment IN the EmployeeProcessor.java so that when “PRESIDENT” is encountered it throws an exception and re-run the job.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Comment OUT the line in EmployeeProcessor.java and re-run the job again.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Verify the output

Open target/output_data.txt one more time and observe that the remaining records were processed and the count should be 14 records.

That’s all for now.

Background

The spring batch framework offers an in-memory Repository that does not persist domain objects once the batch job is complete. This is a severe limitation since:

• Job restarts will not be possible (resume on restart functionality)
• Can not guarantee that two job instances with the same parameters are not launched simultaneously
• Not suitable for multi-threaded job invocations

Requirements

• Java 5 or above and Maven 2
• Successful completion of Hello World with spring batch.

Solution

In order to have the job repository stored in our application we will be using HSQLDB as our database implementation and and DDLUtils to manage the schema.

Modify the pom.xml and insert the following dependencies.

pom.xml

    <dependency>
        <groupId>org.apache.ddlutils</groupId>
        <artifactId>ddlutils</artifactId>
        <version>1.0</version>
    </dependency>
        <dependency>
            <groupId>org.hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <version>1.8.0.10</version>
        </dependency>

DDL

The following xml file is used by Apache DDLUtils to create the Spring Batch JobRepository database. When the application starts for the first time the tables will be created. Each subsequent invocation will cause the DDLUtils to check the database schema against the xml file. If the database is different then it will make the necessary alters to update the database to reflect the schema represented in the below file.

src/main/resources/ddl.xml

<?xml version="1.0"?>
<!DOCTYPE database SYSTEM "http://db.apache.org/torque/dtd/database.dtd">
<database name="testdb">

	<table name="BATCH_JOB_INSTANCE">
		<column name="JOB_INSTANCE_ID" type="BIGINT" primaryKey="true"
			autoIncrement="true"/>
		<column name="VERSION" type="BIGINT" />
		<column name="JOB_NAME" type="VARCHAR" size="100" required="true" />
		<column name="JOB_KEY" type="VARCHAR" size="32" required="true" />
		<unique name="JOB_INST_UN">
		  <unique-column name="JOB_NAME"/>
		  <unique-column name="JOB_KEY"/>
		</unique>
	</table>

    <table name="BATCH_JOB_EXECUTION">
        <column name="JOB_EXECUTION_ID" type="BIGINT" primaryKey="true"/>
        <column name="VERSION" type="BIGINT" />
	    <column name="JOB_INSTANCE_ID" type="BIGINT" required="true"/>
	    <column name="CREATE_TIME" type="TIMESTAMP" required="true"/>
	    <column name="START_TIME" type="TIMESTAMP"/>
	    <column name="END_TIME" type="TIMESTAMP"/>
	    <column name="STATUS" type="VARCHAR" size="10"/>
	    <column name="EXIT_CODE" type="VARCHAR" size="20"/>
	    <column name="EXIT_MESSAGE" type="VARCHAR" size="2500"/>
	    <column name="LAST_UPDATED" type="TIMESTAMP"/>
	    <foreign-key name="JOB_INST_EXEC_FK" foreignTable="BATCH_JOB_INSTANCE">
	       <reference foreign="JOB_INSTANCE_ID" local="JOB_INSTANCE_ID"/>
	    </foreign-key>
    </table>

    <table name="BATCH_JOB_PARAMS">
        <column name="JOB_INSTANCE_ID" type="BIGINT" required="true"/>
	    <column name="TYPE_CD" type="VARCHAR" size="6" required="true"/>
	    <column name="KEY_NAME" type="VARCHAR" size="100" required="true"/> 
	    <column name="STRING_VAL" type="VARCHAR" size="250"/> 
	    <column name="DATE_VAL" type="TIMESTAMP"/> 
	    <column name="LONG_VAL" type="BIGINT"/>
	    <column name="DOUBLE_VAL" type="DOUBLE"/>
        <foreign-key name="JOB_INST_PARAMS_FK" foreignTable="BATCH_JOB_INSTANCE">
           <reference foreign="JOB_INSTANCE_ID" local="JOB_INSTANCE_ID"/>
        </foreign-key>
    </table>

    <table name="BATCH_STEP_EXECUTION">
		<column name="STEP_EXECUTION_ID" type="BIGINT" primaryKey="true" 
		  autoIncrement="true" />
		<column name="VERSION" type="BIGINT" required="true"/>
		<column name="STEP_NAME" type="VARCHAR" size="100" required="true"/>
		<column name="JOB_EXECUTION_ID" type="BIGINT" required="true" />
		<column name="START_TIME" type="TIMESTAMP" required="true"/> 
		<column name="END_TIME" type="TIMESTAMP"/>  
		<column name="STATUS" type=" VARCHAR" size="10"/>
		<column name="COMMIT_COUNT" type="BIGINT"/> 
		<column name="READ_COUNT" type="BIGINT"/>
		<column name="FILTER_COUNT" type="BIGINT"/>
		<column name="WRITE_COUNT" type="BIGINT"/>
		<column name="READ_SKIP_COUNT" type="BIGINT"/>
		<column name="WRITE_SKIP_COUNT" type="BIGINT"/>
		<column name="PROCESS_SKIP_COUNT" type="BIGINT"/>
		<column name="ROLLBACK_COUNT" type="BIGINT"/> 
		<column name="EXIT_CODE" type="VARCHAR" size="20"/>
		<column name="EXIT_MESSAGE" type="VARCHAR" size="2500"/>
		<column name="LAST_UPDATED" type="TIMESTAMP"/>
        <foreign-key name="JOB_EXEC_STEP_FK" foreignTable="BATCH_JOB_EXECUTION">
           <reference foreign="JOB_EXECUTION_ID" local="JOB_EXECUTION_ID"/>
        </foreign-key>
    </table>

    <table name="BATCH_STEP_EXECUTION_CONTEXT">
	    <column name="STEP_EXECUTION_ID" type="BIGINT" primaryKey="true"/>
	    <column name="SHORT_CONTEXT" type="VARCHAR" size="2500" required="true"/>
	    <column name="SERIALIZED_CONTEXT" type="VARCHAR" />
        <foreign-key name="STEP_EXEC_CTX_FK" foreignTable="BATCH_STEP_EXECUTION">
           <reference foreign="STEP_EXECUTION_ID" local="STEP_EXECUTION_ID"/>
        </foreign-key>
    </table>

    <table name="BATCH_JOB_EXECUTION_CONTEXT">
		<column name="JOB_EXECUTION_ID" type="BIGINT" primaryKey="true"/>
		<column name="SHORT_CONTEXT" type="VARCHAR" size="2500" required="true"/>
		<column name="SERIALIZED_CONTEXT" type="VARCHAR"/> 
        <foreign-key name="JOB_EXEC_CTX_FK" foreignTable="BATCH_JOB_EXECUTION">
           <reference foreign="JOB_EXECUTION_ID" local="JOB_EXECUTION_ID"/>
        </foreign-key>        
    </table>
	<table name="BATCH_STEP_EXECUTION_SEQ">
		<column name="ID" type="BIGINT" primaryKey="true" autoIncrement="true" />
	</table>
	<table name="BATCH_JOB_EXECUTION_SEQ">
		<column name="ID" type="BIGINT" primaryKey="true" autoIncrement="true" />
	</table>
	<table name="BATCH_JOB_SEQ">
		<column name="ID" type="BIGINT" primaryKey="true" autoIncrement="true" />
	</table>
    
</database>

Java Code

The following file initializes the Job Repository and creates the schema based on a DDL contained in an xml configuration file. It also allows the system to gracefully shut-down the database and persist the in-memory data back to the file system after batch job completion.

src/main/java/com/test/JobRepositoryInitializer.java

package com.test;

import java.io.InputStreamReader;

import javax.sql.DataSource;

import org.apache.ddlutils.Platform;
import org.apache.ddlutils.PlatformFactory;
import org.apache.ddlutils.io.DatabaseIO;
import org.apache.ddlutils.model.Database;
import org.springframework.jdbc.core.simple.SimpleJdbcTemplate;

public class JobRepositoryInitializer {
	private DataSource dataSource;
	private String configFile;
	
	/**
	 * This method reads the schema from an xml file and determines
	 * if it needs to issue DDL statements to create or modify tables
	 * already in the database.
	 * 
	 * @throws Exception
	 */
	public void checkDDL() throws Exception {
        Platform platform = PlatformFactory
        .createNewPlatformInstance(dataSource);

        // Apache DDLUtils framework is used to create/modify schema.
		Database database = new DatabaseIO().read(new InputStreamReader(
		        getClass().getResourceAsStream(configFile)));
		
		platform.alterTables(database, false);
		System.out.println("HSQLDB is ready: " + platform);
	}

	public void shutdownDatabase() {
        SimpleJdbcTemplate template = new SimpleJdbcTemplate(dataSource);
        template.update("SHUTDOWN;");
        System.out.println("HSQLDB was gracefully shutdown. ");
	}
	public void setDataSource(DataSource dataSource) {
		this.dataSource = dataSource;
	}

	public DataSource getDataSource() {
		return dataSource;
	}

	public void setConfigFile(String configFile) {
		this.configFile = configFile;
	}

	public String getConfigFile() {
		return configFile;
	}	
}

Configuration

In order to get the application to use HSQLDB and generate the JobRepository DDL Schema you need to do the following.

Configure the database

The following bean allows you to define a dataSource that will be used to connect to the HSQLDB database. The database files will be located in the src/main/resources/db folder. (This is specified in the database URL property)

src/main/resources/applicationContext.xml

<bean name="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
        <property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
        <property name="url" value="jdbc:hsqldb:file:src/main/resources/db/testdb"/>
        <property name="username" value="sa"/>
        <property name="password" value=""/>
</bean>

Job Repository Configuration

Update the configuration for the job repository to look like this:

src/main/resources/applicationContext.xml

    <beans:bean id="jobRepository"
        class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
        <beans:property name="databaseType" value="HSQL"/>
        <beans:property name="dataSource" ref="dataSource"/>
        <beans:property name="transactionManager" ref="transactionManager" />
    </beans:bean>

Apache DDLUtil Configuration

When the application starts for the first time the system will need to create the JobRepository tables. The following Spring Bean initializes the job repository and creates tables that are defined in the
src/main/resources/applicationContext.xml

	<beans:bean id="jobRepositoryInitializer" class="com.test.JobRepositoryInitializer"
		init-method="checkDDL" destroy-method="shutdownDatabase">
		<beans:property name="dataSource" ref="dataSource" />
		<beans:property name="configFile" value="/ddl.xml" />
	</beans:bean>

Run the application

To run the job from the command line type the following:

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

You will notice that if you run the job again the console will print the following:

SEVERE: Job Terminated in error: A job instance already exists and is complete for parameters={}. If you want to run this job again, change the parameters.
org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={}. If you want to run this job again, change the parameters.

This is a successful indication that the HSQLDB is working persisting the data. Spring Batch framework is indicating that the helloWorld job invocation has already happened. More than one invocation with the same parameters is not allowed. If you want to read more about this then please read the following article.

You can also read the database script file located here:

src/main/resources/db/testdb.script

INSERT INTO BATCH_JOB_INSTANCE VALUES(0,0,'helloWorldJob','d41d8cd98f00b204e9800998ecf8427e')
INSERT INTO BATCH_JOB_EXECUTION VALUES(0,2,0,'2011-10-02 19:53:54.601000000','2011-10-02 19:53:54.617000000','2011-10-02 19:53:54.642000000','COMPLETED','COMPLETED','','2011-10-02 19:53:54.642000000')
INSERT INTO BATCH_STEP_EXECUTION VALUES(0,3,'step1',0,'2011-10-02 19:53:54.626000000','2011-10-02 19:53:54.639000000','COMPLETED',1,0,0,0,0,0,0,0,'COMPLETED','','2011-10-02 19:53:54.639000000')
INSERT INTO BATCH_STEP_EXECUTION_CONTEXT VALUES(0,'{"map":""}',NULL)
INSERT INTO BATCH_JOB_EXECUTION_CONTEXT VALUES(0,'{"map":""}',NULL)
INSERT INTO BATCH_STEP_EXECUTION_SEQ VALUES(0)
INSERT INTO BATCH_JOB_EXECUTION_SEQ VALUES(0)
INSERT INTO BATCH_JOB_SEQ VALUES(0)

HSQLDB uses this file to persist data to the file system in-between batch invocations.

That’s all for now!

Background

In prior articles we created a batch job that read information and outputted the result to another file. The output file was missing a header and footer lines. In this article we will look into generating this information.

Requirements

• Java 5
• Maven 2
• This page picks up where the following page left off. Please Review and Implement the following article.

Header

The header of the will simply contain the filename and the timestamp.

fileName,Timestamp

Footer

The footer contained in the output be composed of 2 fields.

#ofRecords,salarySum

where:
#ofRecords – Count of the number of records in the output
salarySum – Sum of all values in the salary field

Solution

Create a Item Writer that will wrap the “FlatFileItemWriter”. The “headerCallback” and “footerCallback” methods of the ItemWriter will be called at the beginning and ending of the file. This allows developers to implement code that will allow them to write information to the header and footer of the output file.

src/main/java/com/test/EmployeeItemWriter.java

package com.test;

import java.io.IOException;
import java.io.Writer;
import java.math.BigDecimal;
import java.util.Date;
import java.util.List;

import org.springframework.batch.item.ExecutionContext;
import org.springframework.batch.item.ItemStream;
import org.springframework.batch.item.ItemStreamException;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.FlatFileFooterCallback;
import org.springframework.batch.item.file.FlatFileHeaderCallback;
import org.springframework.batch.item.file.FlatFileItemWriter;

public class EmployeeItemWriter implements ItemWriter<Employee>,
		FlatFileFooterCallback, FlatFileHeaderCallback, ItemStream {

    private FlatFileItemWriter<Employee> delegate;

    private BigDecimal totalAmount = BigDecimal.ZERO;

	private int recordCount = 0;
    
    public void writeFooter(Writer writer) throws IOException {
        writer.write(""+recordCount + "," + totalAmount);
    }
	public void writeHeader(Writer writer) throws IOException {
        writer.write("output_file.txt" + "," + new Date());
	}
    public void write(List<? extends Employee> items) throws Exception {
        BigDecimal chunkTotal = BigDecimal.ZERO;
        int chunkRecord = 0;
        for (Employee employee : items) {
            chunkRecord++;
            chunkTotal = chunkTotal.add(new BigDecimal(employee.getSalary()));
        }
        delegate.write(items);
        // After successfully writing all items
        totalAmount = totalAmount.add(chunkTotal);
        recordCount += chunkRecord;
	}
    
    public void setDelegate(FlatFileItemWriter<Employee> delegate) {
		this.delegate = delegate;
	}

	public void close() throws ItemStreamException {
		this.delegate.close();
	}

	public void open(ExecutionContext arg0) throws ItemStreamException {
		this.delegate.open(arg0);
	}

	public void update(ExecutionContext arg0) throws ItemStreamException {
		this.delegate.update(arg0);
	}
}

Batch XML Configuration

The following xml will allow the ItemWriter to wrap the FlatFileItemWriter and maintain a running total.

The highlighted lines are the changes from the prior version.

/src/main/resources/simpleJob.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
     xmlns:beans="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="
 
http://www.springframework.org/schema/batch
 
http://www.springframework.org/schema/batch/spring-batch-2.1.xsd

http://www.springframework.org/schema/beans
 
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
 
    <beans:import resource="applicationContext.xml"/>
 
<!-- Tokenizer - Converts a delimited string into a Set of Fields -->
<beans:bean name="defaultTokenizer" 
    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"/>

<!-- FieldSetMapper - Populates a bean's attributes with using the FieldSet -->
<beans:bean name="employeeFieldSetMapper" class="com.test.EmployeeFieldSetMapper"/>

<!-- LineMapper - Uses the tokenizer and Mapper to create instances of a Bean. -->
<beans:bean name="employeeLineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
    <beans:property name="lineTokenizer" ref="defaultTokenizer"/>    
    <beans:property name="fieldSetMapper" ref="employeeFieldSetMapper"/>        
</beans:bean>

<!-- Reader - used by the tasklet to process one Item from the input. -->
<beans:bean name="empReader" class="org.springframework.batch.item.file.FlatFileItemReader">
    <beans:property name="lineMapper" ref="employeeLineMapper"/>
    
    <!-- use spring integrations for the following, but for now filename is hard coded -->
    <beans:property name="resource" value="input_data.txt"/>    
</beans:bean>

<!-- Processor -->

<beans:bean name="empProcessor" class="com.test.EmployeeProcessor">
</beans:bean>

<!-- Writer -->
<beans:bean id="empWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
    <beans:property name="resource" value="file:target/output_data.txt" />
    <beans:property name="lineAggregator">
        <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
            <beans:property name="delimiter" value=","/>
            <beans:property name="fieldExtractor">
                <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                    <beans:property name="names" value="empId,lastName,title,salary,rank"/>
                </beans:bean>
            </beans:property>
        </beans:bean>
    </beans:property>
    <beans:property name="footerCallback" ref="empHeaderFooterWriter" />
    <beans:property name="headerCallback" ref="empHeaderFooterWriter" />
</beans:bean>

<beans:bean id="empHeaderFooterWriter" class="com.test.EmployeeItemWriter">
    <beans:property name="delegate" ref="empWriter"/>
</beans:bean>

<job id="helloWorldJob">
    <step id="step1" next="step2">
        <tasklet ref="helloWorldTasklet" />
    </step>
    <step id="step2">
        <tasklet>
            <chunk reader="empReader" processor="empProcessor" writer="empHeaderFooterWriter" commit-interval="1"/>
        </tasklet>
    </step>
</job>
 
<beans:bean name="helloWorldTasklet" class="com.test.HelloWorldTasklet"/>
 
<!--
To run the job from the command line type the following:
mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"
 -->
</beans:beans>

Execute the job

Go to the command line and type the following:

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

View the results

The output file will appear in the target/ folder of the project.

The file should look like this:

output_file.txt,Sat Oct 01 22:50:17 EDT 2011
7876,ADAMS,CLERK,1100,N/A
7499,ALLEN,SALESMAN,1600,N/A
7698,BLAKE,MANAGER,2850,Director
7782,CLARK,MANAGER,2450,N/A
7902,FORD,ANALYST,3000,Director
7900,JAMES,CLERK,950,N/A
7566,JONES,MANAGER,2975,Director
7839,KING,PRESIDENT,5000,Director
7654,MARTIN,SALESMAN,1250,N/A
7934,MILLER,CLERK,1300,N/A
7788,SCOTT,ANALYST,3000,Director
7369,SMITH,CLERK,800,N/A
7844,TURNER,SALESMAN,1500,N/A
7521,WARD,SALESMAN,1250,N/A
14,29025

That’s all for now!

Background

The basic building blocks of any batch process is

1. Reading a Item
2. Performing an operation on it
3. Writing the Item back

Please take some time to review The Domain Language of Batch before proceeding. It covers much of the fundamental concepts we will be covering here.

Batch Steps

This page is focused on an individual step of the batch process.

The following is from the spring batch documentation

A Step is a domain object that encapsulates an independent, sequential phase of a batch job. Therefore, every Job is composed entirely of one or more steps. A Step contains all of the information necessary to define and control the actual batch processing. This is a necessarily vague description because the contents of any given Step are at the discretion of the developer writing a Job. A Step can be as simple or complex as the developer desires. A simple Step might load data from a file into the database, requiring little or no code. (depending upon the implementations used) A more complex Step may have complicated business rules that are applied as part of the processing.

Step Processing types

There are 2 ways a step can process data,

Tasklet

If the step requires only to execute a single task then you can use a tasklet. Typical use case for this is when you need to run a stored procedure, or copy a file from one location to the other. In the “Hello World” example we used a Tasklet to print the message to the console.

Chunk oriented

Chunk oriented processing involves specifying a reader, processor and writer. The input is read one item at a time in sequence and passed to the processor and eventually to the writer in chunks within a transaction boundary. Once the commit interval is reached the items are committed to the writer. Chunk oriented processing is what we will cover on this page.

Requirements

• Java 5
• Maven 2
• Review and implementation of the following article.
• And this one: http://numberformat.wordpress.com/2011/10/01/hello-world-with-spring-batch-2-1-x/

Project Setup

We will be modifying an existing project so please review the articles listed in the section above.

Input Data

The following is the input csv file that will be read. Please create the following file in the projects resource directory.

src/main/resources/input_data.txt

7876,ADAMS,CLERK,1100
7499,ALLEN,SALESMAN,1600
7698,BLAKE,MANAGER,2850
7782,CLARK,MANAGER,2450
7902,FORD,ANALYST,3000
7900,JAMES,CLERK,950
7566,JONES,MANAGER,2975
7839,KING,PRESIDENT,-5000
7654,MARTIN,SALESMAN,1250
7934,MILLER,CLERK,1300
7788,SCOTT,ANALYST,3000
7369,SMITH,CLERK,800
7844,TURNER,SALESMAN,1500
7521,WARD,SALESMAN,1250

Employee Bean

This is a simple bean that represents a single Employee.

src/main/java/com/test/Employee.java

package com.test;

public class Employee {

	private Integer empId;
	private String lastName;
	private String title;
	private Integer salary;
	private String rank;
	
	public Integer getEmpId() {
		return empId;
	}
	public void setEmpId(Integer empId) {
		this.empId = empId;
	}
	public String getLastName() {
		return lastName;
	}
	public void setLastName(String lastName) {
		this.lastName = lastName;
	}
	public String getTitle() {
		return title;
	}
	public void setTitle(String title) {
		this.title = title;
	}
	public Integer getSalary() {
		return salary;
	}
	public void setSalary(Integer salary) {
		this.salary = salary;
	}
	public void setRank(String rank) {
		this.rank = rank;
	}
	public String getRank() {
		return rank;
	}
	@Override
	public String toString() {
		return "Employee [empId=" + empId + ", lastName=" + lastName
				+ ", title=" + title + ", salary=" + salary + ", rank=" + rank
				+ "]";
	}	
}

Reading

In order to read data from a file we will only need to write the FieldSetMapper class that takes a FieldSet object and maps its contents into an bean.

src/main/java/com/test/EmployeeFieldSetMapper.java

package com.test;

import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;

public class EmployeeFieldSetMapper implements FieldSetMapper<Employee> {

	public Employee mapFieldSet(FieldSet fieldSet) throws BindException {
		if(fieldSet == null) return null;
		
		Employee emp = new Employee();	
		// unlike jdbc the index is 0 based	
		emp.setEmpId(fieldSet.readInt(0)); 
		emp.setLastName(fieldSet.readString(1));
		emp.setTitle(fieldSet.readString(2));
		emp.setSalary(fieldSet.readInt(3));
		
		return emp;
	}

}

Arithmetic

Not really! All we are doing is assigning a Rank based on the salary amount. The item processor takes an input Bean and converts it to an output bean. In this case the beans are the same but they don’t have to be.

src/main/java/com/test/EmployeeProcessor.java

package com.test;

import org.springframework.batch.item.ItemProcessor;

public class EmployeeProcessor implements ItemProcessor<Employee, Employee> {

	public Employee process(Employee emp) throws Exception {
		// if salary >= 2500 then set rank as "Director"		
		if(emp.getSalary() >= 2500 ) {
			emp.setRank("Director");			
		} else {
			emp.setRank("N/A");
		}
		return emp;
	}

}

Writing

All the objects needed for writing the file are configured using xml. See the xml file below for more details.

Job Configuration

Modify the job xml file to define the new beans.

src/main/resources/simpleJob.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
     xmlns:beans="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="
 
http://www.springframework.org/schema/batch
 
http://www.springframework.org/schema/batch/spring-batch-2.1.xsd

http://www.springframework.org/schema/beans
 
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
 
    <beans:import resource="applicationContext.xml"/>
 
<!-- Tokenizer - Converts a delimited string into a Set of Fields -->
<beans:bean name="defaultTokenizer" 
    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"/>

<!-- FieldSetMapper - Populates a bean's attributes with using the FieldSet -->
<beans:bean name="employeeFieldSetMapper" class="com.test.EmployeeFieldSetMapper"/>

<!-- LineMapper - Uses the tokenizer and Mapper to create instances of a Bean. -->
<beans:bean name="employeeLineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
    <beans:property name="lineTokenizer" ref="defaultTokenizer"/>    
    <beans:property name="fieldSetMapper" ref="employeeFieldSetMapper"/>        
</beans:bean>

<!-- Reader - used by the tasklet to process one Item from the input. -->
<beans:bean name="empReader" class="org.springframework.batch.item.file.FlatFileItemReader">
    <beans:property name="lineMapper" ref="employeeLineMapper"/>
    
    <!-- use spring integrations for the following, but for now filename is hard coded -->
    <beans:property name="resource" value="input_data.txt"/>    
</beans:bean>

<!-- Processor -->

<beans:bean name="empProcessor" class="com.test.EmployeeProcessor">
</beans:bean>

<!-- Writer -->
<beans:bean id="empWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
    <beans:property name="resource" value="file:target/output_data.txt" />
    <beans:property name="lineAggregator">
        <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
            <beans:property name="delimiter" value=","/>
            <beans:property name="fieldExtractor">
                <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                    <beans:property name="names" value="empId,lastName,title,salary,rank"/>
                </beans:bean>
            </beans:property>
        </beans:bean>
    </beans:property>
</beans:bean>

<job id="helloWorldJob">
    <step id="step1" next="step2">
        <tasklet ref="helloWorldTasklet" />
    </step>
    <step id="step2">
        <tasklet>
            <chunk reader="empReader" processor="empProcessor" writer="empWriter" commit-interval="1"/>
        </tasklet>
    </step>
</job>
 
<beans:bean name="helloWorldTasklet" class="com.test.HelloWorldTasklet"/>
 
<!--
To run the job from the command line type the following:
mvn clean compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"
 -->
</beans:beans>

Execute the job

Go to the command line and type the following:

mvn clean compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

View the Results

The output file will appear in the target/ folder of the project.

Further Reading

To keep things simple we were reading and writing files located in the project own folders. There are many enterprise design patterns that describe the best practices for feeding data into the batch programs. For further reading on this topic please see the Spring Integrations Framework Homepage.

Background

The Hello World With Spring Batch was written a while ago. Since then there was a new version of spring batch released. Instead of changing the original article, I chose to write a new one just describing the changes needed to get the application described there to work.

The original article is located here:

http://numberformat.wordpress.com/2010/02/05/hello-world-with-spring-batch/

Step 1

Update the pom.xml dependency for spring-batch-core.

pom.xml

        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-core</artifactId>
            <version>2.1.8.RELEASE</version>
        </dependency>

Step 2

update the xml:

src/main/resources/simpleJob.xml

http://www.springframework.org/schema/batch/spring-batch-2.0.xsd">

with

http://www.springframework.org/schema/batch/spring-batch-2.1.xsd">

Finally Run the command just like before:

mvn clean compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

That’s all for now!

Requirements

• Maven 2
• Java 5 or above

Background

Type ahead functionality allows users to obtain a list of possible options as they type. Chances that the item the user is looking for increases with each keystroke.

A static working example is located here.

Live search results are typically displayed in combo boxes. A combo boxes are a mix between a drop down box and a text field. The ext-js framework enhances the basic HTML drop down to allow for type ahead functionality.

Simple Database Driven Search

Next we will take the example presented above and enhance it to retrieve results from a servlet invocation. The servlet will call the database to return results.

Smarter Database Driven Search

The data returned after you start to type may sometimes overwhelm the user. A better way to return results is to display them based on ranking. Each word in our sample table also has a rank column. We will start to use this to get smarter search results.

Pagination

The example implemented here uses pagination. The user is able to page thru search results. This capability is enabled by sending back “totalCount” value that represents the total search results available from the database. The “start” and “limit” parameters passed to the servlet help the system “window” the results so that only the required rows get sent to the browser.

In the example below, in order to get the “totalCount” I had to execute the query 2 times. Once to get the count and the other time to get the actual data.

Not all databases support “start” and “limit” fields. Check with your database vendor for availability.

Implementation

The project described here is composed of:

1. index.jsp that displays the extjs component
2. Servlet that does the search against the database and return JSON
3. HSQLDB (HyperSQL) In memory database to hold the data
4. Spring Framework so that we can use the SimpleJdbcTemplate

Start a new Project

We will be creating this project using Maven2. If you have not used maven before please see my maven2 tutorial on the right nav of this site.

mvn archetype:generate -DarchetypeArtifactId=maven-archetype-webapp

groupId: com.test
artifactId: live-search

Answer the rest of the questions with defaults “Just hit the enter key”,

cd to the project base folder.

cd live-search

Create the necessary directories.

mkdir -p src/main/java/com/test
mkdir -p src/main/resources/com/test
mkdir -p src/main/resources/db

The pom.xml file should look something like this.

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.test</groupId>
  <artifactId>live-search</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>live-search Maven Webapp</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20090211</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ibatis</groupId>
            <artifactId>ibatis-sqlmap</artifactId>
            <version>2.3.4.726</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring</artifactId>
            <version>2.5.6</version>
        </dependency>
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.9</version>
    </dependency>        
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.14</version>
        </dependency>
        <dependency>
            <groupId>org.hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <version>2.2.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ddlutils</groupId>
            <artifactId>ddlutils</artifactId>
            <version>1.0</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
            <version>2.5</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jsp-api</artifactId>
            <version>2.0</version>
            <scope>provided</scope>
        </dependency>
  </dependencies>
  <build>
    <finalName>live-search</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.0.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                <artifactId>jetty-maven-plugin</artifactId>
				<version>7.0.0.v20091005</version>
                <configuration>
                    <scanIntervalSeconds>2</scanIntervalSeconds>
                    <webAppConfig>
                        <contextPath>/</contextPath>
                    </webAppConfig>
                    <stopKey>s</stopKey>
                </configuration>
            </plugin>
        </plugins>
  </build>
</project>

Each key press initiates an Ajax request to the server. The server sends back a small list of possible options.

Servlet

The following servlet runs queries against the database and returns results to the caller. In a standard web application you would follow the MVC design pattern. To keep the example short I have done it all in the servlet.

src/main/java/com/test/SearchServlet.java

package com.test;

import java.io.IOException;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;

import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.json.JSONException;
import org.json.JSONObject;
import org.springframework.context.ApplicationContext;
import org.springframework.jdbc.core.simple.ParameterizedRowMapper;
import org.springframework.jdbc.core.simple.SimpleJdbcTemplate;
import org.springframework.web.context.support.WebApplicationContextUtils;

public class SearchServlet extends HttpServlet {
	private static final long serialVersionUID = 1L;
	private SimpleJdbcTemplate jdbcTemplate;

	@Override
	public void init(ServletConfig config) throws ServletException {
		super.init(config);
		ApplicationContext context = WebApplicationContextUtils
			.getRequiredWebApplicationContext(config.getServletContext());
		
		jdbcTemplate = (SimpleJdbcTemplate) context.getBean("jdbcTemplate");		
	}
	
	@Override
	protected void doGet(HttpServletRequest req, HttpServletResponse resp)
			throws ServletException, IOException {

		String query = req.getParameter("query");		
		if(query==null || "".equals(query.trim())) return;		
		int offset = 0, limit = 0;
		
		try {
			offset = Integer.parseInt(req.getParameter("start"));
		} catch (Exception ex) {}
		try {
			limit = Integer.parseInt(req.getParameter("limit"));
		} catch (Exception ex) {}

		if(limit <=0 ) { limit = 20; }
		
		// get the overall total count
		final List<JSONObject> resultList = new ArrayList<JSONObject>();
		jdbcTemplate.<JSONObject> query(
				"SELECT distinct word_x FROM word_lemmatised " +
				"WHERE word_x like ? order by word_x limit ? offset ?;",
				new ParameterizedRowMapper<JSONObject>() {
					public JSONObject mapRow(ResultSet rs, int rowNum)
							throws SQLException {
						JSONObject jsonWord = new JSONObject();
						try {
							jsonWord.put("word_x", rs.getString(1));
							resultList.add(jsonWord);
						} catch (JSONException e) {}
						return jsonWord;
					}
				}, query+"%", limit, offset);
		
		int totalCount = jdbcTemplate.queryForInt(
				"SELECT count(distinct word_x) FROM word_lemmatised "
						+ "WHERE word_x like ?", query + "%");
		boolean success = true;
		
		
		JSONObject response = new JSONObject();		
		try {
			response.put("success", success);
			response.put("totalCount", totalCount);
			response.put("records", resultList);			
		} catch (JSONException e) {}
		resp.getWriter().print(response.toString());
	}

	public void setJdbcTemplate(SimpleJdbcTemplate jdbcTemplate) {
		this.jdbcTemplate = jdbcTemplate;
	}

	public SimpleJdbcTemplate getJdbcTemplate() {
		return jdbcTemplate;
	}
}

src/main/webapp/WEB-INF/web.xml

<!DOCTYPE web-app PUBLIC
 "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd" >

<web-app>
	<display-name>Archetype Created Web Application</display-name>

	<context-param>
		<param-name>contextConfigLocation</param-name>
		<param-value>classpath:applicationContext.xml</param-value>
	</context-param>

	<listener>
		<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
	</listener>
    <listener>
        <listener-class>com.test.DBLifecycleContextListener</listener-class>
    </listener>
    
	<servlet>
		<servlet-name>searchServlet</servlet-name>
		<servlet-class>com.test.SearchServlet</servlet-class>
	</servlet>

	<servlet-mapping>
		<servlet-name>searchServlet</servlet-name>
		<url-pattern>/app/searchServlet</url-pattern>
	</servlet-mapping>
</web-app>

Infrastructure classes

The following classes and configuration files support the application.

src/main/java/com/test/DBLifecycleContextListener.java

package com.test;
 
import java.io.InputStreamReader;
 
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.sql.DataSource;
 
import org.apache.ddlutils.Platform;
import org.apache.ddlutils.PlatformFactory;
import org.apache.ddlutils.io.DatabaseIO;
import org.apache.ddlutils.model.Database;
import org.springframework.jdbc.core.simple.SimpleJdbcTemplate;
import org.springframework.web.context.WebApplicationContext;
import org.springframework.web.context.support.WebApplicationContextUtils;
 
/**
 * This listener initializes or gracefully shuts down the database based on
 * events from the web application.
 */
public class DBLifecycleContextListener implements ServletContextListener {
    private DataSource dataSource = null;
    private WebApplicationContext springContext = null;
 
    public void contextDestroyed(ServletContextEvent event) {
        SimpleJdbcTemplate template = new SimpleJdbcTemplate(dataSource);
        template.update("SHUTDOWN;");
        System.out.println("HSQLDB was shutdown. ");
    }
 
    public void contextInitialized(ServletContextEvent event) {
        springContext = WebApplicationContextUtils
                .getWebApplicationContext(event.getServletContext());
        dataSource = (DataSource) springContext.getBean("dataSource");
 
        Platform platform = PlatformFactory
                .createNewPlatformInstance(dataSource);
 
        Database database = new DatabaseIO().read(new InputStreamReader(
                getClass().getResourceAsStream("/ddl.xml")));
 
        System.out.println(database);
 
        platform.alterTables(database, false);
 
        System.out.println("HSQLDB is ready: " + platform);
 
    }
 
}

HyperSQL in memory database Setup

The following defines the schema that will be used to create the in-memory tables in the Java HyperSQL database.

src/main/resources/ddl.xml

<?xml version="1.0"?>
<!DOCTYPE database SYSTEM "http://db.apache.org/torque/dtd/database.dtd">
<database name="liveSearch">
  <table name="word_lemmatised">
    <column name="word_x"
            type="VARCHAR"
            primaryKey="true"
            size="255"
            required="true"/>
    <column name="wordClass"
            type="VARCHAR"
            primaryKey="true"
            size="10"
            required="true"/>
    <column name="frequency"
            type="INTEGER"
            required="true"/>
    <column name="sortOrder"
            type="INTEGER"
            required="true"/>            

    <index name="word_lemmatised_frequency">
	<index-column name="frequency"/>
    </index>
    <index name="word_lemmatised_sortOrder">
	<index-column name="sortOrder"/>
    </index>

  </table>
 
</database>

The following data is the word list used to populate the in-memory table. This table will be queried by the servlet so it can return json back to the extjs component.

Copy the contents of this file into:

src/main/resources/db/liveSearch.script

CREATE SCHEMA PUBLIC AUTHORIZATION DBA
CREATE MEMORY TABLE WORD_LEMMATISED(WORD_X VARCHAR(255) NOT NULL,WORDCLASS VARCHAR(10) NOT NULL,FREQUENCY INTEGER NOT NULL,SORTORDER INTEGER NOT NULL,PRIMARY KEY(WORD_X,WORDCLASS))
CREATE INDEX WORD_LEMMATISED_FREQUENCY ON WORD_LEMMATISED(FREQUENCY)
CREATE INDEX WORD_LEMMATISED_SORTORDER ON WORD_LEMMATISED(SORTORDER)
CREATE USER SA PASSWORD ""
GRANT DBA TO SA
SET WRITE_DELAY 10
SET SCHEMA PUBLIC
INSERT INTO WORD_LEMMATISED VALUES('a','det',2186369,5)
INSERT INTO WORD_LEMMATISED VALUES('abandon','v',4249,2107)
INSERT INTO WORD_LEMMATISED VALUES('abbey','n',1110,5204)
INSERT INTO WORD_LEMMATISED VALUES('ability','n',10468,966)
INSERT INTO WORD_LEMMATISED VALUES('able','a',30454,321)
INSERT INTO WORD_LEMMATISED VALUES('abnormal','a',809,6277)
INSERT INTO WORD_LEMMATISED VALUES('abolish','v',1744,3862)
INSERT INTO WORD_LEMMATISED VALUES('abolition','n',1154,5085)
INSERT INTO WORD_LEMMATISED VALUES('abortion','n',1471,4341)
INSERT INTO WORD_LEMMATISED VALUES('about','adv',52561,179)
INSERT INTO WORD_LEMMATISED VALUES('about','prep',144554,69)
INSERT INTO WORD_LEMMATISED VALUES('above','a',2139,3341)
INSERT INTO WORD_LEMMATISED VALUES('abroad','adv',3941,2236)
INSERT INTO WORD_LEMMATISED VALUES('abruptly','adv',1146,5106)
INSERT INTO WORD_LEMMATISED VALUES('absence','n',5949,1602)
INSERT INTO WORD_LEMMATISED VALUES('absent','a',1504,4266)
INSERT INTO WORD_LEMMATISED VALUES('absolute','a',3489,2435)
INSERT INTO WORD_LEMMATISED VALUES('absolutely','adv',5782,1651)
INSERT INTO WORD_LEMMATISED VALUES('absorb','v',2684,2907)
INSERT INTO WORD_LEMMATISED VALUES('absorption','n',932,5769)
INSERT INTO WORD_LEMMATISED VALUES('abstract','a',1605,4083)
INSERT INTO WORD_LEMMATISED VALUES('absurd','a',966,5655)
INSERT INTO WORD_LEMMATISED VALUES('abuse','n',3428,2461)
INSERT INTO WORD_LEMMATISED VALUES('abuse','v',1364,4570)
INSERT INTO WORD_LEMMATISED VALUES('academic','a',4594,1974)
INSERT INTO WORD_LEMMATISED VALUES('academy','n',1025,5453)
INSERT INTO WORD_LEMMATISED VALUES('accelerate','v',1114,5188)
INSERT INTO WORD_LEMMATISED VALUES('accent','n',1842,3707)
INSERT INTO WORD_LEMMATISED VALUES('accept','v',20373,507)
INSERT INTO WORD_LEMMATISED VALUES('acceptable','a',3647,2351)
INSERT INTO WORD_LEMMATISED VALUES('acceptance','n',2702,2895)
INSERT INTO WORD_LEMMATISED VALUES('access','n',10099,997)
INSERT INTO WORD_LEMMATISED VALUES('access','v',1389,4516)
INSERT INTO WORD_LEMMATISED VALUES('accessible','a',1637,4024)
INSERT INTO WORD_LEMMATISED VALUES('accident','n',8374,1207)
INSERT INTO WORD_LEMMATISED VALUES('accommodate','v',2065,3418)
INSERT INTO WORD_LEMMATISED VALUES('accommodation','n',4305,2085)
INSERT INTO WORD_LEMMATISED VALUES('accompany','v',4885,1892)
INSERT INTO WORD_LEMMATISED VALUES('accomplish','v',968,5648)
INSERT INTO WORD_LEMMATISED VALUES('accord','n',947,5723)
INSERT INTO WORD_LEMMATISED VALUES('accord','v',976,5613)
INSERT INTO WORD_LEMMATISED VALUES('accordance','n',2042,3443)
INSERT INTO WORD_LEMMATISED VALUES('according','prep',15722,646)
INSERT INTO WORD_LEMMATISED VALUES('accordingly','adv',2288,3214)
INSERT INTO WORD_LEMMATISED VALUES('account','n',19260,536)
INSERT INTO WORD_LEMMATISED VALUES('account','v',6130,1562)
INSERT INTO WORD_LEMMATISED VALUES('accountability','n',1184,5011)
INSERT INTO WORD_LEMMATISED VALUES('accountant','n',2199,3289)
INSERT INTO WORD_LEMMATISED VALUES('accounting','a',1640,4018)
INSERT INTO WORD_LEMMATISED VALUES('accumulate','v',1084,5279)
INSERT INTO WORD_LEMMATISED VALUES('accumulation','n',987,5570)
INSERT INTO WORD_LEMMATISED VALUES('accuracy','n',1692,3934)
INSERT INTO WORD_LEMMATISED VALUES('accurate','a',2928,2743)
INSERT INTO WORD_LEMMATISED VALUES('accurately','adv',1423,4436)
INSERT INTO WORD_LEMMATISED VALUES('accusation','n',1153,5088)
INSERT INTO WORD_LEMMATISED VALUES('accuse','v',4047,2190)
INSERT INTO WORD_LEMMATISED VALUES('accused','a',1387,4521)
INSERT INTO WORD_LEMMATISED VALUES('achieve','v',16628,611)
INSERT INTO WORD_LEMMATISED VALUES('achievement','n',4586,1980)
INSERT INTO WORD_LEMMATISED VALUES('acid','n',5897,1613)
INSERT INTO WORD_LEMMATISED VALUES('acknowledge','v',4151,2139)
INSERT INTO WORD_LEMMATISED VALUES('acquaintance','n',920,5809)
INSERT INTO WORD_LEMMATISED VALUES('acquire','v',6354,1512)
INSERT INTO WORD_LEMMATISED VALUES('acquisition','n',3245,2546)
INSERT INTO WORD_LEMMATISED VALUES('acre','n',2150,3327)
INSERT INTO WORD_LEMMATISED VALUES('across','adv',3439,2457)
INSERT INTO WORD_LEMMATISED VALUES('across','prep',21763,474)
INSERT INTO WORD_LEMMATISED VALUES('act','n',22657,455)
INSERT INTO WORD_LEMMATISED VALUES('act','v',15620,654)
INSERT INTO WORD_LEMMATISED VALUES('action','n',26894,371)
INSERT INTO WORD_LEMMATISED VALUES('activate','v',1293,4708)
INSERT INTO WORD_LEMMATISED VALUES('active','a',7290,1332)
INSERT INTO WORD_LEMMATISED VALUES('actively','adv',1493,4294)
INSERT INTO WORD_LEMMATISED VALUES('activist','n',1285,4736)
INSERT INTO WORD_LEMMATISED VALUES('activity','n',23105,440)
INSERT INTO WORD_LEMMATISED VALUES('actor','n',3540,2413)
INSERT INTO WORD_LEMMATISED VALUES('actress','n',1051,5376)
INSERT INTO WORD_LEMMATISED VALUES('actual','a',6849,1414)
INSERT INTO WORD_LEMMATISED VALUES('actually','adv',25990,385)
INSERT INTO WORD_LEMMATISED VALUES('acute','a',2294,3205)
INSERT INTO WORD_LEMMATISED VALUES('adapt','v',2687,2906)
INSERT INTO WORD_LEMMATISED VALUES('adaptation','n',1132,5153)
INSERT INTO WORD_LEMMATISED VALUES('add','v',27367,363)
INSERT INTO WORD_LEMMATISED VALUES('added','a',1334,4624)
INSERT INTO WORD_LEMMATISED VALUES('addition','n',10664,949)
INSERT INTO WORD_LEMMATISED VALUES('additional','a',7364,1319)
INSERT INTO WORD_LEMMATISED VALUES('address','n',6112,1566)
INSERT INTO WORD_LEMMATISED VALUES('address','v',5872,1625)
INSERT INTO WORD_LEMMATISED VALUES('adequate','a',3571,2397)
INSERT INTO WORD_LEMMATISED VALUES('adequately','adv',1149,5097)
INSERT INTO WORD_LEMMATISED VALUES('adjacent','a',1624,4047)
INSERT INTO WORD_LEMMATISED VALUES('adjective','n',1012,5496)
INSERT INTO WORD_LEMMATISED VALUES('adjust','v',2697,2898)
INSERT INTO WORD_LEMMATISED VALUES('adjustment','n',2114,3370)
INSERT INTO WORD_LEMMATISED VALUES('administer','v',1795,3769)
INSERT INTO WORD_LEMMATISED VALUES('administration','n',6865,1409)
INSERT INTO WORD_LEMMATISED VALUES('administrative','a',3548,2407)
INSERT INTO WORD_LEMMATISED VALUES('administrator','n',1483,4314)
INSERT INTO WORD_LEMMATISED VALUES('admiration','n',966,5654)
INSERT INTO WORD_LEMMATISED VALUES('admire','v',2168,3312)
INSERT INTO WORD_LEMMATISED VALUES('admission','n',2742,2861)
INSERT INTO WORD_LEMMATISED VALUES('admit','v',10905,925)
INSERT INTO WORD_LEMMATISED VALUES('adopt','v',8549,1186)
INSERT INTO WORD_LEMMATISED VALUES('adoption','n',1544,4191)
INSERT INTO WORD_LEMMATISED VALUES('adult','a',1547,4182)
INSERT INTO WORD_LEMMATISED VALUES('adult','n',6855,1411)
INSERT INTO WORD_LEMMATISED VALUES('advance','n',5055,1840)
INSERT INTO WORD_LEMMATISED VALUES('advance','v',3032,2683)
INSERT INTO WORD_LEMMATISED VALUES('advanced','a',3183,2583)
INSERT INTO WORD_LEMMATISED VALUES('advantage','n',10285,981)
INSERT INTO WORD_LEMMATISED VALUES('adventure','n',1993,3491)
INSERT INTO WORD_LEMMATISED VALUES('adverse','a',1184,5010)
INSERT INTO WORD_LEMMATISED VALUES('advertise','v',2099,3386)
INSERT INTO WORD_LEMMATISED VALUES('advertisement','n',2136,3344)
INSERT INTO WORD_LEMMATISED VALUES('advertising','n',3582,2392)
INSERT INTO WORD_LEMMATISED VALUES('advice','n',10437,971)
INSERT INTO WORD_LEMMATISED VALUES('advise','v',5055,1839)
INSERT INTO WORD_LEMMATISED VALUES('adviser','n',3191,2580)
INSERT INTO WORD_LEMMATISED VALUES('advisory','a',1500,4277)
INSERT INTO WORD_LEMMATISED VALUES('advocate','n',911,5848)
INSERT INTO WORD_LEMMATISED VALUES('advocate','v',1636,4029)
INSERT INTO WORD_LEMMATISED VALUES('aesthetic','a',1027,5444)
INSERT INTO WORD_LEMMATISED VALUES('affair','n',10561,954)
INSERT INTO WORD_LEMMATISED VALUES('affect','v',12867,789)
INSERT INTO WORD_LEMMATISED VALUES('affection','n',1601,4092)
INSERT INTO WORD_LEMMATISED VALUES('affinity','n',847,6101)
INSERT INTO WORD_LEMMATISED VALUES('afford','v',5346,1769)
INSERT INTO WORD_LEMMATISED VALUES('afraid','a',5967,1596)
INSERT INTO WORD_LEMMATISED VALUES('after','conj',30855,312)
INSERT INTO WORD_LEMMATISED VALUES('after','prep',85939,111)
INSERT INTO WORD_LEMMATISED VALUES('afternoon','n',8934,1128)
INSERT INTO WORD_LEMMATISED VALUES('afterwards','adv',4544,1991)
INSERT INTO WORD_LEMMATISED VALUES('again','adv',59829,160)
INSERT INTO WORD_LEMMATISED VALUES('against','prep',56208,169)
INSERT INTO WORD_LEMMATISED VALUES('age','n',25340,399)
INSERT INTO WORD_LEMMATISED VALUES('age','v',1565,4142)
INSERT INTO WORD_LEMMATISED VALUES('aged','prep',3096,2641)
INSERT INTO WORD_LEMMATISED VALUES('agency','n',9579,1054)
INSERT INTO WORD_LEMMATISED VALUES('agenda','n',2427,3069)
INSERT INTO WORD_LEMMATISED VALUES('agent','n',8100,1233)
INSERT INTO WORD_LEMMATISED VALUES('aggregate','a',1078,5294)
INSERT INTO WORD_LEMMATISED VALUES('aggression','n',1243,4845)
INSERT INTO WORD_LEMMATISED VALUES('aggressive','a',1925,3581)
INSERT INTO WORD_LEMMATISED VALUES('ago','adv',19808,522)
INSERT INTO WORD_LEMMATISED VALUES('agony','n',1017,5475)
INSERT INTO WORD_LEMMATISED VALUES('agree','v',23497,428)
INSERT INTO WORD_LEMMATISED VALUES('agreed','a',1106,5218)
INSERT INTO WORD_LEMMATISED VALUES('agreement','n',15627,651)
INSERT INTO WORD_LEMMATISED VALUES('agricultural','a',3162,2593)
INSERT INTO WORD_LEMMATISED VALUES('agriculture','n',3858,2265)
INSERT INTO WORD_LEMMATISED VALUES('ahead','adv',8809,1145)
INSERT INTO WORD_LEMMATISED VALUES('aid','n',7903,1249)
INSERT INTO WORD_LEMMATISED VALUES('aid','v',2086,3398)
INSERT INTO WORD_LEMMATISED VALUES('aids','n',2463,3042)
INSERT INTO WORD_LEMMATISED VALUES('aim','n',7460,1306)
INSERT INTO WORD_LEMMATISED VALUES('aim','v',7638,1281)
INSERT INTO WORD_LEMMATISED VALUES('air','n',19046,544)
INSERT INTO WORD_LEMMATISED VALUES('aircraft','n',6200,1548)
INSERT INTO WORD_LEMMATISED VALUES('airline','n',1982,3500)
INSERT INTO WORD_LEMMATISED VALUES('airport','n',2756,2855)
INSERT INTO WORD_LEMMATISED VALUES('alarm','n',2322,3169)
INSERT INTO WORD_LEMMATISED VALUES('alarm','v',1222,4914)
INSERT INTO WORD_LEMMATISED VALUES('albeit','conj',1406,4477)
INSERT INTO WORD_LEMMATISED VALUES('album','n',2581,2957)
INSERT INTO WORD_LEMMATISED VALUES('alcohol','n',3066,2662)
INSERT INTO WORD_LEMMATISED VALUES('alert','a',810,6273)
INSERT INTO WORD_LEMMATISED VALUES('alert','v',1125,5170)
INSERT INTO WORD_LEMMATISED VALUES('alien','a',868,6003)
INSERT INTO WORD_LEMMATISED VALUES('alike','adv',1270,4773)
INSERT INTO WORD_LEMMATISED VALUES('alive','a',4254,2103)
INSERT INTO WORD_LEMMATISED VALUES('all','adv',55704,171)
INSERT INTO WORD_LEMMATISED VALUES('all','det',230737,43)
INSERT INTO WORD_LEMMATISED VALUES('allegation','n',2124,3360)
INSERT INTO WORD_LEMMATISED VALUES('allege','v',1820,3731)
INSERT INTO WORD_LEMMATISED VALUES('alleged','a',1728,3887)
INSERT INTO WORD_LEMMATISED VALUES('allegedly','adv',1041,5398)
INSERT INTO WORD_LEMMATISED VALUES('alliance','n',3518,2422)
INSERT INTO WORD_LEMMATISED VALUES('allied','a',972,5630)
INSERT INTO WORD_LEMMATISED VALUES('allocate','v',2445,3060)
INSERT INTO WORD_LEMMATISED VALUES('allocation','n',2305,3193)
INSERT INTO WORD_LEMMATISED VALUES('allow','v',33687,279)
INSERT INTO WORD_LEMMATISED VALUES('allowance','n',3411,2472)
INSERT INTO WORD_LEMMATISED VALUES('ally','n',2381,3105)
INSERT INTO WORD_LEMMATISED VALUES('almost','adv',31588,300)
INSERT INTO WORD_LEMMATISED VALUES('alone','a',7088,1365)
INSERT INTO WORD_LEMMATISED VALUES('alone','adv',6262,1528)
INSERT INTO WORD_LEMMATISED VALUES('along','adv',8081,1235)
INSERT INTO WORD_LEMMATISED VALUES('along','prep',16233,627)
INSERT INTO WORD_LEMMATISED VALUES('alongside','prep',2720,2883)
INSERT INTO WORD_LEMMATISED VALUES('aloud','adv',999,5533)
INSERT INTO WORD_LEMMATISED VALUES('already','adv',34292,275)
INSERT INTO WORD_LEMMATISED VALUES('alright','a',3592,2386)
INSERT INTO WORD_LEMMATISED VALUES('alright','adv',4777,1926)
INSERT INTO WORD_LEMMATISED VALUES('also','adv',124884,81)
INSERT INTO WORD_LEMMATISED VALUES('altar','n',969,5643)
INSERT INTO WORD_LEMMATISED VALUES('alter','v',4061,2183)
INSERT INTO WORD_LEMMATISED VALUES('alteration','n',1527,4226)
INSERT INTO WORD_LEMMATISED VALUES('alternative','a',5098,1828)
INSERT INTO WORD_LEMMATISED VALUES('alternative','n',5228,1797)
INSERT INTO WORD_LEMMATISED VALUES('alternatively','adv',1723,3894)
INSERT INTO WORD_LEMMATISED VALUES('although','conj',43635,215)
INSERT INTO WORD_LEMMATISED VALUES('altogether','adv',3218,2558)
INSERT INTO WORD_LEMMATISED VALUES('aluminium','n',995,5546)
INSERT INTO WORD_LEMMATISED VALUES('always','adv',46228,201)
INSERT INTO WORD_LEMMATISED VALUES('amateur','a',1025,5452)
INSERT INTO WORD_LEMMATISED VALUES('amateur','n',900,5879)
INSERT INTO WORD_LEMMATISED VALUES('amazing','a',1873,3658)
INSERT INTO WORD_LEMMATISED VALUES('ambassador','n',1264,4788)
INSERT INTO WORD_LEMMATISED VALUES('ambiguity','n',1058,5345)
INSERT INTO WORD_LEMMATISED VALUES('ambiguous','a',837,6146)
INSERT INTO WORD_LEMMATISED VALUES('ambition','n',2319,3177)
INSERT INTO WORD_LEMMATISED VALUES('ambitious','a',1526,4228)
INSERT INTO WORD_LEMMATISED VALUES('ambulance','n',1690,3936)
INSERT INTO WORD_LEMMATISED VALUES('amend','v',1386,4525)
INSERT INTO WORD_LEMMATISED VALUES('amendment','n',2574,2960)
INSERT INTO WORD_LEMMATISED VALUES('amid','prep',1096,5243)
INSERT INTO WORD_LEMMATISED VALUES('among','prep',22864,450)
INSERT INTO WORD_LEMMATISED VALUES('amongst','prep',4552,1989)
INSERT INTO WORD_LEMMATISED VALUES('amount','n',16138,629)
INSERT INTO WORD_LEMMATISED VALUES('amount','v',3633,2358)
INSERT INTO WORD_LEMMATISED VALUES('amp','n',878,5960)
INSERT INTO WORD_LEMMATISED VALUES('ample','a',822,6211)
INSERT INTO WORD_LEMMATISED VALUES('amuse','v',1093,5255)
INSERT INTO WORD_LEMMATISED VALUES('amusement','n',1027,5443)
INSERT INTO WORD_LEMMATISED VALUES('an','det',343063,33)
INSERT INTO WORD_LEMMATISED VALUES('analogy','n',1098,5238)
INSERT INTO WORD_LEMMATISED VALUES('analyse','v',4106,2166)
INSERT INTO WORD_LEMMATISED VALUES('analysis','n',14149,732)
INSERT INTO WORD_LEMMATISED VALUES('analyst','n',1586,4109)
INSERT INTO WORD_LEMMATISED VALUES('ancestor','n',1249,4821)
INSERT INTO WORD_LEMMATISED VALUES('ancient','a',4981,1860)
INSERT INTO WORD_LEMMATISED VALUES('and','conj',2687863,4)
INSERT INTO WORD_LEMMATISED VALUES('angel','n',1897,3616)
INSERT INTO WORD_LEMMATISED VALUES('anger','n',3291,2530)
INSERT INTO WORD_LEMMATISED VALUES('anger','v',892,5902)
INSERT INTO WORD_LEMMATISED VALUES('angle','n',3644,2352)
INSERT INTO WORD_LEMMATISED VALUES('angrily','adv',1093,5254)
INSERT INTO WORD_LEMMATISED VALUES('angry','a',4282,2095)
INSERT INTO WORD_LEMMATISED VALUES('animal','n',15250,671)
INSERT INTO WORD_LEMMATISED VALUES('ankle','n',1363,4573)
INSERT INTO WORD_LEMMATISED VALUES('anniversary','n',2144,3332)
INSERT INTO WORD_LEMMATISED VALUES('announce','v',12582,815)
INSERT INTO WORD_LEMMATISED VALUES('announcement','n',2758,2852)
INSERT INTO WORD_LEMMATISED VALUES('annoy','v',1116,5186)
INSERT INTO WORD_LEMMATISED VALUES('annual','a',7785,1265)
INSERT INTO WORD_LEMMATISED VALUES('annually','adv',1092,5258)
INSERT INTO WORD_LEMMATISED VALUES('anonymous','a',1112,5194)
INSERT INTO WORD_LEMMATISED VALUES('another','det',60182,159)
INSERT INTO WORD_LEMMATISED VALUES('answer','n',12596,810)
INSERT INTO WORD_LEMMATISED VALUES('answer','v',10140,989)
INSERT INTO WORD_LEMMATISED VALUES('ant','n',998,5539)
INSERT INTO WORD_LEMMATISED VALUES('antibody','n',1528,4221)
INSERT INTO WORD_LEMMATISED VALUES('anticipate','v',2316,3184)
INSERT INTO WORD_LEMMATISED VALUES('anticipation','n',835,6155)
INSERT INTO WORD_LEMMATISED VALUES('anxiety','n',3129,2616)
INSERT INTO WORD_LEMMATISED VALUES('anxious','a',3088,2645)
INSERT INTO WORD_LEMMATISED VALUES('any','det',123655,83)
INSERT INTO WORD_LEMMATISED VALUES('anybody','pron',4952,1872)
INSERT INTO WORD_LEMMATISED VALUES('anyone','pron',14956,686)
INSERT INTO WORD_LEMMATISED VALUES('anything','pron',28321,347)
INSERT INTO WORD_LEMMATISED VALUES('anyway','adv',12232,837)
INSERT INTO WORD_LEMMATISED VALUES('anywhere','adv',4105,2167)
INSERT INTO WORD_LEMMATISED VALUES('apart','adv',10040,1004)
INSERT INTO WORD_LEMMATISED VALUES('apartment','n',1901,3612)
INSERT INTO WORD_LEMMATISED VALUES('apologise','v',1057,5353)
INSERT INTO WORD_LEMMATISED VALUES('apology','n',1195,4984)
INSERT INTO WORD_LEMMATISED VALUES('appalling','a',1027,5442)
INSERT INTO WORD_LEMMATISED VALUES('apparatus','n',1099,5236)
INSERT INTO WORD_LEMMATISED VALUES('apparent','a',5306,1782)
INSERT INTO WORD_LEMMATISED VALUES('apparently','adv',7696,1272)
INSERT INTO WORD_LEMMATISED VALUES('appeal','n',10020,1006)
INSERT INTO WORD_LEMMATISED VALUES('appeal','v',4764,1930)
INSERT INTO WORD_LEMMATISED VALUES('appear','v',30595,319)
INSERT INTO WORD_LEMMATISED VALUES('appearance','n',6606,1462)
INSERT INTO WORD_LEMMATISED VALUES('appendix','n',1884,3643)
INSERT INTO WORD_LEMMATISED VALUES('appetite','n',1055,5363)
INSERT INTO WORD_LEMMATISED VALUES('apple','n',2996,2704)
INSERT INTO WORD_LEMMATISED VALUES('applicable','a',1419,4443)
INSERT INTO WORD_LEMMATISED VALUES('applicant','n',2179,3305)
INSERT INTO WORD_LEMMATISED VALUES('application','n',16281,623)
INSERT INTO WORD_LEMMATISED VALUES('applied','a',1155,5081)
INSERT INTO WORD_LEMMATISED VALUES('apply','v',18982,547)
INSERT INTO WORD_LEMMATISED VALUES('appoint','v',6407,1500)
INSERT INTO WORD_LEMMATISED VALUES('appointed','a',1606,4080)
INSERT INTO WORD_LEMMATISED VALUES('appointment','n',5911,1610)
INSERT INTO WORD_LEMMATISED VALUES('appraisal','n',1092,5257)
INSERT INTO WORD_LEMMATISED VALUES('appreciate','v',4312,2082)
INSERT INTO WORD_LEMMATISED VALUES('appreciation','n',1329,4640)
INSERT INTO WORD_LEMMATISED VALUES('approach','n',16331,621)
INSERT INTO WORD_LEMMATISED VALUES('approach','v',7432,1310)
INSERT INTO WORD_LEMMATISED VALUES('appropriate','a',11010,917)
INSERT INTO WORD_LEMMATISED VALUES('appropriately','adv',891,5905)
INSERT INTO WORD_LEMMATISED VALUES('approval','n',4050,2187)
INSERT INTO WORD_LEMMATISED VALUES('approve','v',5312,1780)
INSERT INTO WORD_LEMMATISED VALUES('approved','a',911,5847)
INSERT INTO WORD_LEMMATISED VALUES('approximately','adv',2837,2798)
INSERT INTO WORD_LEMMATISED VALUES('aquarium','n',912,5843)
INSERT INTO WORD_LEMMATISED VALUES('arbitrary','a',1091,5262)
INSERT INTO WORD_LEMMATISED VALUES('arc','n',842,6125)
INSERT INTO WORD_LEMMATISED VALUES('arch','n',1248,4825)
INSERT INTO WORD_LEMMATISED VALUES('archaeological','a',826,6187)
INSERT INTO WORD_LEMMATISED VALUES('archbishop','n',1109,5206)
INSERT INTO WORD_LEMMATISED VALUES('architect','n',2572,2961)
INSERT INTO WORD_LEMMATISED VALUES('architectural','a',885,5929)
INSERT INTO WORD_LEMMATISED VALUES('architecture','n',3018,2688)
INSERT INTO WORD_LEMMATISED VALUES('archive','n',1124,5172)
INSERT INTO WORD_LEMMATISED VALUES('area','n',58449,162)
INSERT INTO WORD_LEMMATISED VALUES('arena','n',952,5704)
INSERT INTO WORD_LEMMATISED VALUES('argue','v',14196,728)
INSERT INTO WORD_LEMMATISED VALUES('argument','n',12125,843)
INSERT INTO WORD_LEMMATISED VALUES('arise','v',8860,1139)
INSERT INTO WORD_LEMMATISED VALUES('arm','n',20089,513)
INSERT INTO WORD_LEMMATISED VALUES('arm','v',1533,4214)
INSERT INTO WORD_LEMMATISED VALUES('armchair','n',886,5924)
INSERT INTO WORD_LEMMATISED VALUES('armed','a',3344,2505)
INSERT INTO WORD_LEMMATISED VALUES('army','n',12379,826)
INSERT INTO WORD_LEMMATISED VALUES('around','adv',23106,439)
INSERT INTO WORD_LEMMATISED VALUES('around','prep',22180,461)
INSERT INTO WORD_LEMMATISED VALUES('arouse','v',1308,4675)
INSERT INTO WORD_LEMMATISED VALUES('arrange','v',7021,1379)
INSERT INTO WORD_LEMMATISED VALUES('arrangement','n',9054,1113)
INSERT INTO WORD_LEMMATISED VALUES('array','n',1258,4803)
INSERT INTO WORD_LEMMATISED VALUES('arrest','n',2095,3391)
INSERT INTO WORD_LEMMATISED VALUES('arrest','v',4059,2185)
INSERT INTO WORD_LEMMATISED VALUES('arrival','n',3815,2278)
INSERT INTO WORD_LEMMATISED VALUES('arrive','v',14093,735)
INSERT INTO WORD_LEMMATISED VALUES('arrow','n',1621,4058)
INSERT INTO WORD_LEMMATISED VALUES('art','n',20168,509)
INSERT INTO WORD_LEMMATISED VALUES('article','n',9710,1037)
INSERT INTO WORD_LEMMATISED VALUES('articulate','v',821,6218)
INSERT INTO WORD_LEMMATISED VALUES('artificial','a',1901,3611)
INSERT INTO WORD_LEMMATISED VALUES('artist','n',8105,1232)
INSERT INTO WORD_LEMMATISED VALUES('artistic','a',1553,4170)
INSERT INTO WORD_LEMMATISED VALUES('as','adv',101583,98)
INSERT INTO WORD_LEMMATISED VALUES('as','conj',364164,31)
INSERT INTO WORD_LEMMATISED VALUES('as','prep',201968,48)
INSERT INTO WORD_LEMMATISED VALUES('ascertain','v',842,6124)
INSERT INTO WORD_LEMMATISED VALUES('ash','n',1051,5375)
INSERT INTO WORD_LEMMATISED VALUES('ashamed','a',1100,5233)
INSERT INTO WORD_LEMMATISED VALUES('aside','adv',3607,2375)
INSERT INTO WORD_LEMMATISED VALUES('ask','v',60879,154)
INSERT INTO WORD_LEMMATISED VALUES('asleep','a',2463,3041)
INSERT INTO WORD_LEMMATISED VALUES('aspect','n',11643,871)
INSERT INTO WORD_LEMMATISED VALUES('aspiration','n',1269,4777)
INSERT INTO WORD_LEMMATISED VALUES('assault','n',2395,3094)
INSERT INTO WORD_LEMMATISED VALUES('assault','v',919,5814)
INSERT INTO WORD_LEMMATISED VALUES('assemble','v',1621,4057)
INSERT INTO WORD_LEMMATISED VALUES('assembly','n',5847,1630)
INSERT INTO WORD_LEMMATISED VALUES('assert','v',2012,3477)
INSERT INTO WORD_LEMMATISED VALUES('assertion','n',1119,5180)
INSERT INTO WORD_LEMMATISED VALUES('assess','v',6088,1570)
INSERT INTO WORD_LEMMATISED VALUES('assessment','n',7602,1290)
INSERT INTO WORD_LEMMATISED VALUES('asset','n',5721,1663)
INSERT INTO WORD_LEMMATISED VALUES('assign','v',1778,3802)
INSERT INTO WORD_LEMMATISED VALUES('assignment','n',1707,3919)
INSERT INTO WORD_LEMMATISED VALUES('assist','v',4021,2202)
INSERT INTO WORD_LEMMATISED VALUES('assistance','n',4344,2064)
INSERT INTO WORD_LEMMATISED VALUES('assistant','a',1745,3860)
INSERT INTO WORD_LEMMATISED VALUES('assistant','n',2655,2921)
INSERT INTO WORD_LEMMATISED VALUES('associate','n',1047,5383)
INSERT INTO WORD_LEMMATISED VALUES('associate','v',8743,1156)
INSERT INTO WORD_LEMMATISED VALUES('associated','a',1447,4391)
INSERT INTO WORD_LEMMATISED VALUES('association','n',13471,757)
INSERT INTO WORD_LEMMATISED VALUES('assume','v',11044,912)
INSERT INTO WORD_LEMMATISED VALUES('assumption','n',5554,1705)
INSERT INTO WORD_LEMMATISED VALUES('assurance','n',2100,3384)
INSERT INTO WORD_LEMMATISED VALUES('assure','v',3116,2625)
INSERT INTO WORD_LEMMATISED VALUES('astonishing','a',848,6096)
INSERT INTO WORD_LEMMATISED VALUES('asylum','n',974,5620)
INSERT INTO WORD_LEMMATISED VALUES('at','prep',534162,19)
INSERT INTO WORD_LEMMATISED VALUES('athlete','n',812,6264)
INSERT INTO WORD_LEMMATISED VALUES('atmosphere','n',4902,1889)
INSERT INTO WORD_LEMMATISED VALUES('atom','n',1473,4331)
INSERT INTO WORD_LEMMATISED VALUES('atomic','a',1106,5217)
INSERT INTO WORD_LEMMATISED VALUES('attach','v',4821,1915)
INSERT INTO WORD_LEMMATISED VALUES('attachment','n',959,5677)
INSERT INTO WORD_LEMMATISED VALUES('attack','n',9963,1012)
INSERT INTO WORD_LEMMATISED VALUES('attack','v',6586,1464)
INSERT INTO WORD_LEMMATISED VALUES('attacker','n',919,5813)
INSERT INTO WORD_LEMMATISED VALUES('attain','v',1182,5016)
INSERT INTO WORD_LEMMATISED VALUES('attainment','n',948,5717)
INSERT INTO WORD_LEMMATISED VALUES('attempt','n',11877,857)
INSERT INTO WORD_LEMMATISED VALUES('attempt','v',9873,1022)
INSERT INTO WORD_LEMMATISED VALUES('attempted','a',849,6090)
INSERT INTO WORD_LEMMATISED VALUES('attend','v',8801,1148)
INSERT INTO WORD_LEMMATISED VALUES('attendance','n',1889,3633)
INSERT INTO WORD_LEMMATISED VALUES('attention','n',13968,740)
INSERT INTO WORD_LEMMATISED VALUES('attitude','n',10758,938)
INSERT INTO WORD_LEMMATISED VALUES('attract','v',6410,1498)
INSERT INTO WORD_LEMMATISED VALUES('attraction','n',2366,3124)
INSERT INTO WORD_LEMMATISED VALUES('attractive','a',5152,1814)
INSERT INTO WORD_LEMMATISED VALUES('attribute','n',1260,4798)
INSERT INTO WORD_LEMMATISED VALUES('attribute','v',2311,3187)
INSERT INTO WORD_LEMMATISED VALUES('auction','n',1466,4352)
INSERT INTO WORD_LEMMATISED VALUES('audience','n',6500,1480)
INSERT INTO WORD_LEMMATISED VALUES('audit','n',2136,3343)
INSERT INTO WORD_LEMMATISED VALUES('auditor','n',1679,3955)
INSERT INTO WORD_LEMMATISED VALUES('aunt','n',1536,4211)
INSERT INTO WORD_LEMMATISED VALUES('author','n',6852,1413)
INSERT INTO WORD_LEMMATISED VALUES('authorise','v',968,5647)
INSERT INTO WORD_LEMMATISED VALUES('authority','n',31231,303)
INSERT INTO WORD_LEMMATISED VALUES('automatic','a',2245,3242)
INSERT INTO WORD_LEMMATISED VALUES('automatically','adv',2787,2832)
INSERT INTO WORD_LEMMATISED VALUES('autonomous','a',1073,5305)
INSERT INTO WORD_LEMMATISED VALUES('autonomy','n',1788,3783)
INSERT INTO WORD_LEMMATISED VALUES('autumn','n',3917,2250)
INSERT INTO WORD_LEMMATISED VALUES('availability','n',1912,3602)
INSERT INTO WORD_LEMMATISED VALUES('available','a',27184,367)
INSERT INTO WORD_LEMMATISED VALUES('avenue','n',943,5736)
INSERT INTO WORD_LEMMATISED VALUES('average','a',5922,1607)
INSERT INTO WORD_LEMMATISED VALUES('average','n',4104,2168)
INSERT INTO WORD_LEMMATISED VALUES('aviation','n',925,5794)
INSERT INTO WORD_LEMMATISED VALUES('avoid','v',11750,866)
INSERT INTO WORD_LEMMATISED VALUES('await','v',1958,3536)
INSERT INTO WORD_LEMMATISED VALUES('awake','a',1040,5404)
INSERT INTO WORD_LEMMATISED VALUES('awake','v',825,6193)
INSERT INTO WORD_LEMMATISED VALUES('award','n',13274,764)
INSERT INTO WORD_LEMMATISED VALUES('award','v',6625,1458)
INSERT INTO WORD_LEMMATISED VALUES('aware','a',10764,936)
INSERT INTO WORD_LEMMATISED VALUES('awareness','n',3516,2424)
INSERT INTO WORD_LEMMATISED VALUES('away','adv',50294,188)
INSERT INTO WORD_LEMMATISED VALUES('awful','a',2960,2731)
INSERT INTO WORD_LEMMATISED VALUES('awkward','a',1489,4301)
INSERT INTO WORD_LEMMATISED VALUES('axis','n',1019,5469)
INSERT INTO WORD_LEMMATISED VALUES('aye','interjecti',5166,1811)
INSERT INTO WORD_LEMMATISED VALUES('baby','n',11503,883)
INSERT INTO WORD_LEMMATISED VALUES('back','a',2335,3160)
INSERT INTO WORD_LEMMATISED VALUES('back','adv',75494,118)
INSERT INTO WORD_LEMMATISED VALUES('back','n',24095,415)
INSERT INTO WORD_LEMMATISED VALUES('back','v',4391,2048)
INSERT INTO WORD_LEMMATISED VALUES('background','n',6923,1398)
INSERT INTO WORD_LEMMATISED VALUES('backing','n',1887,3636)
INSERT INTO WORD_LEMMATISED VALUES('backwards','adv',1802,3757)
INSERT INTO WORD_LEMMATISED VALUES('bacon','n',1037,5411)
INSERT INTO WORD_LEMMATISED VALUES('bacteria','n',1265,4786)
INSERT INTO WORD_LEMMATISED VALUES('bad','a',25608,393)
INSERT INTO WORD_LEMMATISED VALUES('badly','adv',4329,2072)
INSERT INTO WORD_LEMMATISED VALUES('bag','n',6955,1389)
INSERT INTO WORD_LEMMATISED VALUES('bail','n',811,6268)
INSERT INTO WORD_LEMMATISED VALUES('bake','v',930,5773)
INSERT INTO WORD_LEMMATISED VALUES('balance','n',8601,1179)
INSERT INTO WORD_LEMMATISED VALUES('balance','v',2361,3132)
INSERT INTO WORD_LEMMATISED VALUES('balanced','a',898,5884)
INSERT INTO WORD_LEMMATISED VALUES('balcony','n',1127,5165)
INSERT INTO WORD_LEMMATISED VALUES('ball','n',8636,1170)
INSERT INTO WORD_LEMMATISED VALUES('ballet','n',1340,4614)
INSERT INTO WORD_LEMMATISED VALUES('balloon','n',1030,5436)
INSERT INTO WORD_LEMMATISED VALUES('ballot','n',1055,5362)
INSERT INTO WORD_LEMMATISED VALUES('ban','n',2468,3037)
INSERT INTO WORD_LEMMATISED VALUES('ban','v',2851,2791)
INSERT INTO WORD_LEMMATISED VALUES('banana','n',924,5797)
INSERT INTO WORD_LEMMATISED VALUES('band','n',9003,1117)
INSERT INTO WORD_LEMMATISED VALUES('bang','n',980,5597)
INSERT INTO WORD_LEMMATISED VALUES('bang','v',1197,4976)
INSERT INTO WORD_LEMMATISED VALUES('bank','n',20946,491)
INSERT INTO WORD_LEMMATISED VALUES('banker','n',1348,4602)
INSERT INTO WORD_LEMMATISED VALUES('banking','n',1681,3950)
INSERT INTO WORD_LEMMATISED VALUES('bankruptcy','n',1066,5321)
INSERT INTO WORD_LEMMATISED VALUES('banner','n',857,6051)
INSERT INTO WORD_LEMMATISED VALUES('bar','n',9969,1011)
INSERT INTO WORD_LEMMATISED VALUES('bar','v',1210,4947)
INSERT INTO WORD_LEMMATISED VALUES('bare','a',2291,3211)
INSERT INTO WORD_LEMMATISED VALUES('barely','adv',2293,3206)
INSERT INTO WORD_LEMMATISED VALUES('bargain','n',1229,4886)
INSERT INTO WORD_LEMMATISED VALUES('bargain','v',893,5898)
INSERT INTO WORD_LEMMATISED VALUES('barn','n',1465,4356)
INSERT INTO WORD_LEMMATISED VALUES('barrel','n',1411,4464)
INSERT INTO WORD_LEMMATISED VALUES('barrier','n',3011,2691)
INSERT INTO WORD_LEMMATISED VALUES('base','n',9410,1070)
INSERT INTO WORD_LEMMATISED VALUES('base','v',18742,556)
INSERT INTO WORD_LEMMATISED VALUES('basement','n',852,6077)
INSERT INTO WORD_LEMMATISED VALUES('basic','a',10860,928)
INSERT INTO WORD_LEMMATISED VALUES('basically','adv',3116,2624)
INSERT INTO WORD_LEMMATISED VALUES('basin','n',1314,4661)
INSERT INTO WORD_LEMMATISED VALUES('basis','n',14420,715)
INSERT INTO WORD_LEMMATISED VALUES('basket','n',1658,3983)
INSERT INTO WORD_LEMMATISED VALUES('bass','n',1036,5416)
INSERT INTO WORD_LEMMATISED VALUES('bastard','n',1841,3709)
INSERT INTO WORD_LEMMATISED VALUES('bat','n',1249,4820)
INSERT INTO WORD_LEMMATISED VALUES('batch','n',977,5609)
INSERT INTO WORD_LEMMATISED VALUES('bath','n',3484,2439)
INSERT INTO WORD_LEMMATISED VALUES('bathroom','n',2792,2828)
INSERT INTO WORD_LEMMATISED VALUES('battery','n',2050,3433)
INSERT INTO WORD_LEMMATISED VALUES('battle','n',7063,1372)
INSERT INTO WORD_LEMMATISED VALUES('battle','v',983,5583)
INSERT INTO WORD_LEMMATISED VALUES('bay','n',2097,3387)
INSERT INTO WORD_LEMMATISED VALUES('be','v',4239632,2)
INSERT INTO WORD_LEMMATISED VALUES('beach','n',4125,2154)
INSERT INTO WORD_LEMMATISED VALUES('beam','n',1540,4205)
INSERT INTO WORD_LEMMATISED VALUES('bean','n',1767,3824)
INSERT INTO WORD_LEMMATISED VALUES('bear','n',1786,3788)
INSERT INTO WORD_LEMMATISED VALUES('bear','v',17461,592)
INSERT INTO WORD_LEMMATISED VALUES('beard','n',980,5596)
INSERT INTO WORD_LEMMATISED VALUES('bearing','n',1466,4351)
INSERT INTO WORD_LEMMATISED VALUES('beast','n',1232,4877)
INSERT INTO WORD_LEMMATISED VALUES('beat','n',1425,4430)
INSERT INTO WORD_LEMMATISED VALUES('beat','v',8106,1231)
INSERT INTO WORD_LEMMATISED VALUES('beautiful','a',8670,1162)
INSERT INTO WORD_LEMMATISED VALUES('beautifully','adv',1241,4849)
INSERT INTO WORD_LEMMATISED VALUES('beauty','n',4445,2027)
INSERT INTO WORD_LEMMATISED VALUES('because','conj',103003,96)
INSERT INTO WORD_LEMMATISED VALUES('become','v',67219,130)
INSERT INTO WORD_LEMMATISED VALUES('bed','n',17947,578)
INSERT INTO WORD_LEMMATISED VALUES('bedroom','n',5865,1626)
INSERT INTO WORD_LEMMATISED VALUES('bee','n',1237,4859)
INSERT INTO WORD_LEMMATISED VALUES('beef','n',1516,4243)
INSERT INTO WORD_LEMMATISED VALUES('beer','n',3762,2307)
INSERT INTO WORD_LEMMATISED VALUES('before','adv',6285,1522)
INSERT INTO WORD_LEMMATISED VALUES('before','conj',30731,314)
INSERT INTO WORD_LEMMATISED VALUES('before','prep',51259,185)
INSERT INTO WORD_LEMMATISED VALUES('beg','v',1885,3642)
INSERT INTO WORD_LEMMATISED VALUES('begin','v',43740,214)
INSERT INTO WORD_LEMMATISED VALUES('beginning','n',8018,1240)
INSERT INTO WORD_LEMMATISED VALUES('behalf','n',4012,2206)
INSERT INTO WORD_LEMMATISED VALUES('behave','v',3082,2650)
INSERT INTO WORD_LEMMATISED VALUES('behaviour','n',12853,793)
INSERT INTO WORD_LEMMATISED VALUES('behind','adv',2795,2823)
INSERT INTO WORD_LEMMATISED VALUES('behind','prep',20694,498)
INSERT INTO WORD_LEMMATISED VALUES('being','n',3732,2324)
INSERT INTO WORD_LEMMATISED VALUES('belief','n',7509,1302)
INSERT INTO WORD_LEMMATISED VALUES('believe','v',34603,273)
INSERT INTO WORD_LEMMATISED VALUES('bell','n',2758,2851)
INSERT INTO WORD_LEMMATISED VALUES('belly','n',959,5676)
INSERT INTO WORD_LEMMATISED VALUES('belong','v',5716,1664)
INSERT INTO WORD_LEMMATISED VALUES('below','adv',8587,1182)
INSERT INTO WORD_LEMMATISED VALUES('below','prep',5748,1656)
INSERT INTO WORD_LEMMATISED VALUES('belt','n',2328,3163)
INSERT INTO WORD_LEMMATISED VALUES('bench','n',2320,3173)
INSERT INTO WORD_LEMMATISED VALUES('bend','n',881,5946)
INSERT INTO WORD_LEMMATISED VALUES('bend','v',3453,2454)
INSERT INTO WORD_LEMMATISED VALUES('beneath','prep',4917,1883)
INSERT INTO WORD_LEMMATISED VALUES('beneficial','a',1375,4544)
INSERT INTO WORD_LEMMATISED VALUES('beneficiary','n',876,5973)
INSERT INTO WORD_LEMMATISED VALUES('benefit','n',15251,670)
INSERT INTO WORD_LEMMATISED VALUES('benefit','v',4262,2101)
INSERT INTO WORD_LEMMATISED VALUES('beside','prep',5785,1649)
INSERT INTO WORD_LEMMATISED VALUES('besides','adv',1765,3830)
INSERT INTO WORD_LEMMATISED VALUES('besides','prep',848,6095)
INSERT INTO WORD_LEMMATISED VALUES('best','adv',9790,1027)
INSERT INTO WORD_LEMMATISED VALUES('bet','n',950,5710)
INSERT INTO WORD_LEMMATISED VALUES('bet','v',2292,3209)
INSERT INTO WORD_LEMMATISED VALUES('betray','v',1310,4670)
INSERT INTO WORD_LEMMATISED VALUES('better','adv',15626,652)
INSERT INTO WORD_LEMMATISED VALUES('between','prep',91141,105)
INSERT INTO WORD_LEMMATISED VALUES('beyond','adv',1062,5333)
INSERT INTO WORD_LEMMATISED VALUES('beyond','prep',10705,945)
INSERT INTO WORD_LEMMATISED VALUES('bias','n',1372,4548)
INSERT INTO WORD_LEMMATISED VALUES('bible','n',2075,3408)
INSERT INTO WORD_LEMMATISED VALUES('bicycle','n',1134,5144)
INSERT INTO WORD_LEMMATISED VALUES('bid','n',3316,2521)
INSERT INTO WORD_LEMMATISED VALUES('bid','v',1756,3843)
INSERT INTO WORD_LEMMATISED VALUES('big','a',33300,282)
INSERT INTO WORD_LEMMATISED VALUES('bike','n',2224,3266)
INSERT INTO WORD_LEMMATISED VALUES('bile','n',1228,4892)
INSERT INTO WORD_LEMMATISED VALUES('bill','n',10125,993)
INSERT INTO WORD_LEMMATISED VALUES('bin','n',1040,5403)
INSERT INTO WORD_LEMMATISED VALUES('bind','v',6517,1472)
INSERT INTO WORD_LEMMATISED VALUES('binding','a',1246,4832)
INSERT INTO WORD_LEMMATISED VALUES('biography','n',988,5567)
INSERT INTO WORD_LEMMATISED VALUES('biological','a',1965,3523)
INSERT INTO WORD_LEMMATISED VALUES('biology','n',1029,5440)
INSERT INTO WORD_LEMMATISED VALUES('bird','n',9021,1115)
INSERT INTO WORD_LEMMATISED VALUES('birth','n',5889,1615)
INSERT INTO WORD_LEMMATISED VALUES('birthday','n',3352,2500)
INSERT INTO WORD_LEMMATISED VALUES('biscuit','n',1552,4172)
INSERT INTO WORD_LEMMATISED VALUES('bishop','n',3620,2366)
INSERT INTO WORD_LEMMATISED VALUES('bit','n',30675,315)
INSERT INTO WORD_LEMMATISED VALUES('bitch','n',1006,5510)
INSERT INTO WORD_LEMMATISED VALUES('bite','n',887,5920)
INSERT INTO WORD_LEMMATISED VALUES('bite','v',2885,2765)
INSERT INTO WORD_LEMMATISED VALUES('bitter','a',2384,3103)
INSERT INTO WORD_LEMMATISED VALUES('bitterly','adv',1097,5240)
INSERT INTO WORD_LEMMATISED VALUES('bizarre','a',1054,5369)
INSERT INTO WORD_LEMMATISED VALUES('black','a',20609,502)
INSERT INTO WORD_LEMMATISED VALUES('black','n',2310,3189)
INSERT INTO WORD_LEMMATISED VALUES('blade','n',1442,4400)
INSERT INTO WORD_LEMMATISED VALUES('blame','n',852,6076)
INSERT INTO WORD_LEMMATISED VALUES('blame','v',4248,2108)
INSERT INTO WORD_LEMMATISED VALUES('blank','a',1499,4279)
INSERT INTO WORD_LEMMATISED VALUES('blanket','n',1545,4184)
INSERT INTO WORD_LEMMATISED VALUES('blast','n',975,5616)
INSERT INTO WORD_LEMMATISED VALUES('blast','v',866,6009)
INSERT INTO WORD_LEMMATISED VALUES('bleak','a',916,5825)
INSERT INTO WORD_LEMMATISED VALUES('bleed','v',896,5889)
INSERT INTO WORD_LEMMATISED VALUES('bless','v',1228,4891)
INSERT INTO WORD_LEMMATISED VALUES('blessing','n',938,5754)
INSERT INTO WORD_LEMMATISED VALUES('blind','a',2674,2912)
INSERT INTO WORD_LEMMATISED VALUES('blink','v',896,5888)
INSERT INTO WORD_LEMMATISED VALUES('block','n',5705,1669)
INSERT INTO WORD_LEMMATISED VALUES('block','v',2688,2904)
INSERT INTO WORD_LEMMATISED VALUES('bloke','n',1597,4095)
INSERT INTO WORD_LEMMATISED VALUES('blonde','a',912,5842)
INSERT INTO WORD_LEMMATISED VALUES('blood','n',10176,986)
INSERT INTO WORD_LEMMATISED VALUES('bloody','a',5261,1791)
INSERT INTO WORD_LEMMATISED VALUES('bloody','adv',2016,3470)
INSERT INTO WORD_LEMMATISED VALUES('blow','n',2372,3118)
INSERT INTO WORD_LEMMATISED VALUES('blow','v',5013,1856)
INSERT INTO WORD_LEMMATISED VALUES('blue','a',9089,1109)
INSERT INTO WORD_LEMMATISED VALUES('blue','n',2455,3053)
INSERT INTO WORD_LEMMATISED VALUES('board','n',17878,581)
INSERT INTO WORD_LEMMATISED VALUES('board','v',850,6086)
INSERT INTO WORD_LEMMATISED VALUES('boast','v',1179,5026)
INSERT INTO WORD_LEMMATISED VALUES('boat','n',7373,1317)
INSERT INTO WORD_LEMMATISED VALUES('bodily','a',817,6240)
INSERT INTO WORD_LEMMATISED VALUES('body','n',32231,294)
INSERT INTO WORD_LEMMATISED VALUES('boil','v',1148,5101)
INSERT INTO WORD_LEMMATISED VALUES('boiler','n',878,5959)
INSERT INTO WORD_LEMMATISED VALUES('bold','a',1430,4419)
INSERT INTO WORD_LEMMATISED VALUES('bolt','n',1281,4748)
INSERT INTO WORD_LEMMATISED VALUES('bomb','n',3703,2333)
INSERT INTO WORD_LEMMATISED VALUES('bomb','v',1202,4960)
INSERT INTO WORD_LEMMATISED VALUES('bomber','n',1003,5524)
INSERT INTO WORD_LEMMATISED VALUES('bond','n',3753,2313)
INSERT INTO WORD_LEMMATISED VALUES('bone','n',4665,1955)
INSERT INTO WORD_LEMMATISED VALUES('bonus','n',1364,4569)
INSERT INTO WORD_LEMMATISED VALUES('book','n',37675,252)
INSERT INTO WORD_LEMMATISED VALUES('book','v',1894,3623)
INSERT INTO WORD_LEMMATISED VALUES('booking','n',1391,4510)
INSERT INTO WORD_LEMMATISED VALUES('booklet','n',1210,4946)
INSERT INTO WORD_LEMMATISED VALUES('boom','n',1491,4297)
INSERT INTO WORD_LEMMATISED VALUES('boost','n',978,5603)
INSERT INTO WORD_LEMMATISED VALUES('boost','v',1739,3873)
INSERT INTO WORD_LEMMATISED VALUES('boot','n',3930,2240)
INSERT INTO WORD_LEMMATISED VALUES('border','n',5028,1853)
INSERT INTO WORD_LEMMATISED VALUES('bored','a',1040,5402)
INSERT INTO WORD_LEMMATISED VALUES('boring','a',1531,4217)
INSERT INTO WORD_LEMMATISED VALUES('borough','n',2243,3246)
INSERT INTO WORD_LEMMATISED VALUES('borrow','v',3353,2499)
INSERT INTO WORD_LEMMATISED VALUES('boss','n',3739,2320)
INSERT INTO WORD_LEMMATISED VALUES('both','adv',27644,358)
INSERT INTO WORD_LEMMATISED VALUES('both','det',41162,227)
INSERT INTO WORD_LEMMATISED VALUES('bother','v',4048,2188)
INSERT INTO WORD_LEMMATISED VALUES('bottle','n',5872,1624)
INSERT INTO WORD_LEMMATISED VALUES('bottom','a',2301,3198)
INSERT INTO WORD_LEMMATISED VALUES('bottom','n',5441,1742)
INSERT INTO WORD_LEMMATISED VALUES('bounce','v',977,5608)
INSERT INTO WORD_LEMMATISED VALUES('boundary','n',4416,2036)
INSERT INTO WORD_LEMMATISED VALUES('bourgeois','a',929,5782)
INSERT INTO WORD_LEMMATISED VALUES('bow','n',1357,4585)
INSERT INTO WORD_LEMMATISED VALUES('bow','v',1359,4578)
INSERT INTO WORD_LEMMATISED VALUES('bowel','n',1441,4404)
INSERT INTO WORD_LEMMATISED VALUES('bowl','n',2631,2935)
INSERT INTO WORD_LEMMATISED VALUES('bowl','v',1020,5466)
INSERT INTO WORD_LEMMATISED VALUES('bowler','n',997,5541)
INSERT INTO WORD_LEMMATISED VALUES('box','n',11292,900)
INSERT INTO WORD_LEMMATISED VALUES('boxing','n',902,5872)
INSERT INTO WORD_LEMMATISED VALUES('boy','n',21205,486)
INSERT INTO WORD_LEMMATISED VALUES('boyfriend','n',1285,4735)
INSERT INTO WORD_LEMMATISED VALUES('bracket','n',1017,5474)
INSERT INTO WORD_LEMMATISED VALUES('brain','n',5586,1698)
INSERT INTO WORD_LEMMATISED VALUES('brake','n',974,5619)
INSERT INTO WORD_LEMMATISED VALUES('branch','n',8456,1196)
INSERT INTO WORD_LEMMATISED VALUES('brand','n',1819,3733)
INSERT INTO WORD_LEMMATISED VALUES('brandy','n',959,5675)
INSERT INTO WORD_LEMMATISED VALUES('brass','n',1433,4416)
INSERT INTO WORD_LEMMATISED VALUES('brave','a',1760,3838)
INSERT INTO WORD_LEMMATISED VALUES('breach','n',3407,2474)
INSERT INTO WORD_LEMMATISED VALUES('bread','n',3780,2294)
INSERT INTO WORD_LEMMATISED VALUES('break','n',4271,2099)
INSERT INTO WORD_LEMMATISED VALUES('break','v',19512,532)
INSERT INTO WORD_LEMMATISED VALUES('breakdown','n',1591,4104)
INSERT INTO WORD_LEMMATISED VALUES('breakfast','n',4314,2080)
INSERT INTO WORD_LEMMATISED VALUES('breast','n',2648,2926)
INSERT INTO WORD_LEMMATISED VALUES('breath','n',5347,1768)
INSERT INTO WORD_LEMMATISED VALUES('breathe','v',3372,2492)
INSERT INTO WORD_LEMMATISED VALUES('breed','n',1745,3859)
INSERT INTO WORD_LEMMATISED VALUES('breed','v',1974,3513)
INSERT INTO WORD_LEMMATISED VALUES('breeding','n',1403,4488)
INSERT INTO WORD_LEMMATISED VALUES('breeze','n',1455,4376)
INSERT INTO WORD_LEMMATISED VALUES('brewery','n',1012,5495)
INSERT INTO WORD_LEMMATISED VALUES('brick','n',2675,2911)
INSERT INTO WORD_LEMMATISED VALUES('bride','n',1111,5201)
INSERT INTO WORD_LEMMATISED VALUES('bridge','n',5367,1761)
INSERT INTO WORD_LEMMATISED VALUES('brief','a',4977,1861)
INSERT INTO WORD_LEMMATISED VALUES('briefly','adv',3280,2531)
INSERT INTO WORD_LEMMATISED VALUES('brigade','n',1245,4839)
INSERT INTO WORD_LEMMATISED VALUES('bright','a',6190,1552)
INSERT INTO WORD_LEMMATISED VALUES('brilliant','a',3498,2432)
INSERT INTO WORD_LEMMATISED VALUES('bring','v',43894,211)
INSERT INTO WORD_LEMMATISED VALUES('broad','a',6447,1488)
INSERT INTO WORD_LEMMATISED VALUES('broadcast','n',946,5726)
INSERT INTO WORD_LEMMATISED VALUES('broadcast','v',1116,5185)
INSERT INTO WORD_LEMMATISED VALUES('broadly','adv',1602,4091)
INSERT INTO WORD_LEMMATISED VALUES('brochure','n',1176,5037)
INSERT INTO WORD_LEMMATISED VALUES('broken','a',2479,3027)
INSERT INTO WORD_LEMMATISED VALUES('broker','n',1100,5232)
INSERT INTO WORD_LEMMATISED VALUES('bronze','n',1209,4951)
INSERT INTO WORD_LEMMATISED VALUES('brother','n',11757,864)
INSERT INTO WORD_LEMMATISED VALUES('brow','n',1194,4986)
INSERT INTO WORD_LEMMATISED VALUES('brown','a',4300,2088)
INSERT INTO WORD_LEMMATISED VALUES('brush','n',1686,3941)
INSERT INTO WORD_LEMMATISED VALUES('brush','v',2115,3369)
INSERT INTO WORD_LEMMATISED VALUES('bubble','n',843,6120)
INSERT INTO WORD_LEMMATISED VALUES('bucket','n',1401,4490)
INSERT INTO WORD_LEMMATISED VALUES('budget','n',9311,1078)
INSERT INTO WORD_LEMMATISED VALUES('build','v',23931,418)
INSERT INTO WORD_LEMMATISED VALUES('builder','n',1795,3768)
INSERT INTO WORD_LEMMATISED VALUES('building','n',20770,496)
INSERT INTO WORD_LEMMATISED VALUES('bulb','n',1030,5435)
INSERT INTO WORD_LEMMATISED VALUES('bulk','n',2056,3424)
INSERT INTO WORD_LEMMATISED VALUES('bull','n',1257,4807)
INSERT INTO WORD_LEMMATISED VALUES('bullet','n',1227,4894)
INSERT INTO WORD_LEMMATISED VALUES('bulletin','n',861,6036)
INSERT INTO WORD_LEMMATISED VALUES('bump','v',817,6239)
INSERT INTO WORD_LEMMATISED VALUES('bunch','n',1344,4610)
INSERT INTO WORD_LEMMATISED VALUES('bundle','n',881,5945)
INSERT INTO WORD_LEMMATISED VALUES('burden','n',2988,2710)
INSERT INTO WORD_LEMMATISED VALUES('bureau','n',1636,4028)
INSERT INTO WORD_LEMMATISED VALUES('bureaucracy','n',1509,4256)
INSERT INTO WORD_LEMMATISED VALUES('bureaucratic','a',927,5789)
INSERT INTO WORD_LEMMATISED VALUES('burial','n',1052,5373)
INSERT INTO WORD_LEMMATISED VALUES('burn','n',904,5868)
INSERT INTO WORD_LEMMATISED VALUES('burn','v',5091,1829)
INSERT INTO WORD_LEMMATISED VALUES('burning','a',1252,4813)
INSERT INTO WORD_LEMMATISED VALUES('burst','n',936,5761)
INSERT INTO WORD_LEMMATISED VALUES('burst','v',2476,3030)
INSERT INTO WORD_LEMMATISED VALUES('bury','v',2987,2713)
INSERT INTO WORD_LEMMATISED VALUES('bus','n',6761,1428)
INSERT INTO WORD_LEMMATISED VALUES('bush','n',1933,3567)
INSERT INTO WORD_LEMMATISED VALUES('business','n',38204,244)
INSERT INTO WORD_LEMMATISED VALUES('businessman','n',1895,3621)
INSERT INTO WORD_LEMMATISED VALUES('busy','a',5221,1801)
INSERT INTO WORD_LEMMATISED VALUES('but','conj',459622,23)
INSERT INTO WORD_LEMMATISED VALUES('butter','n',2044,3439)
INSERT INTO WORD_LEMMATISED VALUES('butterfly','n',1136,5132)
INSERT INTO WORD_LEMMATISED VALUES('button','n',2503,3012)
INSERT INTO WORD_LEMMATISED VALUES('buy','v',25582,394)
INSERT INTO WORD_LEMMATISED VALUES('buyer','n',4285,2092)
INSERT INTO WORD_LEMMATISED VALUES('by','adv',2898,2760)
INSERT INTO WORD_LEMMATISED VALUES('by','prep',517171,20)
INSERT INTO WORD_LEMMATISED VALUES('bye','interjecti',1626,4042)

The following is the spring application configuration file. If you are using mySQL to connect then just comment out the HSQLDB and comment in the mysql Configuration entry.

src/main/resources/applicationContext.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
 
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
 
http://www.springframework.org/schema/context
 
http://www.springframework.org/schema/context/spring-context-2.5.xsd">


	<!-- Use the following for the HSQLDB in memory Java database (default) -->
	<bean name="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
	    <property name="driverClassName" value="org.hsqldb.jdbcDriver"/>
	    <property name="url" value="jdbc:hsqldb:file:src/main/resources/db/liveSearch"/>
	    <property name="username" value="sa"/>
	    <property name="password" value=""/>
	</bean>
 
    <!-- uncomment this to use mySQL and comment out the HSQLDB above
	<bean name="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
	    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
	    <property name="url" value="jdbc:mysql://xxxx:3306/xxxx"/>
	    <property name="username" value="xxxx"/>
	    <property name="password" value="xxx"/>
	</bean>
 	-->
	 
	<bean name="jdbcTemplate" class="org.springframework.jdbc.core.simple.SimpleJdbcTemplate">
	    <constructor-arg><ref bean="dataSource"/></constructor-arg>
	</bean>
</beans>

Homepage

The following page presents the user with an extjs combo box that makes ajax calls to the servlet to get search results.

src/main/webapp/index.jsp

<html>
<head>
    <title>ComboBox - Live Search</title>
    <link rel="stylesheet" type="text/css"
        href="http://dev.sencha.com/deploy/ext-3.3.1/resources/css/ext-all.css"/>
 
    <script type="text/javascript"
        src="http://dev.sencha.com/deploy/ext-3.3.1/adapter/ext/ext-base-debug.js">
    </script>
    <script type="text/javascript"
        src="http://dev.sencha.com/deploy/ext-3.3.1/ext-all-debug.js">
    </script>

	<script type="text/javascript">
		Ext.onReady(function(){
		 
			// This reader helps process search result data being 
			// returned after the search.
			var myReader = new Ext.data.JsonReader({
				root: 'records',
				totalProperty: 'totalCount',
				fields: [
				        {name: 'word_x', type: 'string'}
				    ]
			});
		 
			// This represents the URL that returns the search results.
			var myStore = new Ext.data.Store({
				// url : 'response.do' // url: uses POST by default
				proxy: new Ext.data.HttpProxy({  // force it to use GET method
					url:    '/app/searchServlet',
					method: 'GET'
				}),
			    reader : myReader
			});
			// Custom rendering Template
			var resultTpl = new Ext.XTemplate(
				'<tpl for="."><div class="search-item">',
				    '<h3>{word_x}</h3>',
				'</div></tpl>'
			);

			var search = new Ext.form.ComboBox({			
				store: myStore,
				displayField:'title',
				typeAhead: false,
				loadingText: 'Searching...',
				width: 570,
				minChars: 2,
				pageSize:5,
				hideTrigger:true,
				tpl: resultTpl,
				applyTo: 'search',
				itemSelector: 'div.search-item',
				onSelect: function(record){ 
					alert(record.data.word_x + ' was selected...');
					// override default onSelect to do redirect
//				    window.location =
//				        String.format('http://extjs.com/forum/showthread.php?t={0}&p={1}', 
//				        	record.data.topicId, record.id);
				}
			});
		});	
	</script>

    <style type="text/css">
        p { width:650px; }
		.search-item {
			font:normal 11px tahoma, arial, helvetica, sans-serif;
			padding:3px 10px 3px 10px;
			border:1px solid #fff;
			border-bottom:1px solid #eeeeee;
			white-space:normal;
			color:#555;
		}
		.search-item h3 {
			display:block;
			font:inherit;
			font-weight:bold;
			color:#222;
		}

		.search-item h3 span {
			float: right;
			font-weight:normal;
			margin:0 0 5px 5px;
			width:100px;
			display:block;
			clear:none;
		}
    </style>     
</head>
 
<body>
<p>
    <b>Combo with Templates and Ajax</b><br />
	
	Type any word that begins with the letters "a" or "b". for example "ab", "be" ...
	
</p>

<!-- The box wrap markup embedded instead of using Element.boxWrap() -->
<div style="width:600px;">
    <div class="x-box-tl"><div class="x-box-tr"><div class="x-box-tc"></div></div></div>
    <div class="x-box-ml"><div class="x-box-mr"><div class="x-box-mc">
        <h3 style="margin-bottom:5px;">Search the Ext Forums</h3>
        <input type="text" size="40" name="search" id="search" />
        <div style="padding-top:4px;">
            Live search requires a minimum of 2 characters.
        </div>
 
    </div></div></div>
    <div class="x-box-bl"><div class="x-box-br"><div class="x-box-bc"></div></div></div>
</div>

</body>
</html>

Test the application

To test the application we will start the jetty servlet engine.

mvn clean compile jetty:run

Navigate to the page and start typing. Search results should start to show.

http://localhost:8080/