07
Oct
11

Restartable Jobs With Spring Batch


This page demonstrates how spring batch handles job failure and how to recover from them.

Background

Please review The Domain Language of Spring Batch prior to reading this further.

StepExecution

In simple terms each “job” is broken up into individual “steps”. The process of executing a step is called a “StepExecution”. Each execution of a step can either result in Success or Failure. If individual steps of a larger job fail it would be nice to know where the step left off. The Designers of this framework recognized this and came up with a way to allow steps to resume. The JobRepository allows Readers and Writers to store status, start and end times, exit codes, and count information for each Execution of a step within a Job. This information is stored using the ExecutionContext at each commit interval.

ExecutionContext

The “ExecutionContext” is a collection of name/value pairs that are persisted to the database. Each instance that a Step is executed by the spring batch framework has its own ExecutionContext. Many of the default Reader/Writer implementations publish information to the ExecutionContext by default.

Outline of this page

Here is a breakdown of what will be demonstrated:

  1. Simulate a job failure by modifying the EmployeeProcessor.java and throw an exception towards the middle of the file.
  2. Examine the output file and verify that all prior records were written out.
  3. Examine the JobRepository database to verify the ExecutionContext record contains the recovery information.
  4. Modify EmployeeProcessor.java to not throw the test exception.
  5. Examine the output file and verify that all the remaining records were written out.
  6. Note that the Footer of the file is incorrect.
  7. Fix the EmployeeItemWriter.java to read the ExecutionContext and output the correct footer.

This page picks up where the last one left of. So if you have not done so already please implement the example application described here.

Requirements

  • Java 5 or above and Maven 2
  • Spring Batch 2.x
  • Successful completion of xxx

Inserting An Error

The below highlighted code stops the process when the record for the “president” is processed.

package com.test;

import org.springframework.batch.item.ItemProcessor;

public class EmployeeProcessor implements ItemProcessor<Employee, Employee> {

	public Employee process(Employee emp) throws Exception {
		if("PRESIDENT".equals(emp.getTitle())) throw new Exception("test exception: found President");
		// if salary >= 2500 then set rank as "Director"		
		if(emp.getSalary() >= 2500 ) {
			emp.setRank("Director");
		} else {
			emp.setRank("N/A");
		}
		return emp;
	}

}

Run the command:

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Open the target/output_data.txt file and observe that only 7 records were processed.

Comment out line 8 from EmployeeProcessor.java above and re-run the batch.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Open target/output_data.txt one more time and observe that the remaining records were processed however note that the footer information is incorrect. It should say 14 records instead of 7.

Making Spring Batch Restartable

If you remember in the previous article we modified the EmployeeItemWriter class to print out a header and footer records. The footer record was generated using state variables (totalAmount, and recordCount). Since the EmployeeItemWriter maintains state, when the batch job restarts the state variables are re-initialized to their original values. Since the second invocation of the job starts from the middle of the input file the writer does not get an opportunity to properly initialize the state variables.

To fix this problem we will use the executionContext. The executionContext gets updated on each commit interval. It is the place where Spring batch keeps information that is important to the currently running job Execution.

We will modify the open() and update() methods of the EmployeeItemWriter to lookup the values from the ExecutionContext.

/src/main/java/com/test/EmployeeItemWriter.java

	public void open(ExecutionContext executionContext) throws ItemStreamException {
		if(executionContext.containsKey("current.recordCount")) {
			recordCount = new Long(executionContext.getLong("current.recordCount")).intValue();
		} else {
			recordCount = 0;
		}		
		
		if(executionContext.containsKey("current.totalAmount")) {
			totalAmount = new BigDecimal(executionContext.getDouble("current.totalAmount"));
		} else {
			totalAmount = BigDecimal.ZERO;
		}		
		this.delegate.open(executionContext);
	}

	public void update(ExecutionContext executionContext) throws ItemStreamException {
		executionContext.putLong("current.recordCount", recordCount);
		executionContext.putDouble("current.totalAmount", totalAmount.doubleValue());
		
		this.delegate.update(executionContext);
	}

We don’t need to modify “empReader” bean since it is backed by “FlatFileItemReader” and it’s implementation is restartable by default.

Clear the database

In order to start fresh again, Delete all files under src/main/resources/db. This is where HSQLDB keeps its database files. Once you delete this the DDLUtils framework will regenerate the database with all the schema information.

Re-run the job

Comment IN the EmployeeProcessor.java so that when “PRESIDENT” is encountered it throws an exception and re-run the job.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Comment OUT the line in EmployeeProcessor.java and re-run the job again.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

Verify the output

Open target/output_data.txt one more time and observe that the remaining records were processed and the count should be 14 records.

That’s all for now.

Advertisements

1 Response to “Restartable Jobs With Spring Batch”



Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 74 other followers

October 2011
S M T W T F S
« Jul   Jan »
 1
2345678
9101112131415
16171819202122
23242526272829
3031  

Blog Stats

  • 801,397 hits

%d bloggers like this: