01
Oct
11

Spring Batch Headers and Footers


This page describes the process of creating running a Spring Batch job that creates an output file with header and footer information.

Background

In prior articles we created a batch job that read information and outputted the result to another file. The output file was missing a header and footer lines. In this article we will look into generating this information.

Requirements

  • Java 5
  • Maven 2
  • This page picks up where the following page left off. Please Review and Implement the following article.

Header

The header of the will simply contain the filename and the timestamp.

fileName,Timestamp

Footer

The footer contained in the output be composed of 2 fields.

#ofRecords,salarySum

where:
#ofRecords – Count of the number of records in the output
salarySum – Sum of all values in the salary field

Solution

Create a Item Writer that will wrap the “FlatFileItemWriter”. The “headerCallback” and “footerCallback” methods of the ItemWriter will be called at the beginning and ending of the file. This allows developers to implement code that will allow them to write information to the header and footer of the output file.

src/main/java/com/test/EmployeeItemWriter.java

package com.test;

import java.io.IOException;
import java.io.Writer;
import java.math.BigDecimal;
import java.util.Date;
import java.util.List;

import org.springframework.batch.item.ExecutionContext;
import org.springframework.batch.item.ItemStream;
import org.springframework.batch.item.ItemStreamException;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.FlatFileFooterCallback;
import org.springframework.batch.item.file.FlatFileHeaderCallback;
import org.springframework.batch.item.file.FlatFileItemWriter;

public class EmployeeItemWriter implements ItemWriter<Employee>,
		FlatFileFooterCallback, FlatFileHeaderCallback, ItemStream {

    private FlatFileItemWriter<Employee> delegate;

    private BigDecimal totalAmount = BigDecimal.ZERO;

	private int recordCount = 0;
    
    public void writeFooter(Writer writer) throws IOException {
        writer.write(""+recordCount + "," + totalAmount);
    }
	public void writeHeader(Writer writer) throws IOException {
        writer.write("output_file.txt" + "," + new Date());
	}
    public void write(List<? extends Employee> items) throws Exception {
        BigDecimal chunkTotal = BigDecimal.ZERO;
        int chunkRecord = 0;
        for (Employee employee : items) {
            chunkRecord++;
            chunkTotal = chunkTotal.add(new BigDecimal(employee.getSalary()));
        }
        delegate.write(items);
        // After successfully writing all items
        totalAmount = totalAmount.add(chunkTotal);
        recordCount += chunkRecord;
	}
    
    public void setDelegate(FlatFileItemWriter<Employee> delegate) {
		this.delegate = delegate;
	}

	public void close() throws ItemStreamException {
		this.delegate.close();
	}

	public void open(ExecutionContext arg0) throws ItemStreamException {
		this.delegate.open(arg0);
	}

	public void update(ExecutionContext arg0) throws ItemStreamException {
		this.delegate.update(arg0);
	}
}

Batch XML Configuration

The following xml will allow the ItemWriter to wrap the FlatFileItemWriter and maintain a running total.

The highlighted lines are the changes from the prior version.

/src/main/resources/simpleJob.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
     xmlns:beans="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="
 
http://www.springframework.org/schema/batch
 
http://www.springframework.org/schema/batch/spring-batch-2.1.xsd

http://www.springframework.org/schema/beans
 
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
 
    <beans:import resource="applicationContext.xml"/>
 
<!-- Tokenizer - Converts a delimited string into a Set of Fields -->
<beans:bean name="defaultTokenizer" 
    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"/>

<!-- FieldSetMapper - Populates a bean's attributes with using the FieldSet -->
<beans:bean name="employeeFieldSetMapper" class="com.test.EmployeeFieldSetMapper"/>

<!-- LineMapper - Uses the tokenizer and Mapper to create instances of a Bean. -->
<beans:bean name="employeeLineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
    <beans:property name="lineTokenizer" ref="defaultTokenizer"/>    
    <beans:property name="fieldSetMapper" ref="employeeFieldSetMapper"/>        
</beans:bean>

<!-- Reader - used by the tasklet to process one Item from the input. -->
<beans:bean name="empReader" class="org.springframework.batch.item.file.FlatFileItemReader">
    <beans:property name="lineMapper" ref="employeeLineMapper"/>
    
    <!-- use spring integrations for the following, but for now filename is hard coded -->
    <beans:property name="resource" value="input_data.txt"/>    
</beans:bean>

<!-- Processor -->

<beans:bean name="empProcessor" class="com.test.EmployeeProcessor">
</beans:bean>

<!-- Writer -->
<beans:bean id="empWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
    <beans:property name="resource" value="file:target/output_data.txt" />
    <beans:property name="lineAggregator">
        <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
            <beans:property name="delimiter" value=","/>
            <beans:property name="fieldExtractor">
                <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                    <beans:property name="names" value="empId,lastName,title,salary,rank"/>
                </beans:bean>
            </beans:property>
        </beans:bean>
    </beans:property>
    <beans:property name="footerCallback" ref="empHeaderFooterWriter" />
    <beans:property name="headerCallback" ref="empHeaderFooterWriter" />
</beans:bean>

<beans:bean id="empHeaderFooterWriter" class="com.test.EmployeeItemWriter">
    <beans:property name="delegate" ref="empWriter"/>
</beans:bean>

<job id="helloWorldJob">
    <step id="step1" next="step2">
        <tasklet ref="helloWorldTasklet" />
    </step>
    <step id="step2">
        <tasklet>
            <chunk reader="empReader" processor="empProcessor" writer="empHeaderFooterWriter" commit-interval="1"/>
        </tasklet>
    </step>
</job>
 
<beans:bean name="helloWorldTasklet" class="com.test.HelloWorldTasklet"/>
 
<!--
To run the job from the command line type the following:
mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"
 -->
</beans:beans>

Execute the job

Go to the command line and type the following:

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"

View the results

The output file will appear in the target/ folder of the project.

The file should look like this:

output_file.txt,Sat Oct 01 22:50:17 EDT 2011
7876,ADAMS,CLERK,1100,N/A
7499,ALLEN,SALESMAN,1600,N/A
7698,BLAKE,MANAGER,2850,Director
7782,CLARK,MANAGER,2450,N/A
7902,FORD,ANALYST,3000,Director
7900,JAMES,CLERK,950,N/A
7566,JONES,MANAGER,2975,Director
7839,KING,PRESIDENT,5000,Director
7654,MARTIN,SALESMAN,1250,N/A
7934,MILLER,CLERK,1300,N/A
7788,SCOTT,ANALYST,3000,Director
7369,SMITH,CLERK,800,N/A
7844,TURNER,SALESMAN,1500,N/A
7521,WARD,SALESMAN,1250,N/A
14,29025

That’s all for now!

Advertisements

4 Responses to “Spring Batch Headers and Footers”


  1. 1 msns3ka
    October 14, 2012 at 3:33 pm

    Awesome Blog! More elegant way of managing Header/Footer data!

  2. February 17, 2017 at 1:42 am

    I tried Implementing the above for header and footer. But I’m getting the below error.

    java.lang.IllegalStateException: Cannot convert value of type [com.sun.proxy.$Proxy50 implementing org.springframework.batch.item.file.ResourceAwareItemWriterItemStream,org.springframework.beans.factory.InitializingBean,org.springframework.batch.item.ItemWriter,org.springframework.batch.item.ItemStream,org.springframework.aop.scope.ScopedObject,java.io.Serializable,org.springframework.aop.framework.AopInfrastructureBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised] to required type [org.springframework.batch.item.file.FlatFileItemWriter] for property ‘delegate’: no matching editors or conversion strategy found


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 77 other followers

October 2011
S M T W T F S
« Jul   Jan »
 1
2345678
9101112131415
16171819202122
23242526272829
3031  

Blog Stats

  • 846,580 hits

%d bloggers like this: