07
Feb
10

Multiple Batch Runs with Spring Batch


This page describes the process of setting up your spring batch programs to run multiple times without specifying a new parameter. Sometimes specifying a new unique parameter is not a big deal. In that case that would be the preferred approach.

If you are getting the following errors this page will definitely help:

  • A job instance already exists and is complete for parameters
  • JobInstanceAlreadyCompleteException

The reason why these errors occur is that Spring Batch ensures against accidental job executions. As you can imagine something like that could be detrimental to the health of your business or your reputation.

example: customer gets billed twice etc…

Spring batch looks for differences in parameters that are used to execute your job:

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob run.id=1"

In the above code you can see that run.id parameter has been entered. Spring Batch will check the JobRepository to see if this job parameter has run before. If it has it will throw the errors you saw in the bulleted list above.

Requirements

Generating a run ID dynamically

As with many systems that run automated, generating a new ID in each batch run could be a problem. There is a method to have spring generate a new run id without changing the command to execute the job. Please keep in mind that by doing this your program will need to take the responsibility of detecting duplicate runs.

Make the following simple changes

Define the job Explorer. This object is like a read only job repository. It is used by various parts of the spring batch to read the status of executed jobs.

	<beans:bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
		<beans:property name="dataSource" ref="dataSource"/>
	</beans:bean>

This is a an incrementor that is used to bump up the run.id parameter.

package com.test;

import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.JobParametersIncrementer;

public class SampleIncrementer implements JobParametersIncrementer {
    public JobParameters getNext(JobParameters parameters) {
    	System.out.println("got job parameters: " + parameters);
        if (parameters==null || parameters.isEmpty()) {
            return new JobParametersBuilder().addLong("run.id", 1L).toJobParameters();
        }
        long id = parameters.getLong("run.id",1L) + 1;
        return new JobParametersBuilder().addLong("run.id", id).toJobParameters();
    }
}

Add the following to your spring Configuration file…

<beans:bean name="sampleIncrementer" class="com.test.SampleIncrementer"/>

Update your job’s definition to include an incrementer.

<job id="helloWorldJob" incrementer="sampleIncrementer">
...
</job>

Run Your Job With “-next” Parameter

Last but not least you need to provide a -next parameter to the job. This allows the CommandLineJobRunner to execute your Incrementer.

Run your job multiple times without the worry of changing job parameters.

mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob -next"

References

Advertisements

8 Responses to “Multiple Batch Runs with Spring Batch”


  1. 1 Dag
    March 5, 2010 at 3:37 am

    Nice approach, but I found it much more convenient to simply add a new unique parameter to the batch, containing e.g. the current time.

    • March 5, 2010 at 9:25 am

      Thanks for your comment!

      You are right. Adding a date would definitely work!

      Doing it the way described on this page allows the programmer to implement a check for duplicate runs in the “SampleIncrementer” class described above. If it is in fact a duplicate run you can stop the execution right there. The logic of checking the duplicate run would nicely be encapsulated in that class.

      • 3 Connor
        January 18, 2013 at 11:37 pm

        Can you describe briefly the logic you’d write to check for duplicate runs from within the Incrementor? Also if my job consists solely of a single step with a tasklet, do I need to do anything special (like pass “-next”) to the job or will it activate the incrementor automatically during each job request?

        I’m trying to basically emulate the flow of a synchronized block of code – allow for as many execution requests for my tasklet as desired, but prevent concurrent runs. I need to rely on a DB lock of some sort because I may have multiple JVMs running spring batch.

  2. 4 Kamath_sv
    April 7, 2011 at 12:29 am

    Thanks this was really very helpful for me as i had similar requirement.
    I used the following command.
    mvn exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args=”META-INF/applicationContext.xml job jobName=generateFile -next”
    and it worked like a charm

    I have one more requirement where I need to use an executable Jar.
    I want to run the job using CommandLineJobRunner by taking the required files from that executablejar not from target folder of project(in production system i can copy only executable jar not source with target folder)

    say i have placed the jar scheduler-1.0.one-jar.jar & batch file scheduler.bat(containing command script to run the job using CommandLineJobRunner) in folder C:\a\
    i must be able to run the batch file which takes required classfiles, classpath from that executable jar

    Can you please help me out in this ?

  3. 6 max
    June 15, 2012 at 3:09 pm

    Hi, i ‘ m trying to use spring batch, but i dont have an unique data source, is there a form to resolve it?
    Thanks


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 75 other followers

February 2010
S M T W T F S
« Jan   Mar »
 123456
78910111213
14151617181920
21222324252627
28  

Blog Stats

  • 813,774 hits

%d bloggers like this: