How to use Spring Batch Restartable Function

SpringBatch provides a restartable mechanics with a Job, in the How to use Spring Batch Restartable Function tutorial, JavaSampleApproach will present to you the principle of this function with SpringBoot.

Related Post:
1. How to configure Spring Batch Step for restart
2. How to start with Spring Batch using Spring Boot – Java Config
3. Spring Batch XML Config by Spring Boot
4. Spring Batch Job with Parallel Steps
5. How to import CSV data to PostgreSQL Database using Spring Batch Job


I. Technologies

– Java 1.8
– Maven 3.3.9
– Spring Boot: 1.4.3.RELEASE
– Spring Tool Suite – Version 3.8.1.RELEASE

II. Overview

Create a Batch Job which has a simple step with a Reader & Writer.
Reader reads a list file-paths then pass them to a Writer. The Write uses file paths to process content of each file.
With created JOB, test a restartable function base on FileNotFound Error.
Note: a Batch Job is created with restartable default. Spring Batch also supports configuration to prevent the restartable function.

1. Project Structure

spring boot springbatch restartable project structure

2. Step to do

– Create SpringBoot project
– Create simple step
– Config a Batch Job
– Create a Job Launch controller
– Config Batch Job Database
– Enable Batch Job
– Scenario Test

III. Spring Batch Restartable Practices
1. Create a SpringBoot project

Open Spring Tool Suite, on main menu, choose File->New->Spring Starter Project, input project info. Press Next then Finish, a Spring Boot project will be created successfully.

– Open pom.xml file, add Web MVC, PostgreSQL and Spring Batch dependencies:


	org.springframework.boot
	spring-boot-starter-web



	org.springframework.boot
	spring-boot-starter-batch




	postgresql
	postgresql
	9.1-901-1.jdbc4


2. Create classes Reader & Writer for simple step

Create a Reader class:

package com.javasampleapproach.springbatch.step;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;
 
public class Reader implements ItemReader{
 
    private String[] files = {"C:\\readfile\\1.txt"};
     
    public static int  count=0;
     
    Logger logger = LoggerFactory.getLogger(this.getClass());
     
    @Override
    public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
         
        if(count < files.length){
            return files[count++];
        }else{
            count=0;
        }
        return null;
    }
}	

Create a Writer class:

package com.javasampleapproach.springbatch.step;
 
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.List;
import java.util.stream.Stream;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemWriter;
 
public class Writer implements ItemWriter {
 
    Logger logger = LoggerFactory.getLogger(this.getClass());
     
    @Override
    public void write(List paths) throws Exception {
        for(String filePath : paths){
        	System.out.println("filePath = " + filePath);
            try (Stream stream = Files.lines(Paths.get(filePath))) {
                stream.forEach(System.out::println);
            } catch (IOException e) {
            	Reader.count = 0;
            	throw(e);
            }
        }
    }
    
}
3. Config a Batch Job

Config a Simple Batch Job:

package com.javasampleapproach.springbatch.config;
 
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.javasampleapproach.springbatch.step.Reader;
import com.javasampleapproach.springbatch.step.Writer;
 
@Configuration
public class BatchConfig {
 
    @Autowired
    public JobBuilderFactory jobBuilderFactory;
 
    @Autowired
    public StepBuilderFactory stepBuilderFactory;
 
     
    @Bean
    public Job job() {
        return jobBuilderFactory.get("job")
                .incrementer(new RunIdIncrementer())
                .flow(step1())
                .end()
                .build();
    }
 
    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                . chunk(1)
                .reader(new Reader())
                .writer(new Writer())
                .build();
    }

}
4. Create a Job Launch controller

- Create a simple Controller for Job Launching.

package com.javasampleapproach.springbatch.controller;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
 
@RestController
public class JobLauncherController {
 
    @Autowired
    JobLauncher jobLauncher;
 
    @Autowired
    Job job;
 
    @RequestMapping("/launchjob")
    public String handle() throws Exception {
 
        Logger logger = LoggerFactory.getLogger(this.getClass());
        JobExecution jobExecution = null;
        try {
        	jobExecution = jobLauncher.run(job, new JobParameters());
        } catch (Exception e) {
            logger.info(e.getMessage());
        }
 
        return "jobExecution's info: Id = " + jobExecution.getId() + " ,status = " + jobExecution.getExitStatus();
    }
}
5. Config Batch Job Database

Config DataBase's info for Spring Batch:

spring.datasource.url=jdbc:postgresql://localhost/testdb
spring.datasource.username=postgres
spring.datasource.password=123
spring.batch.job.enabled=false
6. Enable Batch Job

In main class of Spring Boot project, use @EnableBatchProcessing to enable Spring Batch.

package com.javasampleapproach.springbatch;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
@EnableBatchProcessing
public class SpringBatchRestartableFunctionApplication {

	public static void main(String[] args) {
		SpringApplication.run(SpringBatchRestartableFunctionApplication.class, args);
	}
}
7. Scenario Test

- Build & Run the Spring Boot Batch Job.
The Reader will forward a file name: C:\\readfile\\1.txt to Writer. Delete a file C:\\readfile\\1.txt, then makes a launch request: http://localhost:8080/launchjob
A Job Instance is created with Empty Parameters:

Job: [FlowJob: [name=job]] launched with the following parameters: [{}]

Investigate Server Logs:
Result FAIL:

Job: [FlowJob: [name=job]] completed with the following parameters: [{}] and the following status: [FAILED]

Cause:

2017-01-13 20:21:29.216  INFO 11012 --- [nio-8080-exec-1] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
filePath = C:\readfile\1.txt
2017-01-13 20:21:39.781 ERROR 11012 --- [nio-8080-exec-1] o.s.batch.core.step.AbstractStep         : Encountered an error executing step step1 in job job

java.nio.file.NoSuchFileException: C:\readfile\1.txt
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79) ~[na:1.8.0_101]
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.8.0_101]
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) ~[na:1.8.0_101]
	at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230) ~[na:1.8.0_101]

- Now create a C:\readfile\1.txt with content:

line1
line2
line3
line4
line5

Then restart the Job Instance (with parameters: [{}]) again by http://localhost:8080/launchjob

Result: COMPLETED

Job: [FlowJob: [name=job]] completed with the following parameters: [{}] and the following status: [COMPLETED]
filePath = C:\readfile\1.txt
line1
line2
line3
line4
line5

Make the launching request again: http://localhost:8080/launchjob, result:

Step already complete or not restartable, so no action to execute: StepExecution: id=1000, version=4, name=step1, status=COMPLETED, exitStatus=COMPLETED, readCount=1, filterCount=0, writeCount=1 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=2, rollbackCount=0, exitDescription=
Job: [FlowJob: [name=job]] completed with the following parameters: [{}] and the following status: [COMPLETED]
IV. Prevent Restartable

SpringBatch also supports a config to prevent restartable ability of a Job even if a Job Instance's Execution with FAIL status.

How to configure? Just use a little code strong>.preventRestart(), details with BatchConfig class:

package com.javasampleapproach.springbatch.config;
 
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.javasampleapproach.springbatch.step.Reader;
import com.javasampleapproach.springbatch.step.Writer;
 
@Configuration
public class BatchConfig {
 
    @Autowired
    public JobBuilderFactory jobBuilderFactory;
 
    @Autowired
    public StepBuilderFactory stepBuilderFactory;
 
     
    @Bean
    public Job job() {
        return jobBuilderFactory.get("job")
        		.preventRestart()
                .incrementer(new RunIdIncrementer())
                .flow(step1())
                .end()
                .build();
    }
 
    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                . chunk(1)
                .reader(new Reader())
                .writer(new Writer())
                .build();
    }

}

Build & Run Spring Boot project again, delete C:\\readfile\\1.txt file.
Make a request: http://localhost:8080/launchjob, -> Result: FAIL with Log details:

2017-01-13 20:35:08.664  INFO 5092 --- [nio-8080-exec-1] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
filePath = C:\readfile\1.txt
2017-01-13 20:35:21.296 ERROR 5092 --- [nio-8080-exec-1] o.s.batch.core.step.AbstractStep         : Encountered an error executing step step1 in job job

java.nio.file.NoSuchFileException: C:\readfile\1.txt
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79) ~[na:1.8.0_101]
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.8.0_101]
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) ~[na:1.8.0_101]
	at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230) ~[na:1.8.0_101]

Create a file: C:\\readfile\\1.txt, then makes request http://localhost:8080/launchjob again, result FAIL:

2017-01-13 20:37:31.672  INFO 5092 --- [nio-8080-exec-3] c.j.s.controller.JobLauncherController   : JobInstance already exists and is not restartable
2017-01-13 20:37:31.693 ERROR 5092 --- [nio-8080-exec-3] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause

java.lang.NullPointerException: null
	at com.javasampleapproach.springbatch.controller.JobLauncherController.handle(JobLauncherController.java:33) ~[classes/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_101]
V. Sourcecode

SpringBatchRestartableFunction



By grokonez | January 13, 2017.

Last updated on June 4, 2017.



Related Posts


1 thought on “How to use Spring Batch Restartable Function”

  1. As a Java developer, I’m used to use Spring Batch for batch processing, generally using a streaming library to export large XML files with StAX for exemple.

Got Something To Say:

Your email address will not be published. Required fields are marked *

*