In this article we will see an second example Spring batch application. To create sample Spring Batch application, we will make use of google hot searches feed. Google provides hot trends app where you can choose the country to find the hot searches happening.
The google hot searches are available as atom feed(http://www.google.com/trends/hottrends/atom/feed?pn=p3). To use Spring Batch framework, we should add the below dependencies to pom.xml.
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>3.0.3.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>4.1.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
<version>1.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>4.1.4.RELEASE</version>
</dependency>
We have used feed4j to parse the atom feed. Download the feed4j jars and keep the dependencies in pom.xml.
<dependency>
<groupId>feed4j</groupId>
<artifactId>feed4j</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>C:/Users/test/GT_WS/Google_Trends/lib/feed4j.jar</systemPath>
</dependency>
<dependency>
<groupId>nekohtml</groupId>
<artifactId>nekohtml</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>C:/Users/test/GT_WS/Google_Trends/lib/nekohtml.jar</systemPath>
</dependency>
<dependency>
<groupId>xercesImpl</groupId>
<artifactId>xercesImpl</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xercesImpl.jar</systemPath>
</dependency>
<dependency>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xml-apis.jar</systemPath>
</dependency>
Now, we are ready to implement Item Reader to get the hottrends feed and parse it.
package org.smarttechie.reader;
import it.sauronsoftware.feed4j.FeedParser;
import it.sauronsoftware.feed4j.bean.Feed;
import it.sauronsoftware.feed4j.bean.FeedItem;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;
/**
* The reader gets the atom feed and parses it.
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsReader implements ItemReader<List<GoogleTrendItem>>{
private Log log = LogFactory.getLog(GoogleTrendsReader.class);
/**
* The reader will make call to google hot trends API
*/
@Override
public List<GoogleTrendItem> read() throws Exception, UnexpectedInputException,
ParseException, NonTransientResourceException {
URL url = new URL("http://www.google.com/trends/hottrends/atom/feed?pn=p3");
Feed feed = FeedParser.parse(url);
int items = feed.getItemCount();
log.debug("The number of hot trends " + items);
List<GoogleTrendItem> trends = null;
if (items > 0) {
trends = new ArrayList<GoogleTrendItem>();
for (int i = 0; i < items; i++) {
FeedItem item = feed.getItem(i);
GoogleTrendItem trend = new GoogleTrendItem();
trend.setTitle(item.getTitle());
trend.setLink(item.getLink().toString());
trend.setDescription(item.getDescriptionAsText());
trends.add(trend);
}
}
return trends;
}
}
The Item Writer implementation is given below.
package org.smarttechie.writer;
import java.util.List;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemWriter;
/**
* The writer to write the list of google trends
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsWriter implements ItemWriter<List<GoogleTrendItem>>{
private Log log = LogFactory.getLog(GoogleTrendsWriter.class);
@Override
public void write(List<? extends List<GoogleTrendItem>> trends)
throws Exception {
if (trends != null && !trends.isEmpty()) {
List<GoogleTrendItem> items = trends.get(0);
if (items != null && !items.isEmpty()) {
for (GoogleTrendItem trend : items) {
log.debug("Title: " + trend.getTitle());
log.debug("Link: " + trend.getLink());
log.debug("Plain text description: " + trend.getDescription());
}
}
}
}
}
The sequence diagram of the above components interaction is given below.
The batch configuration is given below.
package org.smarttechie;
import java.util.List;
import org.smarttechie.model.GoogleTrendItem;
import org.smarttechie.reader.GoogleTrendsReader;
import org.smarttechie.writer.GoogleTrendsWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableBatchProcessing
public class GoogleTrendsBatchConfiguration {
@Bean
public ItemReader<List<GoogleTrendItem>> reader() {
GoogleTrendsReader reader = new GoogleTrendsReader();
return reader;
}
@Bean
public ItemWriter<List<GoogleTrendItem>> writer() {
GoogleTrendsWriter writer = new GoogleTrendsWriter();
return writer;
}
@Bean
public Step step1(StepBuilderFactory stepBuilderFactory, ItemReader<List<GoogleTrendItem>> reader,
ItemWriter<List<GoogleTrendItem>> writer) {
return stepBuilderFactory.get("step1")
.<List<GoogleTrendItem>, List<GoogleTrendItem>> chunk(1)
.reader(reader)
.writer(writer)
.build();
}
@Bean
public Job importUserJob(JobBuilderFactory jobs, Step s1) {
return jobs.get("importHotSeachesJob").preventRestart()
.start(s1)
.build();
}
}
The above implementation doesn’t have ItemProcessor which is optional. The source code created as part of this article is available here.
Siva Janapati is an Architect with experience in building Cloud Native Microservices architectures, Reactive Systems, Large scale distributed systems, and Serverless Systems. Siva has hands-on in architecture, design, and implementation of scalable systems using Cloud, Java, Go lang, Apache Kafka, Apache Solr, Spring, Spring Boot, Lightbend reactive tech stack, APIGEE edge & on-premise and other open-source, proprietary technologies. Expertise working with and building RESTful, GraphQL APIs. He has successfully delivered multiple applications in retail, telco, and financial services domains. He manages the GitHub(https://github.com/2013techsmarts) where he put the source code of his work related to his blog posts.
Smart Techie 
Leave a Reply