In this article we will see an second example Spring batch application. To create sample Spring Batch application, we will make use of google hot searches feed. Google provides hot trends app where you can choose the country to find the hot searches happening.
The google hot searches are available as atom feed(http://www.google.com/trends/hottrends/atom/feed?pn=p3). To use Spring Batch framework, we should add the below dependencies to pom.xml.
<dependency> <groupId>org.springframework.batch</groupId> <artifactId>spring-batch-core</artifactId> <version>3.0.3.RELEASE</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-beans</artifactId> <version>4.1.4.RELEASE</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> <version>1.2.2.RELEASE</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>4.1.4.RELEASE</version> </dependency>
We have used feed4j to parse the atom feed. Download the feed4j jars and keep the dependencies in pom.xml.
<dependency> <groupId>feed4j</groupId> <artifactId>feed4j</artifactId> <version>1.0</version> <scope>system</scope> <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/feed4j.jar</systemPath> </dependency> <dependency> <groupId>nekohtml</groupId> <artifactId>nekohtml</artifactId> <version>1.0</version> <scope>system</scope> <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/nekohtml.jar</systemPath> </dependency> <dependency> <groupId>xercesImpl</groupId> <artifactId>xercesImpl</artifactId> <version>1.0</version> <scope>system</scope> <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xercesImpl.jar</systemPath> </dependency> <dependency> <groupId>xml-apis</groupId> <artifactId>xml-apis</artifactId> <version>1.0</version> <scope>system</scope> <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xml-apis.jar</systemPath> </dependency>
Now, we are ready to implement Item Reader to get the hottrends feed and parse it.
package org.smarttechie.reader; import it.sauronsoftware.feed4j.FeedParser; import it.sauronsoftware.feed4j.bean.Feed; import it.sauronsoftware.feed4j.bean.FeedItem; import java.net.URL; import java.util.ArrayList; import java.util.List; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.smarttechie.model.GoogleTrendItem; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.NonTransientResourceException; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; /** * The reader gets the atom feed and parses it. * @author Siva Janapati * @version 1.0 * */ public class GoogleTrendsReader implements ItemReader<List<GoogleTrendItem>>{ private Log log = LogFactory.getLog(GoogleTrendsReader.class); /** * The reader will make call to google hot trends API */ @Override public List<GoogleTrendItem> read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException { URL url = new URL("http://www.google.com/trends/hottrends/atom/feed?pn=p3"); Feed feed = FeedParser.parse(url); int items = feed.getItemCount(); log.debug("The number of hot trends " + items); List<GoogleTrendItem> trends = null; if (items > 0) { trends = new ArrayList<GoogleTrendItem>(); for (int i = 0; i < items; i++) { FeedItem item = feed.getItem(i); GoogleTrendItem trend = new GoogleTrendItem(); trend.setTitle(item.getTitle()); trend.setLink(item.getLink().toString()); trend.setDescription(item.getDescriptionAsText()); trends.add(trend); } } return trends; } }
The Item Writer implementation is given below.
package org.smarttechie.writer; import java.util.List; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.smarttechie.model.GoogleTrendItem; import org.springframework.batch.item.ItemWriter; /** * The writer to write the list of google trends * @author Siva Janapati * @version 1.0 * */ public class GoogleTrendsWriter implements ItemWriter<List<GoogleTrendItem>>{ private Log log = LogFactory.getLog(GoogleTrendsWriter.class); @Override public void write(List<? extends List<GoogleTrendItem>> trends) throws Exception { if (trends != null && !trends.isEmpty()) { List<GoogleTrendItem> items = trends.get(0); if (items != null && !items.isEmpty()) { for (GoogleTrendItem trend : items) { log.debug("Title: " + trend.getTitle()); log.debug("Link: " + trend.getLink()); log.debug("Plain text description: " + trend.getDescription()); } } } } }
The sequence diagram of the above components interaction is given below.
The batch configuration is given below.
package org.smarttechie; import java.util.List; import org.smarttechie.model.GoogleTrendItem; import org.smarttechie.reader.GoogleTrendsReader; import org.smarttechie.writer.GoogleTrendsWriter; import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.ItemWriter; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration @EnableBatchProcessing public class GoogleTrendsBatchConfiguration { @Bean public ItemReader<List<GoogleTrendItem>> reader() { GoogleTrendsReader reader = new GoogleTrendsReader(); return reader; } @Bean public ItemWriter<List<GoogleTrendItem>> writer() { GoogleTrendsWriter writer = new GoogleTrendsWriter(); return writer; } @Bean public Step step1(StepBuilderFactory stepBuilderFactory, ItemReader<List<GoogleTrendItem>> reader, ItemWriter<List<GoogleTrendItem>> writer) { return stepBuilderFactory.get("step1") .<List<GoogleTrendItem>, List<GoogleTrendItem>> chunk(1) .reader(reader) .writer(writer) .build(); } @Bean public Job importUserJob(JobBuilderFactory jobs, Step s1) { return jobs.get("importHotSeachesJob").preventRestart() .start(s1) .build(); } }
The above implementation doesn’t have ItemProcessor which is optional. The source code created as part of this article is available here.
Leave a Reply