How to start Spring Apache Kafka Application with SpringBoot Auto-Configuration

The Spring Apache Kafka (spring-kafka) provides a high-level abstraction for Kafka-based messaging solutions. And Spring Boot 1.5 includes auto-configuration support for Apache Kafka via the spring-kafka project. So in the tutorial, JavaSampleApproach will guide how to start Spring Apache Kafka Application with SpringBoot.

Related Articles:
How to start Apache Kafka
How to start Spring Kafka Application with Spring Boot
Spring JMS – Explicitly Configure Spring ActiveMQ ConnectionFactory | SpringBoot

I. Technologies

– Java 8
– Maven build
– Spring Boot
– Apache Kafka
– Spring Tool Suite editor

II. Overview

As above mention, SpringBoot supports auto-configuration for Apache Kafka development:
– Use spring.kafka.* in application.properties file to modify external configuration.

Spring Apache Kafka Application

– Use Spring auto-configured KafkaTemplate to send Kafka-based messages:

– Use @KafkaListener to setup a Kafka listener:

III. Practice

We create a SpringBoot project with 2 main services: KafkaProducer and KafkaConsumer for sending and receiving messages from Apache Kafka cluster.
And export 2 RestAPIs {‘/producer’, ‘/consumer’} for interaction.

SpringBoot Kafka Application - project structure

Step to do:
– Create a SpringBoot project
– Create Kafka Producer and Consumer
– Add Apache Kafka external configuration
– Export some RestAPIs
– Deployment

1. Create a SpringBoot project

Use SpringToolSuite to create a SpringBoot project, then add dependencies {spring-kafka, spring-boot-starter-web}:

2. Create Kafka Producer and Consumer

– Create a KafkaProducer service:

– Create a KafkaConsumer service:

About MessageStorage, it is an additional implement to store Kafka-based messages after received. See details the implementation of MessageStorage:

3. Add Apache Kafka external configuration

Open application.properties file, add the configurations:

spring.kafka.bootstrap-servers is used to indicate the Kafka Cluster address.
spring.kafka.consumer.group-id is used to indicate the consumer-group-id.
jsa.kafka.topic is an additional configuration. In the the tutorial, we use jsa.kafka.topic to define a Kafka topic name to produce and receive messages.

4. Export some RestAPIs

Create a Web Controller to export 2 RestAPIs {‘/producer’, ‘/consumer’}

/producer is used to send messages from browser to KafkaProducer service.
/consumer is used to get all recieved messages that are buffered in MessageStorage.

5. Deployment

Start Apache Kafka Cluster:
– Start a ZooKeeper:

– Start the Apache Kafka server:

>>> More details at: How to start Apache Kafka

Build and Install the SpringBoot project with commandlines: mvn clean install and mvn spring-boot:run

– Make a producer request: http://localhost:8080/jsa/kafka/producer?data=Hello World
-> Logs:

– Make another producer request: http://localhost:8080/jsa/kafka/producer?data=This is a SpringBoot Kafka Application

-> Logs:

– Make a consumer request: http://localhost:8080/jsa/kafka/consumer, result:

Spring Appache Kafka - consumer request

IV. Sourcecode

SpringBootApacheKafka

By grokonez | June 8, 2017.



Related Posts


4 thoughts on “How to start Spring Apache Kafka Application with SpringBoot Auto-Configuration”

  1. Hi, i am getting error,
    Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:1.5.1.RELEASE:run (default-cli) on project Messaging: Unable to find a suitable main class, please add a ‘mainClass’ property -> [Help 1]

  2. spring -boot version is old here in the pom..also the jars need to be updated….does anyone have an updated pom on this project that works??
    I updated the spring boot version to
    <!– 1.5.4.RELEASE –>
    2.0.2.RELEASE

    compiles alirght but gave test failure
    [INFO]
    [INFO] ——————————————————-
    [INFO] T E S T S
    [INFO] ——————————————————-
    [INFO] Running com.javasampleapproach.apachekafka.SpringApacheKafkaApplicationTests
    [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.042 s <<< FAILURE! – in com.javasampleapproach.apachekafka.SpringApacheKafkaApplicationTests
    [ERROR] initializationError(com.javasampleapproach.apachekafka.SpringApacheKafkaApplicationTests) Time elapsed: 0.014 s <<< ERROR!
    java.lang.TypeNotPresentException: Type org.springframework.test.context.junit4.SpringRunner not present
    Caused by: java.lang.ClassNotFoundException: org.springframework.test.context.junit4.SpringRunner

Got Something To Say:

Your email address will not be published. Required fields are marked *

*