Topics are a special and essential component of Apache Kafka that are used to organize events or messages. In other words, Kafka Topics enable simple data transmission and reception across Kafka Servers by acting as Virtual Groups or Logs that store messages and events in a logical sequence. In this article, we will discuss about how to configure Kafka topics from the spring boot application.
Step by Step Implementation
First, our spring boot project needs Kafka dependencies. Include this part in your pom.xml file. Any Kafka version is accessible. We would prefer that you use the most recent Kafka client. For additional information on the Kafka version.
Java
<dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId> <version>2.9.0</version> <scope>compile</scope></dependency> |
Create a class with whatever name you prefer, but I used TopicConfiguration.java. We need to convert this class into a bean, and since we are configuring Kafka topics, we included the @Configuration annotation from Spring Boot.
Java
import java.io.*; @Configurationpublic class TopicConfiguration { } |
The process of adding new topics to our broker is handled by a KafkaAdmin bean. A KafkaAdmin bean is automatically registered with Spring Boot. The KafkaAdmin bean has to be explicitly registered for non-Spring Boot applications. The next step is to construct a new bean that tells KafkaAdmin what our Kafka URL is. BOOTSTRAP_SERVERS_CONFIG: Host and port on where Kafka broker is running.
BOOTSTRAP_SERVERS_CONFIG: <kafka_url>:<port>
Java
@Bean public KafkaAdmin admin(){ Map<String, Object> configs = new HashMap<>(); configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); return new KafkaAdmin(configs);} |
We can now configure and create Kafka topics. A lot of various settings can be used right here while creating a topic. For more information about TopicBuilder.
Java
import java.io.*; @Bean public NewTopic topic1(){ return TopicBuilder.name("TOPIC-1") .partitions(1) .replicas(1) .build();} @Beanpublic NewTopic topic2(){ return TopicBuilder.name("TOPIC-2") .partitions(1) .replicas(1) .build();} |
File: TopicConfiguration.java
Java
import java.io.*; @Configurationpublic class TopicConfiguration { @Bean public KafkaAdmin admin() { Map<String, Object> configs = new HashMap<>(); configs.put( AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); return new KafkaAdmin(configs); } @Bean public NewTopic topic1() { return TopicBuilder.name("TOPIC-1") .partitions(1) .replicas(1) .build(); } @Bean public NewTopic topic2() { return TopicBuilder.name("TOPIC-2") .partitions(1) .replicas(1) .build(); }} |
In the Springboot application, Kafka topics can be created and configured this way. The spring boot application will automatically create Kafka topics on the specified Kafka broker when it is launched. To get the topic configuration details on the server, run this command.
kafka-topics.sh --bootstrap-server localhost:9092 --topic <topic_name> --describe
Output:
This is the output from the server.
