Wednesday, April 20, 2022

Install Apache Kafka on Windows | Kafka Installation Step-By-Step

 Kafka Setup Instructions:

1. Download opensource Kafka from official website and extract the binary:

a. go to https://kafka.apache.org/downloads

b. download the recommended version from Binary downloads. 

           eg. Scala 2.13  - kafka_2.13-3.5.1.tgz (asc, sha512)

c. Extract using 7-zip and rename it to kafka

*************************

2. Do some configuration:

Update server.properties and zookeeper.properties files located in config directory.

a. update server.properties::

# A comma separated list of directories under which to store log files

log.dirs=C:/IdeaLab/kafka/kafka-logs   #this is the directory where logs will be residing

b. update zookeeper.properties::

# the directory where the snapshot is stored.

dataDir=C:/IdeaLab/kafka/zookeeper-data    #this will be the data directory for our zookeeper

*************************

3. Kickoff zookeeper and kafka server:

a. start zookeeper:

C:\IdeaLab\kafka\bin\windows\zookeeper-server-start.bat C:\IdeaLab\kafka\config\zookeeper.properties

####NOTE: Don't close the zookeeper terminal window. Instead just open new terminal window to start kafka server.

b. kickoff the kafka server:

when you open the terminal inside kafka directory, you don't need to give the full path. You can just give .\bin\... as well.

.\bin\windows\kafka-server-start.bat .\config\server.properties

OR

C:\IdeaLab\kafka\bin\windows\kafka-server-start.bat C:\IdeaLab\kafka\config\server.properties

*************************

4. Produce the data and consume it from the consumer function.

a. we need to first create a test topic and push some data on it.

C:\IdeaLab\kafka\bin\windows>kafka-topics.bat --create --bootstrap-server localhost:9092 --topic vk_kafka_topic

Created topic  vk_kafka_topic

b.

C:\IdeaLab\kafka\bin\windows> .\kafka-console-consumer.bat --topic  vk_kafka_topic --bootstrap-server localhost:9092 --from-beginning



No comments:

Post a Comment