Saturday, April 14, 2018

Install kafka

Steps for installing Kafka

  • You need to setup a Java virtual machine on your system before you can run Apache Kafka properly.
  • We can install OpenJDK Runtime Environment 1.8.0 using YUM:
sudo yum install java-1.8.0-openjdk.x86_64
  • Validate your installation with:
java -version

The output should resemble:
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
  • You also need to setup the "JAVA_HOME" and "JRE_HOME" environment variables:
sudo gedit /etc/profile
  • Append the following lines to the original content of the file
export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk.x86_64

export JRE_HOME=/usr/lib/jvm/jre
        Reload the profile to put your changes into effect:
        source /etc/profile
  •   Restart the cluster

[cloudera@quickstart ~]$ su root
Note:We need to login as root user
[cloudera@quickstart ~]$ reboot

  • Create a directory. we can name it as opt (mkdir op)
  • Install Java 8 and set the environment variable Kafka shell

  • Cmd to install Kafka


  • Unzip tar file

tar -xvf kafka_2.11-1.0.0.tgz

Modify the configuration of your Kafka server
gedit kafka_2.11-1.0.0/bin/
Adjust the memory usage according to your specific system parameters.

For example, if you are using a Vultr server instance with 768MB memory in the test environment, you need to locate the following line: export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
Replace it with: export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"
  • Restart the server and check for the java version

Login as su user 

[root@quickstart kafka_2.11-1.0.0]# bin/  

[cloudera@quickstart ~]$ reboot
kafka shell path

Start ZoopKeeper server
bin/ config/

Install telnet
yum install telnet telnet-server -y

Stat : Gives the status of the ZooKeeper

Shell Topics
[root@quickstart kafka_2.11-1.0.0]# bin/

No comments:

Post a Comment