Saturday, April 14, 2018

Install kafka


Steps for installing Kafka

  • You need to setup a Java virtual machine on your system before you can run Apache Kafka properly.
  • We can install OpenJDK Runtime Environment 1.8.0 using YUM:
sudo yum install java-1.8.0-openjdk.x86_64
  • Validate your installation with:
java -version


The output should resemble:
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
  • You also need to setup the "JAVA_HOME" and "JRE_HOME" environment variables:
sudo gedit /etc/profile
  • Append the following lines to the original content of the file
export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk.x86_64

export JRE_HOME=/usr/lib/jvm/jre
        Reload the profile to put your changes into effect:
        source /etc/profile
  •   Restart the cluster

[cloudera@quickstart ~]$ su root
Password: 
Note:We need to login as root user
[cloudera@quickstart ~]$ reboot

  • Create a directory. we can name it as opt (mkdir op)
  • Install Java 8 and set the environment variable Kafka shell

  • Cmd to install Kafka

 wget http://apache.cs.utah.edu/kafka/1.0.0/kafka_2.11-1.0.0.tgz

  • Unzip tar file

tar -xvf kafka_2.11-1.0.0.tgz

Modify the configuration of your Kafka server
gedit kafka_2.11-1.0.0/bin/kafka-server-start.sh
Adjust the memory usage according to your specific system parameters.

For example, if you are using a Vultr server instance with 768MB memory in the test environment, you need to locate the following line: export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
Replace it with: export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"
  • Restart the server and check for the java version

Login as su user 


1
[root@quickstart kafka_2.11-1.0.0]# bin/zookeeper-server-start.sh  


[cloudera@quickstart ~]$ reboot
kafka shell path
1
/opt/kafka_2.11-1.0.0/bin

Start ZoopKeeper server
1
bin/zookeeper-server-start.sh config/zookeeper.properties

Install telnet
1
yum install telnet telnet-server -y

Stat : Gives the status of the ZooKeeper
1
stat

Shell Topics
1
[root@quickstart kafka_2.11-1.0.0]# bin/kafka-topics.sh

No comments:

Post a Comment