Apache ZooKeeper

APACHE KAFKA DISTRIBUTED ARCHITECTURE

• At the heart of Apache kafka we have a cluster, which consists of hundreds of independent Brokers.
• Closely associated with the kafka cluster, we have a Zookeeper environment,which provides the Brokers within a cluster, the metadata it needs to operate at scale and reliability.As this metadata is constantly changing, connectivity and chatter between the cluster members and Zookeeper is required.

APACHE ZOOKEEPER

• Used in variety of distributed system
• Serves as centralized service for maintaining metadata about the cluster of distributed node
• Maintains configuration information
• Checks Health status

Note  Large distributed systems depend on Zookeeper, like Apache hadoop,Hbase,Mesos, solr, Neo4j. Use it to enable their scale out fault-tolerant capabilities
 Is a Distributed system consisting of multiple nodes in an "ensemble" (cluster for ZooKeeper).

As consumer application come in, Each of these components can scale out to take more demand and increase levels of reliability and availability.

Ex of Netflix
4,000 Brokers across 36 clusters processing over 700 billion messages per day.