Team formation in Kafka

CONTROLLER ELECTION

• Hierarchy starts with a controller/supervisor 
• It is a worker node elected by its peers to officiate in the administrative capacity of a controller
• The worker node selected as controller is the one that is been around the longest

RESPONSIBILITY OF CONTROLLER ELECTION

• Maintain inventory of what workers are available to take on work.
• Maintain a list of work items that has been committed to and assigned to workers
• Maintain active status of the staff and their progress on assigned tasks

TEAM FORMATION

• Once a controller is established and the workers are assigned and available team is formed
• Team formation is the cluster and its members brokers that have assigned themselves to a designated controller within the cluster.

TASK

• When a task comes in from a kafka producer, the controller has to make a decision which worker should take it. There are lot of factor to play

CONTROLLER
The controller within kafka has the following resposibilities

•  Work availability and health
• Needs to know who is available in good health

•  Task redundancy
•  What risk policy should govern its assignment decisions.Thing that determines what level of  replication to employ in case as assigned worker fails
• It has to ensure that if work is assigned to a worker,and that worker becomes unavailable, the work assigned or that work already done is not lost
• Each task given to a worker must also be given to at least one of the workers peers in the event of an unexpected catastrophe, but among a group 
• Promotion
•  For an assignment, if the controller determines redundancy is required, it will promote a worker into a leader, which will take direct ownership of the task assigned.

  RECRUIT TWO PEERS

•  It would be leaders job to recruit two of its peers to take part in the replication

   REPLICATION FACTOR

•  The risk factor to protect against loss is known as its replication factor

    QUOROM

•   Once peers have committed to a leader a Quorom is formed.