➶ Ubuntu 16.04 Vagrant + Apache Kafka [Guide]

Prerequisite

Setup Vagrant Box + Config

Init Box:

  $ vagrant init ubuntu/xenial64
  $ mkdir shared
  $ nano Vagrantfile

Config:

  # Vagrantfile
  Vagrant.configure("2") do |config|
    config.vm.box = "ubuntu/xenial64"
    config.vm.network "private_network", ip: "192.168.33.10"
    config.vm.synced_folder "./shared", "/home/ubuntu/shared"
    config.vm.provider "virtualbox" do |vb|
      vb.memory = "2048"
    end
  end

Start box and SSH:

  $ vagrant up
  $ vagrant ssh

Install JDK8 on Ubuntu 16.04 [Guide]

Install the Java JDK8 onto the Ubuntu Box

  $ sudo add-apt-repository ppa:webupd8team/java
  $ sudo apt-get update
  $ sudo apt install oracle-java8-installer
    # "<OK>", and agree to Oracle's terms "<YES>"
  $ sudo apt install oracle-java8-set-default
  $ javac -version
    # java 1.8.0

Add Apache ZooKeeper Daemon

Zookeeper is a coordination service required for Kafka server.

  $ sudo apt-get install zookeeperd
  $ netstat -ant | grep :2181
    # ensure result

Install Apache Kafka Server

Install the Apache Kafka server onto the vagrant box. Later versions and mirrors.

  $ wget http://mirror.cogentco.com/pub/apache/kafka/1.0.1/kafka_2.11-1.0.1.tgz

Untar package and move to simple directory

  $ tar -xzf kafka_2.11-1.0.1.tgz
  $ mkdir kafka 
  $ mv kafka_2.11-1.0.1/* kafka
  $ cd kafka

Start Kafka Server and Test [Apache Guide]

Start the Kafka server using the built in bin scripts

  $ bin/kafka-server-start.sh config/server.properties &

Now to test, create topic test

  $ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Check if it's running:

  $ bin/kafka-topics.sh --list --zookeeper localhost:2181
    # test

Create some messages and send em:

  $ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
  > This is a message
  > ???
  > PROFIT

Create consumer and dump messages to stdout

  $ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
  This is a message
  ???
  PROFIT

Conclusion

All credits all article writers. Content added sequentially for ease of implementation.

aquaflamingo/1-Apache-Kafka-Vagrant-Ubuntu-16-04.md