Kafka: Message Producer

Kafka: Message Producer

$ cd ~/kafka_2.11-1.1.0/bin

• Each line entered after running producer will be new message. Producer can be terminated using (ctrl+c)from publishing more messages.

  ./kafka-console-producer.sh --broker-list localhost:9092 --topic kafka_test_topic
  Kafka Message 1   
  Kafka Message 2
  Kafka Message 3
  Kafka Message 4
  Kafka Message 5
  
  

• Publishing message using file, keep all the message (1 message per line) in any file (InsertWithFile.txt) and pass it in the producer command as shown below.

  ./kafka-console-producer.sh --broker-list localhost:9092 --topic kafka_test_topic_withpartition < ~/InsertWithFile.txt
  
   File 'InsertWithFile.txt' Contents
   Kafka Topic 2 Message 1   
   Kafka Topic 2 Message 2
   Kafka Topic 2 Message 3
   Kafka Topic 2 Message 4
   Kafka Topic 2 Message 5
  

• In older versions(before 2.2), there was no direct way of purging messages from Kafka topic. Messages were getting purged automatically after the retention period specified for the topic.
  Although there is trick to empty(remove all messages) the topic by changing retention period.
  Change the retention time to very low value and then verify that messages are expired. Later change the retention time again to make it original one

  ./kafka-topics.sh --alter --zookeeper localhost:2181 --topic kafka_test_topic --config retention.ms=1000
  
  Output:
  Updated config for topic "kafka_test_topic".
  
  
  ./kafka-topics.sh --alter --zookeeper localhost:2181 --topic kafka_test_topic --config retention.ms=1000000000
  
  Output:
  Updated config for topic "kafka_test_topic".
  

• Starting with version 2.2, Kafka has introduced a kafka-delete-records API to purge older data till the specified offset of a topic. Click here to know more about Purge operation in Kafka version 2.2 and above.