Kafka partitions deletion

Kafka way:

  • delete.topic.enable=true
  • bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic <topic_name>
  • The Kafka way of deleting topics wont work sometimes as it will just mark the topic for deletion.
  • So we must delete the topics and partitions manually and remove their data from Zookeeper also.

Manual way: Check this link

  1. Stop all the Kafka brokers
    1. sudo supervisorctl stop kafka-broker
  2. Login to all Kafka broker nodes and delete the topics/partitions manually
    • sudo rm -rf <kafka_data_dir>/<topic_name>*
    • The Kafka topic location will be mentioned in Kafka service configs
  3. Now lets delete from Zookeeper
    • Connect to any of the zookeeper node using any of commands below
    • rmr /config/topics/<topic_name>
      rmr /brokers/topics/<topic_name>
      rmr /admin/delete_topics/<topic_name>
  4. Verify the topic directory is deleted in zookeeper:
    • ls <path>
      
  5. Start the Kafka brokers back up:
    1. sudo supervisorctl start kafka-broker
  6. Command to create a Kafka topic again:
    1. bin/kafka-topics.sh --zookeeper ${zookeeper_server} --create --topic $topic --partitions=$pars --replication-factor $rep --config retention.ms=$ret

       

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s