Post

Kafka ops commands

Posted Updated
Preview Image
By Roman Tsypuk 3 min read

Topic operations:

Consumer operations::

Add Kafka to the PATH

1

$ export PATH="/xxx/xxxxxx/kafka_2.13-2.4.1/bin:$PATH"

Topic operations

Create topic

1
2
3
4
5
6

$ kafka-topics.sh --zookeeper zookeeper1:2181 \
--create \
--topic my-topic \
--replication-factor 1 \
--partitions 5 \
--if-not-exists

Change partitions number (can only add more)

1
2
3
4

$ kafka-topics.sh --zookeeper zookeeper1:2181 \
--alter \
--topic my-topic \
--partitions 7

Delete a topic

1
2
3

$ bin/kafka-topics.sh --zookeeper zookeeper1:2181 \
--delete \
--topic my-topic

List all topics

1
2

$ bin/kafka-topics.sh --zookeeper zookeeper1:2181 \
--list

Describe all the topics

1
2

$ bin/kafka-topics.sh --zookeeper zookeeper1:2181 \
--describe

List topics with overrides(configs added to the defaults)

1
2
3

$ bin/kafka-topics.sh --zookeeper zookeeper1:2181 \
--describe \
--topics-with-overrides

List topics that are not in-sync with all replicas

1
2
3

$ bin/kafka-topics.sh --zookeeper zookeeper1:2181 \
--describe \
--under-replicated-partitions

List topics without a leader replica

1
2
3

$ bin/kafka-topics.sh --zookeeper zookeeper1:2181 \
--describe \
--unavailable-partitions

Describe the configurations for all topics (only in addition to the defaults)

1
2
3

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--describe \
--entity-type topics

Describe the configurations for a specific topic (defaults will not show)

1
2
3
4

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--describe \
--entity-type topics \
--entity-name my-topic

Change the topics message retention

1
2
3
4
5

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--alter \
--entity-type topics \
--entity-name connect-test \
--add-config retention.ms=3600000

Describe the configurations for all brokers (defaults will not show)

1
2
3
4

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--entity-type brokers \
--entity-default \
--describe

Describe the configuration for broker 0 (defaults will not show)

1
2
3
4

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--entity-type brokers \
--entity-name 0 \
--describe

Add a custom config to broker 0 (log cleaner thread count)

1
2
3
4
5

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--entity-type brokers \
--entity-name 0 \
--alter \
--add-config log.cleaner.threads=2

Remove all custom configs (not including defaults) from broker 0

1
2
3
4
5

$ bin/kafka-configs.sh --zookeeper zookeeper1:2181 \
--entity-type brokers \
--entity-name 0 \
--alter \
--delete-config log.cleaner.threads

Consumer operations

List all the consumer groups

1
2

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--list

Describe a specific consumer group

1
2
3

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--describe \
--group application1

Describe the active members of the group

1
2
3
4

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--describe \
--group application1 \
--members

If the group has active members get a more verbose output

1
2
3
4
5

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--describe \
--group application1 \
--members \
--verbose

Describe the state of the group

1
2
3
4

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--describe \
--group application1 \
--state

Delete a consumer group (only works if there are no active members).

1
2
3

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--delete \
--group application1

Delete multiple consumer groups

1
2
3
4

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka1:9092 \
--delete \
--group application1 \
--group application2

Reset offsets for a consumer group

1
2
3
4
5

$ bin/kafka-consumer-groups.sh --bootstrap-server kafka:9092 \
--reset-offsets \
--group application1 \
--topic topic-1 \
--to-latest
kafka, linux, cli, administration
kafka linux cli administration
This post is licensed under CC BY 4.0 by the author.