yoshikipom Tech Blog

開発ツール for Kafka (docker-compose, kcat)

Overview

  • docker-composeでの起動
  • CLI tool (kcat)

Environment

  • MacOS
  • docker, docker-compose

docker-composeでの起動

zookeeperも起動する必要があるようなのでdocker-composeで起動。

---
version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.0.1
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
  broker:
    image: confluentinc/cp-kafka:7.0.1
    container_name: broker
    ports:
      - "9092:9092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1

run

docker-compose up

CLI tool (kcat)

インストール

brew install kcat

コマンド

produce (interactive)

$ kcat -b localhost:9092 -t new_topic -P
test
# end by ctrl-D

produce (file)

$ kcat -b localhost:9092 -t new_topic -P -l test.json

consume

# topic list
$ kcat -b localhost:9092 -t new_topic -C
% Reached end of topic new_topic [0] at offset 0
test
test2
% Reached end of topic new_topic [0] at offset 2
test3
% Reached end of topic new_topic [0] at offset 3

topic list

# topic list
$ kcat -b localhost:9092 -L
Metadata for all topics (from broker 1: localhost:9092/1):
 1 brokers:
  broker 1 at localhost:9092 (controller)
 1 topics:
  topic "new_topic" with 1 partitions:
    partition 0, leader 1, replicas: 1, isrs: 1

参考