Skip to content

Kafka

Recipes for working with a Confluent Kafka container running in Docker. All recipes target a container named kafka by default — override with KAFKA_CONTAINER=mycontainer just kafka ....

Getting Started

All recipes require the Kafka container to be running. If it isn't, you'll get a clear error. Start your Compose stack first, then:

just kafka topics              # verify connectivity — lists all topics
just kafka brokers             # check cluster metadata

Inspecting Topics and Consumers

topics lists all topic names. describe shows partition layout, replication, and ISR status for a single topic — use this to check whether partitions are under-replicated or have offline replicas.

groups lists consumer groups. lag shows per-partition offsets and lag for a group — if lag is growing, the consumer isn't keeping up. A consumer with EMPTY state means no active members (crashed or redeployed).

config shows topic-level config overrides (retention, cleanup policy, etc.) — anything not shown is using the broker default.

Producing and Consuming

consume tails a topic from the latest offset. Add a group name to track committed offsets: just kafka consume my-topic my-group. Press Ctrl+C to stop.

produce opens an interactive producer — type one message per line, Ctrl+D to finish. Useful for manual testing.

produce-file sends a JSON file as a single message. The file is piped through jq -c to produce compact single-line JSON.

Managing Topics

create-topic creates a topic with configurable partitions and replication factor (both default to 1 for local dev).

delete-topic requires confirmation. Topic deletion is asynchronous — the topic may briefly appear in topics after deletion.

nuke deletes all user topics (anything not prefixed with __). Internal topics like __consumer_offsets are always preserved.

Common Workflows

"My consumer isn't receiving messages":

  1. just kafka topics — does the topic exist?
  2. just kafka lag my-group — is the consumer connected? Is lag growing?
  3. just kafka consume my-topic — can you see messages manually?

"I need to reset a topic":

  1. just kafka delete-topic my-topic
  2. just kafka create-topic my-topic