📌Overview

Purpose: To facilitate the interaction between AI assistants and Apache Kafka through a Model Context Protocol (MCP) server.

Overview: The mcp-kafka framework enables seamless communication between AI assistants and Kafka clusters, allowing for the creation, management, and interaction with Kafka topics and messages through simple commands.

Key Features:

Create Topic: Allows users to create new Kafka topics with customizable options such as the number of partitions and replication factors.
List Topics: Provides a comprehensive list of all topics available in the Kafka cluster, enhancing operational transparency.
Delete Topic: Enables the removal of existing topics, thereby facilitating effective topic management.
Describe Topic: Delivers detailed insights about a specific topic, including its partition structure, supporting better planning and monitoring.
Produce Message: Facilitates sending messages to Kafka topics, with support for key and header configurations to enhance message context.
Consume Messages: Supports reading messages with customizable timeouts, providing flexibility for message retrieval effectively.

mcp-kafka

A Model Context Protocol (MCP) server for performing Kafka client operations from AI assistants.

Overview

mcp-kafka provides a bridge between AI assistants and Apache Kafka, allowing them to interact with Kafka clusters through the Model Context Protocol. This tool enables AI assistants to create, manage, and interact with Kafka topics and messages directly.

Here is a short video demo on youtube:
https://youtu.be/Jw39kJJOCck

Features

The mcp-kafka server provides the following Kafka operations:

Create Topic: Create a new Kafka topic with configurable partitions and replication factor
List Topics: Get a list of all available Kafka topics in the cluster
Delete Topic: Remove an existing Kafka topic
Describe Topic: Get detailed information about a specific topic, including partition details
Produce Message: Send messages to a Kafka topic with support for message keys and headers
Consume Messages: Read messages from a Kafka topic with configurable timeout

Installation

Prerequisites

Go 1.24 or higher
A running Kafka cluster (default connection: localhost:9092)

Building from Source

Clone the repository:

git clone https://github.com/kanapuli/mcp-kafka.git
cd mcp-kafka

Build the application:
```
make build
```
Optionally, build for a specific platform:
```
make build GOOS=darwin GOARCH=arm64
```

Installing as a Claude Desktop Tool

To use mcp-kafka with Claude Desktop:

Build the application as mentioned above.
Place the executable in a location included in your system PATH or in a dedicated tools directory.
Follow the Claude Desktop Tool Installation Guide: https://modelcontextprotocol.io/quickstart/user

Add the following JSON configuration to your claude_desktop_config.json file. Use the correct username and password for SASL login, or leave them empty for PLAINTEXT authentication.

{
  "mcpServers": {
    "kafka": {
      "command": "/Your-mcp-kafka-executable-path/mcp-kafka-darwin-arm64",
      "args": [
          "--bootstrap-servers=localhost:9092",
          "--consumer-group-id=mcp-kafka-consumer-group",
          "--username=",
          "--password="
        ],
      "env": {}
    }
  }
}

Configuration

The mcp-kafka tool accepts the following configuration parameters:

Parameter	Description	Default
topic	Topic to interact with	(required)
num_partitions	Number of partitions for topic creation	(optional)
replication_factor	Replication factor for topic creation	(optional)
produce_message_key	Key for produced messages	(optional)
produce_message_value	Value for produced messages	(optional)
produce_message_headers	Headers for produced messages	(optional)
consumer_timeout	Timeout in seconds for message consumption	10

These parameters will be automatically derived from your natural language message to the LLM.

CLI flags for the mcp-kafka tool

Use the following flags to configure the Kafka client:

--bootstrap-servers=localhost:9092
--consumer-group-id=mcp-kafka-consumer-group
--username='your_sasl_username'
--password='your_sasl_password'

Note: Currently, SASL_PLAINTEXT is supported along with PLAINTEXT authentication. SASL_SSL is not supported.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License

Acknowledgments

Built using the MCP Golang library