Skip to content
Postman API Testing Tips

How To Send Message To Kafka Topic Using Postman

API Testing Blog

Sending Messages to Kafka Topics Using Postman: A Comprehensive Guide

Postman, a popular API testing tool, can also be used to send messages to Kafka topics, making it a powerful tool for interacting with your Kafka infrastructure. Here’s a comprehensive guide on how to do just that, along with practical examples and step-by-step instructions.

1. Setting Up Your Environment

Before you can send messages to Kafka topics using Postman, you need to ensure you have:

  • A running Kafka cluster: You can use a local setup or a cloud-based service like Confluent Cloud or Amazon MSK.
  • Postman installed and configured: Download Postman from https://www.postman.com/ and install it on your system.
  • A Kafka client: You’ll need a library that allows you to interact with Kafka from your chosen programming language. For this guide, we’ll use the Python Kafka client (confluent-kafka).

2. Understanding the Kafka Protocol

Kafka uses a simple protocol based on HTTP. To send messages to a Kafka topic, you’ll need to perform a POST request to a specific endpoint using the following structure:

http://<kafka_broker_host>:<kafka_broker_port>/topics/<topic_name>

Example:

http://localhost:9092/topics/my-topic

3. Sending Messages Using Postman

Let’s now dive into using Postman to send messages to Kafka topics.

Step 1: Create a New Request

  • Open Postman and click “New” to create a new request.

Step 2: Configure the Request

  • Method: Set the HTTP method to POST.
  • URL: Enter the URL for your Kafka topic as shown above (e.g., http://localhost:9092/topics/my-topic).
  • Headers: Add the following headers to your request:
    • Content-Type: application/vnd.kafka.v2+json
    • Accept: application/vnd.kafka.v2+json

Step 3: Payload Structure

The body of your Postman request contains the message you want to send. It follows a JSON structure, as defined by Kafka’s protocol:

{
  "records": [
    {
      "value": "your message",
      "key": "optional key"
    }
  ]
}

Explanation:

  • records: This array contains one or more messages you want to send.
  • value: This is the actual message content. It can be a string, an object, or any data type supported by your application.
  • key: This is an optional field that can be used to organize messages within a topic. It can be a string or any other data type.

Step 4: Send the Request and Verify

  • Click “Send” to send the message.
  • Success Response: If your message was sent successfully, you should receive a 200 OK response with a JSON body like this:
{
  "offsets": [
    {
      "topic": "my-topic",
      "partition": 0,
      "offset": 151
    }
  ]
}
  • Error Response: If an error occurs, you’ll receive a non-200 HTTP code along with a JSON body describing the error details.

4. Example: Sending a Message With Postman

Here’s a complete example of sending a message to a Kafka topic called “user-events” with a string message and optional key:

Request:

  • Method: POST
  • URL: http://localhost:9092/topics/user-events
  • Headers:
    • Content-Type: application/vnd.kafka.v2+json
    • Accept: application/vnd.kafka.v2+json
  • Body:
{
  "records": [
    {
      "value": "User 'johndoe' logged in",
      "key": "user_login"
    }
  ]
}

Response:

{
  "offsets": [
    {
      "topic": "user-events",
      "partition": 0,
      "offset": 151
    }
  ]
}

5. Advanced Features

Postman’s capabilities for Kafka interactions extend beyond basic message sending:

  • Pre-request Scripts: Automate tasks before sending requests, such as generating random data or authenticating with your Kafka cluster.
  • Tests: Add assertions to your requests to ensure successful message delivery and verify the expected response from the Kafka broker.
  • Environments: Store your Kafka broker information securely in environments within Postman to easily switch between different Kafka setups.

6. Conclusion

Postman offers a user-friendly and powerful interface for sending messages to Kafka topics. Using its features like pre-request scripts, tests, and environments, you can easily integrate Kafka into your API testing workflows and streamline your interaction with Kafka clusters. Remember to explore additional resources like the official Postman documentation and the Confluent documentation for more advanced Kafka interactions and API testing techniques.

API Testing Blog