Skip to main content

Testing Kafka in a Go API with OpenTelemetry and Tracetest

note

Check out the source code on GitHub here.

Tracetest is a testing tool based on OpenTelemetry that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.

Sample Go APIs with OpenTelemetry Collector, Jaeger and Tracetest​

This is a simple quick start on how to configure two Go APIs to communicate via Kafka. They use OpenTelemetry instrumentation with traces, Jaeger as a trace data store, and Tracetest for enhancing your E2E and integration tests with trace-based testing.

Prerequisites​

Tracetest Account:

Docker: Have Docker and Docker Compose installed on your machine.

Run This Quckstart Example​

The example below is provided as part of the Tracetest project. You can download and run the example by following these steps:

Clone the Tracetest project and go to the Node.js Quickstart with Manual Instrumentation:

git clone https://github.com/kubeshop/tracetest
cd tracetest/examples/quick-start-go-and-kafka

Follow these instructions to run the quick start:

  1. Copy the .env.template file to .env.
  2. Fill out the TRACETEST_TOKEN and ENVIRONMENT_ID details by editing your .env file.
  3. Run docker compose run tracetest-run.
  4. Follow the links in the output to view the test results.

Follow along with the sections below for an in detail breakdown of what the example you just ran did and how it works.

Project Structure​

The quick start Kafka + Go project is built with Docker Compose and contains the Tracetest Agent, Jaeger, OpenTelemetry Collector and a both Go apps.

The docker-compose.yaml file in the root directory of the quick start runs the Go apps, Jaeger, OpenTelemetry Collector, and the Tracetest Agent setup.

Go Producer API​

The Go API is a simple HTTP server, contained in the main.go file.

The OpenTelemetry tracing is contained in the ./telemetry/telemetry.go file. Traces will be sent to the OpenTelemetry Collector, and forwarded to Jaeger.

Traces will be sent to either the grpc endpoint. The hostname and port as seen in the env section of the producer-api in the docker-compose.yaml is:

  • OTEL_EXPORTER_OTLP_ENDPOINT=otel-collector:4317

The server starts by running the main.go file as you can see in the Dockerfile.

The docker-compose.yaml contains one service for the Go Producer API.

producer-api:
  image: quick-start-producer-api
  platform: linux/amd64
  build: ./producer-api
  extra_hosts:
    - "host.docker.internal:host-gateway"
  ports:
    - 8080:8080
  environment:
    - OTEL_EXPORTER_OTLP_ENDPOINT=otel-collector:4317
    - OTEL_SERVICE_NAME=producer-api
    - KAFKA_BROKER_URL=kafka:9092
    - KAFKA_TOPIC=messaging
  depends_on:
    otel-collector:
      condition: service_started
    kafka:
      condition: service_healthy

Go Consumer Worker​

The Go API is a simple HTTP server, contained in the main.go file.

The OpenTelemetry tracing is contained in the ./telemetry/telemetry.go file. Traces will be sent to the OpenTelemetry Collector, and forwarded to Jaeger.

Traces will be sent to either the grpc endpoint. The hostname and port as seen in the env section of the consumer-worker in the docker-compose.yaml is:

  • OTEL_EXPORTER_OTLP_ENDPOINT=otel-collector:4317

The server starts by running the main.go file. As you can see in the Dockerfile.

The docker-compose.yaml contains one service for the Go Producer API.

consumer-worker:
  image: quick-start-consumer-worker
  platform: linux/amd64
  build: ./consumer-worker
  extra_hosts:
    - "host.docker.internal:host-gateway"
  environment:
    - OTEL_EXPORTER_OTLP_ENDPOINT=otel-collector:4317
    - OTEL_SERVICE_NAME=consumer-worker
    - KAFKA_BROKER_URL=kafka:9092
    - KAFKA_TOPIC=messaging
  depends_on:
    otel-collector:
      condition: service_started
    kafka:
      condition: service_healthy
    producer-api:
      condition: service_started

Running the Go Apps, Kafka, and Tracetest​

Configure the .env like shown below.

# Get the required information here: https://app.tracetest.io/retrieve-token

TRACETEST_TOKEN="<YOUR_TRACETEST_TOKEN>"
TRACETEST_ENVIRONMENT_ID="<YOUR_ENV_ID>"

To start the full setup, run the following command:

docker compose run tracetest-run

This will:

  1. Start the Go apps, Kafka, the OpenTelemetry Collector, and send the traces to Jaeger.
  2. Start the Tracetest Agent.
  3. Configure the tracing backend and create tests in your environment.
  4. Run the tests.

Follow the links in the output to view the run group and the test result on Tracetest.

Manually Running the Tracetest Tests​

Open the URL and start creating tests! Make sure to use the http://producer-api:8080/publish URL in your test creation, because your Go app and Tracetest Agent are in the same network. And, use the Kafka trigger for the Consumer Worker.

Here are two sample tests you can get started with quickly.

👉 Testing the Consumer

👉 Testing the Producer

Learn More​

Feel free to check out our examples in GitHub and join our Slack Community for more info!