Skip to main content

Performance and Trace-Based Tests with Tracetest and k6

Tracetest is a testing tool based on OpenTelemetry that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.

k6 is an open-source performance testing tool used for testing the performance of APIs, microservices, and websites. It is designed to be developer-centric, making it easy for developers to write and maintain performance tests as code.

Why is this important?

k6 is it's a great tool in its own right that allows you to replicate most of the production challenges you might encounter. But, as with all of the tools that only test the initial transaction between the client side and the server, you can only run validations against the immediate response from the service.

The k6 Tracetest Extension

With the k6 Tracetest extension, you will unlock the power of OpenTelemetry that allows you to run deeper testing based on the traces and spans generated by each of the checkpoints that you define within your services.

Language and Vendor agnostic, with this extension you can use your existing Tracing Data Store and Setup to leverage the work you have already done to instrument your services.

How It Works

The following is high level sequence diagram on how k6 and Tracetest interact with the different pieces of the system.

Today You'll Learn About Performance & Trace-Based Tests

This is a simple quick start guide on how to run the Tracetest k6 extension to run enhanced performance tests with trace-based testing. The infrastructure will use the Pokeshop Demo as a testing ground, triggering requests against it and generating telemetry data.

Requirements

Tracetest Account:

Docker:

Run This Quckstart Example

Clone the official Tracetest Pokeshop Demo App Repo to your local machine.

git clone https://github.com/kubeshop/pokeshop.git
cd pokeshop

Follow these instructions to run the Cypress example:

  1. Copy the .env.template file to .env.
  2. Log into the Tracetest app.
  3. This example is configured to use Jaeger. Ensure the environment you will be utilizing to run this example is also configured to use the Jaeger Tracing Backend by clicking on Settings, Tracing Backend, Jaeger, updating the URL to jaeger:16685, Test Connection and Save.
  4. Fill out the token and agent API key details by editing your .env file. You can find these values in the Settings area for your environment.
  5. Change the POKESHOP_DEMO_URL to http://api:8081 in the .env file.
  6. Run docker compose -f docker-compose.yml -f docker-compose.k6.yml run k6-tracetest.

Follow along with the sections below for an in detail breakdown of what the example you just ran did and how it works.

Project Structure

The project is built with Docker Compose.

Pokeshop Demo App

The Pokeshop Demo App is a complete example of a distributed application using different backend and front-end services, implementation code is written in Typescript.

The docker-compose.yml file in the root directory is for the Pokeshop Demo app and the OpenTelemetry setup. And the docker-compose.k6.yml includes the Tracetest Agent and the K6 Load Tests.

Finally, the k6 load tests can be found in test/k6/import-pokemon.js.

Installing the k6 Tracetest Extension

Installing the k6 Tracetest extension is as easy as running the following command:

Terminal
docker run --rm -it -u "$(id -u):$(id -g)" -v "${PWD}:/xk6" \
grafana/xk6 build v0.43.1 \
--with github.com/kubeshop/xk6-tracetest

This will generate the k6-tracetest binary in the current directory.

The instructions can be also found in the main k6 docs in case you need to combine multiple extensions into one binary.

Using the Tracetest Extension For Load Tests

Once you have installed the k6 Tracetest binary, you can use the base k6 functionality to run load tests against instrumented services and Tracetest to run checks against the resulting telemetry data.

Creating Your k6 Script

The test/k6/import-pokemon.js file contains the k6 script that will be used to run the performance tests. It is a simple script that will trigger a POST request to the /pokemon/import Pokeshop Demo app endpoint.

import { Http, Tracetest } from 'k6/x/tracetest';
import { sleep } from 'k6';

export const options = {
vus: 1,
duration: '5s',
};

const POKESHOP_DEMO_URL = __ENV.POKESHOP_DEMO_URL || 'http://localhost:8081';

const http = new Http();
const tracetest = Tracetest();

let pokemonId = 6; // charizard

export default function () {
const url = `${POKESHOP_DEMO_URL}/pokemon/import`;
const payload = JSON.stringify({
id: pokemonId++,
});
const params = {
headers: {
'Content-Type': 'application/json',
},
};

// tracetest test definition
const definition = `type: Test
spec:
id: k6-tracetest-pokeshop-import-pokemon
name: K6
description: K6
trigger:
type: k6
specs:
- selector: span[tracetest.span.type="general" name="import pokemon"]
name: Should have imported the pokemon
assertions:
- attr:tracetest.selected_spans.count = 1
- selector: |-
span[tracetest.span.type="http" net.peer.name="pokeapi.co" http.method="GET"]
name: Should trigger a request to the POKEAPI
assertions:
- attr:http.url = "https://pokeapi.co/api/v2/pokemon/${pokemonId}"
`;

const response = http.post(url, payload, params);

tracetest.runTest(
response.trace_id,
{
definition,
should_wait: true,
},
{
url,
method: 'GET',
}
);

sleep(1);
}

export function handleSummary() {
return {
stdout: tracetest.summary(),
};
}

export function teardown() {
tracetest.validateResult();
}

Setting the Environment Variables

Copy the .env.template content into a new .env file.

cp .env.template .env

Add the Tracetest API Token and Tracetest Agent API Key to the TRACETEST_API_TOKEN and TRACETEST_AGENT_API_KEY variables.

TRACETEST_API_TOKEN=<YOUR_API_TOKEN> # your environment token
TRACETEST_AGENT_API_KEY=<YOUR_AGENT_API_KEY>
POKESHOP_DEMO_URL=http://api:8081

Starting the Pokeshop Demo App

Having the full setup ready, the final step is to run the k6 script. To do that, run the following command:

docker compose -f docker-compose.yml -f docker-compose.k6.yml run k6-tracetest
WARN[0000] The "TRACETEST_SERVER_URL" variable is not set. Defaulting to a blank string.
[+] Creating 9/9
✔ Network pokeshop_default Created 0.0s
✔ Container pokeshop-jaeger-1 Created 0.1s
✔ Container pokeshop-db-1 Created 0.1s
✔ Container pokeshop-cache-1 Created 0.1s
✔ Container pokeshop-queue-1 Created 0.1s
✔ Container pokeshop-tracetest-agent-1 Created 0.1s
✔ Container pokeshop-otel-collector-1 Created 0.1s
✔ Container pokeshop-worker-1 Created 0.1s
✔ Container pokeshop-api-1 Created 0.1s
[+] Running 8/8
✔ Container pokeshop-queue-1 Healthy 7.7s
✔ Container pokeshop-cache-1 Healthy 2.7s
✔ Container pokeshop-db-1 Healthy 2.7s
✔ Container pokeshop-tracetest-agent-1 Started 0.5s
✔ Container pokeshop-jaeger-1 Healthy 2.1s
✔ Container pokeshop-otel-collector-1 Started 0.1s
✔ Container pokeshop-api-1 Started 0.2s
✔ Container pokeshop-worker-1 Started 0.1s

/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io

execution: local
script: /import-pokemon.js
output: xk6-tracetest-output (TestRunID: 53770)

scenarios: (100.00%) 1 scenario, 1 max VUs, 35s max duration (incl. graceful stop):
* default: 1 looping VUs for 5s (gracefulStop: 30s)

View these tests in our Demo environment

What's Next?

After running the initial set of tests, you can click the run link for any of them, update the assertions and run the scripts once more. This flow enables complete a trace-based TDD flow.

assertions

Learn More

Please visit our examples in GitHub and join our Slack Community for more info!