Performance and Trace-Based Tests with Tracetest and k6
Tracetest is a testing tool based on OpenTelemetry that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.
k6 is an open-source performance testing tool used for testing the performance of APIs, microservices, and websites. It is designed to be developer-centric, making it easy for developers to write and maintain performance tests as code.
Why is this important?​
k6 is it's a great tool in its own right that allows you to replicate most of the production challenges you might encounter. But, as with all of the tools that only test the initial transaction between the client side and the server, you can only run validations against the immediate response from the service.
The k6 Tracetest Extension​
With the k6 Tracetest extension, you will unlock the power of OpenTelemetry that allows you to run deeper testing based on the traces and spans generated by each of the checkpoints that you define within your services.
Language and Vendor agnostic, with this extension you can use your existing Tracing Data Store and Setup to leverage the work you have already done to instrument your services.
How It Works​
The following is high level sequence diagram on how k6 and Tracetest interact with the different pieces of the system.
Today You'll Learn About Performance & Trace-Based Tests​
This is a simple quick start guide on how to run the Tracetest k6 extension to run enhanced performance tests with trace-based testing. The infrastructure will use the Pokeshop Demo as a testing ground, triggering requests against it and generating telemetry data.
Requirements​
Tracetest Account:
- Sign up to
app.tracetest.ioor follow the get started docs. - Create an environment.
- Create an environment token.
- Have access to the environment's agent API key.
Docker:
- Have Docker and Docker Compose installed on your machine.
Run This Quckstart Example​
Clone the official Tracetest Pokeshop Demo App Repo to your local machine.
git clone https://github.com/kubeshop/pokeshop.git
cd pokeshop
Follow these instructions to run the Cypress example:
- Copy the
.env.templatefile to.env. - Log into the Tracetest app.
- This example is configured to use Jaeger. Ensure the environment you will be utilizing to run this example is also configured to use the Jaeger Tracing Backend by clicking on Settings, Tracing Backend, Jaeger, updating the URL to
jaeger:16685, Test Connection and Save. - Fill out the token and agent API key details by editing your
.envfile. You can find these values in the Settings area for your environment. - Change the
POKESHOP_DEMO_URLtohttp://api:8081in the.envfile. - Run
docker compose -f docker-compose.yml -f docker-compose.k6.yml run k6-tracetest.
Follow along with the sections below for an in detail breakdown of what the example you just ran did and how it works.
Project Structure​
The project is built with Docker Compose.
Pokeshop Demo App​
The Pokeshop Demo App is a complete example of a distributed application using different backend and front-end services, implementation code is written in Typescript.
The docker-compose.yml file in the root directory is for the Pokeshop Demo app and the OpenTelemetry setup.
And the docker-compose.k6.yml includes the Tracetest Agent and the K6 Load Tests.
Finally, the k6 load tests can be found in test/k6/import-pokemon.js.
Installing the k6 Tracetest Extension​
Installing the k6 Tracetest extension is as easy as running the following command:
- Linux
- MacOS
- Windows
docker run --rm -it -u "$(id -u):$(id -g)" -v "${PWD}:/xk6" \
grafana/xk6 build v0.43.1 \
--with github.com/kubeshop/xk6-tracetest
docker run --rm -it -e GOOS=darwin -u "$(id -u):$(id -g)" -v "${PWD}:/xk6" \
grafana/xk6 build v0.43.1 \
--with github.com/kubeshop/xk6-tracetest
docker run --rm -it -e GOOS=windows -u "$(id -u):$(id -g)" -v "${PWD}:/xk6" `
grafana/xk6 build v0.43.1 --output k6.exe `
--with github.com/kubeshop/xk6-tracetest
This will generate the k6-tracetest binary in the current directory.
The instructions can be also found in the main k6 docs in case you need to combine multiple extensions into one binary.
Using the Tracetest Extension For Load Tests​
Once you have installed the k6 Tracetest binary, you can use the base k6 functionality to run load tests against instrumented services and Tracetest to run checks against the resulting telemetry data.
Creating Your k6 Script​
The test/k6/import-pokemon.js file contains the k6 script that will be used to run the performance tests. It is a simple script that will trigger a POST request to the /pokemon/import Pokeshop Demo app endpoint.
import { Http, Tracetest } from 'k6/x/tracetest';
import { sleep } from 'k6';
export const options = {
vus: 1,
duration: '5s',
};
const POKESHOP_DEMO_URL = __ENV.POKESHOP_DEMO_URL || 'http://localhost:8081';
const http = new Http();
const tracetest = Tracetest();
let pokemonId = 6; // charizard
export default function () {
const url = `${POKESHOP_DEMO_URL}/pokemon/import`;
const payload = JSON.stringify({
id: pokemonId++,
});
const params = {
headers: {
'Content-Type': 'application/json',
},
};
// tracetest test definition
const definition = `type: Test
spec:
id: k6-tracetest-pokeshop-import-pokemon
name: K6
description: K6
trigger:
type: k6
specs:
- selector: span[tracetest.span.type="general" name="import pokemon"]
name: Should have imported the pokemon
assertions:
- attr:tracetest.selected_spans.count = 1
- selector: |-
span[tracetest.span.type="http" net.peer.name="pokeapi.co" http.method="GET"]
name: Should trigger a request to the POKEAPI
assertions:
- attr:http.url = "https://pokeapi.co/api/v2/pokemon/${pokemonId}"
`;
const response = http.post(url, payload, params);
tracetest.runTest(
response.trace_id,
{
definition,
should_wait: true,
},
{
url,
method: 'GET',
}
);
sleep(1);
}
export function handleSummary() {
return {
stdout: tracetest.summary(),
};
}
export function teardown() {
tracetest.validateResult();
}
Setting the Environment Variables​
Copy the .env.template content into a new .env file.
cp .env.template .env
Add the Tracetest API Token and Tracetest Agent API Key to the TRACETEST_API_TOKEN and TRACETEST_AGENT_API_KEY variables.
TRACETEST_API_TOKEN=<YOUR_API_TOKEN> # your environment token
TRACETEST_AGENT_API_KEY=<YOUR_AGENT_API_KEY>
POKESHOP_DEMO_URL=http://api:8081
Starting the Pokeshop Demo App​
Having the full setup ready, the final step is to run the k6 script. To do that, run the following command:
docker compose -f docker-compose.yml -f docker-compose.k6.yml run k6-tracetest
WARN[0000] The "TRACETEST_SERVER_URL" variable is not set. Defaulting to a blank string.
[+] Creating 9/9
✔ Network pokeshop_default Created 0.0s
✔ Container pokeshop-jaeger-1 Created 0.1s
✔ Container pokeshop-db-1 Created 0.1s
✔ Container pokeshop-cache-1 Created 0.1s
✔ Container pokeshop-queue-1 Created 0.1s
✔ Container pokeshop-tracetest-agent-1 Created 0.1s
✔ Container pokeshop-otel-collector-1 Created 0.1s
✔ Container pokeshop-worker-1 Created 0.1s
✔ Container pokeshop-api-1 Created 0.1s
[+] Running 8/8
✔ Container pokeshop-queue-1 Healthy 7.7s
✔ Container pokeshop-cache-1 Healthy 2.7s
✔ Container pokeshop-db-1 Healthy 2.7s
✔ Container pokeshop-tracetest-agent-1 Started 0.5s
✔ Container pokeshop-jaeger-1 Healthy 2.1s
✔ Container pokeshop-otel-collector-1 Started 0.1s
✔ Container pokeshop-api-1 Started 0.2s
✔ Container pokeshop-worker-1 Started 0.1s
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: /import-pokemon.js
output: xk6-tracetest-output (TestRunID: 53770)
scenarios: (100.00%) 1 scenario, 1 max VUs, 35s max duration (incl. graceful stop):
* default: 1 looping VUs for 5s (gracefulStop: 30s)
Goja stack:
[RunGroup=#nE27wLbIR, Status=failed] - https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/run/nE27wLbIR
[TotalRuns=5, SuccessfulRus=0, FailedRuns=5]
[FAILED Request=GET - http://api:8081/pokemon/import, TraceID=dc07188eb8bab0ea31d6f201f78aa048, RunState=TRIGGER_FAILED FailingSpecs=true, TracetestURL= https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/k6-tracetest-pokeshop-import-pokemon/run/11, LastError=cannot get trigger type "k6": triggerer type not found]
[FAILED Request=GET - http://api:8081/pokemon/import, TraceID=dc0718abc0bab0ea31ca7c85e07c24ff, RunState=TRIGGER_FAILED FailingSpecs=true, TracetestURL= https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/k6-tracetest-pokeshop-import-pokemon/run/12, LastError=cannot get trigger type "k6": triggerer type not found]
[FAILED Request=GET - http://api:8081/pokemon/import, TraceID=dc071896c8bab0ea31ce390164c93b9a, RunState=TRIGGER_FAILED FailingSpecs=true, TracetestURL= https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/k6-tracetest-pokeshop-import-pokemon/run/13, LastError=cannot get trigger type "k6": triggerer type not found]
[FAILED Request=GET - http://api:8081/pokemon/import, TraceID=dc071881d0bab0ea31b55c880c6665ba, RunState=TRIGGER_FAILED FailingSpecs=true, TracetestURL= https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/k6-tracetest-pokeshop-import-pokemon/run/14, LastError=cannot get trigger type "k6": triggerer type not found]
[FAILED Request=GET - http://api:8081/pokemon/import, TraceID=dc0718ebd7bab0ea31059f337a355a79, RunState=TRIGGER_FAILED FailingSpecs=true, TracetestURL= https://app.tracetest.io/organizations/ttorg_2179a9cd8ba8dfa5/environments/ttenv_231b49e808c29e6a/test/k6-tracetest-pokeshop-import-pokemon/run/15, LastError=cannot get trigger type "k6": triggerer type not found]
running (06.3s), 0/1 VUs, 5 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 5s
ERRO[0010] a panic occurred during JS execution: Tracetest: 5 jobs failed
What's Next?​
After running the initial set of tests, you can click the run link for any of them, update the assertions and run the scripts once more. This flow enables complete a trace-based TDD flow.
Learn More​
Please visit our examples in GitHub and join our Slack Community for more info!