Skip to main content

Automatically Provisioning a Developer Environment from the Tracetest CLI

Version Compatibility

The features described here are compatible with the Tracetest CLI v1.2.0 and above.

Tracetest is a testing tool based on OpenTelemetry that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.

Why is this important?

Developer experience is one of the key values we always push forward at Tracetest. This enables teams to have a consistent and reliable way to onboard new developers. By using the Tracetest CLI, you can automate the process of creating a new environment for a developer joining your team. This ensures that the developer has the necessary environment to start working on the project without any manual intervention.

Supported Provisioning Resources

Finding the Resource Definition

Tracetest Definitions are found across the app, for resources under settings you can click the Resource Definition button to find the YAML definition of the resource.

Resource Definition

For Tests and Test Suites, you can find the YAML definition by clicking the Automate tab.

Test Resource Definition

Requirements

Tracetest CLI:

  • Download & Install the Tracetest CLI
  • Login or Signup to your Tracetest account using the CLI with tracetest configure

Run This Quckstart Example

The example below is provided as part of the Tracetest GitHub repo. You can download and run the example by following these steps: Clone the Tracetest project and go to the Privisioning Developer Environment with CLI example directory:

git clone https://github.com/kubeshop/tracetest
cd tracetest/examples/provisioning-developer-environment-cli

Follow these instructions to run Provision an Environment using the example:

  1. Run the tracetest apply environment -f ./environment.yaml command to create the environment and provision the environment.
  2. Run the tracetest list environments command to list the environments.
  3. Configure the Tracetest CLI to point the new environment by running the tracetest configure --organization <your-org-id> --environment automated-pokeshop-demo command.
  4. Execute the test suite by running the tracetest run -f ./resources/suites/pokeshop.yaml --vars tracetesting-vars command.
  5. Follow the link to the Tracetest UI to view the test results.

Project Structure

The project structure for the Provisioning Developer Environment with CLI example is as follows:

environment.yaml
/resources
/tests
add-pokemon.yaml
import-pokemon.yaml
list-pokemon.yaml
/suites
pokeshop.yaml
/config
variableset.yaml
pollingprofile.yaml
runner.yaml
tracing-backend.yaml

The Environment Definition includes a section to specify the resources that will be applied along with the environment. The resources are defined in the /resources directory.

environment.yaml
type: Environment
spec:
id: automated-pokeshop-demo
name: Automated Pokeshop Demo
agentConfiguration:
serverless: true
resources: ./resources

The resources are defined in the /resources directory. The resources include tests, test suites, variable sets, polling profiles, test runners, and tracing backends.

Provisioned Resources

The example provisions the following resources:

Polling Profile

resources/config/pollingprofile.yaml
type: PollingProfile
spec:
id: pokeshop-demo
name: pokeshop-demo
default: true
strategy: periodic
periodic:
retryDelay: 5s
timeout: 1m
selectorMatchRetries: 3

Test Runner

resources/config/runner.yaml
type: TestRunner
spec:
id: current
name: default
requiredGates:
- test-specs

Tracing Backend

resources/config/tracing-backend.yaml
type: DataStore
spec:
id: tracing-backend
name: jaeger
type: jaeger
default: true
jaeger:
endpoint: demo-pokeshop-jaeger-query.demo:16685
tls:
insecure: true
settings: {}

Variable Set

resources/config/variableset.yaml
type: VariableSet
spec:
id: tracetesting-vars
name: tracetesting-vars
values:
- key: POKESHOP_API_URL
value: https://demo-pokeshop.tracetest.io

Tests

resources/tests/add-pokemon.yaml
type: Test
spec:
id: pokeshop-demo-add-pokemon
name: Pokeshop - Add
description: Add a Pokemon
trigger:
type: http
httpRequest:
method: POST
url: ${var:POKESHOP_API_URL}/pokemon
body: |
{
"name": "meowth",
"type":"normal",
"imageUrl":"https://assets.pokemon.com/assets/cms2/img/pokedex/full/052.png",
"isFeatured": true
}
headers:
- key: Content-Type
value: application/json
specs:
- selector: span[tracetest.span.type="http" name="POST /pokemon" http.method="POST"]
name: The POST /pokemon was called correctly
assertions:
- attr:http.status_code = 201
- selector: span[tracetest.span.type="general" name="validate request"]
name: The request sent to API is valid
assertions:
- attr:validation.is_valid = "true"
- selector: span[tracetest.span.type="database" name="create pokeshop.pokemon" db.operation="create" db.sql.table="pokemon"]
name: A Pokemon was inserted on database
assertions:
- attr:db.result | json_path '$.imageUrl' = "https://assets.pokemon.com/assets/cms2/img/pokedex/full/052.png"
resources/tests/import-pokemon.yaml
type: Test
spec:
id: pokeshop-demo-import-pokemon-queue
name: Import a Pokemon using API and MQ Worker
description: Import a Pokemon
trigger:
type: http
httpRequest:
method: POST
url: ${var:POKESHOP_API_URL}/pokemon/import
body: |
{
"id": 143
}
headers:
- key: Content-Type
value: application/json
specs:
- selector: span[tracetest.span.type="http" name="POST /pokemon/import" http.method="POST"]
name: POST /pokemon/import was called successfuly
assertions:
- attr:http.status_code = 200
- attr:http.response.body | json_path '$.id' = "143"
- selector: span[tracetest.span.type="general" name="validate request"]
name: The request was validated correctly
assertions:
- attr:validation.is_valid = "true"
- selector: span[tracetest.span.type="messaging" name="queue.synchronizePokemon publish" messaging.system="rabbitmq" messaging.destination="queue.synchronizePokemon" messaging.operation="publish"]
name: A message was enqueued to the worker
assertions:
- attr:messaging.payload | json_path '$.id' = "143"
- selector: span[tracetest.span.type="messaging" name="queue.synchronizePokemon process" messaging.system="rabbitmq" messaging.destination="queue.synchronizePokemon" messaging.operation="process"]
name: A message was read by the worker
assertions:
- attr:messaging.payload | json_path '$.fields.routingKey' = "queue.synchronizePokemon"
- selector: span[tracetest.span.type="general" name="import pokemon"]
name: A "import pokemon" action was triggered
assertions:
- attr:tracetest.selected_spans.count >= 1
resources/tests/list-pokemon.yaml
type: Test
spec:
id: pokeshop-demo-list-pokemon
name: List Pokemons
description: List Pokemons registered on Pokeshop API
trigger:
type: http
httpRequest:
method: GET
url: ${var:POKESHOP_API_URL}/pokemon?take=100&skip=0
headers:
- key: Content-Type
value: application/json
specs:
- selector: span[tracetest.span.type="http" name="GET /pokemon?take=100&skip=0" http.method="GET"]
name: GET /pokemon endpoint was called and returned valid data
assertions:
- attr:http.status_code = 200
- selector: span[tracetest.span.type="database" name="count pokeshop.pokemon" db.system="postgres" db.name="pokeshop" db.user="ashketchum" db.operation="count" db.sql.table="pokemon"]
name: A count operation was triggered on database
assertions:
- attr:db.operation = "count"
- selector: span[tracetest.span.type="database" name="findMany pokeshop.pokemon" db.system="postgres" db.name="pokeshop" db.user="ashketchum" db.operation="findMany" db.sql.table="pokemon"]
name: A select operation was triggered on database
assertions:
- attr:db.operation = "findMany"

Test Suite

resources/suites/pokeshop.yaml
type: TestSuite
spec:
id: pokeshop-demo-test-suite
name: Pokeshop Demo Test Suite
steps:
- ../tests/add-pokemon.yaml
- ../tests/import-pokemon.yaml
- ../tests/list-pokemons.yaml

Provisioning the Environment

After installing the Tracetest CLI and configuring it with your account, you can run the following command to provision the environment:

tracetest apply environment -f ./environment.yaml

Running the Test Suite

Finally, you can run the test suite by executing the following command:

tracetest configure --environment automated-pokeshop-demo
tracetest run -f ./resources/suites/pokeshop.yaml --vars tracetesting-vars

Learn More

Please visit our examples in GitHub and join our Slack Community for more info!