Skip to main content

Trace-Based Tests with the Tracetest GraphQL Trigger

Version Compatibility

The features described here are compatible with the Tracetest CLI v1.5.2 and above.

note

Check out the source code on GitHub here.

Tracetest is a testing tool based on OpenTelemetry that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions. GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

Why is this important?​

The Tracetest GraphQL trigger enables you to unleash the power of the trace-based testing to easily capture a full distributed trace from your OpenTelemetry instrumented GraphQL back-end system.

By creating a Tracetest GraphQL test, you will be able to create trace-based assertions to be applied across the entire flow like any other Tracetest test. Not only that but it allows you to mix and match it with your existing Monitors, Test Suites and CI/CD validations.

Other impactful benefits of using traces as test specs are:

  • Faster MTTR for failing performance tests.
  • Assert against the Mutiple Queries and Mutations at once from a single test execution.
  • Validate functionality of other parts of your system that may be broken, even when the initial request is passing.

Requirements​

Tracetest Account:

Docker: Have Docker and Docker Compose installed on your machine.

Run This Example​

The example below is provided as part of the Tracetest GitHub repo. You can download and run the example by following these steps:

git clone https://github.com/kubeshop/tracetest
cd tracetest/examples/tracetest-jaeger-graphql-pokeshop

Follow these instructions to run the quick start:

  1. Copy the .env.template file to .env.
  2. Fill out the TRACETEST_TOKEN and ENVIRONMENT_ID details by editing your .env file.
  3. Run docker compose run tracetest-run.
  4. Follow the links in the output to view the test results.

Project Structure​

The project structure for running Tracetest GraphQL tests is as follows:

.env.template
.gitignore
.Dockerfile
collector.config.yaml
docker-compose.yaml
/resources
  apply.sh
  datastore.yaml
  test.yaml
  run.sh
  scheme.graphql
  query.graphql

The Pokeshop Demo App is a complete example of a distributed application using different back-end and front-end services. We will be launching it and running tests against it as part of this example. The docker-compose.yaml file in the root directory of the quick start runs the Pokeshop Demo app, the OpenTelemetry Collector, Jaeger, and the Tracetest Agent setup.

The Tracetest resource definitions and scripts are defined under the /resources directory. The resources include tests and the tracing backend definition, while the scripts include the apply.sh and run.sh scripts to apply the resources and run the tests.

Provisioned Resources​

The example provisions the following resources:

Import Pokemon Test​

resources/test.yaml
type: Test
spec:
  id: re9XOxqSR
  name: Pokeshop - Import
  trigger:
    type: graphql
    graphql:
      url: http://demo-api:8081/graphql
      headers:
        - key: Content-Type
          value: application/json
      auth:
        apiKey: {}
        basic: {}
        bearer: {}
      body:
        query: ./query.graphql
        variables: {}
        operationName: ""
      sslVerification: false
      schema: ./schema.graphql
  specs:
    - name: Import Pokemon Span Exists
      selector: span[tracetest.span.type="general" name="import pokemon"]
      assertions:
        - attr:tracetest.selected_spans.count = 1
    - name: Uses Correct PokemonId
      selector: span[tracetest.span.type="http" name="GET" http.method="GET"]
      assertions:
        - attr:http.url  =  "https://pokeapi.co/api/v2/pokemon/6"
    - name: Matching db result with the Pokemon Name
      selector: span[tracetest.span.type="database" name="create postgres.pokemon"]:first
      assertions:
        - attr:db.result  contains      "charizard"

GraphQL​

resources/query.graphql
mutation import {
  importPokemon(id: 6) {
    id
  }
}

GraphQL Schema​

resources/schema.graphql
schema {
  query: Query
  mutation: Mutation
}

type Pokemon {
  id: Int
  name: String!
  type: String!
  isFeatured: Boolean!
  imageUrl: String
}

"The `Int` scalar type represents non-fractional signed whole numeric values. Int can represent values between -(2^31) and 2^31 - 1."
scalar Int

"The `String` scalar type represents textual data, represented as UTF-8 character sequences. The String type is most often used by GraphQL to represent free-form human-readable text."
scalar String

"The `Boolean` scalar type represents `true` or `false`."
scalar Boolean

type PokemonList {
  items: [Pokemon]
  totalCount: Int
}

type ImportPokemon {
  id: Int!
}

type Query {
  getPokemonList(where: String, skip: Int, take: Int): PokemonList
}

type Mutation {
  createPokemon(name: String!, type: String!, isFeatured: Boolean!, imageUrl: String): Pokemon!
  importPokemon(id: Int!): ImportPokemon!
}
// ...See more in the schema.graphql file

Jaeger Tracing Backend​

resources/datastore.yaml
type: DataStore
spec:
  id: current
  name: jaeger
  type: jaeger
  default: true
  jaeger:
    endpoint: jaeger:16685
    headers:
      "": ""
    tls:
      insecure: true

The Apply Script​

The apply script configures and provisions the resources in the Tracetest environment:

resources/apply.sh
#!/bin/sh

set -e

TOKEN=$TRACETEST_API_KEY
ENVIRONMENT_ID=$TRACETEST_ENVIRONMENT_ID

apply() {
  echo "Configuring TraceTest"
  tracetest configure --token $TOKEN --environment $ENVIRONMENT_ID

  echo "Applying Resources"
  tracetest apply datastore -f /resources/datastore.yaml
  tracetest apply test -f /resources/test.yaml
}

apply

The Run Script​

The run script runs the test suite against the provisioned resources:

resources/run.sh
#!/bin/sh

set -e

TOKEN=$TRACETEST_API_KEY
ENVIRONMENT_ID=$TRACETEST_ENVIRONMENT_ID

run() {
  echo "Configuring Tracetest"
  tracetest configure --token $TOKEN --environment $ENVIRONMENT_ID

  echo "Running Trace-Based Tests..."
  tracetest run test -f /resources/test.yaml
}

run

Setting the Environment Variables​

Copy the .env.template file to .env and add the Tracetest API token and agent tokens to the TRACETEST_API_TOKEN and TRACETEST_ENVIRONMENT_ID variables.

Running the Full Example​

Everything is automated for you to only run the following command:

docker compose run tracetest-run

This command will run the apply.sh script to provision the resources and the run.sh script to run the test suite.

Finding the Results​

The output from the Tracetest Engine script should be visible in the console log after running the test command.

WARN[0000] /Users/oscar/Documents/kubeshop/t/examples/tracetest-jaeger-graphql-pokeshop/docker-compose.yaml: `version` is obsolete
[+] Running 2/2
 ✔ demo-api Pulled                                                                                                                                                0.9s
 ✔ demo-worker Pulled                                                                                                                                             0.9s
[+] Creating 10/9
 ✔ Network tracetest-jaeger-graphql-pokeshop_default              Created                                                                                         0.0s
 ✔ Container tracetest-jaeger-graphql-pokeshop-postgres-1         Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-cache-1            Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-agent-1  Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-queue-1            Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-jaeger-1           Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-demo-worker-1      Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-otel-collector-1   Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-demo-api-1         Created                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-apply-1  Created                                                                                         0.0s
[+] Running 9/9
 ✔ Container tracetest-jaeger-graphql-pokeshop-cache-1            Healthy                                                                                        10.5s
 ✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-agent-1  Started                                                                                         0.3s
 ✔ Container tracetest-jaeger-graphql-pokeshop-jaeger-1           Healthy                                                                                         1.9s
 ✔ Container tracetest-jaeger-graphql-pokeshop-postgres-1         Healthy                                                                                        10.5s
 ✔ Container tracetest-jaeger-graphql-pokeshop-queue-1            Healthy                                                                                        10.5s
 ✔ Container tracetest-jaeger-graphql-pokeshop-otel-collector-1   Started                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-demo-worker-1      Started                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-demo-api-1         Started                                                                                         0.1s
 ✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-apply-1  Started                                                                                         0.1s
[+] Running 2/2
 ✔ demo-api Pulled                                                                                                                                                0.8s
 ✔ demo-worker Pulled                                                                                                                                             0.8s
Configuring Tracetest
 SUCCESS  Successfully configured Tracetest CLI
Running Trace-Based Tests...
✔ RunGroup: #US5klbqSR (https://app-stage.tracetest.io/organizations/ttorg_c71a6b53c3709e95/environments/ttenv_bcf29b43f06a12dc/run/US5klbqSR)
 Summary: 1 passed, 0 failed, 0 pending
  ✔ Pokeshop - Import (https://app-stage.tracetest.io/organizations/ttorg_c71a6b53c3709e95/environments/ttenv_bcf29b43f06a12dc/test/re9XOxqSR/run/11/test) - trace id: 6facf84ee23757eda97930c16fd1d8f9
	✔ Import Pokemon Span Exists
	✔ Uses Correct PokemonId
	✔ Matching db result with the Pokemon Name

What's Next?​

After running the test, you can click the run link, update the assertions, and run the scripts once more. This flow enables complete a trace-based TDD flow.

assertions

Learn More​

Please visit our examples in GitHub and join our Slack Community for more info!