Create Assertions

Now you know how to generate a trace for every test. This page explains how to create assertions by using real trace data from your distributed trace spans. Assertions are the most crucial part of a test. They allow you to validate that your system behaves as expected.

An assertion consists of:

• Selector
• List of assertions
• Name (Optional)

- selector: span[tracetest.span.type="http" name="POST /pokemon/import" http.method="POST"]
    name: POST /pokemon/import was called successfuly
    assertions:
    - attr:http.status_code  =  200
    - attr:http.response.body | json_path '$.id' = "143"

Selector

The selector defines against which spans assertions will be executed. Selectors can match any number of spans—zero, one, twenty, or an unknown number.

Assertions

Assertions are the validations you define. Here are some examples:

• Verify that the HTTP response code of the trigger was 200.
• Parse the response JSON and confirm that the returned object has a non-empty ID field.
• Use JSON path on the response body to get specific fields, validate list length, perform regex matches, etc.
• Check response times.

The biggest value comes from analysing the trace itself. For instance:

• Ensure that any database query took less than 100ms.
• Verify if a specific service was called, and even validate how many times it was invoked.
• Examine spans generated by an asynchronous process after the request passes through a message queue like Kafka, RabbitMQ, etc.
• Validate external asynchronous HTTP requests.

Create Assertions in Two Ways

1. Programmatically, in YAML
2. Visually, in the Web UI

Create Assertions Programatically in YAML

Taking the test you ran in the previous section, you can add assertions with a specs block.

type: Test
spec:
  name: Import a Pokemon using API and MQ Worker
  description: Import a Pokemon
  # Define the trigger
  trigger:
    type: http
    httpRequest:
      method: POST
      # Define the app endpoint
      url: ${var:POKESHOP_API_URL}/pokemon/import
      body: |
        {
          "id": 143
        }
      headers:
      - key: Content-Type
        value: application/json
  
  # Define the assertions to be applied against the resulting trace
  specs:
  - selector: span[tracetest.span.type="http" name="POST /pokemon/import" http.method="POST"]
    name: POST /pokemon/import was called successfuly
    assertions:
    - attr:http.status_code  =  200
    - attr:http.response.body | json_path '$.id' = "143"
  
  - selector: span[tracetest.span.type="general" name="validate request"]
    name: The request was validated correctly
    assertions:
    - attr:validation.is_valid = "true"
  
  - selector: span[tracetest.span.type="messaging" name="queue.synchronizePokemon publish"]
    name: A message was enqueued to the worker
    assertions:
    - attr:messaging.payload | json_path '$.id' = "143"
  
  - selector: span[tracetest.span.type="messaging" name="queue.synchronizePokemon process"]
    name: A message was read by the worker
    assertions:
    - attr:messaging.payload | json_path '$.fields.routingKey' = "queue.synchronizePokemon"
  
  - selector: span[tracetest.span.type="general" name="import pokemon"]
    name: A "import pokemon" action was triggered
    assertions:
    - attr:tracetest.selected_spans.count >= 1

  - selector: span[tracetest.span.type="database"]
    name: All Database Spans Processing time is less than 500ms
    assertions:
    - attr:tracetest.span.duration < 500ms
  
  - selector: span[tracetest.span.type="database" name="create pokeshop.pokemon"]
    name: Validate exactly 1 database create operation was executed
    assertions:
    - attr:db.operation = "create"
    - attr:tracetest.selected_spans.count = 1

  - selector: span[tracetest.span.type="database" name="get pokemon_143"]
    name: Validate the cache was not hit.
    assertions:
    - attr:cache.hit = "false"

  - selector: span[tracetest.span.type="database" name="delete pokeshop.pokemon"]
    name: Validate that no database DELETE operation was made
    assertions:
    - attr:tracetest.selected_spans.count = 0

The Assertions You Defined

1. Validating the response of the HTTP request by validating both the response status code and the response body.
2. Validate the attribute is_valid is set to true in a custom span called validate request.
3. Validate a message with an id of 143 was added to a message queue.
4. Validate a message with a routingKey of queue.synchronizePokemon was read by a message queue.
5. Validate a span called import pokemon exists - Validate an action happened.
6. Validate that all the involved database operations took less than 500ms.
7. Validate that there was exactly one database create operation, that the cache was not hit, and that there were zero delete operations.

Create Assertions Visually with the Web UI

Add assertions quickly by using the assertion snippets, or selecting a span attribute and clicking Create test spec.

This opens the assertions engine.

1. Validating the response of the HTTP request by validating both the response status code and the response body.

2. Validate the attribute is_valid is set to true in a custom span called validate request.

3. Validate a message with an id of 143 was added to a message queue.

4. Validate a message with a routingKey of queue.synchronizePokemon was read by a message queue.

5. Validate a span called import pokemon exists - Validate an action happened.

6. Validate that all the involved database operations took less than 500ms.

7. Validate that there was exactly one database create operation, that the cache was not hit, and that there were zero delete operations.

Why Tracetest Assertions are Powerful

1. You wrote ZERO lines of programming code. It's all defined in YAML.
2. You can save them as part of your GitHub repo because it's defined in YAML.
3. Tracetest has NO access to your database, logs, or any other confidential infrastructure information. Assertions only validate the distributed traces your apps export.
4. Tracetest DOES NOT care about how many services were called internally, or which languages your services are written in, or what database you're using. Tracetest only cares about the distributed trace data you expose to use as assertions.
5. Tracetest can validate that things that DID or DID NOT happen, with just 3 lines of YAML!

How Do Selectors Work?

Selectors works in a similar way as CSS selectors. You can select spans by matching their attributes. Attributes can be generic, like span type, duration, or be defined by your apps.

Here are some examples:

# all database spans
- selector: span[tracetest.span.type="database"]

# all http serving spans
- selector: span[tracetest.span.type="http"]

# all spans that have the custom defined attribute of myapp.user_id
- selector: span[myapp.user_id="123"]

# all spans that have the custom defined attribute of myapp.user_id that are database operations
- selector: span[tracetest.span.type="database" myapp.user_id="123"]

# all spans that have the custom defined attribute of myapp.user_id that are database operations that are children of the user service
- selector: span[service.name="myapp-user-service"] span[tracetest.span.type="database" myapp.user_id="123"]

Check the full selector docs, here.

How Do Assertions Work?

Assertions are the checks you can apply to the values of all the spans matched by the related selector.

Here are some examples:

# validate that all spans from myapp-user-service have a non empty user_id attribute
- selector: span[service.name="myapp-user-service"]
  assertions:
  - attr:myapp.user_id != ""

- selector: span[tracetest.span.type="http" name="GET /pokemon?take=10&skip=0" http.method="GET"]
  assertions:
  - attr:http.status_code = 200
  - attr:tracetest.response.body | json_path '$.items' | length <= 10

Check the full assertions docs, here.

How Do Expressions Work?

Tracetest allows you to add expressions when writing test specs.

General Span Attributes

• tracetes.span.duration
• tracetest.response.body
• and more.

Filters

• json_path
• lenght
• regex
• get_index
• and more.

String Interpolation

• ${}

Check the full expressions docs, here.

Don't have OpenTelemetry installed?

Follow these instructions to install OpenTelemetry in 5 minutes without any code changes!