Node.js and Azure Application Insights with OpenTelemetry Collector

note

Check out the source code on GitHub here.

Tracetest is a testing tool based on OpenTelemetry that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.

Azure Application Insights is an extension of Azure Monitor and provides application performance monitoring (APM) features. APM tools are useful to monitor applications from development, through test, and into production in the following ways:

• Proactively understand how an application is performing.
• Reactively review application execution data to determine the cause of an incident.

OpenTelemetry Collector Contrib - The official OpenTelemetry Distribution for packages outside of the core collector.

Sample Node.js App with Azure Application Insights, OpenTelemetry Collector and Tracetest

This is a simple quick start guide on how to configure a Node.js app to use instrumentation with traces and Tracetest for enhancing your E2E and integration tests with trace-based testing. The infrastructure will use Azure App Insights as the trace data store, the OpenTelemetry Collector to process and route the telemetry data and a Node.js app to generate it.

Prerequisites

You will need Docker and Docker Compose installed on your machine to run this quick start app!

And the App Insights Instrumentation Key from your instance.

Project Structure

The project is built with Docker Compose.

1. Node.js App

The Dockerfile and the docker-compose.yaml in the root directory is for the Node.js app.

2. Tracetest

The docker-compose.yaml file, tracetest.provision.yaml, and tracetest-config.yaml in the tracetest directory are for the setting up the Node.js App and Tracetest.

Docker Compose Network

All services in the docker-compose.yaml are on the same network and will be reachable by hostname from within other services.

Node.js App

The Node.js app is a simple Express app, contained in the src/index.js file.

It is instrumented using the Official OpenTelemetry Node.js SDK wrapping the application code to send telemetry data to the OpenTelemetry Collector.

The following is the instrumentation code from the src/tracing.js file.

const opentelemetry = require('@opentelemetry/sdk-node')
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node')
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc');

const sdk = new opentelemetry.NodeSDK({
  traceExporter: new OTLPTraceExporter({ url: 'http://otel-collector:4317' }),
  instrumentations: [getNodeAutoInstrumentations()],
  serviceName: 'tracetest-azure-app-insights-collector'
})
sdk.start()

To start the server, run this command:

npm start

As you can see the Dockerfile uses the command above.

FROM node:slim
WORKDIR /usr/src/app

COPY ./src/package*.json ./

RUN npm install
COPY ./src .

EXPOSE 3000
CMD [ "npm", "start" ]

Tracetest

The docker-compose.yaml includes three other services.

• Postgres - Postgres is a prerequisite for Tracetest to work. It stores trace data when running the trace-based tests.
• Tracetest - Trace-based testing that generates end-to-end tests automatically from traces.
• OpenTelemetry Collector Contrib - The official Open Telemetry Distribution for packages outside of the core collector.

services:
  postgres:
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: postgres
    healthcheck:
      test:
        - CMD-SHELL
        - pg_isready -U "$$POSTGRES_USER" -d "$$POSTGRES_DB"
      timeout: 5s
      interval: 1s
      retries: 60
    image: postgres:14
    networks:
      default: null
    ports:
      - mode: ingress
        target: 5432
        published: 5432
        protocol: tcp
  tracetest:
    command: --provisioning-file /app/provision.yaml
    platform: linux/amd64
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      TRACETEST_DEV: ${TRACETEST_DEV}
    extra_hosts:
      host.docker.internal: host-gateway
    healthcheck:
      test:
        - CMD
        - wget
        - --spider
        - localhost:11633
      timeout: 3s
      interval: 1s
      retries: 60
    image: kubeshop/tracetest:${TAG:-latest}
    networks:
      default: null
    ports:
      - mode: ingress
        target: 11633
        published: 11633
        protocol: tcp
    volumes:
      - type: bind
        source: tracetest/tracetest.yaml
        target: /app/tracetest.yaml
      - type: bind
        source: tracetest/tracetest-provision.yaml
        target: /app/provision.yaml
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    command:
      - "--config"
      - "/otel-local-config.yaml"
    volumes:
      - ./collector.config.yaml:/otel-local-config.yaml
    environment:
      INSTRUMENTATION_KEY: ${INSTRUMENTATION_KEY}
    ports:
      - 4317:4317
networks:
  default:
    name: _default

Tracetest depends on Postgres and requires config files to be loaded via a volume. The volumes are mapped from the root directory into the root directory and the respective config files.

The tracetest.config.yaml file contains the basic setup of connecting Tracetest to the Postgres instance.

postgres:
  host: postgres
  user: postgres
  password: postgres
  port: 5432
  dbname: postgres
  params: sslmode=disable

The tracetest.provision.yaml file defines the trace data store, set to Azure App Insights using the collector connection, meaning the traces will be sent to the OpenTelemetry collector to be processed and routed to both Tracetest and the Azure cloud.

But how does Tracetest fetch traces?

The OpenTelemetry collector is configured with the azuremonitor and the otlp/tracetest exporter and sending telemetry data to both the Azure Cloud and Tracetest.

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:

exporters:
  azuremonitor:
    instrumentation_key: ${INSTRUMENTATION_KEY}
  otlp/tracetest:
    endpoint: tracetest:4317
    tls:
      insecure: true

service:
  pipelines:
    traces/tracetest:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tracetest]
    traces/appinsights:
      receivers: [otlp]
      exporters: [azuremonitor]

The tracetest/tracetest.provision.yaml file defines the trace data store, set to the Azure App Insights with the collector as connection type.

type: DataStore
spec:
  name: azureappinsights
  type: azureappinsights
  azureappinsights:
    connectionType: collector
    useAzureActiveDirectoryAuth: false

Run Both the Node.js App and Tracetest

To start both the Node.js app and Tracetest, run this command:

docker compose -f ./docker-compose.yaml -f ./tracetest/docker-compose.yaml up -d

This will start your Tracetest instance on http://localhost:11633/. Open it and start creating tests! Make sure to use the http://app:3000/ URL in your test creation because your Node.js app and Tracetest are in the same network.

Learn More

Please visit our examples in GitHub and join our Slack Community for more info!