examples.adoc

Examples

Running the examples

These instructions assume a Unix-like environment, with docker to run Clojure applications and telemetry backend instances in containers. All examples use deps.edn for project configuration.

1. Clone clj-otel repository

   To clone this repository, run the following command

   git clone https://github.com/steffan-westcott/clj-otel.git

2. Get OpenTelemetry instrumentation agent JAR

   Before running any of the examples, the file opentelemetry-javaagent.jar must be downloaded and placed in the examples directory. See the releases page for notes and download links.

   ℹ️	The examples assume opentelemetry-javaagent.jar version 2.0.0 or later is used.

3. Select example

   ⚠️	The examples use :local/root to depend on clj-otel modules. In your own libraries and applications, use :mvn/version instead, as shown in the guides to add manual instrumentation.

   Select an example from the following list and change the current directory to the path shown

   • examples/cube-app

     graph LR
         subgraph docker network
             otel-collector --> jaeger
             otel-collector --remote write--> prometheus
         end
         localhost --> otel-collector

     Loading

     A small application cube-app run with the OpenTelemetry instrumentation agent on localhost, exporting telemetry to a Collector instance.

   • examples/square-app

     graph LR
         subgraph docker network
             otel-collector --> jaeger
             otel-collector --remote write--> prometheus
         end
         localhost --> otel-collector

     Loading

     A small application square-app using autoconfigured OpenTelemetry SDK, run on localhost, exporting telemetry to a Collector instance.

   • examples/factorial-app

     graph LR
         subgraph docker network
             otel-collector --> jaeger
             otel-collector --remote write--> prometheus
         end
         localhost --> otel-collector

     Loading

     A small application factorial-app using programmatically configured OpenTelemetry SDK, run on localhost, exporting telemetry to a Collector instance.

   • examples/divisor-app

     graph LR
         subgraph docker network
             jaeger
             prometheus
         end
         localhost --> jaeger
         prometheus --scrape--> localhost

     Loading

     A small application divisor-app using autoconfigured OpenTelemetry SDK, run on localhost, exporting telemetry directly to Jaeger and Prometheus instances.

   • examples/countries-service

     graph LR
         subgraph docker network
             otel-collector --> jaeger
             otel-collector --remote write--> prometheus
         end
         localhost --> otel-collector

     Loading

     A single HTTP service countries-service run with the OpenTelemetry instrumentation agent on localhost, implemented with Ring middleware, Compojure router, exporting telemetry to a Collector instance.

   • examples/rpg-service

     graph LR
         subgraph docker network
             postgres
             otel-collector --> jaeger
             otel-collector --remote write--> prometheus
         end
         localhost --> postgres
         localhost --> otel-collector

     Loading

     A single HTTP service rpg-service run with the OpenTelemetry instrumentation agent on localhost, implemented with Ring middleware, Reitit router, accessing a PostgreSQL database instance, exporting telemetry to a Collector instance. Load namespace example.rpg-service.repl at the REPL to exercise this example.

   • examples/microservices/auto-instrument/middleware

     graph LR
         subgraph docker network
             sentence-summary-load-gen --> sentence-summary-service
             sentence-summary-service --> word-length-service
             sentence-summary-service & word-length-service --> otel-collector
             otel-collector --> tempo
             otel-collector --remote write--> prometheus
             grafana --> tempo
             grafana --> prometheus
         end

     Loading

     A pair of dependent microservices sentence-summary-service and word-length-service both implemented with Ring middleware, Reitit router, run with the OpenTelemetry instrumentation agent, exporting telemetry to a Collector instance.

   • examples/microservices/auto-instrument/interceptor

     graph LR
         subgraph docker network
             solar-system-load-gen --> solar-system-service
             solar-system-service --> planet-service
             solar-system-service & planet-service --> otel-collector
             otel-collector --> tempo
             otel-collector --remote write--> prometheus
             grafana --> tempo
             grafana --> prometheus
         end

     Loading

     A pair of dependent microservices solar-system-service and planet-service both implemented with Pedestal interceptors, run with the OpenTelemetry instrumentation agent, exporting telemetry to a Collector instance.

   • examples/microservices/manual-instrument/middleware

     graph LR
         subgraph docker network
             puzzle-load-gen --> puzzle-service
             puzzle-service --> random-word-service
             puzzle-service & random-word-service --> otel-collector
             otel-collector --> tempo
             otel-collector --remote write--> prometheus
             grafana --> tempo
             grafana --> prometheus
         end

     Loading

     A pair of dependent microservices puzzle-service and random-word-service both implemented with Ring middleware, Reitit router, not using the agent, exporting telemetry to a Collector instance.

   • examples/microservices/manual-instrument/interceptor

     graph LR
         subgraph docker network
             average-load-gen --> average-service
             average-service --> sum-service
             average-service & sum-service --> otel-collector
             otel-collector --> tempo
             otel-collector --remote write--> prometheus
             grafana --> tempo
             grafana --> prometheus
         end

     Loading

     A pair of dependent microservices average-service and sum-service both implemented with Pedestal interceptors, not using the agent, exporting telemetry to a Collector instance.

4. Start container instances

   compose.yaml is a Docker Compose configuration file that specifies container instances to spin up. These containers may include telemetry backends (Grafana, Jaeger, Prometheus) and the OpenTelemetry Collector. When the Collector is present, otel-collector.yaml is a configuration file that forwards telemetry data to the telemetry backends. The microservice examples include instances for each service and a load generator that automatically exercises the head microservice.

   ℹ️	The microservices examples may take some time to start, particularly those which use the OpenTelemetry instrumentation agent. Wait for the command to complete before proceeding to the next step.

   • For all microservices examples, run one of the following commands corresponding to your choice of implementation for the head microservice:

     SERVER=sync ./up.sh

     SERVER=bound-async ./up.sh

     SERVER=explicit-async ./up.sh

   • For all other examples, run the following command

     ./up.sh

5. Load and exercise example

   • For the microservice examples:

     • A load generator application is included and runs automatically. The load generator issues randomized valid and invalid requests to the head microservice.

     • For altering the running system, all microservice and load generator instances expose an nREPL server for remote access. See the compose.yaml file for details on which port the nREPL server is available. Also see the dev source tree for functions to use in a remote REPL session.

     • The microservices rebuild routes on each request received, so changes will take effect when edited namespaces are reloaded.

   • For the other examples:

     • In the chosen example directory and your favourite Clojure development environment, start a REPL with the otel (and dev when available) alias(es) enabled.

     • Load the example namespace and set as current.

     • Evaluate each expression or enter each terminal command in the comment block at the end of the namespace.

6. View telemetry data in backends

   ℹ️	Traces may appear incomplete until all parts of telemetry data have been received and processed by the telemetry backend. This sometimes impacts recent traces of distributed services, where some but not all contributing services have completed exporting telemetry data.

   ℹ️	The microservices examples use Grafana as the primary backend, with preconfigured dashboards. The other examples use Jaeger and Prometheus.

   • Grafana : Navigate to http://localhost:3000 for the Overview dashboard. Click on the dashboard links for further curated detail. Telemetry data can also be examined by clicking on Home > Explore and selecting tempo (for traces) or prometheus (for metrics).

   • Jaeger : Navigate to http://localhost:16686/search then select an option in the Service dropdown and click the Find Traces button.

   • Prometheus : Navigate to http://localhost:9090/graph then enter a metric name in the expression in the search bar, or click the Open metrics explorer button and select a metric. The Prometheus server is not exposed in examples that use Grafana.

7. Stop container instances

   To stop and tear down the container instances, run the following command:

   ./down.sh

8. Delete example volumes

   The microservice examples cache downloaded dependencies in shared external Docker volumes. After trying out the examples, delete the external Docker volumes with the following commands:

   docker volume rm example.clj-otel.gitlibs
   docker volume rm example.clj-otel.m2