OpenTelemetry

OpenTelemetry (OTel) is the standard way to produce traces, metrics and logs without locking the code to one vendor. You instrument the app once with an OTel SDK, or with an auto-instrumentation agent. The SDK sends data to an OpenTelemetry Collector. The collector forwards it to whichever backend you use, such as Jaeger, Tempo, Honeycomb or Datadog. Switching vendors then means changing the collector's settings, not the application.

The big idea

Think of a power adapter. Your laptop has one plug design, and the wall sockets differ by country. The adapter in between means you do not rebuild the laptop to travel. The SDK is the laptop's plug, and the collector is the adapter.

Walking through it

The pieces, in the order data flows through them:

• SDK creates spans and metrics inside the process, and exports them in batches over OTLP, the OpenTelemetry protocol, over gRPC or HTTP.
• Collector receives, batches, scrubs, samples and fans out the data. Run it as a DaemonSet or a gateway, so the apps never talk to a vendor directly.
• Exporters send data to a backend. Keep credentials and sampling policy in the collector, so the app stays simple.

Auto-instrumentation covers HTTP servers, common database clients and gRPC in many languages. You still add manual spans around business steps, such as "authorize merchant" or "reserve inventory". Generic spans will not tell you which domain step was slow.

from opentelemetry import trace

tracer = trace.get_tracer("checkout")

def confirm_payment(order_id: str) -> None:
    with tracer.start_as_current_span("confirm_payment") as span:
        span.set_attribute("order.id", order_id)
        span.set_attribute("payment.provider", "stripe")
        # ... call the provider ...
        span.set_attribute("payment.status", "succeeded")

This is a conceptual fragment: the provider call is left as a comment. Keep attribute values low-cardinality. Use stable values such as payment.status, and never put card numbers or tokens on a span. The privacy rules for logs apply here too.

Pick a backend by the question you ask most often. If you filter raw events by any field combination, a wide-event store fits. If you follow one request ID across services, a trace UI plus structured logs is enough. Logs go to something like Loki or Elasticsearch, traces to Jaeger, Tempo or Honeycomb, and metrics to Prometheus or Mimir.

Common mistakes

• Hardcoding a vendor agent in each app. Send everything to the collector, so one change covers every service.
• Only auto-instrumentation. Add manual spans for the two or three business steps that fail in unusual ways.
• Free-text attributes. Long error strings create millions of distinct values. Use stable enums.

Check yourself

1. Why run the OTel Collector between the apps and the backend? It keeps credentials and sampling in one place, and lets you change the backend without touching the apps.
2. What is wrong with a span named "HTTP 200 in 90 ms"? It says nothing about which business step was slow. Add a span for each domain step.

Key takeaways

• Instrument once with OTel, send to a collector, and choose the backend later.
• The collector handles batching, scrubbing, sampling and routing.
• Add manual spans for domain steps, with stable, low-cardinality attributes.

Further learning

The OpenTelemetry documentation is the canonical reference for SDKs, the collector and context propagation.

OpenTelemetry documentationOpenTelemetry, also known as OTel, is a vendor-neutral open source Observability framework for instrumenting, generating, collecting, and exporting telemetry data such as traces, metrics, and logs. As an industry-stan…OpenTelemetry

For the Kubernetes and Jaeger walkthrough, this video shows the pieces working together.