Introduction to OpenTelemetry:

OpenTelemetry is an observability framework for cloud native applications. OpenTelemetry provides a collection of APIs, SDKs, and tools to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.

What is Observability?

Observability is the ability to get insights into the internal state of a system by observing its external outputs. Observability helps teams understand how a system is behaving, detect, investigate and remediate the issues that happen in the system.

Pillars of Observability:

Logs: Structured or unstructured records of timestamped events that occur in the system. Logs helps to trace errors, debugging and providers information about the application behavior at specific points in time.

Metrics: Numerical data points that measure the performance and health of the system. Examples include CPU usage, memory consumption, request rates, and error rates. Metrics provide a quantitative view of the system’s health and performance over time. Metrics are mostly system centric.

Traces: Tracks the path of the requests as they propagate through various services and components of a system. Tracing helps visualize how different parts of a system interact, identify latency bottlenecks, and detect failure points in microservices architectures. Traces are usually provides the user centric data.

What is OpenTelemetry ?

1. OpenTelemetry is a collection of tools, APIs and SDKs.
2. It is used to instrument, generate, collect and export telemetry data (Metrics, Logs and Traces) for analysis to an external backend (like Prometheus, Grafana, Jaeger) in order to understand the software's behavior and performance.
3. OpenTelemetry itself is not a backend system which stores the data, It is a standardized layer in between systems and observability backend like Jaeger, Prometheus or Grafana.
4. It has a broad language support, Some of the supported languages are Java, Go, C#, JavaScript, Python, Rust, Swift.
5. In integrated with popular frameworks and libraries like PostgreSQL, Redis, Kafka, Spring, JDBC, RabbitMQ.

OpenTelemetry Reference Architecture:

Running as an Agent
In the below architecture the collector lives along with application in the same container as an agent or library and pushes the data to a Observability Backend.

Running as a Gateway
In the below architecture, The observability data is sent to an Agent that sends the data to the collector that runs as a Service. It receives spans and metrics exported by one or more agents or library or a tool that is emitted in one of the supported protocols.

The Collector is configured to send data to one or more configured observability backends.

Supported Protocols

• gRPC
• HTTP

Instrumentation Types

1. Manual Instrumentation (Code Bases)
opentelemetry-api and opentelemetry-sdk libraries are available to generate telemetry from your application. Manual instrumentation allows us to have a deeper insight and create a telemetry from our application.
Ref: https://opentelemetry.io/docs/languages/java/

2. Automatic Instrumentation (Configuration Based or Zero Code Based)
Automatic instrumentation is suitable when we cannot or don't want to modify the application to get telemetry. This is done by the agents that runs along with the application (In case of Java it is an agent that is attached to the JVM).
Ref: https://opentelemetry.io/docs/zero-code/java/

Both the manual and automatic instrumentation can be used simultaneously to achieve rich telemetry.