In distributed systems, multiple services often work together to handle user requests. These systems typically use protocols like gRPC for efficient inter-service communication and Express.jsfor REST APIs, creating a hybrid application structure. Understanding how requests flow through such systems and diagnosing performance bottlenecks can be challenging without proper observability tools.

OpenTelemetry is a powerful framework for collecting telemetry data, such as traces, metrics, and logs. It provides end-to-end visibility into your system, enabling you to monitor performance, identify issues, and optimize processes. This guide explains how to integrate OpenTelemetry with a Node.js hybrid application that uses gRPC and Express.js. Each step includes code and detailed explanations to ensure a smooth learning experience.

What is OpenTelemetry?

OpenTelemetry is an open-source observability framework designed to standardize the generation, collection, and export of telemetry data. Its primary features include:

- Tracing: Tracks requests across services, APIs, and databases.

- Metrics: Captures system and application performance data.

- Logging: Records events for debugging and analysis.

Why Use OpenTelemetry in Hybrid Applications?

Hybrid applications combine protocols like gRPC and HTTP/REST, making it crucial to monitor request flow across both. OpenTelemetry simplifies observability by:

1. Providing End-to-End Tracing: Tracks a request from its origin (Express.js) through its journey to a gRPC service.
2. Automatic Context Propagation: Ensures all parts of the trace are connected, even when switching between protocols.
3. Integrating with Backends: Works seamlessly with tools like Jaeger, Zipkin, and Grafana for trace visualization.

Prerequisites

Before we dive in, make sure you have the following:

1. Node.js installed (version 14 or higher).

2. Basic understanding of gRPC, Express.js, and the client-server architecture.

3. A tracing backend like Jaeger for viewing traces.

4. Required npm packages installed:
   

npm install @grpc/grpc-js @grpc/proto-loader express @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node @opentelemetry/exporter-trace-otlp-grpc

Setting Up a gRPC Service

We’ll begin by creating a gRPC service that greets users.

Define the Protocol Buffer (greet.proto)
The Protocol Buffer (proto) file describes the structure of your gRPC service and its methods.

syntax = "proto3";

service Greeter {
  rpc Greet (GreetRequest) returns (GreetResponse);
}

message GreetRequest {
  string name = 1;
}

message GreetResponse {
  string message = 1;
}

How the code works: The service Greeter defines a service with a single method Greet. Greet accepts a GreetRequest message with a name field and returns a GreetResponse message containing a message field. The proto3 syntax ensures compatibility with modern tools and libraries.

Implement the gRPC Server (grpc-server.js)

const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');

const PROTO_PATH = './greet.proto';
const packageDefinition = protoLoader.loadSync(PROTO_PATH);
const greetProto = grpc.loadPackageDefinition(packageDefinition).Greeter;

const server = new grpc.Server();

server.addService(greetProto.service, {
  Greet: (call, callback) => {
    console.log(`Received request for: ${call.request.name}`);
    callback(null, { message: `Hello, ${call.request.name}!` });
  },
});

server.bindAsync('localhost:50051', grpc.ServerCredentials.createInsecure(), () => {
  console.log('gRPC server is running on port 50051');
  server.start();
});

How the code works: Import Modules: The @grpc/grpc-js module provides gRPC functionalities, and @grpc/proto-loader loads the greet.proto file.
Load Proto File: The loadSync function parses the proto file and generates a JavaScript object for the service.
Create Server: A gRPC server is created using new grpc.Server().
Add Service: The Greet method is added to the server. It logs the name from the request and sends a greeting message in the response.
Start Server: The server listens on localhost:50051 for incoming requests.

Setting Up an Express Server

Next, we’ll create an Express.js server that acts as a client to the gRPC service.

Express Server (express-server.js)

const express = require('express');
const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');

const PROTO_PATH = './greet.proto';
const packageDefinition = protoLoader.loadSync(PROTO_PATH);
const greetProto = grpc.loadPackageDefinition(packageDefinition).Greeter;

const app = express();
app.use(express.json());

const client = new greetProto('localhost:50051', grpc.credentials.createInsecure());

app.post('/greet', (req, res) => {
  const { name } = req.body;
  client.Greet({ name }, (err, response) => {
    if (err) return res.status(500).send(err.message);
    res.send(response);
  });
});

app.listen(3000, () => {
  console.log('Express server is running at http://localhost:3000');
});

How the code works:
Load Proto File: Similar to the gRPC server, the greet.proto file is loaded to generate a client object.
Create gRPC Client: The greetProto client connects to the gRPC server at localhost:50051.
Setup Endpoint: The /greet POST endpoint accepts a name in the request body and calls the gRPC server using the client.
Handle Response: The gRPC server's response is sent back to the HTTP client.

Adding OpenTelemetry

Integrate OpenTelemetry to trace requests across both servers.

Setup Tracing (tracing.js)

const { NodeSDK } = require('@opentelemetry/sdk-node');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc');

const traceExporter = new OTLPTraceExporter();

const sdk = new NodeSDK({
  traceExporter,
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start().then(() => console.log('OpenTelemetry initialized'));
process.on('SIGTERM', () => sdk.shutdown());

How the code works: Explanation:
NodeSDK: Initializes OpenTelemetry with auto-instrumentation for gRPC and HTTP libraries.
OTLPTraceExporter: Configures OpenTelemetry to send telemetry data to a backend like Jaeger using the OTLP protocol.
Auto-Instrumentation: Automatically tracks traces for supported libraries without needing manual instrumentation.

Include Tracing in Servers

At the top of both grpc-server.js and express-server.js, add:

require('./tracing');

How the code works: This ensures tracing is initialized before the server starts, capturing all requests and responses.

Viewing Traces in Jaeger

Run Jaeger Locally
Start a Jaeger container to visualize traces:

docker run -d --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 -p 4317:4317 \
  jaegertracing/all-in-one:latest

Visit http://localhost:16686 to access the Jaeger UI.

Send a Test Request

curl -X POST http://localhost:3000/greet -H "Content-Type: application/json" -d '{"name": "OpenTelemetry"}'

View Traces

In Jaeger, search for the trace and observe spans for both the HTTP and gRPC requests, showing the full lifecycle of the operation

Conclusion

By integrating OpenTelemetry with gRPC and Express.js in a hybrid Node.js application, you achieve full visibility into request flows. This allows you to debug, monitor, and optimize your services effectively. OpenTelemetry’s simplicity and flexibility make it a critical tool for modern distributed systems.

Start using OpenTelemetry today to gain unparalleled insights into your applications!