Logo

dev-resources.site

for different kinds of informations.

Why should we use Protobuf in Web API as data transfer protocol.

Published at
8/28/2024
Categories
grpc
protobuf
rest
Author
sreeni5018
Categories
3 categories in total
grpc
open
protobuf
open
rest
open
Author
10 person written this
sreeni5018
open
Why should we use Protobuf in Web API as data transfer protocol.

What is Protobuf?

Protobuf, or Protocol Buffers, is a structured data serialization protocol developed by Google, similar to JSON or XML. Unlike JSON or XML, Protobuf is binary-based, making it not human-readable but highly efficient and fast for server-to-server communication.

Key Features

  • Language Neutral and Platform Independent: Protobuf allows you to define your data structure once using a protocol buffer compiler (protoc), and then generate language-specific code for various programming languages such as C#, Java, Go, Node.js (JavaScript or TypeScript), Ruby, Python, and more.
  • Efficiency: Due to its binary format, Protobuf is faster and more efficient in serialization and deserialization compared to text-based formats like JSON.
  • Widely Used in gRPC: Protobuf is the data serialization format used in gRPC. Google adopts Protobuf for all its service-to-service communications, making it a reliable choice for enterprise adoption.

Example Usage

Protobuf is particularly suitable for applications requiring low latency and high efficiency. Below is a simple example of how to use Protobuf in a C# console application for data serialization and deserialization, and how it compares with JSON.

Defining a Protobuf Message Contract in C

Here, we model a UserInfo message, which is sent to UC to provision Microsoft Teams and add 100 users to it.

syntax = "proto3";
message UserInfo {
  int32 user_id = 1;
  string user_name = 2;
  string email = 3;
}
Enter fullscreen mode Exit fullscreen mode

In the above Protobuf message contract, all attributes are self-descriptive. Each attribute is assigned a unique number, which is used during serialization and deserialization, not the attribute name. This helps maintain a small message size and ensures forward and backward compatibility.

Note: Clients and services will ignore field numbers they do not recognize. For more details about Protobuf, visit protobuf.dev.

Using Protobuf in C# with Protobuf-net

You can use the Protobuf compiler to generate code for serialization and deserialization, or you can use the .NET NuGet package Protobuf-net to write a C# style class as a Protobuf contract.

To get started, download the Protobuf-net NuGet package from NuGet.

Here is an example of how to define a Protobuf message in C#:

[ProtoContract]
public class UserInfo {
  [ProtoMember(1)]
  public int UserId { get; set; }

  [ProtoMember(2)]
  public string UserName { get; set; }

  [ProtoMember(3)]
  public string Email { get; set; }
}
Enter fullscreen mode Exit fullscreen mode

Here is my C# code which does both JSON and protobuf data Serialization and Deserialization and save the result to file.
// See https://aka.ms/new-console-template for more information
using System.Text.Json;
using ProtoBuf;

List users = new List();

for (int i = 0; i < 100; i++)
;

users.Add(user);
Enter fullscreen mode Exit fullscreen mode

}

// JSON Seralizer
string jsonString = JsonSerializer.Serialize(users);

using (var file = File.Create("users.json"))

using (var file = File.Create("users.data"))

List users;
using (var file = File.OpenRead("users.data"))

See the file size of .json and users.data . JSON format is 9.8k and Protobuf format size is 6.7k

Performance Comparison: JSON vs. Protobuf

Protobuf is significantly more efficient than JSON in terms of serialization and deserialization speed and the size of the data transferred over the wire. Here's a percentage difference for 100 users' data:

  • JSON: Larger size, slower serialization/deserialization.
  • Protobuf: Smaller size, faster serialization/deserialization.

Image description

When to Use Protobuf

  1. Microservices: Ideal for service-to-service communication.
  2. Language Interoperability: When the application involves multiple programming languages.
  3. Performance: When fast and efficient serialization/deserialization is required.
  4. Compatibility: When backward and forward compatibility between client and server contracts is necessary.
  5. Storage Efficiency: For saving data to disk with less storage and performing ETL operations.
  6. Large Payloads in APIs: For REST API routes with large payloads, where JSON is slow and cumbersome over the wire.

Note: If you want to use the Google Protobuf compiler for code generation, download it from GitHub based on your OS. Follow this link for code generation instructions.

For more information, visit Protobuf Documentation.


Thanks,

Sreeni Ramadurai

grpc Article's
30 articles in total
Favicon
How to Handle Excessive Warning Messages When Running `pecl install grpc`
Favicon
Protocol Buffers as a Serialization Format
Favicon
gRPC, Haskell, Nix, love, hate
Favicon
🚀Build, Implement, and Test gRPC Services with .NET9
Favicon
Using OpenTelemetry with gRPC in Node.js and Express Hybrid Applications
Favicon
Exploring gRPC: The High-Performance Communication in Distributed Systems
Favicon
Wednesday Links - Edition 2024-11-06
Favicon
Part 3: Compiling the Protos and Setting up the gRPC server
Favicon
Mocking gRPC Clients in C#: Fake It Till You Make It
Favicon
How gRPC Works
Favicon
Part 2: Defining the Authentication gRPC Interface
Favicon
How Does gRPC Actually Work?
Favicon
gRPC Streaming: Best Practices and Performance Insights
Favicon
Why is gRPC so much faster than a JSON-based REST API?
Favicon
Strongly typed web APIs with gRPC
Favicon
Use RBAC to protect your gRPC service right on proto definition
Favicon
gRPC and Go: Building High-Performance Web Services
Favicon
Connecting NestJS and ASP.NET Core with gRPC: A Step-by-Step Guide
Favicon
gRPC: what is it? An introduction...
Favicon
gRPC: onde vive? o que come?
Favicon
Understanding the Importance of gRPC and Its Applications in Modern Software Development
Favicon
Mastering Go gRPC Services with Docker: A One-Stop Guide
Favicon
Hybrid NestJs Microservice Responding to Both HTTP and gRPC Requests
Favicon
Part 1: How to build Auth API in Rust using gRPC
Favicon
gRPC between Web and Server.
Favicon
FauxRPC
Favicon
Why should we use Protobuf in Web API as data transfer protocol.
Favicon
Browser Client to gRPC Server Routing options: Connect, gRPC-web, gRPC-gateway and more!
Favicon
gRPC vs REST: A Comprehensive Comparison
Favicon
gRPC - Unimplemented Error 12

Featured ones: