A quick tutorial on working with ProtoBuf

What is ProtoBuf

ProtoBuf stands for Protocol Buffer, a library/framework that generates sources in different languages such as C++, Python, Java, Kotlin, etc. for serialization and deserialization of data. Then, different applications in different platforms can communicate through ProtoBuf regardless of the languages that they are developed with.

Protocol Buffer had been developed by Google. The initial internal version was introduced in 2001 while the main purpose was using it internally. Then it was published under the BSD license as a public open-source project in 2008. That’s a flexible, fast and small protocol that can be used like JSON or XML files however you don’t need to handle data format conversion. Instead, you only work with classes, functions and structures generated by Proro Compiler. ProtoBuf can be used for communicating over a network or even storing data.

What is the .proto file?

A .proto file containing messages defined by you or a developer that Proto Compiler will generate source files for the target programming language from that. The picture below shows the output of the Proto compiler for C++

Steps to create source code from the .proto file

Create a .proto file(s) for the message(s)
Define messages in .proto file/files
Compile the .proto file through the command line or the build system to generate source files by Proto Compiler
Import/Include the generated source files into your project

Example of .proto file

syntax = "proto3";
package proto_example;

message Message {
  int32 id = 1;
  string title = 2;
  string date_time = 3;
  string sender_name = 4;

  enum Priority {
    URGENT = 0;
    IMPORTANT = 1;
    REGULAR = 2;
    ADVERTISE = 3;
  }
  Priority priority = 5;
  message OtherReceiver {
    string receiver_name = 1;
  }
  repeated OtherReceiver receivers = 6;
}

The first line shows which proto version should be considered for this .proto file. The second line tells the compiler the package it belongs to that later in the generated source files will be used. Also, if it’s needed you can import other .proto files in the beginning to utilize newly defined types.

This .proto file includes one message. The “Message” has 6 different fields, some of them are common types(float, double, int32, int64, …) while two of them have newly defined types. You can define your new types through different .proto files and easily import them everywhere you need that It makes .proto files reusable and make it possible to organize them as you wish.

Let’s have a look again at the example. The last parameter can be repeated. It means that it works like a dynamic array and ProtoBuf will take care of the length of that in the serialization/deserialization process.

How to use Proto Compiler

First step is installing Proto Compiler on your system. Here is an example of Ubuntu:

$ sudo apt-get install protobuf-compiler

Now you can generate source codes from .proto.

C++

$ protoc --cpp_out=. path-to/message.proto

Python

$ protoc --python_out=. path-to/message.proto

Note: replace “path-to/” with the path on your local machine.

Two parameters should be provided for protoc through the command line. The first is the output directories describing the output source code’s language and the second one is the .proto file.

Now you can take an action and go through the steps!

Source Code of the Protouf-Example

Read more

proto3 Language guide

Protocol Buffer Overview

ProtoBuf tutorial for different languages