Protobuf - Basic App

Let us now use Google Protocol Buffer and see how it works with a simple Greeting app. In this example, we will create a simple application which would do the following −

  • Greeting the Writer −

    • Take greeting and username from the user

    • Store the above information in a file in the disk

  • Greeting Reader −

    • Reads the same file which we stored in the above file

    • Convert that data into an object and print the data

Protocol Buffer Definition file

The protocol buffer "definition file" contains the schema definition of the data we want to serialize. The data is stored in a human readable file with the extension ".proto".

Let us store the following data in "greeting.proto" and we will use this in our first application.

syntax = "proto3";
package tutorial;
option java_package = "com.tutorialspoint.greeting";

message Greet {
   string greeting = 1;
   string username = 2;
}

Now, let us take a closer look at the data and see what each line of code does in the above code block.

syntax = "proto3";

The "syntax" here represents what version of Protobuf we are using. So, we are using the latest version 3 and the schema thus can use all the syntax which is valid for version 3.

package tutorial;

The package here is used for conflict resolution if, say, we have multiple classes/members with same name.

option java_package = "com.tutorialspoint.greeting";

This argument is specific to Java, i.e., the package where the code from the ".proto" file will be auto-generated.

message Greet

Name of the base class for the object which would be created/recreated.

string greeting = 1;
string username = 2;

These are the attributes of the Greet class along with the data type and the position of the tag in the schema. If a new tag is to be added, it should have "3" as the position. Note that this position integer is important to ensure that the actual data is compact and there is scope of schema evolution.

Protocol Buffer Code Generation

Now that we have defined, let us install the "proto" binary which we will use to autogenerate the code for the above Greet class. The binaries can be found at "https://github.com/protocolbuffers/protobuf/releases/".

Choose the correct binary based on the OS. We will install proto binary on Windows but the steps are not very different for Linux.

Once installed, ensure that you are able to access it via command line −

protoc --version

libprotoc 3.15.6

It confirms that Protobuf is correctly installed. Now let us move to creating the Greeting app described above for Java.

Project Structure

Here is the overall project structure that we would have −

Code related to individual languages go to their respective directories. And we have a separate directory to store our "proto" files.

Project Structure

And here is the project structure that we would be having for Java −

Project Structure1

Greeting App in Java

Now that we have installed protoc, we can auto-generate the code from the proto files using protoc. Let us first create a Java project though.

Following is the Maven configuration that we will use for our Java project. Note that it contains the required library for Protobuf as well.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.tutorials.point</groupId>
   <artifactId>protobuf-tutorial</artifactId>
   <version>1.0</version>
   <packaging>jar</packaging>

   <properties>
      <maven.compiler.source>1.8</maven.compiler.source>
      <maven.compiler.target>1.8</maven.compiler.target>
   </properties>

   <dependencies>
      <!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java -->
      <dependency>
         <groupId>com.google.protobuf</groupId>
         <artifactId>protobuf-java</artifactId>
         <version>3.15.8</version>
      </dependency>
   </dependencies>

   <build>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.2.4</version>
            <configuration>
               <!--Put your configurations here-->
            </configuration>
            <executions>
               <execution>
                  <phase>package</phase>
                     <goals>
                     <goal>shade</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
      </plugins>
   </build>
</project>

All of our code would be present under src/main/java.

With the project structure out of the way, let us generate the code for the Greet class −

protoc  --java_out=java/src/main/java proto_files/greeting.proto

Post execution of the command, you will notice two auto-generated classes.

  • Greeting.java

  • GreetOrBuilder.java

These two classes would help us with serialization and deserialization of the Greet object.

Now, let us write the writer of the data, which will take the username and the greeting as its inputs −

package com.tutorialspoint.greeting;

import java.io.FileOutputStream;
import java.io.IOException;
import com.tutorialspoint.greeting.Greeting.Greet;

public class GreetWriter{
   public static void main(String[] args) throws IOException {
      Greet greeting = Greet.newBuilder()
         .setGreeting(args[0])
         .setUsername(args[1])
         .build();
		
      String filename = "greeting_protobuf_output";
      System.out.println("Saving greeting to file: " + filename);
		
      try(FileOutputStream output = new FileOutputStream(filename)){
         greeting.writeTo(output);
      }
      System.out.println("Saved greeting with following data to disk: \n" + greeting);
   }
}

The writer simply takes CLI arguments, creates the Greet object, serializes it and then dumps it to a file.

Now let us write a reader which will read the file −

package com.tutorialspoint.greeting;

import java.io.FileInputStream;
import java.io.IOException;
import com.tutorialspoint.greeting.Greeting.Greet;

public class GreetReader{
   public static void main(String[] args) throws IOException {
      Greet.Builder greetBuilder = Greet.newBuilder();

      String filename = "greeting_protobuf_output";
      System.out.println("Reading from file " + filename);
	    
      try(FileInputStream input = new FileInputStream(filename)) {
         Greet greet = greetBuilder.mergeFrom(input).build();
         System.out.println("Greeting: " + greet.getGreeting() + "\n" + "Username: " + greet.getUsername());
      }
   }
}

The reader simply reads from the same file, deserializes it, and prints the data about the greeting.

Now that we have set up the reader and the writer, let us compile the project.

mvn clean install

And now, let us first execute the writer.

java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.greeting.GreetWriter Hello John

Saving greeting to file: 
greeting_protobuf_output

Saved greeting with following data to disk:
greeting: Hello
username: John

And then, let us execute the reader.

java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.greeting.GreetReader

Reading from file greeting_protobuf_output
Greeting: Hello
Username: John

So, as we see the data that was serialized by the writer and saved to the file, that exact data is correctly deserialized by the reader and printed accordingly.

Greeting App in Python

Let us now write the same example as a Python project −

We will need to install protobuf pip package before we proceed.

pip install protobuf

All of our code would be present under "google-protobuf/python".

With the project structure out of the way, let us generate the code for Greet class −

protoc  --python_out=python proto_files/greeting.proto

Post execution of this command, you will notice an auto-generated class " * proto_files/greeting_pb2.py " under the Python directory. This class would help us with serialization and deserialization of the Greet object.

Now, let us write the writer of the data, which will take the username and the greeting as its input −

from .proto_files import greeting_pb2

import sys

greet = greeting_pb2.Greet()
greet.username = sys.argv[1]
greet.greeting = sys.argv[2]

filename = "greeting_protobuf_output";
print("Saving to file: " + filename)

f = open(filename, "wb")
f.write(greet.SerializeToString())
f.close()
print("Saved following greeting to disk: \n" + str(greet))

The writer simply takes CLI arguments, creates the Greet object, serializes it, and then dumps it to a file.

Now let us create a reader which will read the file −

from proto_files import greeting_pb2

greet = greeting_pb2.Greet()

filename = "greeting_protobuf_output";
print("Reading from file: " + filename)

f = open(filename, "rb")
greet.ParseFromString(f.read())
f.close()

print("Read greeting from disk: \n" + str(greet))

The reader simply reads from the same file, deserializes it, and prints the data about the greeting.

Now, let us first execute the writer.

python greetWriter.py Hola Jane

Saving to file: greeting_protobuf_output
Saved following greeting to disk:
greeting: "Hola"
username: "Jane"

And then, let us execute the reader.

python greetReader.py

Reading from file: greeting_protobuf_output
Read greeting from disk:
greeting: "Hola"
username: "Jane"

So, as we see, the data that was serialized by the writer and saved to a file. Next, the same data is correctly deserialized by the reader and printed accordingly.

Advertisements