Protocol Buffers - Compound Data Types



Overview

There are two more compound data types which may be useful for complicated use cases. They are "OneOf" and "Any". In this chapter, we will see how to use these two data types of Protobuf.

OneOf

We pass a few parameters to this OneOf data type and Protobuf ensures that only one of them is set. If we set one of them and try to set the other one, the first attribute gets reset. Let's us understand this via an example.

Continuing with our theater example from Protocol Buffers - String chapter, say, we have an API which is used to fetch the count of available employees. The value returned from this API is then set to 'count' tag in the following file. But if that API errors out, we can't really 'count', instead we attach the error log.

Ideally, we will always have one of them set, i.e., either the call is successful and we get the count OR the count calculation fails and we get the error message.

Following is the syntax that we need to have to instruct Protobuf that we will be creating an OneOf attribute −

theater.proto

syntax = "proto3";
package theater;
option java_package = "com.tutorialspoint.theater";

message Theater { 
   oneof availableEmployees {
      int32 count = 4;
      string errorLog = 5;
   }
}

Serialization using Java

To use Protobuf, we will now have to use protoc binary to create the required classes from this ".proto" file. Let us see how to do that −

protoc  --java_out=. theater.proto

This will create a TheaterOuterClass.java class in com > tutorialspoint > theater folder in current directory. We're using this class in our application similar to as done in Protocol Buffers - Basic App chapter.

Using Java Classes created from Proto File

First, we will create a writer to write the theater information −

TheaterWriter.java

package com.tutorialspoint.theater;

import java.io.FileOutputStream;
import java.io.IOException;

import com.tutorialspoint.theater.TheaterOuterClass.Theater;

public class TheaterWriter{
   public static void main(String[] args) throws IOException {
    	   
      Theater theater = Theater.newBuilder()
         .setCount(3)
         .setErrorLog("No employee found")
         .build();
		
      String filename = "theater_protobuf_output";
      System.out.println("Saving theater information to file: " + filename);
		
      try(FileOutputStream output = new FileOutputStream(filename)){
         theater.writeTo(output);
      }
      System.out.println("Saved theater information with following data to disk: \n" + theater);
   }
}

Next, we will have a reader to read the theater information −

TheaterReader.java

package com.tutorialspoint.theater;

import java.io.FileInputStream;
import java.io.IOException;

import com.tutorialspoint.theater.TheaterOuterClass.Theater;
import com.tutorialspoint.theater.TheaterOuterClass.Theater.Builder;

public class TheaterReader{
   public static void main(String[] args) throws IOException {
      Builder theaterBuilder = Theater.newBuilder();

      String filename = "theater_protobuf_output";
      System.out.println("Reading from file " + filename);
        
      try(FileInputStream input = new FileInputStream(filename)) {
         Theater theater = theaterBuilder.mergeFrom(input).build();
         System.out.println(theater);
      }
   }
}

Compile the project

Now that we have set up the reader and the writer, let us compile the project.

mvn clean install

Serialize the Java Object

Now, post compilation, let us execute the writer first −

> java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterWriter

Saving theater information to file: theater_protobuf_output
Saved theater information with following data to disk:
errorLog: "No employee found"

Deserialize the Serialized Object

Now, let us execute the reader to read from the same file −

java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterReader

Reading from file theater_protobuf_output
errorLog: "No employee found"

So, as we see, we are able to see only the error log being set as later.

Any

The next data type that can be of use for complicated uses cases is Any. We can pass any type/message/class to this data type and Protobuf would not complain. Let us understand this via an example.

Continuing with the theater example, say, we want to track people inside the theater. Some of them could be employees and others could be viewers. But ultimately they are people, so we will pass them in a single list which would contain both the types.

Now our class/message contains an Any attribute 'peopleInside' list along with Viewer and Employee class, i.e., information about the people inside theater. Let us see this in action.

Following is the syntax that we need to have to instruct Protobuf that we will be creating an Any attribute −

theater.proto

syntax = "proto3";
package theater;
option java_package = "com.tutorialspoint.theater";

import "google/protobuf/any.proto";

message Theater {
   string name = 1;
   string address = 2;
  
   repeated google.protobuf.Any peopleInside = 3;
}
message Employee{
   string name = 1;
   string address = 2;
}
message Viewer{
   string name = 1;
   int32 age = 2;
   string sex = 3;
}

Serialization using Java

To use Protobuf, we will now have to use protoc binary to create the required classes from this ".proto" file. Let us see how to do that −

protoc  --java_out=. theater.proto

This will create a TheaterOuterClass.java class in com > tutorialspoint > theater folder in current directory. We're using this class in our application similar to as done in Protocol Buffers - Basic App chapter.

Using Java Classes created from Proto File

First, we will create a writer to write the theater information −

TheaterWriter.java

package com.tutorialspoint.theater;

import java.util.List;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;

import com.google.protobuf.Any;
import com.tutorialspoint.theater.TheaterOuterClass.Employee;
import com.tutorialspoint.theater.TheaterOuterClass.Theater;
import com.tutorialspoint.theater.TheaterOuterClass.Viewer;

public class TheaterWriter{
   public static void main(String[] args) throws IOException {
	    
   List<Any> people = new ArrayList<>();
   people.add(Any.pack(Employee.newBuilder().setName("John").build()));
   people.add(Any.pack(Viewer.newBuilder().setName("Jane").setAge(30).build()));
   people.add(Any.pack(Employee.newBuilder().setName("Simon").build()));
   people.add(Any.pack(Viewer.newBuilder().setName("Janice").setAge(25).build()));
	    
   Theater theater = Theater.newBuilder()
      .setName("SilverScreen")
      .addAllPeopleInside(people)
      .build();
		
      String filename = "theater_protobuf_output_silver";
      System.out.println("Saving theater information to file: " + filename);
		
      try(FileOutputStream output = new FileOutputStream(filename)){
         theater.writeTo(output);
      }
	    
      System.out.println("Saved theater information with following data to disk: \n" + theater);
   }
}

Next, we will have a reader to read the theater information −

TheaterReader.java

package com.tutorialspoint.theater;

import java.io.FileInputStream;
import java.io.IOException;

import com.google.protobuf.Any;
import com.tutorialspoint.theater.TheaterOuterClass.Employee;
import com.tutorialspoint.theater.TheaterOuterClass.Theater;
import com.tutorialspoint.theater.TheaterOuterClass.Theater.Builder;
import com.tutorialspoint.theater.TheaterOuterClass.Viewer;

public class TheaterReader{
   public static void main(String[] args) throws IOException {
	   Builder theaterBuilder = Theater.newBuilder();

      String filename = "theater_protobuf_output_silver";
      System.out.println("Reading from file " + filename);
        
      try(FileInputStream input = new FileInputStream(filename)) {
         Theater theater = theaterBuilder.mergeFrom(input).build();
            
         System.out.println("Name:" + theater.getName() + "\n");
            
         for (Any anyPeople : theater.getPeopleInsideList()) {
            if(anyPeople.is(Employee.class)) {
               Employee employee = anyPeople.unpack(Employee.class);
               System.out.println("Employee:" + employee + "\n");
            }
                
            if(anyPeople.is(Viewer.class)) {
               Viewer viewer = anyPeople.unpack(Viewer.class);
               System.out.println("Viewer:" + viewer + "\n");
            }
         }
      }
   }
}	

Compile the project

Now that we have set up the reader and the writer, let us compile the project.

mvn clean install

Serialize the Java Object

Now, post compilation, let us execute the writer first −

> java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterWriter

Saving theater information to file: theater_protobuf_output
Saved theater information with following data to disk:
name: "SilverScreen"
peopleInside {
   type_url: "type.googleapis.com/theater.Employee"
   value: "\n\004John"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Viewer"
   value: "\n\004Jane\020\036"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Employee"
   value: "\n\005Simon"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Viewer"
   value: "\n\006Janice\020\031"
}

Note − There are two points to note −

  • In case of Any, Protobuf packs/serializes the contents inside any tag to bytes and then stores it as 'value'. Basically, that allows us to send any message type with this 'Any' tag.

  • We also see "type.googleapis.com/theater.Viewer" and "type.googleapis.com/theater.Employee". This is used by Protobuf to save the type of object along with the data as the type of data in the Any data type can vary.

Deserialize the Serialized Object

Now let us execute the reader to read from the same file −

java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterReader

Reading from file theater_protobuf_output_silver
Name:SilverScreen

Employee:name: "John"

Viewer:name: "Jane"
age: 30

Employee:name: "Simon"

Viewer:name: "Janice"
age: 25

So, as we see, our reader code is successfully able to differentiate between Employee and the Viewer, even though they come in the same array.

Advertisements