
- Protocol Buffers - Home
- Protocol Buffers - Introduction
- Protocol Buffers - Environment Setup
- Protocol Buffers - Basic App
- Protocol Buffers - Constructs
- Protocol Buffers - message
- Protocol Buffers - string
- Protocol Buffers - Numbers
- Protocol Buffers - bool
- Protocol Buffers - enum
- Protocol Buffers - repeated
- Protocol Buffers - map
- Protocol Buffers - Nested Class
- Protocol Buffers - Optionality & Defaults
- Protocol Buffers - Language Independence
- Protocol Buffers - Compound Data Types
- Protocol Buffers - Command Line Usage
- Protocol Buffers - Rules to Update Definition
- Protocol Buffers - Integration with Kafka
- Protocol Buffers - In Other Languages
- Protocol Buffers Useful Resources
- Protocol Buffers - Quick Guide
- Protocol Buffers - Useful Resources
- Protocol Buffers - Discussion
Protocol Buffers - Compound Data Types
Overview
There are two more compound data types which may be useful for complicated use cases. They are "OneOf" and "Any". In this chapter, we will see how to use these two data types of Protobuf.
OneOf
We pass a few parameters to this OneOf data type and Protobuf ensures that only one of them is set. If we set one of them and try to set the other one, the first attribute gets reset. Let's us understand this via an example.
Continuing with our theater example from Protocol Buffers - String chapter, say, we have an API which is used to fetch the count of available employees. The value returned from this API is then set to 'count' tag in the following file. But if that API errors out, we can't really 'count', instead we attach the error log.
Ideally, we will always have one of them set, i.e., either the call is successful and we get the count OR the count calculation fails and we get the error message.
Following is the syntax that we need to have to instruct Protobuf that we will be creating an OneOf attribute −
theater.proto
syntax = "proto3"; package theater; option java_package = "com.tutorialspoint.theater"; message Theater { oneof availableEmployees { int32 count = 4; string errorLog = 5; } }
Serialization using Java
To use Protobuf, we will now have to use protoc binary to create the required classes from this ".proto" file. Let us see how to do that −
protoc --java_out=. theater.proto
This will create a TheaterOuterClass.java class in com > tutorialspoint > theater folder in current directory. We're using this class in our application similar to as done in Protocol Buffers - Basic App chapter.
Using Java Classes created from Proto File
First, we will create a writer to write the theater information −
TheaterWriter.java
package com.tutorialspoint.theater; import java.io.FileOutputStream; import java.io.IOException; import com.tutorialspoint.theater.TheaterOuterClass.Theater; public class TheaterWriter{ public static void main(String[] args) throws IOException { Theater theater = Theater.newBuilder() .setCount(3) .setErrorLog("No employee found") .build(); String filename = "theater_protobuf_output"; System.out.println("Saving theater information to file: " + filename); try(FileOutputStream output = new FileOutputStream(filename)){ theater.writeTo(output); } System.out.println("Saved theater information with following data to disk: \n" + theater); } }
Next, we will have a reader to read the theater information −
TheaterReader.java
package com.tutorialspoint.theater; import java.io.FileInputStream; import java.io.IOException; import com.tutorialspoint.theater.TheaterOuterClass.Theater; import com.tutorialspoint.theater.TheaterOuterClass.Theater.Builder; public class TheaterReader{ public static void main(String[] args) throws IOException { Builder theaterBuilder = Theater.newBuilder(); String filename = "theater_protobuf_output"; System.out.println("Reading from file " + filename); try(FileInputStream input = new FileInputStream(filename)) { Theater theater = theaterBuilder.mergeFrom(input).build(); System.out.println(theater); } } }
Compile the project
Now that we have set up the reader and the writer, let us compile the project.
mvn clean install
Serialize the Java Object
Now, post compilation, let us execute the writer first −
> java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterWriter Saving theater information to file: theater_protobuf_output Saved theater information with following data to disk: errorLog: "No employee found"
Deserialize the Serialized Object
Now, let us execute the reader to read from the same file −
java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterReader Reading from file theater_protobuf_output errorLog: "No employee found"
So, as we see, we are able to see only the error log being set as later.
Any
The next data type that can be of use for complicated uses cases is Any. We can pass any type/message/class to this data type and Protobuf would not complain. Let us understand this via an example.
Continuing with the theater example, say, we want to track people inside the theater. Some of them could be employees and others could be viewers. But ultimately they are people, so we will pass them in a single list which would contain both the types.
Now our class/message contains an Any attribute 'peopleInside' list along with Viewer and Employee class, i.e., information about the people inside theater. Let us see this in action.
Following is the syntax that we need to have to instruct Protobuf that we will be creating an Any attribute −
theater.proto
syntax = "proto3"; package theater; option java_package = "com.tutorialspoint.theater"; import "google/protobuf/any.proto"; message Theater { string name = 1; string address = 2; repeated google.protobuf.Any peopleInside = 3; } message Employee{ string name = 1; string address = 2; } message Viewer{ string name = 1; int32 age = 2; string sex = 3; }
Serialization using Java
To use Protobuf, we will now have to use protoc binary to create the required classes from this ".proto" file. Let us see how to do that −
protoc --java_out=. theater.proto
This will create a TheaterOuterClass.java class in com > tutorialspoint > theater folder in current directory. We're using this class in our application similar to as done in Protocol Buffers - Basic App chapter.
Using Java Classes created from Proto File
First, we will create a writer to write the theater information −
TheaterWriter.java
package com.tutorialspoint.theater; import java.util.List; import java.io.FileOutputStream; import java.io.IOException; import java.util.ArrayList; import com.google.protobuf.Any; import com.tutorialspoint.theater.TheaterOuterClass.Employee; import com.tutorialspoint.theater.TheaterOuterClass.Theater; import com.tutorialspoint.theater.TheaterOuterClass.Viewer; public class TheaterWriter{ public static void main(String[] args) throws IOException { List<Any> people = new ArrayList<>(); people.add(Any.pack(Employee.newBuilder().setName("John").build())); people.add(Any.pack(Viewer.newBuilder().setName("Jane").setAge(30).build())); people.add(Any.pack(Employee.newBuilder().setName("Simon").build())); people.add(Any.pack(Viewer.newBuilder().setName("Janice").setAge(25).build())); Theater theater = Theater.newBuilder() .setName("SilverScreen") .addAllPeopleInside(people) .build(); String filename = "theater_protobuf_output_silver"; System.out.println("Saving theater information to file: " + filename); try(FileOutputStream output = new FileOutputStream(filename)){ theater.writeTo(output); } System.out.println("Saved theater information with following data to disk: \n" + theater); } }
Next, we will have a reader to read the theater information −
TheaterReader.java
package com.tutorialspoint.theater; import java.io.FileInputStream; import java.io.IOException; import com.google.protobuf.Any; import com.tutorialspoint.theater.TheaterOuterClass.Employee; import com.tutorialspoint.theater.TheaterOuterClass.Theater; import com.tutorialspoint.theater.TheaterOuterClass.Theater.Builder; import com.tutorialspoint.theater.TheaterOuterClass.Viewer; public class TheaterReader{ public static void main(String[] args) throws IOException { Builder theaterBuilder = Theater.newBuilder(); String filename = "theater_protobuf_output_silver"; System.out.println("Reading from file " + filename); try(FileInputStream input = new FileInputStream(filename)) { Theater theater = theaterBuilder.mergeFrom(input).build(); System.out.println("Name:" + theater.getName() + "\n"); for (Any anyPeople : theater.getPeopleInsideList()) { if(anyPeople.is(Employee.class)) { Employee employee = anyPeople.unpack(Employee.class); System.out.println("Employee:" + employee + "\n"); } if(anyPeople.is(Viewer.class)) { Viewer viewer = anyPeople.unpack(Viewer.class); System.out.println("Viewer:" + viewer + "\n"); } } } } }
Compile the project
Now that we have set up the reader and the writer, let us compile the project.
mvn clean install
Serialize the Java Object
Now, post compilation, let us execute the writer first −
> java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterWriter Saving theater information to file: theater_protobuf_output Saved theater information with following data to disk: name: "SilverScreen" peopleInside { type_url: "type.googleapis.com/theater.Employee" value: "\n\004John" } peopleInside { type_url: "type.googleapis.com/theater.Viewer" value: "\n\004Jane\020\036" } peopleInside { type_url: "type.googleapis.com/theater.Employee" value: "\n\005Simon" } peopleInside { type_url: "type.googleapis.com/theater.Viewer" value: "\n\006Janice\020\031" }
Note − There are two points to note −
In case of Any, Protobuf packs/serializes the contents inside any tag to bytes and then stores it as 'value'. Basically, that allows us to send any message type with this 'Any' tag.
We also see "type.googleapis.com/theater.Viewer" and "type.googleapis.com/theater.Employee". This is used by Protobuf to save the type of object along with the data as the type of data in the Any data type can vary.
Deserialize the Serialized Object
Now let us execute the reader to read from the same file −
java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterReader Reading from file theater_protobuf_output_silver Name:SilverScreen Employee:name: "John" Viewer:name: "Jane" age: 30 Employee:name: "Simon" Viewer:name: "Janice" age: 25
So, as we see, our reader code is successfully able to differentiate between Employee and the Viewer, even though they come in the same array.