protocol buffer介紹(protobuf)

@(HADOOP)[hadoop, 大數據]

一、理論概述

0、參考資料

入門資料：https://developers.google.com/protocol-buffers/docs/javatutorial

更詳細的資料：

For more detailed reference information, see the Protocol Buffer Language Guide, the Java API Reference, the Java Generated Code Guide, and the Encoding Reference.

1、protobuf是什么？

看看官方的解釋

Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

也就是說：protobuf是一個google的開源項目，它是一個語言獨立、平臺獨立，可擴展的數據序列化機制，類似于XML，但它更小、更快和更簡單。

2、protobuf能做什么？

很明顯，它可用于對象序列化與反序列化，主要用于數據存儲與數據傳輸格式的定義。
目前被大量用于hadoop的RPC通信協議中，所有的RPC函數參數均是使用protobuf定義的。

3、與XML相比的優缺點

優點：更小，更快（XML的反序列化效率極低），而且可以利用工具自動生成代碼。

缺點：由于用二進制保存數據，導致可讀性差

二、API

1、使用protobuf的基本步驟如下：

（1）定義消息的格式（一般使用.proto后綴）

（2）使用protobuf提供的compiler，根據.proto文件生成類

（3）使用API進行消息的讀寫

2、定義消息的格式（一般使用.proto后綴）

package tutorial;option java_package = "org.ljh.protobufdemo"; option java_outer_classname = "AddressBookProtos";message Person {required string name = 1;required int32 id = 2;optional string email = 3;enum PhoneType {MOBILE = 0;HOME = 1;WORK = 2;}message PhoneNumber {required string number = 1;optional PhoneType type = 2 [default = HOME];}repeated PhoneNumber phone = 4; }message AddressBook {repeated Person person = 1; }

proto文件說明，完整說明請見https://developers.google.com/protocol-buffers/docs/proto
（1）package用于指明命名空間，以防與其它項目沖突。
（2）如果指定java_package，則它作為下一步要生成的java類的package，否則package中定義的值將作為java類的package。
（3）java_outer_classname指定了生成類的類名，如果沒指定，則使用.proto文件的文件名作為類名。
（4）message表示消息定義，消息之間可以互相嵌套或者調用。
（5）每個字段后的等號定義的是該字段的tag，由于1～15少用了一個字節，因此，這些標簽最好留給用得很多的字段，尤其是使用repeated定義的字段。
（6）每個字段必須使用以下3個修飾符之一：required, optional, repeated。

3、從標準輸入中讀入信息，構建person，然后序列化到一個文件中

package org.ljh.protobufdemo; import java.io.BufferedReader; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.InputStreamReader; import java.io.IOException; import java.io.PrintStream;import org.ljh.protobufdemo.AddressBookProtos.AddressBook; import org.ljh.protobufdemo.AddressBookProtos.Person;public class AddPerson {// This function fills in a Person message based on user input.static Person PromptForAddress(BufferedReader stdin,PrintStream stdout) throws IOException {Person.Builder person = Person.newBuilder();stdout.print("Enter person ID: ");person.setId(Integer.valueOf(stdin.readLine()));stdout.print("Enter name: ");person.setName(stdin.readLine());stdout.print("Enter email address (blank for none): ");String email = stdin.readLine();if (email.length() > 0) {person.setEmail(email);}while (true) {stdout.print("Enter a phone number (or leave blank to finish): ");String number = stdin.readLine();if (number.length() == 0) {break;}Person.PhoneNumber.Builder phoneNumber =Person.PhoneNumber.newBuilder().setNumber(number);stdout.print("Is this a mobile, home, or work phone? ");String type = stdin.readLine();if (type.equals("mobile")) {phoneNumber.setType(Person.PhoneType.MOBILE);} else if (type.equals("home")) {phoneNumber.setType(Person.PhoneType.HOME);} else if (type.equals("work")) {phoneNumber.setType(Person.PhoneType.WORK);} else {stdout.println("Unknown phone type. Using default.");}person.addPhone(phoneNumber);}return person.build();}// Main function: Reads the entire address book from a file,// adds one person based on user input, then writes it back out to the same// file.public static void main(String[] args) throws Exception {if (args.length != 1) {System.err.println("Usage: AddPerson ADDRESS_BOOK_FILE");System.exit(-1);}AddressBook.Builder addressBook = AddressBook.newBuilder();// Read the existing address book.try {addressBook.mergeFrom(new FileInputStream(args[0]));} catch (FileNotFoundException e) {System.out.println(args[0] + ": File not found. Creating a new file.");}// Add an address.addressBook.addPerson(PromptForAddress(new BufferedReader(new InputStreamReader(System.in)),System.out));// Write the new address book back to disk.FileOutputStream output = new FileOutputStream(args[0]);addressBook.build().writeTo(output);output.close();} }

4、從文件中讀取內容，然后反序列化到一個實例

package org.ljh.protobufdemo;import java.io.FileInputStream; import java.io.IOException; import java.io.PrintStream;import org.ljh.protobufdemo.AddressBookProtos.AddressBook; import org.ljh.protobufdemo.AddressBookProtos.Person;public class ListPeople {// Iterates though all people in the AddressBook and prints info about them.static void Print(AddressBook addressBook) {for (Person person: addressBook.getPersonList()) {System.out.println("Person ID: " + person.getId());System.out.println(" Name: " + person.getName());if (person.hasEmail()) {System.out.println(" E-mail address: " + person.getEmail());}for (Person.PhoneNumber phoneNumber : person.getPhoneList()) {switch (phoneNumber.getType()) {case MOBILE:System.out.print(" Mobile phone #: ");break;case HOME:System.out.print(" Hls"+ "ome phone #: ");break;case WORK:System.out.print(" Work phone #: ");break;}System.out.println(phoneNumber.getNumber());}}}// Main function: Reads the entire address book from a file and prints all// the information inside.public static void main(String[] args) throws Exception {if (args.length != 1) {System.err.println("Usage: ListPeople ADDRESS_BOOK_FILE");System.exit(-1);}// Read the existing address book.AddressBook addressBook =AddressBook.parseFrom(new FileInputStream(args[0]));Print(addressBook);} }

總結

以上是生活随笔為你收集整理的protocol buffer介绍(protobuf)的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。