README.md 15.1 KB
Newer Older
1
Protocol Buffers - Google's data interchange format
2 3 4 5
===================================================

[![Build Status](https://travis-ci.org/google/protobuf.svg?branch=master)](https://travis-ci.org/google/protobuf)

6 7
Copyright 2008 Google Inc.

8
This directory contains the Java Protocol Buffers Nano runtime library.
9 10

Installation - With Maven
11
-------------------------
12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

The Protocol Buffers build is managed using Maven.  If you would
rather build without Maven, see below.

1) Install Apache Maven if you don't have it:

     http://maven.apache.org/

2) Build the C++ code, or obtain a binary distribution of protoc.  If
   you install a binary distribution, make sure that it is the same
   version as this package.  If in doubt, run:

     $ protoc --version

   You will need to place the protoc executable in ../src.  (If you
   built it yourself, it should already be there.)

3) Run the tests:

     $ mvn test

   If some tests fail, this library may not work correctly on your
   system.  Continue at your own risk.

4) Install the library into your Maven repository:

     $ mvn install

5) If you do not use Maven to manage your own build, you can build a
   .jar file to use:

     $ mvn package

   The .jar will be placed in the "target" directory.

Installation - Without Maven
48
----------------------------
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

If you would rather not install Maven to build the library, you may
follow these instructions instead.  Note that these instructions skip
running unit tests.

1) Build the C++ code, or obtain a binary distribution of protoc.  If
   you install a binary distribution, make sure that it is the same
   version as this package.  If in doubt, run:

     $ protoc --version

   If you built the C++ code without installing, the compiler binary
   should be located in ../src.

2) Invoke protoc to build DescriptorProtos.java:

     $ protoc --java_out=src/main/java -I../src \
         ../src/google/protobuf/descriptor.proto
67

68 69 70 71
3) Compile the code in src/main/java using whatever means you prefer.

4) Install the classes wherever you prefer.

Ulas Kirazci's avatar
Ulas Kirazci committed
72
Nano version
73
------------
Ulas Kirazci's avatar
Ulas Kirazci committed
74

75 76 77 78
JavaNano is a special code generator and runtime library designed specially for
resource-restricted systems, like Android. It is very resource-friendly in both
the amount of code and the runtime overhead. Here is an overview of JavaNano
features compared with the official Java protobuf:
79 80 81 82 83

- No descriptors or message builders.
- All messages are mutable; fields are public Java fields.
- For optional fields only, encapsulation behind setter/getter/hazzer/
  clearer functions is opt-in, which provide proper 'has' state support.
84 85 86 87 88
- For proto2, if not opted in, has state (field presence) is not available.
  Serialization outputs all fields not equal to their defaults
  (see important implications below).
  The behavior is consistent with proto3 semantics.
- Required fields (proto2 only) are always serialized.
89 90
- Enum constants are integers; protection against invalid values only
  when parsing from the wire.
91 92
- Enum constants can be generated into container interfaces bearing
  the enum's name (so the referencing code is in Java style).
93 94 95 96
- CodedInputByteBufferNano can only take byte[] (not InputStream).
- Similarly CodedOutputByteBufferNano can only write to byte[].
- Repeated fields are in arrays, not ArrayList or Vector. Null array
  elements are allowed and silently ignored.
97 98
- Full support for serializing/deserializing repeated packed fields.
- Support  extensions (in proto2).
Ulas Kirazci's avatar
Ulas Kirazci committed
99 100 101 102
- Unset messages/groups are null, not an immutable empty default
  instance.
- toByteArray(...) and mergeFrom(...) are now static functions of
  MessageNano.
103
- The 'bytes' type translates to the Java type byte[].
Ulas Kirazci's avatar
Ulas Kirazci committed
104

105 106 107 108 109
The generated messages are not thread-safe for writes, but may be
used simultaneously from multiple threads in a read-only manner.
In other words, an appropriate synchronization mechanism (such as
a ReadWriteLock) must be used to ensure that a message, its
ancestors, and descendants are not accessed by any other threads
110 111 112 113
while the message is being modified. Field reads, getter methods
(but not getExtension(...)), toByteArray(...), writeTo(...),
getCachedSize(), and getSerializedSize() are all considered read-only
operations.
114

115
IMPORTANT: If you have fields with defaults and opt out of accessors
Ulas Kirazci's avatar
Ulas Kirazci committed
116 117 118 119 120 121 122 123

How fields with defaults are serialized has changed. Because we don't
keep "has" state, any field equal to its default is assumed to be not
set and therefore is not serialized. Consider the situation where we
change the default value of a field. Senders compiled against an older
version of the proto continue to match against the old default, and
don't send values to the receiver even though the receiver assumes the
new default value. Therefore, think carefully about the implications
124 125
of changing the default value. Alternatively, turn on accessors and
enjoy the benefit of the explicit has() checks.
Ulas Kirazci's avatar
Ulas Kirazci committed
126 127 128 129 130 131 132 133 134 135

IMPORTANT: If you have "bytes" fields with non-empty defaults

Because the byte buffer is now of mutable type byte[], the default
static final cannot be exposed through a public field. Each time a
message's constructor or clear() function is called, the default value
(kept in a private byte[]) is cloned. This causes a small memory
penalty. This is not a problem if the field has no default or is an
empty default.

136
Nano Generator options
137
----------------------
138

139
```
140 141 142
java_package           -> <file-name>|<package-name>
java_outer_classname   -> <file-name>|<package-name>
java_multiple_files    -> true or false
143 144
java_nano_generate_has -> true or false [DEPRECATED]
optional_field_style   -> default or accessors
145
enum_style             -> c or java
146
ignore_services        -> true or false
147
parcelable_messages    -> true or false
148 149 150
```

**java_package=\<file-name\>|\<package-name\>** (no default)
151

152 153 154 155 156 157 158 159
  This allows overriding the 'java_package' option value
  for the given file from the command line. Use multiple
  java_package options to override the option for multiple
  files. The final Java package for each file is the value
  of this command line option if present, or the value of
  the same option defined in the file if present, or the
  proto package if present, or the default Java package.

160 161
**java_outer_classname=\<file-name\>|\<outer-classname\>** (no default)

162 163 164 165 166 167 168 169 170 171
  This allows overriding the 'java_outer_classname' option
  for the given file from the command line. Use multiple
  java_outer_classname options to override the option for
  multiple files. The final Java outer class name for each
  file is the value of this command line option if present,
  or the value of the same option defined in the file if
  present, or the file name converted to CamelCase. This
  outer class will nest all classes and integer constants
  generated from file-scope messages and enums.

172 173
**java_multiple_files={true,false}** (no default)

174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189
  This allows overriding the 'java_multiple_files' option
  in all source files and their imported files from the
  command line. The final value of this option for each
  file is the value defined in this command line option, or
  the value of the same option defined in the file if
  present, or false. This specifies whether to generate
  package-level classes for the file-scope messages in the
  same Java package as the outer class (instead of nested
  classes in the outer class). File-scope enum constants
  are still generated as integer constants in the outer
  class. This affects the fully qualified references in the
  Java code. NOTE: because the command line option
  overrides the value for all files and their imported
  files, using this option inconsistently may result in
  incorrect references to the imported messages and enum
  constants.
190

191 192
**java_nano_generate_has={true,false}** (default: false)

193 194
  DEPRECATED. Use optional_field_style=accessors.

195
  If true, generates a public boolean variable has\<fieldname\>
196
  accompanying each optional or required field (not present for
197 198 199 200 201 202 203 204 205 206 207
  repeated fields, groups or messages). It is set to false initially
  and upon clear(). If parseFrom(...) reads the field from the wire,
  it is set to true. This is a way for clients to inspect the "has"
  value upon parse. If it is set to true, writeTo(...) will ALWAYS
  output that field (even if field value is equal to its
  default).

  IMPORTANT: This option costs an extra 4 bytes per primitive field in
  the message. Think carefully about whether you really need this. In
  many cases reading the default works and determining whether the
  field was received over the wire is irrelevant.
Ulas Kirazci's avatar
Ulas Kirazci committed
208

209 210
**optional_field_style={default,accessors,reftypes}** (default: default)

211 212
  Defines the style of the generated code for fields.

213
  * default
214

215 216
  In the default style, optional fields translate into public mutable
  Java fields, and the serialization process is as discussed in the
217
  "IMPORTANT" section above.
218

219
  * accessors
220 221

  When set to 'accessors', each optional field is encapsulated behind
222 223
  4 accessors, namely get\<fieldname\>(), set\<fieldname\>(), has\<fieldname\>()
  and clear\<fieldname\>() methods, with the standard semantics. The hazzer's
224 225 226 227
  return value determines whether a field is serialized, so this style is
  useful when you need to serialize a field with the default value, or check
  if a field has been explicitly set to its default value from the wire.

228 229 230
  In the 'accessors' style, required and nested message fields are still
  translated to one public mutable Java field each, repeated fields are still
  translated to arrays. No accessors are generated for them.
231 232

  IMPORTANT: When using the 'accessors' style, ProGuard should always
233 234 235 236 237 238
  be enabled with optimization (don't use -dontoptimize) and allowing
  access modification (use -allowaccessmodification). This removes the
  unused accessors and maybe inline the rest at the call sites,
  reducing the final code size.
  TODO(maxtroy): find ProGuard config that would work the best.

239
  * reftypes
240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268

  When set to 'reftypes', each proto field is generated as a public Java
  field. For primitive types, these fields use the Java reference types
  such as java.lang.Integer instead of primitive types such as int.

  In the 'reftypes' style, fields are initialized to null (or empty
  arrays for repeated fields), and their default values are not available.
  They are serialized over the wire based on equality to null.

  The 'reftypes' mode has some additional cost due to autoboxing and usage
  of reference types. In practice, many boxed types are cached, and so don't
  result in object creation. However, references do take slightly more memory
  than primitives.

  The 'reftypes' mode is useful when you want to be able to serialize fields
  with default values, or check if a field has been explicitly set to the
  default over the wire without paying the extra method cost of the
  'accessors' mode.

  Note that if you attempt to write null to a required field in the reftypes
  mode, serialization of the proto will cause a NullPointerException. This is
  an intentional indicator that you must set required fields.

  NOTE
  optional_field_style=accessors or reftypes cannot be used together with
  java_nano_generate_has=true. If you need the 'has' flag for any
  required field (you have no reason to), you can only use
  java_nano_generate_has=true.

269 270
**enum_style={c,java}** (default: c)

271 272
  Defines where to put the int constants generated from enum members.

273
  * c
274 275 276 277 278 279 280

  Use C-style, so the enum constants are available at the scope where
  the enum is defined. A file-scope enum's members are referenced like
  'FileOuterClass.ENUM_VALUE'; a message-scope enum's members are
  referenced as 'Message.ENUM_VALUE'. The enum name is unavailable.
  This complies with the Micro code generator's behavior.

281
  * java
282 283 284 285 286 287 288 289 290 291 292

  Use Java-style, so the enum constants are available under the enum
  name and referenced like 'EnumName.ENUM_VALUE' (they are still int
  constants). The enum name becomes the name of a public interface, at
  the scope where the enum is defined. If the enum is file-scope and
  the java_multiple_files option is on, the interface will be defined
  in its own file. To reduce code size, this interface should not be
  implemented and ProGuard shrinking should be used, so after the Java
  compiler inlines all referenced enum constants into the call sites,
  the interface remains unused and can be removed by ProGuard.

293 294
**ignore_services={true,false}** (default: false)

295 296 297 298 299 300
  Skips services definitions.

  Nano doesn't support services. By default, if a service is defined
  it will generate a compilation error. If this flag is set to true,
  services will be silently ignored, instead.

301 302
**parcelable_messages={true,false}** (default: false)

303 304
  Android-specific option to generate Parcelable messages.

305 306

To use nano protobufs within the Android repo:
307
----------------------------------------------
308 309 310 311 312 313 314

- Set 'LOCAL_PROTOC_OPTIMIZE_TYPE := nano' in your local .mk file.
  When building a Java library or an app (package) target, the build
  system will add the Java nano runtime library to the
  LOCAL_STATIC_JAVA_LIBRARIES variable, so you don't need to.
- Set 'LOCAL_PROTO_JAVA_OUTPUT_PARAMS := ...' in your local .mk file
  for any command-line options you need. Use commas to join multiple
315 316 317
  options. In the nano flavor only, whitespace surrounding the option
  names and values are ignored, so you can use backslash-newline or
  '+=' to structure your make files nicely.
318 319 320 321 322 323 324 325 326 327 328 329 330 331
- The options will be applied to *all* proto files in LOCAL_SRC_FILES
  when you build a Java library or package. In case different options
  are needed for different proto files, build separate Java libraries
  and reference them in your main target. Note: you should make sure
  that, for each separate target, all proto files imported from any
  proto file in LOCAL_SRC_FILES are included in LOCAL_SRC_FILES. This
  is because the generator has to assume that the imported files are
  built using the same options, and will generate code that reference
  the fields and enums from the imported files using the same code
  style.
- Hint: 'include $(CLEAR_VARS)' resets all LOCAL_ variables, including
  the two above.

To use nano protobufs outside of Android repo:
332
----------------------------------------------
Ulas Kirazci's avatar
Ulas Kirazci committed
333 334

- Link with the generated jar file
335
  \<protobuf-root\>java/target/protobuf-java-2.3.0-nano.jar.
Ulas Kirazci's avatar
Ulas Kirazci committed
336
- Invoke with --javanano_out, e.g.:
337
```
338
./protoc '--javanano_out=\
339 340 341
    java_package=src/proto/simple-data.proto|my_package,\
    java_outer_classname=src/proto/simple-data.proto|OuterName\
  :.' src/proto/simple-data.proto
342
```
343

344
Contributing to nano:
345
---------------------
346 347 348 349 350 351 352 353 354 355 356 357 358

Please add/edit tests in NanoTest.java.

Please run the following steps to test:

- cd external/protobuf
- ./configure
- Run "make -j12 check" and verify all tests pass.
- cd java
- Run "mvn test" and verify all tests pass.
- cd ../../..
- . build/envsetup.sh
- lunch 1
359
- "make -j12 aprotoc libprotobuf-java-2.3.0-nano aprotoc-test-nano-params NanoAndroidTest" and
360
  check for build errors.
361 362 363 364 365
- Plug in an Android device or start an emulator.
- adb install -r out/target/product/generic/data/app/NanoAndroidTest.apk
- Run:
  "adb shell am instrument -w com.google.protobuf.nano.test/android.test.InstrumentationTestRunner"
  and verify all tests pass.
366 367 368
- repo sync -c -j256
- "make -j12" and check for build errors

369
Usage
370
-----
371 372 373 374

The complete documentation for Protocol Buffers is available via the
web at:

375
    https://developers.google.com/protocol-buffers/