1. 24 Jul, 2019 5 commits
  2. 23 Jul, 2019 4 commits
  3. 22 Jul, 2019 4 commits
  4. 19 Jul, 2019 5 commits
  5. 18 Jul, 2019 1 commit
  6. 17 Jul, 2019 3 commits
    • Adam Cozzette's avatar
      Updated cmake path to io_win32.h (#6397) · 582640ef
      Adam Cozzette authored
      Fixes #6382.
      582640ef
    • Adam Liddell's avatar
      Remove zlib copts flags when on Windows · 19ca196e
      Adam Liddell authored
      19ca196e
    • Po-Chuan Hsieh's avatar
      Fix test on FreeBSD · 640b932c
      Po-Chuan Hsieh authored
      google/protobuf/testing/zcgzip.cc:61:25: error: unknown type name 'STDOUT_FILENO'
        FileOutputStream fout(STDOUT_FILENO);
                              ^
      google/protobuf/testing/zcgzip.cc:61:24: warning: parentheses were disambiguated as a function declaration [-Wvexing-parse]
        FileOutputStream fout(STDOUT_FILENO);
                             ^~~~~~~~~~~~~~~
      google/protobuf/testing/zcgzip.cc:61:25: note: add a pair of parentheses to declare a variable
        FileOutputStream fout(STDOUT_FILENO);
                              ^
                              (
      google/protobuf/testing/zcgzip.cc:75:20: error: use of undeclared identifier 'STDIN_FILENO'
          readlen = read(STDIN_FILENO, outptr, outlen);
                         ^
      1 warning and 2 errors generated.
      gmake[3]: *** [Makefile:4009: google/protobuf/testing/zcgzip.o] Error 1
      
      google/protobuf/testing/zcgunzip.cc:62:23: error: unknown type name 'STDIN_FILENO'
        FileInputStream fin(STDIN_FILENO);
                            ^
      google/protobuf/testing/zcgunzip.cc:62:22: warning: parentheses were disambiguated as a function declaration [-Wvexing-parse]
        FileInputStream fin(STDIN_FILENO);
                           ^~~~~~~~~~~~~~
      google/protobuf/testing/zcgunzip.cc:62:23: note: add a pair of parentheses to declare a variable
        FileInputStream fin(STDIN_FILENO);
                            ^
                            (
      google/protobuf/testing/zcgunzip.cc:74:23: error: use of undeclared identifier 'STDOUT_FILENO'
            int err = write(STDOUT_FILENO, inptr, inlen);
                            ^
      1 warning and 2 errors generated.
      gmake[3]: *** [Makefile:4009: google/protobuf/testing/zcgunzip.o] Error 1
      
      Reference:	https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=215346
      		https://svnweb.FreeBSD.org/changeset/ports/428734
      640b932c
  7. 15 Jul, 2019 2 commits
  8. 12 Jul, 2019 1 commit
  9. 10 Jul, 2019 5 commits
    • Hao Nguyen's avatar
      Merge pull request #6331 from elharo/patch-2 · fc9fd4d4
      Hao Nguyen authored
      Update guava
      fc9fd4d4
    • Hao Nguyen's avatar
      Merge pull request #6330 from elharo/patch-1 · a387c102
      Hao Nguyen authored
      Update gson
      a387c102
    • Hao Nguyen's avatar
      Merge pull request #6348 from elharo/patch-4 · 90bb99c8
      Hao Nguyen authored
      Update version in README
      90bb99c8
    • Chris Gaffney's avatar
      ruby: Improve performance of Google::Protobuf::Timestamp#to_time (#6360) · 7da7bec4
      Chris Gaffney authored
      This changes to_time to use Ruby's built in Time.at with nanos support
      rather than calculating a float and passing it to Time.at. The new
      version runs about 3 times faster than the original version and
      allocates fewer objects.
      
      Warming up --------------------------------------
          protobuf#to_time    57.296k i/100ms
            faster#to_time   133.229k i/100ms
      Calculating -------------------------------------
          protobuf#to_time    635.361k (± 2.1%) i/s -      3.209M in   5.052169s
            faster#to_time      1.873M (± 3.3%) i/s -      9.459M in   5.055169s
      
      Comparison:
            faster#to_time:  1873368.8 i/s
          protobuf#to_time:   635361.4 i/s - 2.95x  slower
      
      Calculating -------------------------------------
          protobuf#to_time   326.000  memsize (   126.000  retained)
                               7.000  objects (     2.000  retained)
                               0.000  strings (     0.000  retained)
            faster#to_time    86.000  memsize (     0.000  retained)
                               1.000  objects (     0.000  retained)
                               0.000  strings (     0.000  retained)
      
      Comparison:
            faster#to_time:         86 allocated
          protobuf#to_time:        326 allocated - 3.79x more
      7da7bec4
    • Ben Gordon's avatar
      Requesting extension id 1072 · f5362e11
      Ben Gordon authored
      This library supports an idiomatic proto3 protobuf generator for kotlin.
      The library will be open sourced by Toast Inc under the Apache2 license, and is currently used in production at Toast.
      The following is the readme.md that will be released with the code by the end of Q4 2019.
      
      Supports only the Protocol Buffers language version 3.
      
      #### Features
      - Clean data class generation
      - Oneof types handled as sealed classes
      - JavaDoc comments on generated code
      - Deprecation option pass-through to Kotlin's `@Deprecated` annotation
      - Protokt-specific options: non-null types, wrapper types, interface implementation,
      and more
      - Tight integration with Protobuf's Java library: compatibility with its well-known
      types and usage of CodedInputStream and CodedOutputStream for best performance
      
      #### Not yet implemented
      - Kotlin native support
      - Kotlin JS support
      - Support for gRPC service generation
      - Protobuf JSON support
      
      See examples in [protokt-testing](https://github.com/toasttab/protokt/tree/master/protokt-testing).
      
      ### Generated Code
      Generated code is placed in `<buildDir>/generated-sources/main/protokt`.
      
      A simple example:
      ```proto
      syntax = "proto3";
      
      package com.protokt.sample;
      
      message Sample {
        string sample_field = 1;
      }
      ```
      
      will produce:
      ```kotlin
      /*
       * Generated by protokt. Do not modify.
       */
      package com.protokt.sample
      
      import com.toasttab.protokt.rt.*
      
      data class Sample(
          val sampleField: String,
          val unknown: Map<Int, Unknown> = emptyMap()
      ) : KtMessage {
          @Suppress("UNUSED")
          constructor(
              sampleField: String = ""
          ) : this(
              sampleField,
              emptyMap()
          )
      
          override val messageSize by lazy { sizeof() }
      
          override fun serialize(serializer: KtMessageSerializer) {
              if (sampleField.isNotEmpty()) {
                  serializer.write(Tag(10)).write(sampleField)
              }
              if (unknown.isNotEmpty()) {
                  serializer.writeUnknown(unknown)
              }
          }
      
          private fun sizeof(): Int {
              var res = 0
              if (sampleField.isNotEmpty()) {
                  res += sizeof(Tag(1)) + sizeof(sampleField)
              }
              res += unknown.entries.sumBy { it.value.sizeof() }
              return res
          }
      
          companion object Deserializer : KtDeserializer<Sample> {
              override fun deserialize(deserializer: KtMessageDeserializer): Sample {
                  var sampleField = ""
                  val unknown = mutableMapOf<Int, Unknown>()
                  while (true) {
                      when (deserializer.readTag()) {
                          0 ->
                              return Sample(
                                  sampleField,
                                  unknown
                              )
                          10 -> sampleField = deserializer.readString()
                          else -> {
                              val unk = deserializer.readUnknown()
                              unknown[unk.fieldNum] = unknown[unk.fieldNum].let {
                                  when (it) {
                                      null -> unk
                                      else ->
                                          when (val v = it.value) {
                                              is ListVal ->
                                                  Unknown(unk.fieldNum, ListVal(v.value + unk.value))
                                              else ->
                                                  Unknown(unk.fieldNum, ListVal(listOf(v, unk.value)))
                                          }
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }
      ```
      
      #### Runtime Notes
      ##### Package
      The Kotlin package of a generated file can be overridden from protobuf package with the `(protokt).package` option:
      ```proto
      syntax = "proto3";
      
      import "protokt.proto";
      
      package com.example;
      
      option (protokt).package = "com.package";
      ```
      
      ##### Message
      Each protokt message implements the `KtMessage` interface. `KtMessage` defines the `serialize()`
      method and its overloads which can serialize to a byte array, a `KtMessageSerializer`, or on the JVM,
      an `OutputStream`.
      
      Each protokt message has a companion object `Deserializer` that implements the `KtDeserializer`
      interface, which provides the `deserialize()` method and its overloads to construct an
      instance of the message from a byte array, a Java InputStream, or others.
      
      In order to enjoy the full benefits of Kotlin data classes, byte arrays are wrapped in the
      protokt `Bytes` class, which provides appropriate `equals()` and `hashCode()` implementations.
      
      ##### Enums
      Enum fields are generated as data classes with a single integer field. Kotlin enum classes are
      closed and cannot retain unknown values, and protobuf requires that unknown enum values are
      preserved for reserialization. This compromise exposes a constructor taking an integer, but the
      `from(value: Int)` on an enum's `Deserializer` should be preferred as it avoids instantiation
      when possible.
      
      
      Other notes:
      - `optimize_for` is ignored.
      - `repeated` fields are deserialized to Lists.
      - `map` fields are deserialized to Maps.
      - `oneof` fields are represented as data class subtypes of a sealed base class with a single property.
      
      ### Extensions
      See examples of each option in the [protokt-options](https://github.com/toasttab/protokt/tree/master/protokt-testing/protokt-options/src/main/proto)
      module. All protokt-specific options require importing `protokt.proto` in the protocol file.
      #### Wrapper Types
      Sometimes a field on a protobuf message corresponds to a concrete nonprimitive type. In
      standard protobuf the user would be responsible for this extra transformation, but the
      protokt wrapper type option allows specification of a converter that will automatically
      encode and decode custom types to protobuf primitives and well-known types. Some standard
      types are implemented in
      [protokt-extensions](https://github.com/toasttab/protokt/tree/master/protokt-extensions/src/main/kotlin/com/toasttab/protokt/ext).
      
      Wrap a field by invoking the `(protokt_property).wrap` option:
      ```proto
      message DateWrapperMessage {
        int64 date = 1 [
          (protokt_property).wrap = "java.util.Date"
        ];
      }
      ```
      
      Converters implement the `Converter` interface:
      ```kotlin
      interface Converter<S: Any, T: Any> {
          val wrapper: KClass<S>
      
          fun wrap(unwrapped: T): S
      
          fun unwrap(wrapped: S): T
      }
      ```
      
      and protokt will reference the converter's methods to wrap and unwrap from protobuf primitives:
      ```kotlin
      object DateConverter : Converter<Date, Long> {
          override val wrapper = Date::class
      
          override fun wrap(unwrapped: Long) =
              Date(unwrapped)
      
          override fun unwrap(wrapped: Date) =
              wrapped.time
      }
      ```
      
      ```kotlin
      data class WrapperModel(
          val date: java.util.Date,
          ...
      ) : KtMessage {
          ...
          override fun serialize(serializer: KtMessageSerializer) {
              serializer.write(Tag(10)).write(Int64(DateConverter.unwrap(date)))
              ...
          }
      
          override fun deserialize(deserializer: KtMessageDeserializer): WrapperModel {
              var date = 0L
      
              while (true) {
                  when (deserializer.readTag()) {
                      0 ->
                          return WrapperModel(
                              DateConverter.wrap(date),
                              ...
                          )
                      ...
                  }
              }
          }
      }
      ```
      
      Converters can also implement the `OptimizedSizeofConverter` interface adding `sizeof()`,
      which allows them to optimize the calculation of the wrapper's size rather than unwrap
      the object twice. For example, a UUID is always 16 bytes:
      
      ```kotlin
      object UuidConverter : OptimizedSizeofConverter<UUID, ByteArray> {
          override val wrapper = UUID::class
      
          private val sizeofProxy = ByteArray(16)
      
          override fun sizeof(wrapped: UUID) =
              sizeof(sizeofProxy)
      
          override fun wrap(unwrapped: ByteArray): UUID {
              require(unwrapped.size == 16) {
                  "input must have size 16; had ${unwrapped.size}"
              }
      
              return ByteBuffer.wrap(unwrapped)
                  .run { UUID(long, long) }
          }
      
          override fun unwrap(wrapped: UUID) =
              ByteBuffer.allocate(16)
                  .putLong(wrapped.mostSignificantBits)
                  .putLong(wrapped.leastSignificantBits)
                  .array()
      }
      ```
      
      Rather than convert a UUID to a byte array both for size calculation and for serialization
      (which is what a naïve implementation would do), UuidConverter always returns the size of a
      constant 16-byte array.
      
      If the wrapper type is in the same package as the generated protobuf message, then it
      does not need a fully-qualified name. Custom wrapper type converters can be in the same module as
      protobuf types that reference them. In order to use any wrapper type defined in
      `protokt-extensions`, the module must be included as a dependency:
      
      ```groovy
      dependencies {
          implementation 'com.toasttab.protokt:protokt-extensions:0.0.3'
      }
      ```
      
      #### Interface implementation
      To avoid the need to create domain-specific objects from protobuf messages you can declare
      that a protobuf message implements a custom interface with properties and default methods.
      
      ```kotlin
      package com.protokt.sample
      
      interface Model {
          val id: String
      }
      ```
      
      ```proto
      package com.protokt.sample;
      
      message ImplementsSampleMessage {
        option (protokt_class).implements = "Model";
      
        string id = 1;
      }
      ```
      
      If the wrapper interface is in the same package as the generated protobuf message, then it
      does not need a fully-qualified name. Wrapper interfaces cannot be used by protobuf messages
      in the same module that defines them; the dependency must be declared with`protoktExtensions`
      in `build.gradle`:
      
      ```groovy
      dependencies {
          protoktExtensions project(':api-module')
      }
      ```
      
      #### Nonnull fields
      If there is a message that has no meaning whatsoever when a particular field is missing, you
      can emulate proto2's `required` key word by using the `(protokt_oneof).non_null` option:
      
      ```proto
      message Sample {
      }
      
      message NonNullSampleMessage {
        Sample non_null_sample = 1 [
          (protokt_property).non_null = true
        ];
      }
      ```
      
      Generated code will not have a nullable type so the field can be referenced without using
      Kotlin's `!!`.
      
      Oneof fields can also be declared non-null:
      
      ```proto
      message NonNullSampleMessage {
        oneof non_null_oneof {
          option (protokt_oneof).non_null = true;
      
          string message = 2;
        }
      }
      ```
      
      Note that deserialization of a message with a non-nullable field will fail if the
      message being decoded does not contain an instance of the required field.
      
      #### BytesSlice
      When reading messages that contain other serialized messages as `bytes` fields, protokt can
      keep a reference to the originating byte array to prevent a large copy operation on
      deserialization. This can be desirable when the wrapping message is a thin metadata shim and
      doesn't include much memory overhead:
      
      ```proto
      message SliceModel {
        int64 version = 1;
      
        bytes encoded_message = 2 [
          (protokt_property).bytes_slice = true
        ];
      }
      ```
      
      ### Usage
      
      #### Gradle
      
      ```groovy
      buildscript {
          dependencies {
              classpath "com.toasttab.protokt:protokt-gradle-plugin:0.0.3"
          }
      }
      
      apply plugin: 'com.toasttab.protokt'
      ```
      
      This will automatically download and install protokt, apply the Google protobuf plugin,
      and configure all the necessary boilerplate. By default it will also add `protokt-runtime`
      to the api scope of the project, and `protobuf-java` to the implementation scope.
      
      If your project is pure Kotlin you may run into the following error:
      
      ```
      Execution failed for task ':compileJava'.
      > error: no source files
      ```
      
      To work around it, disable all `JavaCompile` tasks in the project:
      
      ```groovy
      tasks.withType(JavaCompile) {
          enabled = false
      }
      ```
      
      or:
      ```groovy
      compileJava.enabled = false
      ```
      
      #### Command line code generation
      
      ```bash
      protokt-codegen$ ./gradlew assemble [OR ./gradlew installDist]
      
      protokt-codegen$ ./run-protokt.sh -h
      
      protokt-codegen$ ./run-protokt.sh \
        -out=../kotlin \
        -pkg=com.toasttab.protokt.conformance \
        -file=conformance.proto \
        -cp=../build/libs/protokt-codegen-0.0.3-SNAPSHOT-all.jar \
        -plugin=../bin/protokt.sh
      ```
      
      ### Contribution
      To enable rapid development of the code generator, the protobuf conformance tests have been 
      compiled and included in the protokt-testing project. They run on Mac OS 10.14+ and Ubuntu
      16.04 x86-64.
      
      Publish the plugin to the integration repository:
      ```bash
      protokt$ ./gradlew publishToIntegrationRepository
      ```
      
      Then run the tests from `protokt-testing`:
      ```bash
      protokt-testing$ ./gradlew protokt-conformance-tests:test
      ```
      
      All integration tests can be run with:
      ```
      protokt-testing$ ./gradlew test
      ```
      f5362e11
  10. 09 Jul, 2019 2 commits
  11. 08 Jul, 2019 4 commits
  12. 07 Jul, 2019 1 commit
  13. 05 Jul, 2019 1 commit
  14. 03 Jul, 2019 2 commits