- 24 Jul, 2019 5 commits
-
-
Yannic Bonenberger authored
-
Yannic Bonenberger authored
-
Yannic Bonenberger authored
-
Yannic Bonenberger authored
-
Yannic Bonenberger authored
-
- 23 Jul, 2019 4 commits
-
-
Paul Yang authored
* Add binary conformance test for enum fields. * Add failed conformance test to js failure list
-
rmstar authored
-
Paul Yang authored
* Add binary conformance test case for bytes field * Add failed conformance test to php's failure list
-
Joe Bolinger authored
-
- 22 Jul, 2019 4 commits
-
-
-
Paul Yang authored
* Add conformance test for binary string * Add failing tests to php failure list
-
-
Wang Kirin authored
-
- 19 Jul, 2019 5 commits
-
-
Sydney Acksman authored
* Fix binary compatibility in FieldCodec factory messages * Make default value parameter for current factories required * Route old methods through default value overloads
-
Leon Barrett authored
Currently, if you access a ListValue from a Struct and attempted to assign it to another Struct, you would get an exception: > s1 = spb.Struct() > s1['a'] = [1] > s2 = spb.Struct() > s2['a'] = s1['a'] ValueError: Unexpected type This fixes that case.
-
Paul Yang authored
* Down integrate to GitHub * Fix broken tests
-
Adam Cozzette authored
I'm not exactly sure why, but this fixes the failing Bazel presubmit test. Using the most recent version seems like a good idea anyway so that we can make sure we're compatible with any new Bazel changes.
-
Thomas Van Lenten authored
Since true for objc is external, changes should always start on the github side, so the fact that something got pushed out is a sign something was broken in the process.
-
- 18 Jul, 2019 1 commit
-
-
Yannic Bonenberger authored
-
- 17 Jul, 2019 3 commits
-
-
Adam Cozzette authored
Fixes #6382.
-
Adam Liddell authored
-
Po-Chuan Hsieh authored
google/protobuf/testing/zcgzip.cc:61:25: error: unknown type name 'STDOUT_FILENO' FileOutputStream fout(STDOUT_FILENO); ^ google/protobuf/testing/zcgzip.cc:61:24: warning: parentheses were disambiguated as a function declaration [-Wvexing-parse] FileOutputStream fout(STDOUT_FILENO); ^~~~~~~~~~~~~~~ google/protobuf/testing/zcgzip.cc:61:25: note: add a pair of parentheses to declare a variable FileOutputStream fout(STDOUT_FILENO); ^ ( google/protobuf/testing/zcgzip.cc:75:20: error: use of undeclared identifier 'STDIN_FILENO' readlen = read(STDIN_FILENO, outptr, outlen); ^ 1 warning and 2 errors generated. gmake[3]: *** [Makefile:4009: google/protobuf/testing/zcgzip.o] Error 1 google/protobuf/testing/zcgunzip.cc:62:23: error: unknown type name 'STDIN_FILENO' FileInputStream fin(STDIN_FILENO); ^ google/protobuf/testing/zcgunzip.cc:62:22: warning: parentheses were disambiguated as a function declaration [-Wvexing-parse] FileInputStream fin(STDIN_FILENO); ^~~~~~~~~~~~~~ google/protobuf/testing/zcgunzip.cc:62:23: note: add a pair of parentheses to declare a variable FileInputStream fin(STDIN_FILENO); ^ ( google/protobuf/testing/zcgunzip.cc:74:23: error: use of undeclared identifier 'STDOUT_FILENO' int err = write(STDOUT_FILENO, inptr, inlen); ^ 1 warning and 2 errors generated. gmake[3]: *** [Makefile:4009: google/protobuf/testing/zcgunzip.o] Error 1 Reference: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=215346 https://svnweb.FreeBSD.org/changeset/ports/428734
-
- 15 Jul, 2019 2 commits
-
-
Seth Greenstein authored
Following the instructions as is does not work, the new_local_repository rule needs to be called "python_headers"
-
Adam Liddell authored
* Fix reference to six in //:protobuf_python rule * Add six to protobuf_deps.bzl * Use six archive directly as repo @six
-
- 12 Jul, 2019 1 commit
-
-
Hao Nguyen authored
Requesting extension id 1072
-
- 10 Jul, 2019 5 commits
-
-
Hao Nguyen authored
Update guava
-
Hao Nguyen authored
Update gson
-
Hao Nguyen authored
Update version in README
-
Chris Gaffney authored
This changes to_time to use Ruby's built in Time.at with nanos support rather than calculating a float and passing it to Time.at. The new version runs about 3 times faster than the original version and allocates fewer objects. Warming up -------------------------------------- protobuf#to_time 57.296k i/100ms faster#to_time 133.229k i/100ms Calculating ------------------------------------- protobuf#to_time 635.361k (± 2.1%) i/s - 3.209M in 5.052169s faster#to_time 1.873M (± 3.3%) i/s - 9.459M in 5.055169s Comparison: faster#to_time: 1873368.8 i/s protobuf#to_time: 635361.4 i/s - 2.95x slower Calculating ------------------------------------- protobuf#to_time 326.000 memsize ( 126.000 retained) 7.000 objects ( 2.000 retained) 0.000 strings ( 0.000 retained) faster#to_time 86.000 memsize ( 0.000 retained) 1.000 objects ( 0.000 retained) 0.000 strings ( 0.000 retained) Comparison: faster#to_time: 86 allocated protobuf#to_time: 326 allocated - 3.79x more
-
Ben Gordon authored
This library supports an idiomatic proto3 protobuf generator for kotlin. The library will be open sourced by Toast Inc under the Apache2 license, and is currently used in production at Toast. The following is the readme.md that will be released with the code by the end of Q4 2019. Supports only the Protocol Buffers language version 3. #### Features - Clean data class generation - Oneof types handled as sealed classes - JavaDoc comments on generated code - Deprecation option pass-through to Kotlin's `@Deprecated` annotation - Protokt-specific options: non-null types, wrapper types, interface implementation, and more - Tight integration with Protobuf's Java library: compatibility with its well-known types and usage of CodedInputStream and CodedOutputStream for best performance #### Not yet implemented - Kotlin native support - Kotlin JS support - Support for gRPC service generation - Protobuf JSON support See examples in [protokt-testing](https://github.com/toasttab/protokt/tree/master/protokt-testing). ### Generated Code Generated code is placed in `<buildDir>/generated-sources/main/protokt`. A simple example: ```proto syntax = "proto3"; package com.protokt.sample; message Sample { string sample_field = 1; } ``` will produce: ```kotlin /* * Generated by protokt. Do not modify. */ package com.protokt.sample import com.toasttab.protokt.rt.* data class Sample( val sampleField: String, val unknown: Map<Int, Unknown> = emptyMap() ) : KtMessage { @Suppress("UNUSED") constructor( sampleField: String = "" ) : this( sampleField, emptyMap() ) override val messageSize by lazy { sizeof() } override fun serialize(serializer: KtMessageSerializer) { if (sampleField.isNotEmpty()) { serializer.write(Tag(10)).write(sampleField) } if (unknown.isNotEmpty()) { serializer.writeUnknown(unknown) } } private fun sizeof(): Int { var res = 0 if (sampleField.isNotEmpty()) { res += sizeof(Tag(1)) + sizeof(sampleField) } res += unknown.entries.sumBy { it.value.sizeof() } return res } companion object Deserializer : KtDeserializer<Sample> { override fun deserialize(deserializer: KtMessageDeserializer): Sample { var sampleField = "" val unknown = mutableMapOf<Int, Unknown>() while (true) { when (deserializer.readTag()) { 0 -> return Sample( sampleField, unknown ) 10 -> sampleField = deserializer.readString() else -> { val unk = deserializer.readUnknown() unknown[unk.fieldNum] = unknown[unk.fieldNum].let { when (it) { null -> unk else -> when (val v = it.value) { is ListVal -> Unknown(unk.fieldNum, ListVal(v.value + unk.value)) else -> Unknown(unk.fieldNum, ListVal(listOf(v, unk.value))) } } } } } } } } } ``` #### Runtime Notes ##### Package The Kotlin package of a generated file can be overridden from protobuf package with the `(protokt).package` option: ```proto syntax = "proto3"; import "protokt.proto"; package com.example; option (protokt).package = "com.package"; ``` ##### Message Each protokt message implements the `KtMessage` interface. `KtMessage` defines the `serialize()` method and its overloads which can serialize to a byte array, a `KtMessageSerializer`, or on the JVM, an `OutputStream`. Each protokt message has a companion object `Deserializer` that implements the `KtDeserializer` interface, which provides the `deserialize()` method and its overloads to construct an instance of the message from a byte array, a Java InputStream, or others. In order to enjoy the full benefits of Kotlin data classes, byte arrays are wrapped in the protokt `Bytes` class, which provides appropriate `equals()` and `hashCode()` implementations. ##### Enums Enum fields are generated as data classes with a single integer field. Kotlin enum classes are closed and cannot retain unknown values, and protobuf requires that unknown enum values are preserved for reserialization. This compromise exposes a constructor taking an integer, but the `from(value: Int)` on an enum's `Deserializer` should be preferred as it avoids instantiation when possible. Other notes: - `optimize_for` is ignored. - `repeated` fields are deserialized to Lists. - `map` fields are deserialized to Maps. - `oneof` fields are represented as data class subtypes of a sealed base class with a single property. ### Extensions See examples of each option in the [protokt-options](https://github.com/toasttab/protokt/tree/master/protokt-testing/protokt-options/src/main/proto) module. All protokt-specific options require importing `protokt.proto` in the protocol file. #### Wrapper Types Sometimes a field on a protobuf message corresponds to a concrete nonprimitive type. In standard protobuf the user would be responsible for this extra transformation, but the protokt wrapper type option allows specification of a converter that will automatically encode and decode custom types to protobuf primitives and well-known types. Some standard types are implemented in [protokt-extensions](https://github.com/toasttab/protokt/tree/master/protokt-extensions/src/main/kotlin/com/toasttab/protokt/ext). Wrap a field by invoking the `(protokt_property).wrap` option: ```proto message DateWrapperMessage { int64 date = 1 [ (protokt_property).wrap = "java.util.Date" ]; } ``` Converters implement the `Converter` interface: ```kotlin interface Converter<S: Any, T: Any> { val wrapper: KClass<S> fun wrap(unwrapped: T): S fun unwrap(wrapped: S): T } ``` and protokt will reference the converter's methods to wrap and unwrap from protobuf primitives: ```kotlin object DateConverter : Converter<Date, Long> { override val wrapper = Date::class override fun wrap(unwrapped: Long) = Date(unwrapped) override fun unwrap(wrapped: Date) = wrapped.time } ``` ```kotlin data class WrapperModel( val date: java.util.Date, ... ) : KtMessage { ... override fun serialize(serializer: KtMessageSerializer) { serializer.write(Tag(10)).write(Int64(DateConverter.unwrap(date))) ... } override fun deserialize(deserializer: KtMessageDeserializer): WrapperModel { var date = 0L while (true) { when (deserializer.readTag()) { 0 -> return WrapperModel( DateConverter.wrap(date), ... ) ... } } } } ``` Converters can also implement the `OptimizedSizeofConverter` interface adding `sizeof()`, which allows them to optimize the calculation of the wrapper's size rather than unwrap the object twice. For example, a UUID is always 16 bytes: ```kotlin object UuidConverter : OptimizedSizeofConverter<UUID, ByteArray> { override val wrapper = UUID::class private val sizeofProxy = ByteArray(16) override fun sizeof(wrapped: UUID) = sizeof(sizeofProxy) override fun wrap(unwrapped: ByteArray): UUID { require(unwrapped.size == 16) { "input must have size 16; had ${unwrapped.size}" } return ByteBuffer.wrap(unwrapped) .run { UUID(long, long) } } override fun unwrap(wrapped: UUID) = ByteBuffer.allocate(16) .putLong(wrapped.mostSignificantBits) .putLong(wrapped.leastSignificantBits) .array() } ``` Rather than convert a UUID to a byte array both for size calculation and for serialization (which is what a naïve implementation would do), UuidConverter always returns the size of a constant 16-byte array. If the wrapper type is in the same package as the generated protobuf message, then it does not need a fully-qualified name. Custom wrapper type converters can be in the same module as protobuf types that reference them. In order to use any wrapper type defined in `protokt-extensions`, the module must be included as a dependency: ```groovy dependencies { implementation 'com.toasttab.protokt:protokt-extensions:0.0.3' } ``` #### Interface implementation To avoid the need to create domain-specific objects from protobuf messages you can declare that a protobuf message implements a custom interface with properties and default methods. ```kotlin package com.protokt.sample interface Model { val id: String } ``` ```proto package com.protokt.sample; message ImplementsSampleMessage { option (protokt_class).implements = "Model"; string id = 1; } ``` If the wrapper interface is in the same package as the generated protobuf message, then it does not need a fully-qualified name. Wrapper interfaces cannot be used by protobuf messages in the same module that defines them; the dependency must be declared with`protoktExtensions` in `build.gradle`: ```groovy dependencies { protoktExtensions project(':api-module') } ``` #### Nonnull fields If there is a message that has no meaning whatsoever when a particular field is missing, you can emulate proto2's `required` key word by using the `(protokt_oneof).non_null` option: ```proto message Sample { } message NonNullSampleMessage { Sample non_null_sample = 1 [ (protokt_property).non_null = true ]; } ``` Generated code will not have a nullable type so the field can be referenced without using Kotlin's `!!`. Oneof fields can also be declared non-null: ```proto message NonNullSampleMessage { oneof non_null_oneof { option (protokt_oneof).non_null = true; string message = 2; } } ``` Note that deserialization of a message with a non-nullable field will fail if the message being decoded does not contain an instance of the required field. #### BytesSlice When reading messages that contain other serialized messages as `bytes` fields, protokt can keep a reference to the originating byte array to prevent a large copy operation on deserialization. This can be desirable when the wrapping message is a thin metadata shim and doesn't include much memory overhead: ```proto message SliceModel { int64 version = 1; bytes encoded_message = 2 [ (protokt_property).bytes_slice = true ]; } ``` ### Usage #### Gradle ```groovy buildscript { dependencies { classpath "com.toasttab.protokt:protokt-gradle-plugin:0.0.3" } } apply plugin: 'com.toasttab.protokt' ``` This will automatically download and install protokt, apply the Google protobuf plugin, and configure all the necessary boilerplate. By default it will also add `protokt-runtime` to the api scope of the project, and `protobuf-java` to the implementation scope. If your project is pure Kotlin you may run into the following error: ``` Execution failed for task ':compileJava'. > error: no source files ``` To work around it, disable all `JavaCompile` tasks in the project: ```groovy tasks.withType(JavaCompile) { enabled = false } ``` or: ```groovy compileJava.enabled = false ``` #### Command line code generation ```bash protokt-codegen$ ./gradlew assemble [OR ./gradlew installDist] protokt-codegen$ ./run-protokt.sh -h protokt-codegen$ ./run-protokt.sh \ -out=../kotlin \ -pkg=com.toasttab.protokt.conformance \ -file=conformance.proto \ -cp=../build/libs/protokt-codegen-0.0.3-SNAPSHOT-all.jar \ -plugin=../bin/protokt.sh ``` ### Contribution To enable rapid development of the code generator, the protobuf conformance tests have been compiled and included in the protokt-testing project. They run on Mac OS 10.14+ and Ubuntu 16.04 x86-64. Publish the plugin to the integration repository: ```bash protokt$ ./gradlew publishToIntegrationRepository ``` Then run the tests from `protokt-testing`: ```bash protokt-testing$ ./gradlew protokt-conformance-tests:test ``` All integration tests can be run with: ``` protokt-testing$ ./gradlew test ```
-
- 09 Jul, 2019 2 commits
- 08 Jul, 2019 4 commits
-
-
Yilun Chong authored
fix dart example readme typo
-
Yilun Chong authored
allow copy and paste from readme
-
Yilun Chong authored
Use proto_library in proto_lang_toolchain.blacklisted_protos
-
Elliotte Rusty Harold authored
to 3.8.0
-
- 07 Jul, 2019 1 commit
-
-
Phani Rithvij authored
-
- 05 Jul, 2019 1 commit
-
-
John D. Pope authored
spare people the time deleting $ from text.
-
- 03 Jul, 2019 2 commits
-
-
boscosiu authored
-
Asra Ali authored
Signed-off-by: Asra Ali <asraa@google.com>
-