Commit 3c54fd96 authored by Robert's avatar Robert Committed by rw

Port FlatBuffers to Rust (#4898)

This is a port of FlatBuffers to Rust. It provides code generation and a
runtime library derived from the C++ implementation. It utilizes the
Rust type system to provide safe and fast traversal of FlatBuffers data.

There are 188 tests, including many fuzz tests of roundtrips for various
serialization scenarios. Initial benchmarks indicate that the canonical
example payload can be written in ~700ns, and traversed in ~100ns.

Rustaceans may be interested in the Follow, Push, and SafeSliceAccess
traits. These traits lift traversals, reads, writes, and slice accesses
into the type system, providing abstraction with no runtime penalty.
parent e7578548
......@@ -16,6 +16,7 @@
*.keystore
**/.vs/**
**/bin/**
!tests/rust_usage_test/bin/**
**/gen/**
**/libs/**
**/obj/**
......@@ -56,6 +57,7 @@ tests/monsterdata_java_wire_sp.mon
tests/monsterdata_go_wire.mon
tests/monsterdata_javascript_wire.mon
tests/monsterdata_lobster_wire.mon
tests/monsterdata_rust_wire.mon
tests/unicode_test.mon
tests/ts/
tests/php/
......
......@@ -92,6 +92,7 @@ cc_binary(
"src/idl_gen_lobster.cpp",
"src/idl_gen_php.cpp",
"src/idl_gen_python.cpp",
"src/idl_gen_rust.cpp",
"src/idl_gen_text.cpp",
],
includes = [
......
......@@ -54,6 +54,7 @@ set(FlatBuffers_Compiler_SRCS
src/idl_gen_python.cpp
src/idl_gen_lobster.cpp
src/idl_gen_lua.cpp
src/idl_gen_rust.cpp
src/idl_gen_fbs.cpp
src/idl_gen_grpc.cpp
src/idl_gen_json_schema.cpp
......
......@@ -43,6 +43,8 @@ For any schema input files, one or more generators can be specified:
- `--lobster`: Generate Lobster code.
- `--rust`, `-r` : Generate Rust code.
For any data input files:
- `--binary`, `-b` : If data is contained in this file, generate a
......
......@@ -4,7 +4,7 @@ FlatBuffers {#flatbuffers_index}
# Overview {#flatbuffers_overview}
[FlatBuffers](@ref flatbuffers_overview) is an efficient cross platform
serialization library for C++, C#, C, Go, Java, JavaScript, Lobster, Lua, TypeScript, PHP, and Python.
serialization library for C++, C#, C, Go, Java, JavaScript, Lobster, Lua, TypeScript, PHP, Python, and Rust.
It was originally created at Google for game development and other
performance-critical applications.
......@@ -144,6 +144,8 @@ sections provide a more in-depth usage guide.
own programs.
- How to [use the generated Lobster code](@ref flatbuffers_guide_use_lobster) in your
own programs.
- How to [use the generated Rust code](@ref flatbuffers_guide_use_rust) in your
own programs.
- [Support matrix](@ref flatbuffers_support) for platforms/languages/features.
- Some [benchmarks](@ref flatbuffers_benchmarks) showing the advantage of
using FlatBuffers.
......
Use in Rust {#flatbuffers_guide_use_rust}
==========
## Before you get started
Before diving into the FlatBuffers usage in Rust, it should be noted that
the [Tutorial](@ref flatbuffers_guide_tutorial) page has a complete guide
to general FlatBuffers usage in all of the supported languages (including Rust).
This page is designed to cover the nuances of FlatBuffers usage, specific to
Rust.
#### Prerequisites
This page assumes you have written a FlatBuffers schema and compiled it
with the Schema Compiler. If you have not, please see
[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler)
and [Writing a schema](@ref flatbuffers_guide_writing_schema).
Assuming you wrote a schema, say `mygame.fbs` (though the extension doesn't
matter), you've generated a Rust file called `mygame_generated.rs` using the
compiler (e.g. `flatc --rust mygame.fbs`), you can now start using this in
your program by including the file. As noted, this header relies on the crate
`flatbuffers`, which should be in your include `Cargo.toml`.
## FlatBuffers Rust library code location
The code for the FlatBuffers Rust library can be found at
`flatbuffers/rust`. You can browse the library code on the
[FlatBuffers GitHub page](https://github.com/google/flatbuffers/tree/master/rust).
## Testing the FlatBuffers Rust library
The code to test the Rust library can be found at `flatbuffers/tests/rust_usage_test`.
The test code itself is located in
[integration_test.rs](https://github.com/google/flatbuffers/blob/master/tests/rust_usage_test/tests/integration_test.rs)
This test file requires `flatc` to be present. To review how to build the project,
please read the [Building](@ref flatbuffers_guide_building) documenation.
To run the tests, execute `RustTest.sh` from the `flatbuffers/tests` directory.
For example, on [Linux](https://en.wikipedia.org/wiki/Linux), you would simply
run: `cd tests && ./RustTest.sh`.
*Note: The shell script requires [Rust](https://www.rust-lang.org) to
be installed.*
## Using the FlatBuffers Rust library
*Note: See [Tutorial](@ref flatbuffers_guide_tutorial) for a more in-depth
example of how to use FlatBuffers in Rust.*
FlatBuffers supports both reading and writing FlatBuffers in Rust.
To use FlatBuffers in your code, first generate the Rust modules from your
schema with the `--rust` option to `flatc`. Then you can import both FlatBuffers
and the generated code to read or write FlatBuffers.
For example, here is how you would read a FlatBuffer binary file in Rust:
First, include the library and generated code. Then read the file into
a `u8` vector, which you pass, as a byte slice, to `get_root_as_monster()`.
This full example program is available in the Rust test suite:
[monster_example.rs](https://github.com/google/flatbuffers/blob/master/tests/rust_usage_test/bin/monster_example.rs)
It can be run by `cd`ing to the `rust_usage_test` directory and executing: `cargo run monster_example`.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.rs}
extern crate flatbuffers;
#[path = "../../monster_test_generated.rs"]
mod monster_test_generated;
pub use monster_test_generated::my_game;
use std::io::Read;
fn main() {
let mut f = std::fs::File::open("../monsterdata_test.mon").unwrap();
let mut buf = Vec::new();
f.read_to_end(&mut buf).expect("file reading failed");
let monster = my_game::example::get_root_as_monster(&buf[..]);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`monster` is of type `Monster`, and points to somewhere *inside* your
buffer (root object pointers are not the same as `buffer_pointer` !).
If you look in your generated header, you'll see it has
convenient accessors for all fields, e.g. `hp()`, `mana()`, etc:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.rs}
println!("{}", monster.hp()); // `80`
println!("{}", monster.mana()); // default value of `150`
println!("{:?}", monster.name()); // Some("MyMonster")
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*Note: That we never stored a `mana` value, so it will return the default.*
## Direct memory access
As you can see from the above examples, all elements in a buffer are
accessed through generated accessors. This is because everything is
stored in little endian format on all platforms (the accessor
performs a swap operation on big endian machines), and also because
the layout of things is generally not known to the user.
For structs, layout is deterministic and guaranteed to be the same
across platforms (scalars are aligned to their
own size, and structs themselves to their largest member), and you
are allowed to access this memory directly by using `safe_slice` and
on the reference to a struct, or even an array of structs.
To compute offsets to sub-elements of a struct, make sure they
are structs themselves, as then you can use the pointers to
figure out the offset without having to hardcode it. This is
handy for use of arrays of structs with calls like `glVertexAttribPointer`
in OpenGL or similar APIs.
It is important to note is that structs are still little endian on all
machines, so only use tricks like this if you can guarantee you're not
shipping on a big endian machine (using an `#[cfg(target_endian = "little")]`
attribute would be wise).
The special function `safe_slice` is implemented on Vector objects that are
represented in memory the same way as they are represented on the wire. This
function is always available on vectors of struct, bool, u8, and i8. It is
conditionally-compiled on little-endian systems for all the remaining scalar
types.
The FlatBufferBuilder function `create_vector_direct` is implemented for all
types that are endian-safe to write with a `memcpy`. It is the write-equivalent
of `safe_slice`.
## Access of untrusted buffers
The generated accessor functions access fields over offsets, which is
very quick. These offsets are used to index into Rust slices, so they are
bounds-checked by the Rust runtime. However, our Rust implementation may
change: we may convert access functions to use direct pointer dereferencing, to
improve lookup speed. As a result, users should not rely on the aforementioned
bounds-checking behavior.
When you're processing large amounts of data from a source you know (e.g.
your own generated data on disk), this is acceptable, but when reading
data from the network that can potentially have been modified by an
attacker, this is undesirable.
The C++ port provides a buffer verifier. At this time, Rust does not. Rust may
provide a verifier in a future version. In the meantime, Rust users can access
the buffer verifier generated by the C++ port through a foreign function
interface (FFI).
## Threading
Reading a FlatBuffer does not touch any memory outside the original buffer,
and is entirely read-only (all immutable), so is safe to access from multiple
threads even without synchronisation primitives.
Creating a FlatBuffer is not thread safe. All state related to building
a FlatBuffer is contained in a FlatBufferBuilder instance, and no memory
outside of it is touched. To make this thread safe, either do not
share instances of FlatBufferBuilder between threads (recommended), or
manually wrap it in synchronisation primitives. There's no automatic way to
accomplish this, by design, as we feel multithreaded construction
of a single buffer will be rare, and synchronisation overhead would be costly.
<br>
......@@ -18,23 +18,23 @@ In general:
NOTE: this table is a start, it needs to be extended.
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | -------
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard*
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ----
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ?
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ?
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ?
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw
* aard = aardappel (previously: gwvo)
* ev = evolutional
......
This diff is collapsed.
......@@ -760,6 +760,7 @@ INPUT = "FlatBuffers.md" \
"PythonUsage.md" \
"LuaUsage.md" \
"LobsterUsage.md" \
"RustUsage.md" \
"Support.md" \
"Benchmarks.md" \
"WhitePaper.md" \
......@@ -778,6 +779,7 @@ INPUT = "FlatBuffers.md" \
"../../net/FlatBuffers/FlatBufferBuilder.cs" \
"../../include/flatbuffers/flatbuffers.h" \
"../../go/builder.go"
"../../rust/flatbuffers/src/builder.rs"
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
......
......@@ -41,24 +41,24 @@ namespace flatbuffers {
// of type tokens.
// clang-format off
#define FLATBUFFERS_GEN_TYPES_SCALAR(TD) \
TD(NONE, "", uint8_t, byte, byte, byte, uint8) \
TD(UTYPE, "", uint8_t, byte, byte, byte, uint8) /* begin scalar/int */ \
TD(BOOL, "bool", uint8_t, boolean,byte, bool, bool) \
TD(CHAR, "byte", int8_t, byte, int8, sbyte, int8) \
TD(UCHAR, "ubyte", uint8_t, byte, byte, byte, uint8) \
TD(SHORT, "short", int16_t, short, int16, short, int16) \
TD(USHORT, "ushort", uint16_t, short, uint16, ushort, uint16) \
TD(INT, "int", int32_t, int, int32, int, int32) \
TD(UINT, "uint", uint32_t, int, uint32, uint, uint32) \
TD(LONG, "long", int64_t, long, int64, long, int64) \
TD(ULONG, "ulong", uint64_t, long, uint64, ulong, uint64) /* end int */ \
TD(FLOAT, "float", float, float, float32, float, float32) /* begin float */ \
TD(DOUBLE, "double", double, double, float64, double, float64) /* end float/scalar */
TD(NONE, "", uint8_t, byte, byte, byte, uint8, u8) \
TD(UTYPE, "", uint8_t, byte, byte, byte, uint8, u8) /* begin scalar/int */ \
TD(BOOL, "bool", uint8_t, boolean,byte, bool, bool, bool) \
TD(CHAR, "byte", int8_t, byte, int8, sbyte, int8, i8) \
TD(UCHAR, "ubyte", uint8_t, byte, byte, byte, uint8, u8) \
TD(SHORT, "short", int16_t, short, int16, short, int16, i16) \
TD(USHORT, "ushort", uint16_t, short, uint16, ushort, uint16, u16) \
TD(INT, "int", int32_t, int, int32, int, int32, i32) \
TD(UINT, "uint", uint32_t, int, uint32, uint, uint32, u32) \
TD(LONG, "long", int64_t, long, int64, long, int64, i64) \
TD(ULONG, "ulong", uint64_t, long, uint64, ulong, uint64, u64) /* end int */ \
TD(FLOAT, "float", float, float, float32, float, float32, f32) /* begin float */ \
TD(DOUBLE, "double", double, double, float64, double, float64, f64) /* end float/scalar */
#define FLATBUFFERS_GEN_TYPES_POINTER(TD) \
TD(STRING, "string", Offset<void>, int, int, StringOffset, int) \
TD(VECTOR, "", Offset<void>, int, int, VectorOffset, int) \
TD(STRUCT, "", Offset<void>, int, int, int, int) \
TD(UNION, "", Offset<void>, int, int, int, int)
TD(STRING, "string", Offset<void>, int, int, StringOffset, int, unused) \
TD(VECTOR, "", Offset<void>, int, int, VectorOffset, int, unused) \
TD(STRUCT, "", Offset<void>, int, int, int, int, unused) \
TD(UNION, "", Offset<void>, int, int, int, int, unused)
// The fields are:
// - enum
......@@ -68,12 +68,14 @@ namespace flatbuffers {
// - Go type.
// - C# / .Net type.
// - Python type.
// - Rust type.
// using these macros, we can now write code dealing with types just once, e.g.
/*
switch (type) {
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, \
RTYPE) \
case BASE_TYPE_ ## ENUM: \
// do something specific to CTYPE here
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
......@@ -90,13 +92,15 @@ switch (type) {
__extension__ // Stop GCC complaining about trailing comma with -Wpendantic.
#endif
enum BaseType {
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, \
RTYPE) \
BASE_TYPE_ ## ENUM,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
};
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, \
RTYPE) \
static_assert(sizeof(CTYPE) <= sizeof(largest_scalar_t), \
"define largest_scalar_t as " #CTYPE);
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
......@@ -111,6 +115,8 @@ inline bool IsFloat (BaseType t) { return t == BASE_TYPE_FLOAT ||
inline bool IsLong (BaseType t) { return t == BASE_TYPE_LONG ||
t == BASE_TYPE_ULONG; }
inline bool IsBool (BaseType t) { return t == BASE_TYPE_BOOL; }
inline bool IsOneByte(BaseType t) { return t >= BASE_TYPE_UTYPE &&
t <= BASE_TYPE_UCHAR; }
// clang-format on
extern const char *const kTypeNames[];
......@@ -410,6 +416,7 @@ struct IDLOptions {
kDart = 1 << 11,
kLua = 1 << 12,
kLobster = 1 << 13,
kRust = 1 << 14,
kMAX
};
......@@ -834,6 +841,12 @@ extern bool GenerateLua(const Parser &parser,
const std::string &path,
const std::string &file_name);
// Generate Rust files from the definitions in the Parser object.
// See idl_gen_rust.cpp.
extern bool GenerateRust(const Parser &parser,
const std::string &path,
const std::string &file_name);
// Generate Json schema file
// See idl_gen_json_schema.cpp.
extern bool GenerateJsonSchema(const Parser &parser,
......@@ -872,6 +885,12 @@ extern std::string DartMakeRule(const Parser &parser,
const std::string &path,
const std::string &file_name);
// Generate a make rule for the generated Rust code.
// See idl_gen_rust.cpp.
extern std::string RustMakeRule(const Parser &parser,
const std::string &path,
const std::string &file_name);
// Generate a make rule for the generated Java/C#/... files.
// See idl_gen_general.cpp.
extern std::string GeneralMakeRule(const Parser &parser,
......
[[package]]
name = "flatbuffers"
version = "0.1.0"
[package]
name = "flatbuffers"
version = "0.1.0"
authors = ["Robert Winslow <hello@rwinslow.com>", "FlatBuffers Maintainers"]
[dependencies]
smallvec = "0.6"
This diff is collapsed.
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use std::mem::size_of;
/// Trait for values that must be stored in little-endian byte order, but
/// might be represented in memory as big-endian. Every type that implements
/// EndianScalar is a valid FlatBuffers scalar value.
///
/// The Rust stdlib does not provide a trait to represent scalars, so this trait
/// serves that purpose, too.
///
/// Note that we do not use the num-traits crate for this, because it provides
/// "too much". For example, num-traits provides i128 support, but that is an
/// invalid FlatBuffers type.
pub trait EndianScalar: Sized + PartialEq + Copy + Clone {
fn to_little_endian(self) -> Self;
fn from_little_endian(self) -> Self;
}
/// Macro for implementing a no-op endian conversion. This is used for types
/// that are one byte wide.
macro_rules! impl_endian_scalar_noop {
($ty:ident) => (
impl EndianScalar for $ty {
#[inline]
fn to_little_endian(self) -> Self {
self
}
#[inline]
fn from_little_endian(self) -> Self {
self
}
}
)
}
/// Macro for implementing an endian conversion using the stdlib `to_le` and
/// `from_le` functions. This is used for integer types. It is not used for
/// floats, because the `to_le` and `from_le` are not implemented for them in
/// the stdlib.
macro_rules! impl_endian_scalar_stdlib_le_conversion {
($ty:ident) => (
impl EndianScalar for $ty {
#[inline]
fn to_little_endian(self) -> Self {
Self::to_le(self)
}
#[inline]
fn from_little_endian(self) -> Self {
Self::from_le(self)
}
}
)
}
impl_endian_scalar_noop!(bool);
impl_endian_scalar_noop!(u8);
impl_endian_scalar_noop!(i8);
impl_endian_scalar_stdlib_le_conversion!(u16);
impl_endian_scalar_stdlib_le_conversion!(u32);
impl_endian_scalar_stdlib_le_conversion!(u64);
impl_endian_scalar_stdlib_le_conversion!(i16);
impl_endian_scalar_stdlib_le_conversion!(i32);
impl_endian_scalar_stdlib_le_conversion!(i64);
impl EndianScalar for f32 {
/// Convert f32 from host endian-ness to little-endian.
#[inline]
fn to_little_endian(self) -> Self {
#[cfg(target_endian = "little")]
{
self
}
#[cfg(not(target_endian = "little"))]
{
byte_swap_f32(&self)
}
}
/// Convert f32 from little-endian to host endian-ness.
#[inline]
fn from_little_endian(self) -> Self {
#[cfg(target_endian = "little")]
{
self
}
#[cfg(not(target_endian = "little"))]
{
byte_swap_f32(&self)
}
}
}
impl EndianScalar for f64 {
/// Convert f64 from host endian-ness to little-endian.
#[inline]
fn to_little_endian(self) -> Self {
#[cfg(target_endian = "little")]
{
self
}
#[cfg(not(target_endian = "little"))]
{
byte_swap_f64(&self)
}
}
/// Convert f64 from little-endian to host endian-ness.
#[inline]
fn from_little_endian(self) -> Self {
#[cfg(target_endian = "little")]
{
self
}
#[cfg(not(target_endian = "little"))]
{
byte_swap_f64(&self)
}
}
}
/// Swaps the bytes of an f32.
#[allow(dead_code)]
#[inline]
pub fn byte_swap_f32(x: f32) -> f32 {
f32::from_bits(x.to_bits().swap_bytes())
}
/// Swaps the bytes of an f64.
#[allow(dead_code)]
#[inline]
pub fn byte_swap_f64(x: f64) -> f64 {
f64::from_bits(x.to_bits().swap_bytes())
}
/// Place an EndianScalar into the provided mutable byte slice. Performs
/// endian conversion, if necessary.
#[inline]
pub fn emplace_scalar<T: EndianScalar>(s: &mut [u8], x: T) {
let sz = size_of::<T>();
let mut_ptr = (&mut s[..sz]).as_mut_ptr() as *mut T;
let val = x.to_little_endian();
unsafe {
*mut_ptr = val;
}
}
/// Read an EndianScalar from the provided byte slice at the specified location.
/// Performs endian conversion, if necessary.
#[inline]
pub fn read_scalar_at<T: EndianScalar>(s: &[u8], loc: usize) -> T {
let buf = &s[loc..loc + size_of::<T>()];
read_scalar(buf)
}
/// Read an EndianScalar from the provided byte slice. Performs endian
/// conversion, if necessary.
#[inline]
pub fn read_scalar<T: EndianScalar>(s: &[u8]) -> T {
let sz = size_of::<T>();
let p = (&s[..sz]).as_ptr() as *const T;
let x = unsafe { *p };
x.from_little_endian()
}
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use std::marker::PhantomData;
/// Follow is a trait that allows us to access FlatBuffers in a declarative,
/// type safe, and fast way. They compile down to almost no code (after
/// optimizations). Conceptually, Follow lifts the offset-based access
/// patterns of FlatBuffers data into the type system. This trait is used
/// pervasively at read time, to access tables, vtables, vectors, strings, and
/// all other data. At this time, Follow is not utilized much on the write
/// path.
///
/// Writing a new Follow implementation primarily involves deciding whether
/// you want to return data (of the type Self::Inner) or do you want to
/// continue traversing the FlatBuffer.
pub trait Follow<'a> {
type Inner;
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner;
}
/// Execute a follow as a top-level function.
#[allow(dead_code)]
#[inline]
pub fn lifted_follow<'a, T: Follow<'a>>(buf: &'a [u8], loc: usize) -> T::Inner {
T::follow(buf, loc)
}
/// FollowStart wraps a Follow impl in a struct type. This can make certain
/// programming patterns more ergonomic.
#[derive(Debug)]
pub struct FollowStart<T>(PhantomData<T>);
impl<'a, T: Follow<'a> + 'a> FollowStart<T> {
#[inline]
pub fn new() -> Self {
Self { 0: PhantomData }
}
#[inline]
pub fn self_follow(&'a self, buf: &'a [u8], loc: usize) -> T::Inner {
T::follow(buf, loc)
}
}
impl<'a, T: Follow<'a>> Follow<'a> for FollowStart<T> {
type Inner = T::Inner;
#[inline]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
T::follow(buf, loc)
}
}
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
mod builder;
mod endian_scalar;
mod follow;
mod primitives;
mod push;
mod table;
mod vector;
mod vtable;
mod vtable_writer;
pub use builder::FlatBufferBuilder;
pub use endian_scalar::{EndianScalar, emplace_scalar, read_scalar, read_scalar_at, byte_swap_f32, byte_swap_f64};
pub use follow::{Follow, FollowStart};
pub use primitives::*;
pub use push::Push;
pub use table::{Table, buffer_has_identifier, get_root, get_size_prefixed_root};
pub use vector::{SafeSliceAccess, Vector, follow_cast_ref};
pub use vtable::field_index_to_field_offset;
// TODO(rw): Unify `create_vector` and `create_vector_direct` by using
// `Into<Vector<...>>`.
// TODO(rw): Split fill ops in builder into fill_small, fill_big like in C++.
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use std::marker::PhantomData;
use std::mem::size_of;
use std::ops::Deref;
use endian_scalar::{emplace_scalar, read_scalar, read_scalar_at};
use follow::Follow;
use push::Push;
pub const FLATBUFFERS_MAX_BUFFER_SIZE: usize = (1u64 << 31) as usize;
pub const FILE_IDENTIFIER_LENGTH: usize = 4;
pub const VTABLE_METADATA_FIELDS: usize = 2;
pub const SIZE_U8: usize = size_of::<u8>();
pub const SIZE_I8: usize = size_of::<i8>();
pub const SIZE_U16: usize = size_of::<u16>();
pub const SIZE_I16: usize = size_of::<i16>();
pub const SIZE_U32: usize = size_of::<u32>();
pub const SIZE_I32: usize = size_of::<i32>();
pub const SIZE_U64: usize = size_of::<u64>();
pub const SIZE_I64: usize = size_of::<i64>();
pub const SIZE_F32: usize = size_of::<f32>();
pub const SIZE_F64: usize = size_of::<f64>();
pub const SIZE_SOFFSET: usize = SIZE_I32;
pub const SIZE_UOFFSET: usize = SIZE_U32;
pub const SIZE_VOFFSET: usize = SIZE_I16;
pub const SIZE_SIZEPREFIX: usize = SIZE_UOFFSET;
/// SOffsetT is an i32 that is used by tables to reference their vtables.
pub type SOffsetT = i32;
/// UOffsetT is a u32 that is used by pervasively to represent both pointers
/// and lengths of vectors.
pub type UOffsetT = u32;
/// VOffsetT is a i32 that is used by vtables to store field data.
pub type VOffsetT = i16;
/// TableFinishedWIPOffset marks a WIPOffset as being for a finished table.
pub struct TableFinishedWIPOffset {}
/// TableUnfinishedWIPOffset marks a WIPOffset as being for an unfinished table.
pub struct TableUnfinishedWIPOffset {}
/// UnionWIPOffset marks a WIPOffset as being for a union value.
pub struct UnionWIPOffset {}
/// VTableWIPOffset marks a WIPOffset as being for a vtable.
pub struct VTableWIPOffset {}
/// WIPOffset contains an UOffsetT with a special meaning: it is the location of
/// data relative to the *end* of an in-progress FlatBuffer. The
/// FlatBufferBuilder uses this to track the location of objects in an absolute
/// way. The impl of Push converts a WIPOffset into a ForwardsUOffset.
#[derive(Debug)]
pub struct WIPOffset<T>(UOffsetT, PhantomData<T>);
// TODO(rw): why do we need to reimplement (with a default impl) Copy to
// avoid ownership errors?
impl<T> Copy for WIPOffset<T> {}
impl<T> Clone for WIPOffset<T> {
#[inline]
fn clone(&self) -> WIPOffset<T> {
WIPOffset::new(self.0.clone())
}
}
impl<T> PartialEq for WIPOffset<T> {
fn eq(&self, o: &WIPOffset<T>) -> bool {
self.value() == o.value()
}
}
impl<T> Deref for WIPOffset<T> {
type Target = UOffsetT;
#[inline]
fn deref(&self) -> &UOffsetT {
&self.0
}
}
impl<'a, T: 'a> WIPOffset<T> {
/// Create a new WIPOffset.
#[inline]
pub fn new(o: UOffsetT) -> WIPOffset<T> {
WIPOffset {
0: o,
1: PhantomData,
}
}
/// Return a wrapped value that brings its meaning as a union WIPOffset
/// into the type system.
#[inline(always)]
pub fn as_union_value(&self) -> WIPOffset<UnionWIPOffset> {
WIPOffset::new(self.0)
}
/// Get the underlying value.
#[inline(always)]
pub fn value(&self) -> UOffsetT {
self.0
}
}
impl<T> Push for WIPOffset<T> {
type Output = ForwardsUOffset<T>;
#[inline(always)]
fn push(&self, dst: &mut [u8], rest: &[u8]) {
let n = (SIZE_UOFFSET + rest.len() - self.value() as usize) as UOffsetT;
emplace_scalar::<UOffsetT>(dst, n);
}
}
impl<T> Push for ForwardsUOffset<T> {
type Output = Self;
#[inline(always)]
fn push(&self, dst: &mut [u8], rest: &[u8]) {
self.value().push(dst, rest);
}
}
/// ForwardsUOffset is used by Follow to traverse a FlatBuffer: the pointer
/// is incremented by the value contained in this type.
#[derive(Debug)]
pub struct ForwardsUOffset<T>(UOffsetT, PhantomData<T>);
impl<T> ForwardsUOffset<T> {
#[inline(always)]
pub fn value(&self) -> UOffsetT {
self.0
}
}
impl<'a, T: Follow<'a>> Follow<'a> for ForwardsUOffset<T> {
type Inner = T::Inner;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
let slice = &buf[loc..loc + SIZE_UOFFSET];
let off = read_scalar::<u32>(slice) as usize;
T::follow(buf, loc + off)
}
}
/// ForwardsVOffset is used by Follow to traverse a FlatBuffer: the pointer
/// is incremented by the value contained in this type.
#[derive(Debug)]
pub struct ForwardsVOffset<T>(VOffsetT, PhantomData<T>);
impl<T> ForwardsVOffset<T> {
#[inline(always)]
pub fn value(&self) -> VOffsetT {
self.0
}
}
impl<'a, T: Follow<'a>> Follow<'a> for ForwardsVOffset<T> {
type Inner = T::Inner;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
let slice = &buf[loc..loc + SIZE_VOFFSET];
let off = read_scalar::<VOffsetT>(slice) as usize;
T::follow(buf, loc + off)
}
}
impl<T> Push for ForwardsVOffset<T> {
type Output = Self;
#[inline]
fn push(&self, dst: &mut [u8], rest: &[u8]) {
self.value().push(dst, rest);
}
}
/// ForwardsSOffset is used by Follow to traverse a FlatBuffer: the pointer
/// is incremented by the *negative* of the value contained in this type.
#[derive(Debug)]
pub struct BackwardsSOffset<T>(SOffsetT, PhantomData<T>);
impl<T> BackwardsSOffset<T> {
#[inline(always)]
pub fn value(&self) -> SOffsetT {
self.0
}
}
impl<'a, T: Follow<'a>> Follow<'a> for BackwardsSOffset<T> {
type Inner = T::Inner;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
let slice = &buf[loc..loc + SIZE_SOFFSET];
let off = read_scalar::<SOffsetT>(slice);
T::follow(buf, (loc as SOffsetT - off) as usize)
}
}
impl<T> Push for BackwardsSOffset<T> {
type Output = Self;
#[inline]
fn push(&self, dst: &mut [u8], rest: &[u8]) {
self.value().push(dst, rest);
}
}
/// SkipSizePrefix is used by Follow to traverse a FlatBuffer: the pointer is
/// incremented by a fixed constant in order to skip over the size prefix value.
pub struct SkipSizePrefix<T>(PhantomData<T>);
impl<'a, T: Follow<'a> + 'a> Follow<'a> for SkipSizePrefix<T> {
type Inner = T::Inner;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
T::follow(buf, loc + SIZE_SIZEPREFIX)
}
}
/// SkipRootOffset is used by Follow to traverse a FlatBuffer: the pointer is
/// incremented by a fixed constant in order to skip over the root offset value.
pub struct SkipRootOffset<T>(PhantomData<T>);
impl<'a, T: Follow<'a> + 'a> Follow<'a> for SkipRootOffset<T> {
type Inner = T::Inner;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
T::follow(buf, loc + SIZE_UOFFSET)
}
}
/// FileIdentifier is used by Follow to traverse a FlatBuffer: the pointer is
/// dereferenced into a byte slice, whose bytes are the file identifer value.
pub struct FileIdentifier;
impl<'a> Follow<'a> for FileIdentifier {
type Inner = &'a [u8];
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
&buf[loc..loc + FILE_IDENTIFIER_LENGTH]
}
}
/// SkipFileIdentifier is used by Follow to traverse a FlatBuffer: the pointer
/// is incremented by a fixed constant in order to skip over the file
/// identifier value.
pub struct SkipFileIdentifier<T>(PhantomData<T>);
impl<'a, T: Follow<'a> + 'a> Follow<'a> for SkipFileIdentifier<T> {
type Inner = T::Inner;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
T::follow(buf, loc + FILE_IDENTIFIER_LENGTH)
}
}
/// Follow trait impls for primitive types.
///
/// Ideally, these would be implemented as a single impl using trait bounds on
/// EndianScalar, but implementing Follow that way causes a conflict with
/// other impls.
macro_rules! impl_follow_for_endian_scalar {
($ty:ident) => (
impl<'a> Follow<'a> for $ty {
type Inner = $ty;
#[inline(always)]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
read_scalar_at::<$ty>(buf, loc)
}
}
)
}
impl_follow_for_endian_scalar!(bool);
impl_follow_for_endian_scalar!(u8);
impl_follow_for_endian_scalar!(u16);
impl_follow_for_endian_scalar!(u32);
impl_follow_for_endian_scalar!(u64);
impl_follow_for_endian_scalar!(i8);
impl_follow_for_endian_scalar!(i16);
impl_follow_for_endian_scalar!(i32);
impl_follow_for_endian_scalar!(i64);
impl_follow_for_endian_scalar!(f32);
impl_follow_for_endian_scalar!(f64);
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use std::cmp::max;
use std::mem::{align_of, size_of};
use endian_scalar::emplace_scalar;
/// Trait to abstract over functionality needed to write values (either owned
/// or referenced). Used in FlatBufferBuilder and implemented for generated
/// types.
pub trait Push: Sized {
type Output;
fn push(&self, dst: &mut [u8], _rest: &[u8]);
#[inline]
fn size() -> usize {
size_of::<Self::Output>()
}
#[inline]
fn alignment() -> PushAlignment {
PushAlignment::new(align_of::<Self::Output>())
}
}
/// Ensure Push alignment calculations are typesafe (because this helps reduce
/// implementation issues when using FlatBufferBuilder::align).
pub struct PushAlignment(usize);
impl PushAlignment {
#[inline]
pub fn new(x: usize) -> Self {
PushAlignment { 0: x }
}
#[inline]
pub fn value(&self) -> usize {
self.0
}
#[inline]
pub fn max_of(&self, o: usize) -> Self {
PushAlignment::new(max(self.0, o))
}
}
/// Macro to implement Push for EndianScalar types.
macro_rules! impl_push_for_endian_scalar {
($ty:ident) => (
impl Push for $ty {
type Output = $ty;
#[inline]
fn push(&self, dst: &mut [u8], _rest: &[u8]) {
emplace_scalar::<$ty>(dst, *self);
}
}
)
}
impl_push_for_endian_scalar!(bool);
impl_push_for_endian_scalar!(u8);
impl_push_for_endian_scalar!(i8);
impl_push_for_endian_scalar!(u16);
impl_push_for_endian_scalar!(i16);
impl_push_for_endian_scalar!(u32);
impl_push_for_endian_scalar!(i32);
impl_push_for_endian_scalar!(u64);
impl_push_for_endian_scalar!(i64);
impl_push_for_endian_scalar!(f32);
impl_push_for_endian_scalar!(f64);
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use follow::Follow;
use primitives::*;
use vtable::VTable;
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Table<'a> {
pub buf: &'a [u8],
pub loc: usize,
}
impl<'a> Table<'a> {
#[inline]
pub fn new(buf: &'a [u8], loc: usize) -> Self {
Table { buf: buf, loc: loc }
}
#[inline]
pub fn vtable(&'a self) -> VTable<'a> {
<BackwardsSOffset<VTable<'a>>>::follow(self.buf, self.loc)
}
#[inline]
pub fn get<T: Follow<'a> + 'a>(
&'a self,
slot_byte_loc: VOffsetT,
default: Option<T::Inner>,
) -> Option<T::Inner> {
let o = self.vtable().get(slot_byte_loc) as usize;
if o == 0 {
return default;
}
Some(<T>::follow(self.buf, self.loc + o))
}
}
impl<'a> Follow<'a> for Table<'a> {
type Inner = Table<'a>;
#[inline]
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
Table { buf: buf, loc: loc }
}
}
#[inline]
pub fn get_root<'a, T: Follow<'a> + 'a>(data: &'a [u8]) -> T::Inner {
<ForwardsUOffset<T>>::follow(data, 0)
}
#[inline]
pub fn get_size_prefixed_root<'a, T: Follow<'a> + 'a>(data: &'a [u8]) -> T::Inner {
<SkipSizePrefix<ForwardsUOffset<T>>>::follow(data, 0)
}
#[inline]
pub fn buffer_has_identifier(data: &[u8], ident: &str, size_prefixed: bool) -> bool {
assert_eq!(ident.len(), FILE_IDENTIFIER_LENGTH);
let got = if size_prefixed {
<SkipSizePrefix<SkipRootOffset<FileIdentifier>>>::follow(data, 0)
} else {
<SkipRootOffset<FileIdentifier>>::follow(data, 0)
};
ident.as_bytes() == got
}
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use std::marker::PhantomData;
use std::mem::size_of;
use std::slice::from_raw_parts;
use std::str::from_utf8_unchecked;
use endian_scalar::{EndianScalar, read_scalar};
use follow::Follow;
use primitives::*;
#[derive(Debug)]
pub struct Vector<'a, T: 'a>(&'a [u8], usize, PhantomData<T>);
impl<'a, T: 'a> Vector<'a, T> {
#[inline(always)]
pub fn new(buf: &'a [u8], loc: usize) -> Self {
Vector {
0: buf,
1: loc,
2: PhantomData,
}
}
#[inline(always)]
pub fn len(&self) -> usize {
read_scalar::<UOffsetT>(&self.0[self.1 as usize..]) as usize
}
}
impl<'a, T: Follow<'a> + 'a> Vector<'a, T> {
#[inline(always)]
pub fn get(&self, idx: usize) -> T::Inner {
debug_assert!(idx < read_scalar::<u32>(&self.0[self.1 as usize..]) as usize);
let sz = size_of::<T>();
debug_assert!(sz > 0);
T::follow(self.0, self.1 as usize + SIZE_UOFFSET + sz * idx)
}
}
pub trait SafeSliceAccess {}
impl<'a, T: SafeSliceAccess + 'a> Vector<'a, T> {
pub fn safe_slice(self) -> &'a [T] {
let buf = self.0;
let loc = self.1;
let sz = size_of::<T>();
debug_assert!(sz > 0);
let len = read_scalar::<UOffsetT>(&buf[loc..loc + SIZE_UOFFSET]) as usize;
let data_buf = &buf[loc + SIZE_UOFFSET..loc + SIZE_UOFFSET + len * sz];
let ptr = data_buf.as_ptr() as *const T;
let s: &'a [T] = unsafe { from_raw_parts(ptr, len) };
s
}
}
impl SafeSliceAccess for u8 {}
impl SafeSliceAccess for i8 {}
impl SafeSliceAccess for bool {}
#[cfg(target_endian = "little")]
mod le_safe_slice_impls {
impl super::SafeSliceAccess for u16 {}
impl super::SafeSliceAccess for u32 {}
impl super::SafeSliceAccess for u64 {}
impl super::SafeSliceAccess for i16 {}
impl super::SafeSliceAccess for i32 {}
impl super::SafeSliceAccess for i64 {}
impl super::SafeSliceAccess for f32 {}
impl super::SafeSliceAccess for f64 {}
}
pub use self::le_safe_slice_impls::*;
pub fn follow_cast_ref<'a, T: Sized + 'a>(buf: &'a [u8], loc: usize) -> &'a T {
let sz = size_of::<T>();
let buf = &buf[loc..loc + sz];
let ptr = buf.as_ptr() as *const T;
unsafe { &*ptr }
}
impl<'a> Follow<'a> for &'a str {
type Inner = &'a str;
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
let len = read_scalar::<UOffsetT>(&buf[loc..loc + SIZE_UOFFSET]) as usize;
let slice = &buf[loc + SIZE_UOFFSET..loc + SIZE_UOFFSET + len];
let s = unsafe { from_utf8_unchecked(slice) };
s
}
}
fn follow_slice_helper<T>(buf: &[u8], loc: usize) -> &[T] {
let sz = size_of::<T>();
debug_assert!(sz > 0);
let len = read_scalar::<UOffsetT>(&buf[loc..loc + SIZE_UOFFSET]) as usize;
let data_buf = &buf[loc + SIZE_UOFFSET..loc + SIZE_UOFFSET + len * sz];
let ptr = data_buf.as_ptr() as *const T;
let s: &[T] = unsafe { from_raw_parts(ptr, len) };
s
}
/// Implement direct slice access if the host is little-endian.
#[cfg(target_endian = "little")]
impl<'a, T: EndianScalar> Follow<'a> for &'a [T] {
type Inner = &'a [T];
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
follow_slice_helper::<T>(buf, loc)
}
}
/// Implement Follow for all possible Vectors that have Follow-able elements.
impl<'a, T: Follow<'a> + 'a> Follow<'a> for Vector<'a, T> {
type Inner = Vector<'a, T>;
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
Vector::new(buf, loc)
}
}
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use endian_scalar::read_scalar_at;
use follow::Follow;
use primitives::*;
/// VTable encapsulates read-only usage of a vtable. It is only to be used
/// by generated code.
#[derive(Debug)]
pub struct VTable<'a> {
buf: &'a [u8],
loc: usize,
}
impl<'a> PartialEq for VTable<'a> {
fn eq(&self, other: &VTable) -> bool {
self.as_bytes().eq(other.as_bytes())
}
}
impl<'a> VTable<'a> {
pub fn init(buf: &'a [u8], loc: usize) -> Self {
VTable {
buf: buf,
loc: loc,
}
}
pub fn num_fields(&self) -> usize {
(self.num_bytes() / SIZE_VOFFSET) - 2
}
pub fn num_bytes(&self) -> usize {
read_scalar_at::<VOffsetT>(self.buf, self.loc) as usize
}
pub fn object_inline_num_bytes(&self) -> usize {
let n = read_scalar_at::<VOffsetT>(self.buf, self.loc + SIZE_VOFFSET);
n as usize
}
pub fn get_field(&self, idx: usize) -> VOffsetT {
// TODO(rw): distinguish between None and 0?
if idx > self.num_fields() {
return 0;
}
read_scalar_at::<VOffsetT>(
self.buf,
self.loc + SIZE_VOFFSET + SIZE_VOFFSET + SIZE_VOFFSET * idx,
)
}
pub fn get(&self, byte_loc: VOffsetT) -> VOffsetT {
// TODO(rw): distinguish between None and 0?
if byte_loc as usize >= self.num_bytes() {
return 0;
}
read_scalar_at::<VOffsetT>(self.buf, self.loc + byte_loc as usize)
}
pub fn as_bytes(&self) -> &[u8] {
let len = self.num_bytes();
&self.buf[self.loc..self.loc + len]
}
}
#[allow(dead_code)]
pub fn field_index_to_field_offset(field_id: VOffsetT) -> VOffsetT {
// Should correspond to what end_table() below builds up.
let fixed_fields = 2; // Vtable size and Object Size.
((field_id + fixed_fields) * (SIZE_VOFFSET as VOffsetT)) as VOffsetT
}
#[allow(dead_code)]
pub fn field_offset_to_field_index(field_o: VOffsetT) -> VOffsetT {
debug_assert!(field_o >= 2);
let fixed_fields = 2; // VTable size and Object Size.
(field_o / (SIZE_VOFFSET as VOffsetT)) - fixed_fields
}
impl<'a> Follow<'a> for VTable<'a> {
type Inner = VTable<'a>;
fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
VTable::init(buf, loc)
}
}
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
use std::ptr::write_bytes;
use endian_scalar::{emplace_scalar, read_scalar};
use primitives::*;
/// VTableWriter compartmentalizes actions needed to create a vtable.
#[derive(Debug)]
pub struct VTableWriter<'a> {
buf: &'a mut [u8],
}
impl<'a> VTableWriter<'a> {
#[inline(always)]
pub fn init(buf: &'a mut [u8]) -> Self {
VTableWriter { buf: buf }
}
/// Writes the vtable length (in bytes) into the vtable.
///
/// Note that callers already need to have computed this to initialize
/// a VTableWriter.
///
/// In debug mode, asserts that the length of the underlying data is equal
/// to the provided value.
#[inline(always)]
pub fn write_vtable_byte_length(&mut self, n: VOffsetT) {
emplace_scalar::<VOffsetT>(&mut self.buf[..SIZE_VOFFSET], n);
debug_assert_eq!(n as usize, self.buf.len());
}
/// Writes an object length (in bytes) into the vtable.
#[inline(always)]
pub fn write_object_inline_size(&mut self, n: VOffsetT) {
emplace_scalar::<VOffsetT>(&mut self.buf[SIZE_VOFFSET..2 * SIZE_VOFFSET], n);
}
/// Gets an object field offset from the vtable. Only used for debugging.
///
/// Note that this expects field offsets (which are like pointers), not
/// field ids (which are like array indices).
#[inline(always)]
pub fn get_field_offset(&self, vtable_offset: VOffsetT) -> VOffsetT {
let idx = vtable_offset as usize;
read_scalar::<VOffsetT>(&self.buf[idx..idx + SIZE_VOFFSET])
}
/// Writes an object field offset into the vtable.
///
/// Note that this expects field offsets (which are like pointers), not
/// field ids (which are like array indices).
#[inline(always)]
pub fn write_field_offset(&mut self, vtable_offset: VOffsetT, object_data_offset: VOffsetT) {
let idx = vtable_offset as usize;
emplace_scalar::<VOffsetT>(&mut self.buf[idx..idx + SIZE_VOFFSET], object_data_offset);
}
/// Clears all data in this VTableWriter. Used to cleanly undo a
/// vtable write.
#[inline(always)]
pub fn clear(&mut self) {
// This is the closest thing to memset in Rust right now.
let len = self.buf.len();
let p = self.buf.as_mut_ptr() as *mut u8;
unsafe {
write_bytes(p, 0, len);
}
}
}
This diff is collapsed.
/*
* Copyright 2018 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// import the flatbuffers runtime library
extern crate flatbuffers;
// import the generated code
#[path = "./monster_generated.rs"]
mod monster_generated;
pub use monster_generated::my_game::sample::{get_root_as_monster,
Color, Equipment,
Monster, MonsterArgs,
Vec3,
Weapon, WeaponArgs};
// Example how to use FlatBuffers to create and read binary buffers.
fn main() {
// Build up a serialized buffer algorithmically.
// Initialize it with a capacity of 1024 bytes.
let mut builder = flatbuffers::FlatBufferBuilder::new_with_capacity(1024);
// Serialize some weapons for the Monster: A 'sword' and an 'axe'.
let weapon_one_name = builder.create_string("Sword");
let weapon_two_name = builder.create_string("Axe");
// Use the `Weapon::create` shortcut to create Weapons with named field
// arguments.
let sword = Weapon::create(&mut builder, &WeaponArgs{
name: Some(weapon_one_name),
damage: 3,
});
let axe = Weapon::create(&mut builder, &WeaponArgs{
name: Some(weapon_two_name),
damage: 5,
});
// Name of the Monster.
let name = builder.create_string("Orc");
// Inventory.
let inventory = builder.create_vector(&[0u8, 1, 2, 3, 4, 5, 6, 7, 8, 9]);
// Create a FlatBuffer `vector` that contains offsets to the sword and axe
// we created above.
let weapons = builder.create_vector(&[sword, axe]);
// Create the path vector of Vec3 objects:
//let x = Vec3::new(1.0, 2.0, 3.0);
//let y = Vec3::new(4.0, 5.0, 6.0);
//let path = builder.create_vector(&[x, y]);
// Note that, for convenience, it is also valid to create a vector of
// references to structs, like this:
// let path = builder.create_vector(&[&x, &y]);
// Create the monster using the `Monster::create` helper function. This
// function accepts a `MonsterArgs` struct, which supplies all of the data
// needed to build a `Monster`. To supply empty/default fields, just use the
// Rust built-in `Default::default()` function, as demononstrated below.
let orc = Monster::create(&mut builder, &MonsterArgs{
pos: Some(&Vec3::new(1.0f32, 2.0f32, 3.0f32)),
mana: 150,
hp: 80,
name: Some(name),
inventory: Some(inventory),
color: Color::Red,
weapons: Some(weapons),
equipped_type: Equipment::Weapon,
equipped: Some(axe.as_union_value()),
//path: Some(path),
..Default::default()
});
// Serialize the root of the object, without providing a file identifier.
builder.finish(orc, None);
// We now have a FlatBuffer we can store on disk or send over a network.
// ** file/network code goes here :) **
// Instead, we're going to access it right away (as if we just received it).
// This must be called after `finish()`.
let buf = builder.finished_data(); // Of type `&[u8]`
// Get access to the root:
let monster = get_root_as_monster(buf);
// Get and test some scalar types from the FlatBuffer.
let hp = monster.hp();
let mana = monster.mana();
let name = monster.name();
assert_eq!(hp, 80);
assert_eq!(mana, 150); // default
assert_eq!(name, Some("Orc"));
// Get and test a field of the FlatBuffer's `struct`.
assert!(monster.pos().is_some());
let pos = monster.pos().unwrap();
let x = pos.x();
let y = pos.y();
let z = pos.z();
assert_eq!(x, 1.0f32);
assert_eq!(y, 2.0f32);
assert_eq!(z, 3.0f32);
// Get an element from the `inventory` FlatBuffer's `vector`.
assert!(monster.inventory().is_some());
let inv = monster.inventory().unwrap();
// Note that this vector is returned as a slice, because direct access for
// this type, a u8 vector, is safe on all platforms:
let third_item = inv[2];
assert_eq!(third_item, 2);
// Get and test the `weapons` FlatBuffers's `vector`.
assert!(monster.weapons().is_some());
let weps = monster.weapons().unwrap();
//let weps_len = weps.len();
let wep2 = weps.get(1);
let second_weapon_name = wep2.name();
let second_weapon_damage = wep2.damage();
assert_eq!(second_weapon_name, Some("Axe"));
assert_eq!(second_weapon_damage, 5);
// Get and test the `Equipment` union (`equipped` field).
assert_eq!(monster.equipped_type(), Equipment::Weapon);
let equipped = monster.equipped_as_weapon().unwrap();
let weapon_name = equipped.name();
let weapon_damage = equipped.damage();
assert_eq!(weapon_name, Some("Axe"));
assert_eq!(weapon_damage, 5);
// Get and test the `path` FlatBuffers's `vector`.
//assert_eq!(monster.path().unwrap().len(), 2);
//assert_eq!(monster.path().unwrap()[0].x(), 1.0);
//assert_eq!(monster.path().unwrap()[1].x(), 4.0);
println!("The FlatBuffer was successfully created and accessed!");
}
......@@ -79,6 +79,10 @@ int main(int argc, const char *argv[]) {
flatbuffers::IDLOptions::kLua,
"Generate Lua files for tables/structs",
flatbuffers::GeneralMakeRule },
{ flatbuffers::GenerateRust, "-r", "--rust", "Rust", true, nullptr,
flatbuffers::IDLOptions::kRust,
"Generate Rust files for tables/structs",
flatbuffers::RustMakeRule },
{ flatbuffers::GeneratePhp, nullptr, "--php", "PHP", true, nullptr,
flatbuffers::IDLOptions::kPhp, "Generate PHP files for tables/structs",
flatbuffers::GeneralMakeRule },
......
......@@ -477,7 +477,8 @@ class CppGenerator : public BaseGenerator {
std::string GenTypeBasic(const Type &type, bool user_facing_type) const {
static const char * const ctypename[] = {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, \
RTYPE) \
#CTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
......@@ -247,7 +247,7 @@ class GeneralGenerator : public BaseGenerator {
// clang-format off
static const char * const java_typename[] = {
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#JTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......@@ -255,7 +255,7 @@ class GeneralGenerator : public BaseGenerator {
static const char * const csharp_typename[] = {
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#NTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
......@@ -690,7 +690,7 @@ static std::string GenTypeBasic(const Type &type) {
static const char *ctypename[] = {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#GTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
......@@ -81,7 +81,7 @@ class LobsterGenerator : public BaseGenerator {
static const char *ctypename[] = {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#PTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
......@@ -604,7 +604,7 @@ namespace lua {
static const char *ctypename[] = {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#PTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
......@@ -864,7 +864,7 @@ class PhpGenerator : public BaseGenerator {
static const char *ctypename[] = {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#NTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
......@@ -615,7 +615,7 @@ class PythonGenerator : public BaseGenerator {
static const char *ctypename[] = {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
#PTYPE,
FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
This diff is collapsed.
......@@ -131,7 +131,7 @@ bool Print<const void *>(const void *val, Type type, int indent,
switch (type.base_type) {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
case BASE_TYPE_ ## ENUM: \
if (!PrintVector<CTYPE>( \
*reinterpret_cast<const Vector<CTYPE> *>(val), \
......@@ -223,7 +223,7 @@ static bool GenStruct(const StructDef &struct_def, const Table *table,
switch (fd.value.type.base_type) {
// clang-format off
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
case BASE_TYPE_ ## ENUM: \
if (!GenField<CTYPE>(fd, table, struct_def.fixed, \
opts, indent + Indent(opts), _text)) { \
......@@ -234,7 +234,7 @@ static bool GenStruct(const StructDef &struct_def, const Table *table,
#undef FLATBUFFERS_TD
// Generate drop-thru case statements for all pointer types:
#define FLATBUFFERS_TD(ENUM, IDLTYPE, \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
case BASE_TYPE_ ## ENUM:
FLATBUFFERS_GEN_TYPES_POINTER(FLATBUFFERS_TD)
#undef FLATBUFFERS_TD
......
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
......@@ -14,8 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
../flatc --cpp --java --csharp --dart --go --binary --lobster --lua --python --js --ts --php --grpc --gen-mutable --reflect-names --gen-object-api --no-includes --cpp-ptr-type flatbuffers::unique_ptr --no-fb-import -I include_test monster_test.fbs monsterdata_test.json
../flatc --cpp --java --csharp --dart --go --binary --lobster --lua --python --js --ts --php --gen-mutable --reflect-names --no-fb-import --cpp-ptr-type flatbuffers::unique_ptr -o namespace_test namespace_test/namespace_test1.fbs namespace_test/namespace_test2.fbs
../flatc --cpp --java --csharp --dart --go --binary --lobster --lua --python --js --ts --php --rust --grpc --gen-mutable --reflect-names --gen-object-api --no-includes --cpp-ptr-type flatbuffers::unique_ptr --no-fb-import -I include_test monster_test.fbs monsterdata_test.json
../flatc --cpp --java --csharp --dart --go --binary --lobster --lua --python --js --ts --php --rust --gen-mutable --reflect-names --no-fb-import --cpp-ptr-type flatbuffers::unique_ptr -o namespace_test namespace_test/namespace_test1.fbs namespace_test/namespace_test2.fbs
../flatc --cpp --js --ts --php --gen-mutable --reflect-names --gen-object-api --cpp-ptr-type flatbuffers::unique_ptr -o union_vector ./union_vector/union_vector.fbs
../flatc -b --schema --bfbs-comments -I include_test monster_test.fbs
../flatc --jsonschema --schema -I include_test monster_test.fbs
......
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment