Initial commit of the FlatBuffers code.

Change-Id: I4c9f0f722490b374257adb3fec63e44ae93da920
Tested: using VS2010 / Xcode / gcc on Linux.
parent c1b43e22
* text=auto
*_wire.txt
*_wire.bin
.DS_Store
*.o
*.o.d
*.class
*.a
*~
*.vcxproj
*.vcxproj.filters
*.vcxproj.user
*.sln
*.suo
*.keystore
**/bin/**
**/gen/**
**/libs/**
**/obj/**
**/*.dir/**
**/CMakeFiles/**
**/cmake_install.cmake
**/CMakeCache.txt
**/Debug/**
**/Release/**
build.xml
local.properties
project.properties
proguard-project.txt
linklint_results
Makefile
flatc
flattests
flatsamplebinary
flatsampletext
snapshot.sh
cmake_minimum_required(VERSION 2.8)
project(FlatBuffers)
# NOTE: Code coverage only works on Linux & OSX.
option(FLATBUFFERS_CODE_COVERAGE "Enable the code coverage build option." OFF)
set(FlatBuffers_Compiler_SRCS
include/flatbuffers/flatbuffers.h
include/flatbuffers/idl.h
include/flatbuffers/util.h
src/idl_parser.cpp
src/idl_gen_cpp.cpp
src/idl_gen_java.cpp
src/idl_gen_text.cpp
src/flatc.cpp
)
set(FlatBuffers_Tests_SRCS
include/flatbuffers/flatbuffers.h
include/flatbuffers/idl.h
include/flatbuffers/util.h
src/idl_parser.cpp
src/idl_gen_text.cpp
tests/test.cpp
# file generate by running compiler on tests/monster_test.fbs
tests/monster_test_generated.h
)
set(FlatBuffers_Sample_Binary_SRCS
include/flatbuffers/flatbuffers.h
samples/sample_binary.cpp
# file generate by running compiler on samples/monster.fbs
samples/monster_generated.h
)
set(FlatBuffers_Sample_Text_SRCS
include/flatbuffers/flatbuffers.h
include/flatbuffers/idl.h
include/flatbuffers/util.h
src/idl_parser.cpp
src/idl_gen_text.cpp
samples/sample_text.cpp
# file generate by running compiler on samples/monster.fbs
samples/monster_generated.h
)
set(CMAKE_BUILD_TYPE Debug)
# source_group(Compiler FILES ${FlatBuffers_Compiler_SRCS})
# source_group(Tests FILES ${FlatBuffers_Tests_SRCS})
if(CMAKE_COMPILER_IS_GNUCXX)
add_definitions("-std=c++0x")
add_definitions("-Wall")
endif()
if("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
add_definitions("-std=c++0x")
endif()
if(FLATBUFFERS_CODE_COVERAGE)
add_definitions("-g -fprofile-arcs -ftest-coverage")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fprofile-arcs -ftest-coverage")
endif()
include_directories(include)
add_executable(flatc ${FlatBuffers_Compiler_SRCS})
add_executable(flattests ${FlatBuffers_Tests_SRCS})
add_executable(flatsamplebinary ${FlatBuffers_Sample_Binary_SRCS})
add_executable(flatsampletext ${FlatBuffers_Sample_Text_SRCS})
add_test(NAME flattest
CONFIGURATIONS Debug
WORKING_DIRECTORY tests
COMMAND flattests)
This diff is collapsed.
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright (c) 2014 Google, Inc.
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
-->
<projectDescription>
<name>FlatBufferTest</name>
</projectDescription>
<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) 2013 Google, Inc.
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
-->
<!-- BEGIN_INCLUDE(manifest) -->
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.FlatBufferTest"
android:versionCode="1"
android:versionName="1.0">
<uses-feature android:glEsVersion="0x00020000"></uses-feature>
<!-- This is the platform API where NativeActivity was introduced. -->
<uses-sdk android:minSdkVersion="9" />
<!-- This .apk has no Java code itself, so set hasCode to false. -->
<application android:label="@string/app_name" android:hasCode="false">
<!-- Our activity is the built-in NativeActivity framework class.
This will take care of integrating with our NDK code. -->
<activity android:name="android.app.NativeActivity"
android:label="@string/app_name"
android:configChanges="orientation|keyboardHidden"
android:screenOrientation="landscape">
<!-- Tell NativeActivity the name of or .so -->
<meta-data android:name="android.app.lib_name"
android:value="FlatBufferTest" />
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>
<!-- END_INCLUDE(manifest) -->
This diff is collapsed.
# Copyright (c) 2013 Google, Inc.
#
# This software is provided 'as-is', without any express or implied
# warranty. In no event will the authors be held liable for any damages
# arising from the use of this software.
# Permission is granted to anyone to use this software for any purpose,
# including commercial applications, and to alter it and redistribute it
# freely, subject to the following restrictions:
# 1. The origin of this software must not be misrepresented; you must not
# claim that you wrote the original software. If you use this software
# in a product, an acknowledgment in the product documentation would be
# appreciated but is not required.
# 2. Altered source versions must be plainly marked as such, and must not be
# misrepresented as being the original software.
# 3. This notice may not be removed or altered from any source distribution.
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := FlatBufferTest
LOCAL_C_INCLUDES := $(LOCAL_PATH)/../../include
LOCAL_SRC_FILES := main.cpp ../../tests/test.cpp ../../src/idl_parser.cpp ../../src/idl_gen_text.cpp
LOCAL_LDLIBS := -llog -landroid
LOCAL_STATIC_LIBRARIES := android_native_app_glue
LOCAL_ARM_MODE:=arm
LOCAL_CPPFLAGS += -std=c++11 -fexceptions -Wall -Wno-literal-suffix
include $(BUILD_SHARED_LIBRARY)
$(call import-module,android/native_app_glue)
$(call import-add-path,../..)
# Copyright (c) 2014 Google, Inc.
#
# This software is provided 'as-is', without any express or implied
# warranty. In no event will the authors be held liable for any damages
# arising from the use of this software.
# Permission is granted to anyone to use this software for any purpose,
# including commercial applications, and to alter it and redistribute it
# freely, subject to the following restrictions:
# 1. The origin of this software must not be misrepresented; you must not
# claim that you wrote the original software. If you use this software
# in a product, an acknowledgment in the product documentation would be
# appreciated but is not required.
# 2. Altered source versions must be plainly marked as such, and must not be
# misrepresented as being the original software.
# 3. This notice may not be removed or altered from any source distribution.
APP_PLATFORM := android-10
APP_PROJECT_PATH := $(call my-dir)/..
APP_STL := gnustl_static
APP_ABI := armeabi-v7a
NDK_TOOLCHAIN_VERSION := 4.8
APP_CPPFLAGS += -std=c++11
/*
* Copyright 2014 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <android_native_app_glue.h>
extern int main(int argc, char **argv);
void android_main(android_app *app) {
// Make sure glue isn't stripped.
app_dummy();
main(0, NULL);
}
<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) 2014 Google, Inc.
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
-->
<resources>
<string name="app_name">FlatBufferTest</string>
</resources>
<html>
<head>
<meta http-equiv="refresh" content="0;url=html/index.html">
</head>
<body>
<a href="html/index.html">Click here if you are not redirected.</a>
</body>
</html>
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
This diff is collapsed.
function toggleVisibility(linkObj)
{
var base = $(linkObj).attr('id');
var summary = $('#'+base+'-summary');
var content = $('#'+base+'-content');
var trigger = $('#'+base+'-trigger');
var src=$(trigger).attr('src');
if (content.is(':visible')===true) {
content.hide();
summary.show();
$(linkObj).addClass('closed').removeClass('opened');
$(trigger).attr('src',src.substring(0,src.length-8)+'closed.png');
} else {
content.show();
summary.hide();
$(linkObj).removeClass('closed').addClass('opened');
$(trigger).attr('src',src.substring(0,src.length-10)+'open.png');
}
return false;
}
function updateStripes()
{
$('table.directory tr').
removeClass('even').filter(':visible:even').addClass('even');
}
function toggleLevel(level)
{
$('table.directory tr').each(function(){
var l = this.id.split('_').length-1;
var i = $('#img'+this.id.substring(3));
var a = $('#arr'+this.id.substring(3));
if (l<level+1) {
i.attr('src','ftv2folderopen.png');
a.attr('src','ftv2mnode.png');
$(this).show();
} else if (l==level+1) {
i.attr('src','ftv2folderclosed.png');
a.attr('src','ftv2pnode.png');
$(this).show();
} else {
$(this).hide();
}
});
updateStripes();
}
function toggleFolder(id)
{
//The clicked row
var currentRow = $('#row_'+id);
var currentRowImages = currentRow.find("img");
//All rows after the clicked row
var rows = currentRow.nextAll("tr");
//Only match elements AFTER this one (can't hide elements before)
var childRows = rows.filter(function() {
var re = new RegExp('^row_'+id+'\\d+_$', "i"); //only one sub
return this.id.match(re);
});
//First row is visible we are HIDING
if (childRows.filter(':first').is(':visible')===true) {
currentRowImages.filter("[id^=arr]").attr('src', 'ftv2pnode.png');
currentRowImages.filter("[id^=img]").attr('src', 'ftv2folderclosed.png');
rows.filter("[id^=row_"+id+"]").hide();
} else { //We are SHOWING
//All sub images
var childImages = childRows.find("img");
var childImg = childImages.filter("[id^=img]");
var childArr = childImages.filter("[id^=arr]");
currentRow.find("[id^=arr]").attr('src', 'ftv2mnode.png'); //open row
currentRow.find("[id^=img]").attr('src', 'ftv2folderopen.png'); //open row
childImg.attr('src','ftv2folderclosed.png'); //children closed
childArr.attr('src','ftv2pnode.png'); //children closed
childRows.show(); //show all children
}
updateStripes();
}
function toggleInherit(id)
{
var rows = $('tr.inherit.'+id);
var img = $('tr.inherit_header.'+id+' img');
var src = $(img).attr('src');
if (rows.filter(':first').is(':visible')===true) {
rows.css('display','none');
$(img).attr('src',src.substring(0,src.length-8)+'closed.png');
} else {
rows.css('display','table-row'); // using show() causes jump in firefox
$(img).attr('src',src.substring(0,src.length-10)+'open.png');
}
}
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Main Page</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('index.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">FlatBuffers Documentation</div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>FlatBuffers is an efficient cross platform serialization library in for C++ and Java. It was created at Google specifically for game development and other performance-critical applications.</p>
<p>It is available as open source under the Apache license, v2 (see LICENSE.txt).</p>
<h2>Why use FlatBuffers?</h2>
<ul>
<li><b>Access to serialized data without parsing/unpacking</b> - What sets FlatBuffers apart is that it represents hierarchical data in a flat binary buffer in such a way that it can still be accessed directly without parsing/unpacking, while also still supporting data structure evolution (forwards/backwards compatibility).</li>
<li><b>Memory efficiency and speed</b> - The only memory needed to access your data is that of the buffer. It requires 0 additional allocations. FlatBuffers is also very suitable for use with mmap (or streaming), requiring only part of the buffer to be in memory. Access is close to the speed of raw struct access with only one extra indirection (a kind of vtable) to allow for format evolution and optional fields. It is aimed at projects where spending time and space (many memory allocations) to be able to access or construct serialized data is undesirable, such as in games or any other performance sensitive applications. See the <a href="md__benchmarks.html">benchmarks</a> for details.</li>
<li><b>Flexible</b> - Optional fields means not only do you get great forwards and backwards compatibility (increasingly important for long-lived games: don't have to update all data with each new version!). It also means you have a lot of choice in what data you write and what data you don't, and how you design data structures.</li>
<li><b>Tiny code footprint</b> - Small amounts of generated code, and just a single small header as the minimum dependency, which is very easy to integrate. Again, see the benchmark section for details.</li>
<li><b>Strongly typed</b> - Errors happen at compile time rather than manually having to write repetitive and error prone run-time checks. Useful code can be generated for you.</li>
<li><p class="startli"><b>Convenient to use</b> - Generated C++ code allows for terse access &amp; construction code. Then there's optional functionality for parsing schemas and JSON-like text representations at runtime efficiently if needed (faster and more memory efficient than other JSON parsers).</p>
<p class="startli">Java code supports object-reuse.</p>
</li>
<li><b>Cross platform C++11/Java code with no dependencies</b> - will work with any recent gcc/clang and VS2010. Comes with build files for the tests &amp; samples (Android .mk files, and cmake for all other platforms).</li>
</ul>
<h3>Why not use Protocol Buffers, or .. ?</h3>
<p>Protocol Buffers is indeed relatively similar to FlatBuffers, with the primary difference being that FlatBuffers does not need a parsing/ unpacking step to a secondary representation before you can access data, often coupled with per-object memory allocation. The code is an order of magnitude bigger, too. Protocol Buffers has neither optional text import/export nor schema language features like unions.</p>
<h3>But all the cool kids use JSON!</h3>
<p>JSON is very readable (which is why we use it as our optional text format) and very convenient when used together with dynamically typed languages (such as JavaScript). When serializing data from statically typed languages, however, JSON not only has the obvious drawback of runtime inefficiency, but also forces you to write <em>more</em> code to access data (counterintuitively) due to its dynamic-typing serialization system. In this context, it is only a better choice for systems that have very little to no information ahead of time about what data needs to be stored.</p>
<p>Read more about the "why" of FlatBuffers in the <a href="md__white_paper.html">white paper</a>.</p>
<h2>Usage in brief</h2>
<p>This section is a quick rundown of how to use this system. Subsequent sections provide a more in-depth usage guide.</p>
<ul>
<li>Write a schema file that allows you to define the data structures you may want to serialize. Fields can have a scalar type (ints/floats of all sizes), or they can be a: string; array of any type; reference to yet another object; or, a set of possible objects (unions). Fields are optional and have defaults, so they don't need to be present for every object instance.</li>
<li>Use <code>flatc</code> (the FlatBuffer compiler) to generate a C++ header (or Java classes) with helper classes to access and construct serialized data. This header (say <code>mydata_generated.h</code>) only depends on <code>flatbuffers.h</code>, which defines the core functionality.</li>
<li>Use the <code>FlatBufferBuilder</code> class to construct a flat binary buffer. The generated functions allow you to add objects to this buffer recursively, often as simply as making a single function call.</li>
<li>Store or send your buffer somewhere!</li>
<li>When reading it back, you can obtain the pointer to the root object from the binary buffer, and from there traverse it conveniently in-place with <code>object-&gt;field()</code>.</li>
</ul>
<h2>In-depth documentation</h2>
<ul>
<li>How to <a href="md__building.html">build the compiler</a> and samples on various platforms.</li>
<li>How to <a href="md__compiler.html">use the compiler</a>.</li>
<li>How to <a href="md__schemas.html">write a schema</a>.</li>
<li>How to <a href="md__cpp_usage.html">use the generated C++ code</a> in your own programs.</li>
<li>How to <a href="md__java_usage.html">use the generated Java code</a> in your own programs.</li>
<li>Some <a href="md__benchmarks.html">benchmarks</a> showing the advantage of using FlatBuffers.</li>
<li>A <a href="md__white_paper.html">white paper</a> explaining the "why" of FlatBuffers.</li>
<li>A description of the <a href="md__internals.html">internals</a> of FlatBuffers.</li>
<li>A formal <a href="md__grammar.html">grammar</a> of the schema language.</li>
</ul>
<h2>Online resources</h2>
<ul>
<li><a href="http://github.com/google/flatbuffers">github repository</a></li>
<li><a href="http://google.github.io/flatbuffers">landing page</a></li>
<li><a href="http://group.google.com/group/flatbuffers">FlatBuffers Google Group</a></li>
<li><a href="http://github.com/google/flatbuffers/issues">FlatBuffers Issues Tracker</a> </li>
</ul>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
This source diff could not be displayed because it is too large. You can view the blob instead.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Benchmarks</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__benchmarks.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Benchmarks </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>Comparing against other serialization solutions, running on Windows 7 64bit. We use the LITE runtime for Protocol Buffers (less code / lower overhead), and Rapid JSON, one of the fastest C++ JSON parsers around.</p>
<p>We compare against Flatbuffers with the binary wire format (as intended), and also with JSON as the wire format with the optional JSON parser (which, using a schema, parses JSON into a binary buffer that can then be accessed as before).</p>
<p>The benchmark object is a set of about 10 objects containing an array, 4 strings, and a large variety of int/float scalar values of all sizes, meant to be representative of game data, e.g. a scene format.</p>
<table class="doxtable">
<tr>
<th></th><th>FlatBuffers (binary) </th><th>Protocol Buffers LITE </th><th>Rapid JSON </th><th>FlatBuffers (JSON) </th></tr>
<tr>
<td>Decode + Traverse + Dealloc (1 million times, seconds) </td><td>0.08 </td><td>305 </td><td>583 </td><td>105 </td></tr>
<tr>
<td>Decode / Traverse / Dealloc (breakdown) </td><td>0 / 0.08 / 0 </td><td>220 / 3.6 / 81 </td><td>294 / 0.9 / 287 </td><td>70 / 0.08 / 35 </td></tr>
<tr>
<td>Encode (1 million times, seconds) </td><td>3.2 </td><td>185 </td><td>650 </td><td>169 </td></tr>
<tr>
<td>Wire format size (normal / zlib, bytes) </td><td>344 / 220 </td><td>228 / 174 </td><td>1475 / 322 </td><td>1029 / 298 </td></tr>
<tr>
<td>Memory needed to store decoded wire (bytes / blocks) </td><td>0 / 0 </td><td>760 / 20 </td><td>65689 / 40 </td><td>328 / 1 </td></tr>
<tr>
<td>Transient memory allocated during decode (KB) </td><td>0 </td><td>1 </td><td>131 </td><td>4 </td></tr>
<tr>
<td>Generated source code size (KB) </td><td>4 </td><td>61 </td><td>0 </td><td>4 </td></tr>
<tr>
<td>Field access in handwritten traversal code </td><td>accessors </td><td>accessors </td><td>manual error checking </td><td>accessors </td></tr>
<tr>
<td>Library source code (KB) </td><td>15 </td><td>some subset of 3800 </td><td>87 </td><td>43 </td></tr>
</table>
<h3>Some other serialization systems we compared against but did not benchmark (yet), in rough order of applicability:</h3>
<ul>
<li>Cap'n'Proto promises to reduce Protocol Buffers much like FlatBuffers does, though with a more complicated binary encoding and less flexibility (no optional fields to allow deprecating fields or serializing with missing fields for which defaults exist). It currently also isn't fully cross-platform portable (lack of VS support).</li>
<li>msgpack: has very minimal forwards/backwards compatability support when used with the typed C++ interface. Also lacks VS2010 support.</li>
<li>Thrift: very similar to Protocol Buffers, but appears to be less efficient, and have more dependencies.</li>
<li>XML: typically even slower than JSON, but has the advantage that it can be parsed with a schema to reduce error-checking boilerplate code.</li>
<li>YAML: a superset of JSON and otherwise very similar. Used by e.g. Unity.</li>
<li>C# comes with built-in serialization functionality, as used by Unity also. Being tied to the language, and having no automatic versioning support limits its applicability.</li>
<li>Project Anarchy (the free mobile engine by Havok) comes with a serialization system, that however does no automatic versioning (have to code around new fields manually), is very much tied to the rest of the engine, and works without a schema to generate code (tied to your C++ class definition). </li>
</ul>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Building</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__building.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Building </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>The system comes with a <code>cmake</code> file that should allow you to build the compiler <code>flatc</code> and the tests (optionally). For details on <code>cmake</code>, see <a href="http://www.cmake.org">http://www.cmake.org</a>. In brief, depending on your platform, use one of e.g.: </p>
<pre class="fragment">cmake -G "Unix Makefiles"
cmake -G "Visual Studio 10"
cmake -G "Xcode"
</pre><p>Then, build as normal for your platform. This should result in a <code>flatc</code> executable, essential for the next steps. Note that to use clang instead of gcc, you may need to set up your environment variables, e.g. <code>CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake -G "Unix Makefiles"</code>.</p>
<p>Optionally, run the <code>flattests</code> executable. to ensure everything is working correctly on your system. If this fails, please contact us!</p>
<p>The cmake file will also build two sample executables, <code>sample_binary</code> and <code>sample_text</code>, see the corresponding <code>.cpp</code> file in the samples directory.</p>
<p>There is an <code>android</code> directory that contains all you need to build the test executable on android (use the included <code>build_apk.sh</code> script, or use <code>ndk_build</code> / <code>adb</code> etc. as usual). Upon running, it will output to the log if tests succeeded or not.</p>
<p>There is usually no runtime to compile, as the code consists of a single header, <code>include/flatbuffers/flatbuffers.h</code>. You should add the <code>include</code> folder to your include paths. If you wish to be able to load schemas and/or parse text into binary buffers at runtime, you additionally need the other headers in <code>include/flatbuffers</code>. You must also compile/link <code>src/idl_parser.cpp</code> (and <code>src/idl_gen_text.cpp</code> if you also want to be able convert binary to text).</p>
<p>For applications on Google Play that integrate this library, usage is tracked. This tracking is done automatically using the embedded version string (flatbuffer_version_string), and helps us continue to optimize it. Aside from consuming a few extra bytes in your application binary, it shouldn't affect your application at all. We use this information to let us know if FlatBuffers is useful and if we should continue to invest in it. Since this is open source, you are free to remove the version string but we would appreciate if you would leave it in. </p>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Using the schema compiler</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__compiler.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Using the schema compiler </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>Usage: </p>
<pre class="fragment">flatc [ -c ] [ -j ] [ -b ] [ -t ] file1 file2 ..
</pre><p>The files are read and parsed in order, and can contain either schemas or data (see below). Later files can make use of definitions in earlier files. Depending on the flags passed, additional files may be generated for each file processed:</p>
<ul>
<li><code>-c</code> : Generate a C++ header for all definitions in this file (as <code>filename_generated.h</code>). Skips data.</li>
<li><code>-j</code> : Generate Java classes.</li>
<li><code>-b</code> : If data is contained in this file, generate a <code>filename_wire.bin</code> containing the binary flatbuffer.</li>
<li><code>-t</code> : If data is contained in this file, generate a <code>filename_wire.txt</code> (for debugging). </li>
</ul>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
This diff is collapsed.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Formal Grammar of the schema language</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__grammar.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Formal Grammar of the schema language </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>schema = namespace_decl | type_decl | enum_decl | root_decl | object</p>
<p>namespace_decl = <code>namespace</code> ident ( <code>.</code> ident )* <code>;</code></p>
<p>type_decl = ( <code>table</code> | <code>struct</code> ) ident metadata <code>{</code> field_decl+ <code>}</code></p>
<p>enum_decl = ( <code>enum</code> | <code>union</code> ) ident [ <code>:</code> type ] metadata <code>{</code> commasep( enumval_decl ) <code>}</code></p>
<p>root_decl = <code>root_type</code> ident <code>;</code></p>
<p>field_decl = type <code>:</code> ident [ <code>=</code> scalar ] metadata <code>;</code></p>
<p>type = <code>bool</code> | <code>byte</code> | <code>ubyte</code> | <code>short</code> | <code>ushort</code> | <code>int</code> | <code>uint</code> | <code>float</code> | <code>long</code> | <code>ulong</code> | <code>double</code> | <code>string</code> | <code>[</code> type <code>]</code> | ident</p>
<p>enumval_decl = ident [ <code>=</code> integer_constant ]</p>
<p>metadata = [ <code>(</code> commasep( ident [ <code>:</code> scalar ] ) <code>)</code> ]</p>
<p>scalar = integer_constant | float_constant | <code>true</code> | <code>false</code></p>
<p>object = { commasep( ident <code>:</code> value ) }</p>
<p>value = scalar | object | string_constant | <code>[</code> commasep( value ) <code>]</code></p>
<p>commasep(x) = [ x ( <code>,</code> x )* ] </p>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
This diff is collapsed.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Use in Java</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__java_usage.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Use in Java </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>There's experimental support for reading FlatBuffers in Java. Generate code for Java with the <code>-j</code> option to <code>flatc</code>.</p>
<p>See <code>javaTest.java</code> for an example. Essentially, you read a FlatBuffer binary file into a <code>byte[]</code>, which you then turn into a <code>ByteBuffer</code>, which you pass to the <code>getRootAsMonster</code> function: </p>
<pre class="fragment">ByteBuffer bb = ByteBuffer.wrap(data);
Monster monster = Monster.getRootAsMonster(bb);
</pre><p>Now you can access values much like C++: </p>
<pre class="fragment">short hp = monster.hp();
Vec3 pos = monster.pos();
</pre><p>Note that whenever you access a new object like in the <code>pos</code> example above, a new temporary accessor object gets created. If your code is very performance sensitive (you iterate through a lot of objects), there's a second <code>pos()</code> method to which you can pass a <code>Vec3</code> object you've already created. This allows you to reuse it across many calls and reduce the amount of object allocation (and thus garbage collection) your program does.</p>
<p>Sadly the string accessors currently always create a new string when accessed, since FlatBuffer's UTF-8 strings can't be read in-place by Java.</p>
<p>Vector access is also a bit different from C++: you pass an extra index to the vector field accessor. Then a second method with the same name suffixed by <code>_length</code> let's you know the number of elements you can access: </p>
<pre class="fragment">for (int i = 0; i &lt; monster.inventory_length(); i++)
monster.inventory(i); // do something here
</pre><p>You can also construct these buffers in Java using the static methods found in the generated code, and the FlatBufferBuilder class: </p>
<pre class="fragment">FlatBufferBuilder fbb = new FlatBufferBuilder();
</pre><p>Create strings: </p>
<pre class="fragment">int str = fbb.createString("MyMonster");
</pre><p>Create a table with a struct contained therein: </p>
<pre class="fragment">Monster.startMonster(fbb);
Monster.addPos(fbb, Vec3.createVec3(fbb, 1.0f, 2.0f, 3.0f, 3.0, (byte)4, (short)5, (byte)6));
Monster.addHp(fbb, (short)80);
Monster.addName(fbb, str);
Monster.addInventory(fbb, inv);
Monster.addTest_type(fbb, (byte)1);
Monster.addTest(fbb, mon2);
Monster.addTest4(fbb, test4s);
int mon = Monster.endMonster(fbb);
</pre><p>As you can see, the Java code for tables does not use a convenient <code>createMonster</code> call like the C++ code. This is to create the buffer without using temporary object allocation (since the <code>Vec3</code> is an inline component of <code>Monster</code>, it has to be created right where it is added, whereas the name and the inventory are not inline). Structs do have convenient methods that even have arguments for nested structs.</p>
<p>Vectors also use this start/end pattern to allow vectors of both scalar types and structs: </p>
<pre class="fragment">Monster.startInventoryVector(fbb, 5);
for (byte i = 4; i &gt;=0; i--) fbb.addByte(i);
int inv = fbb.endVector();
</pre><p>You can use the generated method <code>startInventoryVector</code> to conveniently call <code>startVector</code> with the right element size. You pass the number of elements you want to write. You write the elements backwards since the buffer is being constructed back to front.</p>
<h2>Text Parsing</h2>
<p>There currently is no support for parsing text (Schema's and JSON) directly from Java, though you could use the C++ parser through JNI. Please see the C++ documentation for more on text parsing. </p>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
This diff is collapsed.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: FlatBuffers white paper</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__white_paper.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">FlatBuffers white paper </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>This document tries to shed some light on to the "why" of FlatBuffers, a new serialization library.</p>
<h2>Motivation</h2>
<p>Back in the good old days, performance was all about instructions and cycles. Nowadays, processing units have run so far ahead of the memory subsystem, that making an efficient application should start and finish with thinking about memory. How much you use of it. How you lay it out and access it. How you allocate it. When you copy it.</p>
<p>Serialization is a pervasive activity in a lot programs, and a common source of memory inefficiency, with lots of temporary data structures needed to parse and represent data, and inefficient allocation patterns and locality.</p>
<p>If it would be possible to do serialization with no temporary objects, no additional allocation, no copying, and good locality, this could be of great value. The reason serialization systems usually don't manage this is because it goes counter to forwards/backwards compatability, and platform specifics like endianness and alignment.</p>
<p>FlatBuffers is what you get if you try anyway.</p>
<p>In particular, FlatBuffers focus is on mobile hardware (where memory size and memory bandwidth is even more constrained than on desktop hardware), and applications that have the highest performance needs: games.</p>
<h2>FlatBuffers</h2>
<p><em>This is a summary of FlatBuffers functionality, with some rationale. A more detailed description can be found in the FlatBuffers documentation.</em></p>
<h3>Summary</h3>
<p>A FlatBuffer is a binary buffer containing nested objects (structs, tables, vectors,..) organized using offsets so that the data can be traversed in-place just like any pointer-based data structure. Unlike most in-memory data structures however, it uses strict rules of alignment and endianness (always little) to ensure these buffers are cross platform. Additionally, for objects that are tables, FlatBuffers provides forwards/backwards compatibility and general optionality of fields, to support most forms of format evolution.</p>
<p>You define your object types in a schema, which can then be compiled to C++ or Java for low to zero overhead reading &amp; writing. Optionally, JSON data can be dynamically parsed into buffers.</p>
<h3>Tables</h3>
<p>Tables are the cornerstone of FlatBuffers, since format evolution is essential for most applications of serialization. Typically, dealing with format changes is something that can be done transparently during the parsing process of most serialization solutions out there. But a FlatBuffer isn't parsed before it is accessed.</p>
<p>Tables get around this by using an extra indirection to access fields, through a <em>vtable</em>. Each table comes with a vtable (which may be shared between multiple tables with the same layout), and contains information where fields for this particular kind of instance of vtable are stored. The vtable may also indicate that the field is not present (because this FlatBuffer was written with an older version of the software, of simply because the information was not necessary for this instance, or deemed deprecated), in which case a default value is returned.</p>
<p>Tables have a low overhead in memory (since vtables are small and shared) and in access cost (an extra indirection), but provide great flexibility. Tables may even cost less memory than the equivalent struct, since fields do not need to be stored when they are equal to their default.</p>
<p>FlatBuffers additionally offers "naked" structs, which do not offer forwards/backwards compatibility, but can be even smaller (useful for very small objects that are unlikely to change, like e.g. a coordinate pair or a RGBA color).</p>
<h3>Schemas</h3>
<p>While schemas reduce some generality (you can't just read any data without having its schema), they have a lot of upsides:</p>
<ul>
<li>Most information about the format can be factored into the generated code, reducing memory needed to store data, and time to access it.</li>
<li>The strong typing of the data definitions means less error checking/handling at runtime (less can go wrong).</li>
<li>A schema enables us to access a buffer without parsing.</li>
</ul>
<p>FlatBuffer schemas are fairly similar to those of the incumbent, Protocol Buffers, and generally should be readable to those familiar with the C family of languages. We chose to improve upon the features offered by .proto files in the following ways:</p>
<ul>
<li>Deprecation of fields instead of manual field id assignment. Extending an object in a .proto means hunting for a free slot among the numbers (preferring lower numbers since they have a more compact representation). Besides being inconvenient, it also makes removing fields problematic: you either have to keep them, not making it obvious that this field shouldn't be read/written anymore, and still generating accessors. Or you remove it, but now you risk that there's still old data around that uses that field by the time someone reuses that field id, with nasty consequences.</li>
<li>Differentiating between tables and structs (see above). Effectively all table fields are <code>optional</code>, and all struct fields are <code>required</code>.</li>
<li>Having a native vector type instead of <code>repeated</code>. This gives you a length without having to collect all items, and in the case of scalars provides for a more compact representation, and one that guarantees adjacency.</li>
<li>Having a native <code>union</code> type instead of using a series of <code>optional</code> fields, all of which must be checked individually.</li>
<li>Being able to define defaults for all scalars, instead of having to deal with their optionality at each access.</li>
<li>A parser that can deal with both schemas and data definitions (JSON compatible) uniformly. </li>
</ul>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
#nav-tree .children_ul {
margin:0;
padding:4px;
}
#nav-tree ul {
list-style:none outside none;
margin:0px;
padding:0px;
}
#nav-tree li {
white-space:nowrap;
margin:0px;
padding:0px;
}
#nav-tree .plus {
margin:0px;
}
#nav-tree .selected {
background-image: url('tab_a.png');
background-repeat:repeat-x;
color: #fff;
text-shadow: 0px 1px 1px rgba(0, 0, 0, 1.0);
}
#nav-tree img {
margin:0px;
padding:0px;
border:0px;
vertical-align: middle;
}
#nav-tree a {
text-decoration:none;
padding:0px;
margin:0px;
outline:none;
}
#nav-tree .label {
margin:0px;
padding:0px;
font: 12px 'Lucida Grande',Geneva,Helvetica,Arial,sans-serif;
}
#nav-tree .label a {
padding:2px;
}
#nav-tree .selected a {
text-decoration:none;
color:#fff;
}
#nav-tree .children_ul {
margin:0px;
padding:0px;
}
#nav-tree .item {
margin:0px;
padding:0px;
}
#nav-tree {
padding: 0px 0px;
background-color: #FAFAFF;
font-size:14px;
overflow:auto;
}
#doc-content {
overflow:auto;
display:block;
padding:0px;
margin:0px;
-webkit-overflow-scrolling : touch; /* iOS 5+ */
}
#side-nav {
padding:0 6px 0 0;
margin: 0px;
display:block;
position: absolute;
left: 0px;
width: 250px;
}
.ui-resizable .ui-resizable-handle {
display:block;
}
.ui-resizable-e {
background:url("ftv2splitbar.png") repeat scroll right center transparent;
cursor:e-resize;
height:100%;
right:0;
top:0;
width:6px;
}
.ui-resizable-handle {
display:none;
font-size:0.1px;
position:absolute;
z-index:1;
}
#nav-tree-contents {
margin: 6px 0px 0px 0px;
}
#nav-tree {
background-image:url('nav_h.png');
background-repeat:repeat-x;
background-color: #F9FAFC;
-webkit-overflow-scrolling : touch; /* iOS 5+ */
}
#nav-sync {
position:absolute;
top:5px;
right:24px;
z-index:0;
}
#nav-sync img {
opacity:0.3;
}
#nav-sync img:hover {
opacity:0.9;
}
@media print
{
#nav-tree { display: none; }
div.ui-resizable-handle { display: none; position: relative; }
}
This diff is collapsed.
var NAVTREEINDEX0 =
{
"index.html":[],
"md__benchmarks.html":[5],
"md__building.html":[0],
"md__compiler.html":[1],
"md__cpp_usage.html":[3],
"md__grammar.html":[8],
"md__internals.html":[7],
"md__java_usage.html":[4],
"md__schemas.html":[2],
"md__white_paper.html":[6],
"pages.html":[]
};
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.5"/>
<title>FlatBuffers: Related Pages</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">FlatBuffers
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.5 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('pages.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Related Pages</div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock">Here is a list of all related documentation pages:</div><div class="directory">
<table class="directory">
<tr id="row_0_" class="even"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__building.html" target="_self">Building</a></td><td class="desc"></td></tr>
<tr id="row_1_"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__compiler.html" target="_self">Using the schema compiler</a></td><td class="desc"></td></tr>
<tr id="row_2_" class="even"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__schemas.html" target="_self">Writing a schema</a></td><td class="desc"></td></tr>
<tr id="row_3_"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__cpp_usage.html" target="_self">Use in C++</a></td><td class="desc"></td></tr>
<tr id="row_4_" class="even"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__java_usage.html" target="_self">Use in Java</a></td><td class="desc"></td></tr>
<tr id="row_5_"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__benchmarks.html" target="_self">Benchmarks</a></td><td class="desc"></td></tr>
<tr id="row_6_" class="even"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__white_paper.html" target="_self">FlatBuffers white paper</a></td><td class="desc"></td></tr>
<tr id="row_7_"><td class="entry"><img src="ftv2node.png" alt="o" width="16" height="22" /><a class="el" href="md__internals.html" target="_self">FlatBuffer Internals</a></td><td class="desc"></td></tr>
<tr id="row_8_" class="even"><td class="entry"><img src="ftv2lastnode.png" alt="\" width="16" height="22" /><a class="el" href="md__grammar.html" target="_self">Formal Grammar of the schema language</a></td><td class="desc"></td></tr>
</table>
</div><!-- directory -->
</div><!-- contents -->
</div><!-- doc-content -->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-49880327-7', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>
var cookie_namespace = 'doxygen';
var sidenav,navtree,content,header;
function readCookie(cookie)
{
var myCookie = cookie_namespace+"_"+cookie+"=";
if (document.cookie)
{
var index = document.cookie.indexOf(myCookie);
if (index != -1)
{
var valStart = index + myCookie.length;
var valEnd = document.cookie.indexOf(";", valStart);
if (valEnd == -1)
{
valEnd = document.cookie.length;
}
var val = document.cookie.substring(valStart, valEnd);
return val;
}
}
return 0;
}
function writeCookie(cookie, val, expiration)
{
if (val==undefined) return;
if (expiration == null)
{
var date = new Date();
date.setTime(date.getTime()+(10*365*24*60*60*1000)); // default expiration is one week
expiration = date.toGMTString();
}
document.cookie = cookie_namespace + "_" + cookie + "=" + val + "; expires=" + expiration+"; path=/";
}
function resizeWidth()
{
var windowWidth = $(window).width() + "px";
var sidenavWidth = $(sidenav).outerWidth();
content.css({marginLeft:parseInt(sidenavWidth)+"px"});
writeCookie('width',sidenavWidth, null);
}
function restoreWidth(navWidth)
{
var windowWidth = $(window).width() + "px";
content.css({marginLeft:parseInt(navWidth)+6+"px"});
sidenav.css({width:navWidth + "px"});
}
function resizeHeight()
{
var headerHeight = header.outerHeight();
var footerHeight = footer.outerHeight();
var windowHeight = $(window).height() - headerHeight - footerHeight;
content.css({height:windowHeight + "px"});
navtree.css({height:windowHeight + "px"});
sidenav.css({height:windowHeight + "px",top: headerHeight+"px"});
}
function initResizable()
{
header = $("#top");
sidenav = $("#side-nav");
content = $("#doc-content");
navtree = $("#nav-tree");
footer = $("#nav-path");
$(".side-nav-resizable").resizable({resize: function(e, ui) { resizeWidth(); } });
$(window).resize(function() { resizeHeight(); });
var width = readCookie('width');
if (width) { restoreWidth(width); } else { resizeWidth(); }
resizeHeight();
var url = location.href;
var i=url.indexOf("#");
if (i>=0) window.location.hash=url.substr(i);
var _preventDefault = function(evt) { evt.preventDefault(); };
$("#splitbar").bind("dragstart", _preventDefault).bind("selectstart", _preventDefault);
$(document).bind('touchmove',function(e){
try {
var target = e.target;
while (target) {
if ($(target).css('-webkit-overflow-scrolling')=='touch') return;
target = target.parentNode;
}
e.preventDefault();
} catch(err) {
e.preventDefault();
}
});
}
.tabs, .tabs2, .tabs3 {
background-image: url('tab_b.png');
width: 100%;
z-index: 101;
font-size: 13px;
font-family: 'Lucida Grande',Geneva,Helvetica,Arial,sans-serif;
}
.tabs2 {
font-size: 10px;
}
.tabs3 {
font-size: 9px;
}
.tablist {
margin: 0;
padding: 0;
display: table;
}
.tablist li {
float: left;
display: table-cell;
background-image: url('tab_b.png');
line-height: 36px;
list-style: none;
}
.tablist a {
display: block;
padding: 0 20px;
font-weight: bold;
background-image:url('tab_s.png');
background-repeat:no-repeat;
background-position:right;
color: #283A5D;
text-shadow: 0px 1px 1px rgba(255, 255, 255, 0.9);
text-decoration: none;
outline: none;
}
.tabs3 .tablist a {
padding: 0 10px;
}
.tablist a:hover {
background-image: url('tab_h.png');
background-repeat:repeat-x;
color: #fff;
text-shadow: 0px 1px 1px rgba(0, 0, 0, 1.0);
text-decoration: none;
}
.tablist li.current a {
background-image: url('tab_a.png');
background-repeat:repeat-x;
color: #fff;
text-shadow: 0px 1px 1px rgba(0, 0, 0, 1.0);
}
# Benchmarks
Comparing against other serialization solutions, running on Windows 7
64bit. We use the LITE runtime for Protocol Buffers (less code / lower
overhead), and Rapid JSON, one of the fastest C++ JSON parsers around.
We compare against Flatbuffers with the binary wire format (as
intended), and also with JSON as the wire format with the optional JSON
parser (which, using a schema, parses JSON into a binary buffer that can
then be accessed as before).
The benchmark object is a set of about 10 objects containing an array, 4
strings, and a large variety of int/float scalar values of all sizes,
meant to be representative of game data, e.g. a scene format.
| | FlatBuffers (binary) | Protocol Buffers LITE | Rapid JSON | FlatBuffers (JSON) |
|--------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|
| Decode + Traverse + Dealloc (1 million times, seconds) | 0.08 | 305 | 583 | 105 |
| Decode / Traverse / Dealloc (breakdown) | 0 / 0.08 / 0 | 220 / 3.6 / 81 | 294 / 0.9 / 287 | 70 / 0.08 / 35 |
| Encode (1 million times, seconds) | 3.2 | 185 | 650 | 169 |
| Wire format size (normal / zlib, bytes) | 344 / 220 | 228 / 174 | 1475 / 322 | 1029 / 298 |
| Memory needed to store decoded wire (bytes / blocks) | 0 / 0 | 760 / 20 | 65689 / 40 | 328 / 1 |
| Transient memory allocated during decode (KB) | 0 | 1 | 131 | 4 |
| Generated source code size (KB) | 4 | 61 | 0 | 4 |
| Field access in handwritten traversal code | accessors | accessors | manual error checking | accessors |
| Library source code (KB) | 15 | some subset of 3800 | 87 | 43 |
### Some other serialization systems we compared against but did not benchmark (yet), in rough order of applicability:
- Cap'n'Proto promises to reduce Protocol Buffers much like FlatBuffers does,
though with a more complicated binary encoding and less flexibility (no
optional fields to allow deprecating fields or serializing with missing
fields for which defaults exist).
It currently also isn't fully cross-platform portable (lack of VS support).
- msgpack: has very minimal forwards/backwards compatability support when used
with the typed C++ interface. Also lacks VS2010 support.
- Thrift: very similar to Protocol Buffers, but appears to be less efficient,
and have more dependencies.
- XML: typically even slower than JSON, but has the advantage that it can be
parsed with a schema to reduce error-checking boilerplate code.
- YAML: a superset of JSON and otherwise very similar. Used by e.g. Unity.
- C# comes with built-in serialization functionality, as used by Unity also.
Being tied to the language, and having no automatic versioning support
limits its applicability.
- Project Anarchy (the free mobile engine by Havok) comes with a serialization
system, that however does no automatic versioning (have to code around new
fields manually), is very much tied to the rest of the engine, and works
without a schema to generate code (tied to your C++ class definition).
# Building
The system comes with a `cmake` file that should allow you to build the
compiler `flatc` and the tests (optionally). For details on `cmake`, see
<http://www.cmake.org>. In brief, depending on your platform, use one of
e.g.:
cmake -G "Unix Makefiles"
cmake -G "Visual Studio 10"
cmake -G "Xcode"
Then, build as normal for your platform. This should result in a `flatc`
executable, essential for the next steps.
Note that to use clang instead of gcc, you may need to set up your environment
variables, e.g.
`CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake -G "Unix Makefiles"`.
Optionally, run the `flattests` executable.
to ensure everything is working correctly on your system. If this fails,
please contact us!
The cmake file will also build two sample executables, `sample_binary` and
`sample_text`, see the corresponding `.cpp` file in the samples directory.
There is an `android` directory that contains all you need to build the test
executable on android (use the included `build_apk.sh` script, or use
`ndk_build` / `adb` etc. as usual). Upon running, it will output to the log
if tests succeeded or not.
There is usually no runtime to compile, as the code consists of a single
header, `include/flatbuffers/flatbuffers.h`. You should add the
`include` folder to your include paths. If you wish to be
able to load schemas and/or parse text into binary buffers at runtime,
you additionally need the other headers in `include/flatbuffers`. You must
also compile/link `src/idl_parser.cpp` (and `src/idl_gen_text.cpp` if you
also want to be able convert binary to text).
For applications on Google Play that integrate this library, usage is tracked.
This tracking is done automatically using the embedded version string
(flatbuffer_version_string), and helps us continue to optimize it.
Aside from consuming a few extra bytes in your application binary, it shouldn't
affect your application at all. We use this information to let us know if
FlatBuffers is useful and if we should continue to invest in it. Since this is
open source, you are free to remove the version string but we would appreciate
if you would leave it in.
# Using the schema compiler
Usage:
flatc [ -c ] [ -j ] [ -b ] [ -t ] file1 file2 ..
The files are read and parsed in order, and can contain either schemas
or data (see below). Later files can make use of definitions in earlier
files. Depending on the flags passed, additional files may
be generated for each file processed:
- `-c` : Generate a C++ header for all definitions in this file (as
`filename_generated.h`). Skips data.
- `-j` : Generate Java classes.
- `-b` : If data is contained in this file, generate a
`filename_wire.bin` containing the binary flatbuffer.
- `-t` : If data is contained in this file, generate a
`filename_wire.txt` (for debugging).
# Use in C++
Assuming you have written a schema using the above language in say
`mygame.fbs` (FlatBuffer Schema, though the extension doesn't matter),
you've generated a C++ header called `mygame_generated.h` using the
compiler (e.g. `flatc -c mygame.fbs`), you can now start using this in
your program by including the header. As noted, this header relies on
`flatbuffers/flatbuffers.h`, which should be in your include path.
### Writing in C++
To start creating a buffer, create an instance of `FlatBufferBuilder`
which will contain the buffer as it grows:
FlatBufferBuilder fbb;
Before we serialize a Monster, we need to first serialize any objects
that are contained there-in, i.e. we serialize the data tree using
depth first, pre-order traversal. This is generally easy to do on
any tree structures. For example:
auto name = fbb.CreateString("MyMonster");
unsigned char inv[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
auto inventory = fbb.CreateVector(inv, 10);
`CreateString` and `CreateVector` serialize these two built-in
datatypes, and return offsets into the serialized data indicating where
they are stored, such that `Monster` below can refer to them.
`CreateString` can also take an `std::string`, or a `const char *` with
an explicit length, and is suitable for holding UTF-8 and binary
data if needed.
`CreateVector` can also take an `std::vector`. The
offset it returns is typed, i.e. can only be used to set fields of the
correct type below. To create a vector of struct objects (which will
be stored as contiguous memory in the buffer, use `CreateVectorOfStructs`
instead.
Vec3 vec(1, 2, 3);
`Vec3` is the first example of code from our generated
header. Structs (unlike tables) translate to simple structs in C++, so
we can construct them in a familiar way.
We have now serialized the non-scalar components of of the monster
example, so we could create the monster something like this:
auto mloc = CreateMonster(fbb, &vec, 150, 80, name, inventory, Color_Red, Offset<void>(0), Any_NONE);
Note that we're passing `150` for the `mana` field, which happens to be the
default value: this means the field will not actually be written to the buffer,
since we'll get that value anyway when we query it. This is a nice space
savings, since it is very common for fields to be at their default. It means
we also don't need to be scared to add fields only used in a minority of cases,
since they won't bloat up the buffer sizes if they're not actually used.
We do something similarly for the union field `test` by specifying a `0` offset
and the `NONE` enum value (part of every union) to indicate we don't actually
want to write this field.
Tables (like `Monster`) give you full flexibility on what fields you write
(unlike `Vec3`, which always has all fields set because it is a `struct`).
If you want even more control over this (i.e. skip fields even when they are
not default), instead of the convenient `CreateMonster` call we can also
build the object field-by-field manually:
MonsterBuilder mb(fbb);
mb.add_pos(&vec);
mb.add_hp(80);
mb.add_name(name);
mb.add_inventory(inventory);
auto mloc = mb.Finish();
We start with a temporary helper class `MonsterBuilder` (which is
defined in our generated code also), then call the various `add_`
methods to set fields, and `Finish` to complete the object. This is
pretty much the same code as you find inside `CreateMonster`, except
we're leaving out a few fields. Fields may also be added in any order,
though orderings with fields of the same size adjacent
to each other most efficient in size, due to alignment. You should
not nest these Builder classes (serialize your
data in pre-order).
Regardless of whether you used `CreateMonster` or `MonsterBuilder`, you
now have an offset to the root of your data, and you can finish the
buffer using:
fbb.Finish(mloc);
The buffer is now ready to be stored somewhere, sent over the network,
be compressed, or whatever you'd like to do with it. You can access the
start of the buffer with `fbb.GetBufferPointer()`, and it's size from
`fbb.GetSize()`.
`samples/sample_binary.cpp` is a complete code sample similar to
the code above, that also includes the reading code below.
### Reading in C++
If you've received a buffer from somewhere (disk, network, etc.) you can
directly start traversing it using:
auto monster = GetMonster(buffer_pointer);
`monster` is of type `Monster *`, and points to somewhere inside your
buffer. If you look in your generated header, you'll see it has
convenient accessors for all fields, e.g.
assert(monster->hp() == 80);
assert(monster->mana() == 150); // default
assert(strcmp(monster->name()->c_str(), "MyMonster") == 0);
These should all be true. Note that we never stored a `mana` value, so
it will return the default.
To access sub-objects, in this case the `Vec3`:
auto pos = monster->pos();
assert(pos);
assert(pos->z() == 3);
If we had not set the `pos` field during serialization, it would be
`NULL`.
Similarly, we can access elements of the inventory array:
auto inv = monster->inventory();
assert(inv);
assert(inv->Get(9) == 9);
### Direct memory access
As you can see from the above examples, all elements in a buffer are
accessed through generated accessors. This is because everything is
stored in little endian format on all platforms (the accessor
performs a swap operation on big endian machines), and also because
the layout of things is generally not known to the user.
For structs, layout is deterministic and guaranteed to be the same
accross platforms (scalars are aligned to their
own size, and structs themselves to their largest member), and you
are allowed to access this memory directly by using `sizeof()` and
`memcpy` on the pointer to a struct, or even an array of structs.
To compute offsets to sub-elements of a struct, make sure they
are a structs themselves, as then you can use the pointers to
figure out the offset without having to hardcode it. This is
handy for use of arrays of structs with calls like `glVertexAttribPointer`
in OpenGL or similar APIs.
It is important to note is that structs are still little endian on all
machines, so only use tricks like this if you can guarantee you're not
shipping on a big endian machine (an `assert(FLATBUFFERS_LITTLEENDIAN)`
would be wise).
## Text & schema parsing
Using binary buffers with the generated header provides a super low
overhead use of FlatBuffer data. There are, however, times when you want
to use text formats, for example because it interacts better with source
control, or you want to give your users easy access to data.
Another reason might be that you already have a lot of data in JSON
format, or a tool that generates JSON, and if you can write a schema for
it, this will provide you an easy way to use that data directly.
There are two ways to use text formats:
### Using the compiler as a conversion tool
This is the preferred path, as it doesn't require you to add any new
code to your program, and is maximally efficient since you can ship with
binary data. The disadvantage is that it is an extra step for your
users/developers to perform, though you might be able to automate it.
flatc -b myschema.fbs mydata.json
This will generate the binary file `mydata_wire.bin` which can be loaded
as before.
### Making your program capable of loading text directly
This gives you maximum flexibility. You could even opt to support both,
i.e. check for both files, and regenerate the binary from text when
required, otherwise just load the binary.
This option is currently only available for C++, or Java through JNI.
As mentioned in the section "Building" above, this technique requires
you to link a few more files into your program, and you'll want to include
`flatbuffers/idl.h`.
Load text (either a schema or json) into an in-memory buffer (there is a
convenient `LoadFile()` utility function in `flatbuffers/util.h` if you
wish). Construct a parser:
flatbuffers::Parser parser;
Now you can parse any number of text files in sequence:
parser.Parse(text_file.c_str());
This works similarly to how the command-line compiler works: a sequence
of files parsed by the same `Parser` object allow later files to
reference definitions in earlier files. Typically this means you first
load a schema file (which populates `Parser` with definitions), followed
by one or more JSON files.
If there were any parsing errors, `Parse` will return `false`, and
`Parser::err` contains a human readable error string with a line number
etc, which you should present to the creator of that file.
After each JSON file, the `Parser::fbb` member variable is the
`FlatBufferBuilder` that contains the binary buffer version of that
file, that you can access as described above.
`samples/sample_text.cpp` is a code sample showing the above operations.
### Threading
None of the code is thread-safe, by design. That said, since currently a
FlatBuffer is read-only and entirely `const`, reading by multiple threads
is possible.
This diff is collapsed.
# Formal Grammar of the schema language
schema = namespace\_decl | type\_decl | enum\_decl | root\_decl | object
namespace\_decl = `namespace` ident ( `.` ident )* `;`
type\_decl = ( `table` | `struct` ) ident metadata `{` field\_decl+ `}`
enum\_decl = ( `enum` | `union` ) ident [ `:` type ] metadata `{` commasep(
enumval\_decl ) `}`
root\_decl = `root_type` ident `;`
field\_decl = type `:` ident [ `=` scalar ] metadata `;`
type = `bool` | `byte` | `ubyte` | `short` | `ushort` | `int` | `uint` |
`float` | `long` | `ulong` | `double`
| `string` | `[` type `]` | ident
enumval\_decl = ident [ `=` integer\_constant ]
metadata = [ `(` commasep( ident [ `:` scalar ] ) `)` ]
scalar = integer\_constant | float\_constant | `true` | `false`
object = { commasep( ident `:` value ) }
value = scalar | object | string\_constant | `[` commasep( value ) `]`
commasep(x) = [ x ( `,` x )\* ]
This diff is collapsed.
# Use in Java
There's experimental support for reading FlatBuffers in Java. Generate code
for Java with the `-j` option to `flatc`.
See `javaTest.java` for an example. Essentially, you read a FlatBuffer binary
file into a `byte[]`, which you then turn into a `ByteBuffer`, which you pass to
the `getRootAsMonster` function:
ByteBuffer bb = ByteBuffer.wrap(data);
Monster monster = Monster.getRootAsMonster(bb);
Now you can access values much like C++:
short hp = monster.hp();
Vec3 pos = monster.pos();
Note that whenever you access a new object like in the `pos` example above,
a new temporary accessor object gets created. If your code is very performance
sensitive (you iterate through a lot of objects), there's a second `pos()`
method to which you can pass a `Vec3` object you've already created. This allows
you to reuse it across many calls and reduce the amount of object allocation (and
thus garbage collection) your program does.
Sadly the string accessors currently always create a new string when accessed,
since FlatBuffer's UTF-8 strings can't be read in-place by Java.
Vector access is also a bit different from C++: you pass an extra index
to the vector field accessor. Then a second method with the same name
suffixed by `_length` let's you know the number of elements you can access:
for (int i = 0; i < monster.inventory_length(); i++)
monster.inventory(i); // do something here
You can also construct these buffers in Java using the static methods found
in the generated code, and the FlatBufferBuilder class:
FlatBufferBuilder fbb = new FlatBufferBuilder();
Create strings:
int str = fbb.createString("MyMonster");
Create a table with a struct contained therein:
Monster.startMonster(fbb);
Monster.addPos(fbb, Vec3.createVec3(fbb, 1.0f, 2.0f, 3.0f, 3.0, (byte)4, (short)5, (byte)6));
Monster.addHp(fbb, (short)80);
Monster.addName(fbb, str);
Monster.addInventory(fbb, inv);
Monster.addTest_type(fbb, (byte)1);
Monster.addTest(fbb, mon2);
Monster.addTest4(fbb, test4s);
int mon = Monster.endMonster(fbb);
As you can see, the Java code for tables does not use a convenient
`createMonster` call like the C++ code. This is to create the buffer without
using temporary object allocation (since the `Vec3` is an inline component of
`Monster`, it has to be created right where it is added, whereas the name and
the inventory are not inline).
Structs do have convenient methods that even have arguments for nested structs.
Vectors also use this start/end pattern to allow vectors of both scalar types
and structs:
Monster.startInventoryVector(fbb, 5);
for (byte i = 4; i >=0; i--) fbb.addByte(i);
int inv = fbb.endVector();
You can use the generated method `startInventoryVector` to conveniently call
`startVector` with the right element size. You pass the number of
elements you want to write. You write the elements backwards since the buffer
is being constructed back to front.
## Text Parsing
There currently is no support for parsing text (Schema's and JSON) directly
from Java, though you could use the C++ parser through JNI. Please see the
C++ documentation for more on text parsing.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
{
pos: {
x: 1,
y: 2,
z: 3
},
hp: 80,
name: "MyMonster"
}
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment