Commit c633a6f8 authored by Kenton Varda's avatar Kenton Varda

Update docs for new compiler.

parent 2d209702
...@@ -166,7 +166,7 @@ name "KJ" has no particular meaning; it was chosen to be short and easy-to-type. ...@@ -166,7 +166,7 @@ name "KJ" has no particular meaning; it was chosen to be short and easy-to-type.
To generate C++ code from your `.capnp` [interface definition](language.html), run: To generate C++ code from your `.capnp` [interface definition](language.html), run:
capnpc -oc++ myproto.capnp capnp compile -oc++ myproto.capnp
This will create `myproto.capnp.h` and `myproto.capnp.c++` in the same directory as `myproto.capnp`. This will create `myproto.capnp.h` and `myproto.capnp.c++` in the same directory as `myproto.capnp`.
...@@ -181,10 +181,10 @@ $Cxx.namespace("foo::bar::baz"); ...@@ -181,10 +181,10 @@ $Cxx.namespace("foo::bar::baz");
{% endhighlight %} {% endhighlight %}
Note that `capnp/c++.capnp` is installed in `$PREFIX/include` (`/usr/local/include` by default) Note that `capnp/c++.capnp` is installed in `$PREFIX/include` (`/usr/local/include` by default)
when you install the C++ runtime. `capnpc` automatically searches `/usr/include` and when you install the C++ runtime. The `capnp` tool automatically searches `/usr/include` and
`/usr/local/include` for imports that start with a `/`, so it should "just work". If you installed `/usr/local/include` for imports that start with a `/`, so it should "just work". If you installed
somewhere else, you may need to add it to the search path with the `-I` flag to `capnpc`, which somewhere else, you may need to add it to the search path with the `-I` flag to `capnp compile`,
works much like the compiler flag of the same name. which works much like the compiler flag of the same name.
## Types ## Types
......
...@@ -32,33 +32,11 @@ If you'd like to hack on Cap'n Proto, you should join the ...@@ -32,33 +32,11 @@ If you'd like to hack on Cap'n Proto, you should join the
If you'd just like to receive updates as things progress, add yourself to the If you'd just like to receive updates as things progress, add yourself to the
[announce list](https://groups.google.com/group/capnproto-announce). [announce list](https://groups.google.com/group/capnproto-announce).
## Installing the Cap'n Proto Compiler ## Installing the Cap'n Proto tools and C++ Runtime
`capnpc`, which takes `.capnp` files and generates source code for them (e.g. in C++), is itself The Cap'n Proto tools, including the compiler which takes `.capnp` files and generates source code
written in Haskell. for them, are written in C++. Therefore, you must install the C++ package even if your actual
development language is something else.
First, install [Cabal](http://www.haskell.org/cabal/), e.g. on Ubuntu:
sudo apt-get install cabal-install
Now you can download and install the release tarball:
<pre><code>curl -O <a href="http://capnproto.org/capnproto-compiler-0.1.0.tar.gz">http://capnproto.org/capnproto-compiler-0.1.0.tar.gz</a>
cabal install capnproto-compiler-0.1.0.tar.gz</code></pre>
Be sure that the Cabal bin directory (e.g. `$HOME/.cabal/bin` on Ubuntu or
`$HOME/Library/Haskell/bin` on Mac OSX) is in your `PATH` before you attempt to build the C++
runtime.
### Building the compiler from Git mainline
If you want to try out the latest -- possibly broken! -- compiler, do:
git clone https://github.com/kentonv/capnproto.git
cd capnproto/compiler
cabal install capnproto-compiler.cabal
## Installing the C++ Runtime
### GCC 4.7 or Clang 3.2 Needed ### GCC 4.7 or Clang 3.2 Needed
...@@ -80,8 +58,10 @@ There are two options: ...@@ -80,8 +58,10 @@ There are two options:
1. Use [Macports](http://www.macports.org/) or [Fink](http://www.finkproject.org/) to get an 1. Use [Macports](http://www.macports.org/) or [Fink](http://www.finkproject.org/) to get an
up-to-date GCC. up-to-date GCC.
2. Obtain Clang 3.2 (or better) 2. Obtain Clang 3.2
[directly from the LLVM project](http://llvm.org/releases/download.html). [directly from the LLVM project](http://llvm.org/releases/download.html). (Unfortunately,
Clang 3.3 apparently does NOT work, because the libc++ headers shipped with XCode contain
bugs that Clang 3.3 refuses to compile.)
Option 2 is the one preferred by Cap'n Proto's developers. Here are step-by-step instructions Option 2 is the one preferred by Cap'n Proto's developers. Here are step-by-step instructions
for setting this up: for setting this up:
...@@ -103,27 +83,26 @@ for setting this up: ...@@ -103,27 +83,26 @@ for setting this up:
You may now follow the instructions below, but make sure to tell `configure` to use your You may now follow the instructions below, but make sure to tell `configure` to use your
newly-downloaded Clang binary: newly-downloaded Clang binary:
./configure CXX=~/clang-3.2/bin/clang++ ./configure CXX=$HOME/clang-3.2/bin/clang++
Hopefully, Xcode 5.0 will be released soon with a newer Clang, making this extra work unnecessary. Hopefully, Xcode 5.0 will be released soon with a newer Clang, making this extra work unnecessary.
### Building from a release package ### Building from a release package
First, make sure you've installed the Cap'n Proto compiler, described above. You MUST use the You may download and install the release version of Cap'n Proto like so:
exact same version of the compiler and the runtime library! That means if you installed `capnpc`
from Git, you must install the runtime from Git.
You may download and install the release version of the C++ runtime like so:
<pre><code>curl -O <a href="http://capnproto.org/capnproto-c++-0.1.0.tar.gz">http://capnproto.org/capnproto-c++-0.1.0.tar.gz</a> <pre><code>curl -O <a href="http://capnproto.org/capnproto-c++-0.2.0.tar.gz">http://capnproto.org/capnproto-c++-0.2.0.tar.gz</a>
tar zxf capnproto-c++-0.1.0.tar.gz tar zxf capnproto-c++-0.2.0.tar.gz
cd capnproto-c++-0.1.0 cd capnproto-c++-0.2.0
./configure ./configure
make -j6 check make -j6 check
sudo make install</code></pre> sudo make install</code></pre>
This will install `libcapnp` in `/usr/local/lib` and headers in `/usr/local/include/capnp` and This will install `capnp`, the Cap'n Proto command-line tool. It will also install `libcapnp` in
`/usr/local/include/kj`. `/usr/local/lib` and headers in `/usr/local/include/capnp` and `/usr/local/include/kj`.
On Linux, if running `capnp` immediately after installation produces an error saying that the
`libcapnp` library does not exist, run `sudo ldconfig` and try again.
### Building from Git with Autotools ### Building from Git with Autotools
......
...@@ -7,13 +7,13 @@ title: Schema Language ...@@ -7,13 +7,13 @@ title: Schema Language
Like Protocol Buffers and Thrift (but unlike JSON or MessagePack), Cap'n Proto messages are Like Protocol Buffers and Thrift (but unlike JSON or MessagePack), Cap'n Proto messages are
strongly-typed and not self-describing. You must define your message structure in a special strongly-typed and not self-describing. You must define your message structure in a special
language, then invoke the Cap'n Proto compiler (`capnpc`) to generate source code to manipulate language, then invoke the Cap'n Proto compiler (`capnp compile`) to generate source code to
that message type in your desired language. manipulate that message type in your desired language.
For example: For example:
{% highlight python %} {% highlight python %}
# unique file ID, generated by capnpc -i # unique file ID, generated by `capnp id`
@0xdbb9ad1f14bf0b36; @0xdbb9ad1f14bf0b36;
struct Person { struct Person {
...@@ -341,8 +341,8 @@ struct Foo { ...@@ -341,8 +341,8 @@ struct Foo {
{% endhighlight %} {% endhighlight %}
The above imports specify relative paths. If the path begins with a `/`, it is absolute -- in The above imports specify relative paths. If the path begins with a `/`, it is absolute -- in
this case, `capnpc` searches for the file in each of the search path directories specified with this case, the `capnp` tool searches for the file in each of the search path directories specified
`-I`. with `-I`.
### Annotations ### Annotations
...@@ -412,7 +412,7 @@ $corge(string = "hello", number = 123); ...@@ -412,7 +412,7 @@ $corge(string = "hello", number = 123);
### Unique IDs ### Unique IDs
A Cap'n Proto file must have a unique 64-bit ID, and each type and annotation defined therein may A Cap'n Proto file must have a unique 64-bit ID, and each type and annotation defined therein may
also have an ID. Use `capnpc -i` to generate a new ID randomly. ID specifications begin with `@`: also have an ID. Use `capnp id` to generate a new ID randomly. ID specifications begin with `@`:
{% highlight capnp %} {% highlight capnp %}
# file ID # file ID
...@@ -436,10 +436,11 @@ annotation qux @0xf8a1bedf44c89f00 (field) :Text; ...@@ -436,10 +436,11 @@ annotation qux @0xf8a1bedf44c89f00 (field) :Text;
If you omit the ID for a type or annotation, one will be assigned automatically. This default If you omit the ID for a type or annotation, one will be assigned automatically. This default
ID is derived by taking the first 8 bytes of the MD5 hash of the parent scope's ID concatenated ID is derived by taking the first 8 bytes of the MD5 hash of the parent scope's ID concatenated
with the declaration's name (where the "parent scope" is the file for top-level delarations, or with the declaration's name (where the "parent scope" is the file for top-level delarations, or
the outer type for nested declarations). You can see the automatically-generated IDs by running the outer type for nested declarations). You can see the automatically-generated IDs by "compiling"
`capnpc -v` on a file. In general, you would only specify an explicit ID for a declaration if that your file with the `-ocapnp` flag, which echos the schema back to the terminal annotated with
declaration has been renamed or moved and you want the ID to stay the same for extra information, e.g. `capnp compile -ocapnp myschema.capnp`. In general, you would only specify
backwards-compatibility. an explicit ID for a declaration if that declaration has been renamed or moved and you want the ID
to stay the same for backwards-compatibility.
IDs exist to provide a relatively short yet unambiguous way to refer to a type or annotation from IDs exist to provide a relatively short yet unambiguous way to refer to a type or annotation from
another context. They may be used for representing schemas, for tagging dynamically-typed fields, another context. They may be used for representing schemas, for tagging dynamically-typed fields,
...@@ -447,7 +448,7 @@ etc. Most languages prefer instead to define a symbolic global namespace e.g. f ...@@ -447,7 +448,7 @@ etc. Most languages prefer instead to define a symbolic global namespace e.g. f
but this would have some important disadvantages in the context of Cap'n Proto: but this would have some important disadvantages in the context of Cap'n Proto:
* Programmers often feel the need to change symbolic names and organization in order to make their * Programmers often feel the need to change symbolic names and organization in order to make their
code cleaner. code cleaner, but the renamed code should still work with existing encoded data.
* It's easy for symbolic names to collide, and these collisions could be hard to detect in a large * It's easy for symbolic names to collide, and these collisions could be hard to detect in a large
distributed system with many different binaries using different versions of protocols. distributed system with many different binaries using different versions of protocols.
* Fully-qualified type names may be large and waste space when transmitted on the wire. * Fully-qualified type names may be large and waste space when transmitted on the wire.
......
...@@ -32,16 +32,18 @@ If you'd like to own the implementation of Cap'n Proto in some particular langua ...@@ -32,16 +32,18 @@ If you'd like to own the implementation of Cap'n Proto in some particular langua
useful tips that will save you time. :) useful tips that will save you time. :)
**Do not implement your own schema parser.** The schema language is more complicated than it **Do not implement your own schema parser.** The schema language is more complicated than it
looks, and the algorithm to determine offsets of fields is subtle. If you reuse `capnpc`'s parser, looks, and the algorithm to determine offsets of fields is subtle. If you reuse the official
you won't risk getting these wrong, and you won't have to spend time keeping your parser up-to-date. parser, you won't risk getting these wrong, and you won't have to spend time keeping your parser
In fact, you can still write your code generator in any language you want, using compiler plugins! up-to-date. In fact, you can still write your code generator in any language you want, using
compiler plugins!
### How to Write Compiler Plugins ### How to Write Compiler Plugins
The Cap'n Proto compiler / code generator binary, `capnpc`, supports a "plugin" interface for The Cap'n Proto tool, `capnp`, does not actually know how to generate code. It only parses schemas,
custom code generators. Plugins are independent executables (written in any language) which read then hands the parse tree off to another binary -- known as a "plugin" -- which generates the code.
a description of the schema from standard input and then generate the necessary code. The Plugins are independent executables (written in any language) which read a description of the
description is itself a Cap'n Proto message, defined by schema from standard input and then generate the necessary code. The description is itself a
Cap'n Proto message, defined by
[schema.capnp](https://github.com/kentonv/capnproto/blob/master/c%2B%2B/src/capnp/schema.capnp). [schema.capnp](https://github.com/kentonv/capnproto/blob/master/c%2B%2B/src/capnp/schema.capnp).
Specifically, the plugin receives a `CodeGeneratorRequest`, using Specifically, the plugin receives a `CodeGeneratorRequest`, using
[standard serialization](http://kentonv.github.io/capnproto/encoding.html#serialization_over_a_stream) [standard serialization](http://kentonv.github.io/capnproto/encoding.html#serialization_over_a_stream)
...@@ -52,42 +54,39 @@ Of course, because the input to a plugin is itself in Cap'n Proto format, if you ...@@ -52,42 +54,39 @@ Of course, because the input to a plugin is itself in Cap'n Proto format, if you
plugin directly in the language you wish to support, you may have a bootstrapping problem: you plugin directly in the language you wish to support, you may have a bootstrapping problem: you
somehow need to generate code for `schema.capnp` before you write your code generator. Luckily, somehow need to generate code for `schema.capnp` before you write your code generator. Luckily,
because of the simplicity of the Cap'n Proto format, it is generally not too hard to do this by because of the simplicity of the Cap'n Proto format, it is generally not too hard to do this by
hand. Remember that you can use `capnpc -v schema.capnp` to get a dump of the sizes and offsets hand. Remember that you can use `capnp compile -ocapnp schema.capnp` to get a dump of the sizes
of all structs and fields defined in the file. and offsets of all structs and fields defined in the file.
`capnpc` normally looks for plugins in `$PATH` with the name `capnpc-[language]`. However, if the `capnp compile` normally looks for plugins in `$PATH` with the name `capnpc-[language]`, e.g.
language name given on the command line contains a slash character, `capnpc` assumes that it is an `capnpc-c++` or `capnpc-capnp`. However, if the language name given on the command line contains
exact path to the plugin executable, and does not search `$PATH`. Examples: a slash character, `capnp` assumes that it is an exact path to the plugin executable, and does not
search `$PATH`. Examples:
# Searches $PATH for executable "capnpc-mylang". # Searches $PATH for executable "capnpc-mylang".
capnpc -o mylang addressbook.capnp capnp compile -o mylang addressbook.capnp
# Uses plugin executable "myplugin" from the current directory. # Uses plugin executable "myplugin" from the current directory.
capnpc -o ./myplugin addressbook.capnp capnp compile -o ./myplugin addressbook.capnp
If the user specifies an output directory, the compiler will run the plugin with that directory If the user specifies an output directory, the compiler will run the plugin with that directory
as the working directory, so you do not need to worry about this. as the working directory, so you do not need to worry about this.
[This example plugin](https://github.com/kentonv/capnproto/blob/master/c%2B%2B/src/capnp/compiler/capnpc-capnp.c%2B%2B) For examples of plugins, take a look at
writes the schema back to standard output in Cap'n Proto schema language, similar to what [capnpc-capnp](https://github.com/kentonv/capnproto/blob/master/c%2B%2B/src/capnp/compiler/capnpc-capnp.c%2B%2B)
`capnpc -v` does. or [capnpc-c++](https://github.com/kentonv/capnproto/blob/master/c%2B%2B/src/capnp/compiler/capnpc-c%2B%2B.c%2B%2B).
### Supporting Dynamic Languages ### Supporting Dynamic Languages
Dynamic languages have no compile step. This makes it difficult to work `capnpc` into the workflow Dynamic languages have no compile step. This makes it difficult to work `capnp compile` into the
for such languages. Additionally, dynamic languages are often scripting languages that do not workflow for such languages. Additionally, dynamic languages are often scripting languages that do
support pointer arithmetic or any reasonably-performant alternative. not support pointer arithmetic or any reasonably-performant alternative.
Fortunately, dynamic languages usually have facilities for calling native code. The best way to Fortunately, dynamic languages usually have facilities for calling native code. The best way to
support Cap'n Proto in a dynamic language, then, is to wrap the C++ library, in particular the support Cap'n Proto in a dynamic language, then, is to wrap the C++ library, in particular the
[C++ dynamic API](cxx.html#dynamic_reflection). This way you get reasonable performance while [C++ dynamic API](cxx.html#dynamic_reflection). This way you get reasonable performance while
still avoiding the need to generate any code specific to each schema. still avoiding the need to generate any code specific to each schema.
Of course, you still need to parse the schema. As of v0.1, the C++ runtime only has the ability Of course, you still need to parse the schema. Version 0.3 of Cap'n Proto will introduce a public
to load schemas that have already been parsed and translated to a binary format (defined by C++ API to the schema parser which your bindings can invoke. By the time you read this, the API
`schema.capnp`, the same format used by compiler plugins). It cannot parse `.capnp` files directly. is probably already available at git head, or will be within a few days;
In a future version of Cap'n Proto, the schema parser will be rewritten from Haskell to C++ to [send us a note](https://groups.google.com/group/capnproto) if you want to try it out.
solve this problem. For now, you can hack around it by invoking `capnpc` as a subprocess. Running
`capnpc -o/bin/cat myschema.capnp`, for example, will cause `capnpc` to invoke `cat` as if it were
a compiler plugin causing the compiled schemas to be written back to stdout, from which your calling
process can parse and load them.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment