README.md 6.76 KB
Newer Older
1 2 3 4 5 6
# Build scripts that publish pre-compiled protoc artifacts
``protoc`` is the compiler for ``.proto`` files. It generates language bindings
for the messages and/or RPC services from ``.proto`` files.

Because ``protoc`` is a native executable, the scripts under this directory
build and publish a ``protoc`` executable (a.k.a. artifact) to Maven
7 8 9 10 11 12
repositories. The artifact can be used by build automation tools so that users
would not need to compile and install ``protoc`` for their systems.

## Versioning
The version of the ``protoc`` artifact must be the same as the version of the
Protobuf project.
13 14 15 16 17

## Artifact name
The name of a published ``protoc`` artifact is in the following format:
``protoc-<version>-<os>-<arch>.exe``, e.g., ``protoc-3.0.0-alpha-3-windows-x86_64.exe``.

18
## System requirement
Kun Zhang's avatar
Kun Zhang committed
19 20 21
Install [Apache Maven](http://maven.apache.org/) if you don't have it.

The scripts only work under Unix-like environments, e.g., Linux, MacOSX, and
22 23 24
Cygwin or MinGW for Windows. Please see ``README.md`` of the Protobuf project
for how to set up the build environment.

25 26 27
## Building from a freshly checked-out source

If you just checked out the Protobuf source from github, you need to
28
generate the configure script.
29 30 31 32 33 34 35

Under the protobuf project directory:

```
$ ./autogen.sh && ./configure && make
```

36 37 38
## To install artifacts locally
The following command will install the ``protoc`` artifact to your local Maven repository.
```
Kun Zhang's avatar
Kun Zhang committed
39
$ mvn install
40 41
```

42 43
## Cross-compilation
The Maven script will try to detect the OS and the architecture from Java
Kun Zhang's avatar
Kun Zhang committed
44
system properties. It's possible to build a protoc binary for an architecture
45
that is different from what Java has detected, as long as you have the proper
46 47 48 49 50 51 52 53 54 55 56
compilers installed.

You can override the Maven properties ``os.detected.name`` and
``os.detected.arch`` to force the script to generate binaries for a specific OS
and/or architecture. Valid values are defined as the return values of
``normalizeOs()`` and ``normalizeArch()`` of ``Detector`` from
[os-maven-plugin](https://github.com/trustin/os-maven-plugin/blob/master/src/main/java/kr/motd/maven/os/Detector.java).
Frequently used values are:
- ``os.detected.name``: ``linux``, ``osx``, ``windows``.
- ``os.detected.arch``: ``x86_32``, ``x86_64``

57
For example, MinGW32 only ships with 32-bit compilers, but you can still build
58
32-bit protoc under 64-bit Windows, with the following command:
59 60 61 62
```
$ mvn install -Dos.detected.arch=x86_32
```

63
## To push artifacts to Maven Central
Kun Zhang's avatar
Kun Zhang committed
64 65
Before you can upload artifacts to Maven Central repository, make sure you have
read [this page](http://central.sonatype.org/pages/apache-maven.html) on how to
Kun Zhang's avatar
Kun Zhang committed
66
configure GPG and Sonatype account.
67

68
You need to perform the deployment for every platform that you want to
69
support. DO NOT close the staging repository until you have done the
70
deployment for all platforms. Currently the following platforms are supported:
Jisi Liu's avatar
Jisi Liu committed
71
- Linux (x86_32, x86_64 and cross compiled aarch_64)
72
- Windows (x86_32 and x86_64) with
Jisi Liu's avatar
Jisi Liu committed
73 74 75
  - Cygwin64 with MinGW compilers (x86_64)
  - MSYS with MinGW32 (x86_32)
  - Cross compile in Linux with MinGW-w64 (x86_32, x86_64)
76
- MacOSX (x86_32 and x86_64)
77

78 79 80 81
As for MSYS2/MinGW64 for Windows: protoc will build, but it insists on
adding a dependency of `libwinpthread-1.dll`, which isn't shipped with
Windows.

82 83
Use the following command to deploy artifacts for the host platform to a
staging repository.
84
```
Kun Zhang's avatar
Kun Zhang committed
85
$ mvn clean deploy -P release
86
```
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
It creates a new staging repository. Go to
https://oss.sonatype.org/#stagingRepositories and find the repository, usually
in the name like ``comgoogle-123``.

You will want to run this command on a different platform. Remember, in
subsequent deployments you will need to provide the repository name that you
have found in the first deployment so that all artifacts go to the same
repository:
```
$ mvn clean deploy -P release -Dstaging.repository=comgoogle-123
```

A 32-bit artifact can be deployed from a 64-bit host with
``-Dos.detected.arch=x86_32``

Jisi Liu's avatar
Jisi Liu committed
102 103 104
An arm64 artifact can be deployed from x86 host with
``-Dos.detected.arch=aarch_64``

Jisi Liu's avatar
Jisi Liu committed
105 106 107
A windows artifact can be deployed from a linux machine with
``-Dos.detected.name=windows``

108 109 110 111
When you have done deployment for all platforms, go to
https://oss.sonatype.org/#stagingRepositories, verify that the staging
repository has all the binaries, close and release this repository.

112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130
## Upload zip packages to github release page.
After uploading protoc artifacts to Maven Central repository, run the
build-zip.sh script to bulid zip packages for these protoc binaries
and upload these zip packages to the download section of the github
release. For example:
```
$ ./build-zip.sh 3.0.0-beta-4
```
The above command will create 5 zip files:
```
dist/protoc-3.0.0-beta-4-win32.zip
dist/protoc-3.0.0-beta-4-osx-x86_32.zip
dist/protoc-3.0.0-beta-4-osx-x86_64.zip
dist/protoc-3.0.0-beta-4-linux-x86_32.zip
dist/protoc-3.0.0-beta-4-linux-x86_64.zip
```
Before running the script, make sure the artifacts are accessible from:
http://repo1.maven.org/maven2/com/google/protobuf/protoc/

131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147
### Tips for deploying on Linux
We build on Centos 6.6 to provide a good compatibility for not very new
systems. We have provided a ``Dockerfile`` under this directory to build the
environment. It has been tested with Docker 1.6.1.

To build a image:
```
$ docker build -t protoc-artifacts .
```

To run the image:
```
$ docker run -it --rm=true protoc-artifacts
```

The Protobuf repository has been cloned into ``/protobuf``.

148 149 150 151 152 153 154 155 156
### Tips for deploying on Windows
Under Windows the following error may occur: ``gpg: cannot open tty `no tty':
No such file or directory``. This can be fixed by configuring gpg through an
active profile in ``.m2\settings.xml`` where also the Sonatype password is
stored:
```xml
<settings>
  <servers>
    <server>
157
      <id>sonatype-nexus-staging</id>
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175
      <username>[username]</username>
      <password>[password]</password>
    </server>
  </servers>
  <profiles>
    <profile>
      <id>gpg</id>
      <properties>
        <gpg.executable>gpg</gpg.executable>
        <gpg.passphrase>[password]</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>gpg</activeProfile>
  </activeProfiles>
</settings>
```
Kun Zhang's avatar
Kun Zhang committed
176 177

### Tested build environments
178
We have successfully built artifacts on the following environments:
179
- Linux x86_32 and x86_64:
Jisi Liu's avatar
Jisi Liu committed
180 181 182
  - Centos 6.6 (within Docker 1.6.1)
  - Ubuntu 14.04.2 64-bit
- Linux aarch_64: Cross compiled with `g++-aarch64-linux-gnu` on Ubuntu 14.04.2 64-bit
Kun Zhang's avatar
Kun Zhang committed
183
- Windows x86_32: MSYS with ``mingw32-gcc-g++ 4.8.1-4`` on Windows 7 64-bit
Jisi Liu's avatar
Jisi Liu committed
184
- Windows x86_32: Cross compile with ``i686-w64-mingw32-g++ 4.8.2`` on Ubuntu 14.04.2 64-bit
Kun Zhang's avatar
Kun Zhang committed
185
- Windows x86_64: Cygwin64 with ``mingw64-x86_64-gcc-g++ 4.8.3-1`` on Windows 7 64-bit
Jisi Liu's avatar
Jisi Liu committed
186
- Windows x86_64: Cross compile with ``x86_64-w64-mingw32-g++ 4.8.2`` on Ubuntu 14.04.2 64-bit
Kun Zhang's avatar
Kun Zhang committed
187
- Mac OS X x86_32 and x86_64: Mac OS X 10.9.5