• David Z. Chen's avatar
    Bazel build: Keep generated sources and Python runtime in the same directory. · 02cd45ca
    David Z. Chen authored
    Users often encounter a Python import error when trying to build Python
    protos if protobuf is installed locally on the machine. In this case,
    Python ends up looking in the wrong directory when importing files (see
    bazelbuild/bazel#1209 and tensorflow/tensorflow#2021). It seems that the
    problem is caused by Python getting confused when there are Python
    source files that are meant to be part of the same package but are
    in separate directories.
    
    Prior to #1233, the Bazel build setup would copy the Python
    runtime sources and all generated sources for the builtin protos into
    the root directory (assuming that the protobuf tree is vendored in a
    google/protobuf directory).
    
    With #1233, the two sets of sources are kept in their respective
    directories but both `src/` and `python/` are added to the `PYTHONPATH`
    using the new `imports` attribute of the Bazel Python rules. However,
    both the runtime sources and the generated sources are under the same
    package: `google.protobuf`, causing Python to become confused when
    trying to import modules that are in the other directory.
    
    This patch adds a workaround to the Bazel build to add a modified
    version of the original `internal_copied_filegroup` macro to copy the
    `.proto` files under `src/` to `python/` before building the
    `py_proto_library` targets for the builtin protos. This ensures that the
    generated sources for the builtin protos will be in the same directory
    as the corresponding runtime sources.
    
    This patch was tested with the following:
    * All Python tests in protobuf
    * All Python tests in tensorflow
    * All tests in [Skydoc](https://github.com/bazelbuild/skydoc)
    * Importing protobuf as `//google/protobuf`
    * Importing and binding targets under `//external`
    * Importing protobuf as `//third_party/protobuf`
    02cd45ca
protobuf.bzl 9.49 KB