Merge branch 'master' of github.com:brpc/brpc into cmake_support

.travis.yml

@@ -14,10 +14,6 @@ env:
 before_script:
 - ulimit -c unlimited -S       # enable core dumps
 
-after_failure:
-- COREFILE=$(find . -maxdepth 2 -name "core*" | head -n 1) # find core file
-- if [[ -f "$COREFILE" ]]; then gdb -c "$COREFILE" example -ex "thread apply all bt" -ex "set pagination 0" -batch; fi
-
 before_install:
 - wget --no-clobber https://github.com/bazelbuild/bazel/releases/download/0.8.1/bazel_0.8.1-linux-x86_64.deb
 - sudo dpkg -i bazel_0.8.1-linux-x86_64.deb

README.md

@@ -75,6 +75,7 @@ You can use it to:
     * [FlatMap](docs/cn/flatmap.md)
     * [brpc外功修炼宝典](docs/cn/brpc_intro.pptx)(training material)
     * [A tutorial on building large-scale services](docs/en/tutorial_on_building_services.pptx)(training material)
+    * [brpc internal](docs/en/brpc_internal.pptx)(training material)
   * RPC in depth
     * [New Protocol](docs/en/new_protocol.md)
     * [Atomic instructions](docs/en/atomic_instructions.md)

README_cn.md

@@ -76,6 +76,7 @@
     * [FlatMap](docs/cn/flatmap.md)
     * [brpc外功修炼宝典](docs/cn/brpc_intro.pptx)(培训材料)
     * [搭建大型服务入门](docs/en/tutorial_on_building_services.pptx)(培训材料)
+    * [brpc内功修炼宝典](docs/en/brpc_internal.pptx)(培训材料)
   * 深入RPC
     * [New Protocol](docs/cn/new_protocol.md)
     * [Atomic instructions](docs/cn/atomic_instructions.md)

docs/cn/client.md

@@ -96,6 +96,18 @@ BNS是百度内常用的名字服务，比如bns://rdev.matrix.all，其中"bns"
 
 连接一个域名下所有的机器, 例如http://www.baidu.com:80 ，注意连接单点的Init（两个参数）虽然也可传入域名，但只会连接域名下的一台机器。
 
+### consul://\<service-name\>
+
+通过consul获取服务名称为service-name的服务列表。consul的默认地址是localhost:8500，可通过gflags设置-consul\_agent\_addr来修改。consul的连接超时时间默认是200ms，可通过-consul\_connect\_timeout\_ms来修改。
+
+默认在consul请求参数中添加[stale](https://www.consul.io/api/index.html#consistency-modes)和passing（仅返回状态为passing的服务列表），可通过gflags中-consul\_url\_parameter改变[consul请求参数](https://www.consul.io/api/health.html#parameters-2)。
+
+除了对consul的首次请求，后续对consul的请求都采用[long polling](https://www.consul.io/api/index.html#blocking-queries)的方式，即仅当服务列表更新或请求超时后consul才返回结果，这里超时时间默认为60s，可通过-consul\_blocking\_query\_wait\_secs来设置。
+
+若consul返回的服务列表[响应格式](https://www.consul.io/api/health.html#sample-response-2)有错误，或者列表中所有服务都因为地址、端口等关键字段缺失或无法解析而被过滤，consul naming server会拒绝更新服务列表，并在一段时间后（默认500ms，可通过-consul\_retry\_interval\_ms设置）重新访问consul。
+
+如果consul不可访问，服务可自动降级到file naming service获取服务列表。此功能默认关闭，可通过设置-consul\_enable\_degrade\_to\_file\_naming\_service来打开。服务列表文件目录通过-consul \_file\_naming\_service\_dir来设置，使用service-name作为文件名。该文件可通过consul-template生成，里面会保存consul不可用之前最新的下游服务节点。当consul恢复时可自动恢复到consul naming service。
+
 ### 名字服务过滤器
 
 当名字服务获得机器列表后，可以自定义一个过滤器进行筛选，最后把结果传递给负载均衡：

docs/cn/memory_management.md

@@ -3,7 +3,7 @@
 - 线程间竞争少。内存分配的粒度大都比较小，对性能敏感，如果不同的线程在大多数分配时会竞争同一份资源或同一把锁，性能将会非常糟糕，原因无外乎和cache一致性有关，已被大量的malloc方案证明。
 - 浪费的空间少。如果每个线程各申请各的，速度也许不错，但万一一个线程总是申请，另一个线程总是释放，内存就爆炸了。线程之间总是要共享内存的，如何共享就是方案的关键了。
 
-一般的应用可以使用[tcmalloc](http://goog-perftools.sourceforge.net/doc/tcmalloc.html)、[jemalloc](https://github.com/jemalloc/jemalloc)等成熟的内存分配方案，但这对于较为底层，关注性能长尾的应用是不够的。多线程框架广泛地通过传递对象的ownership来让问题异步化，如何让分配这些小对象的开销变的更小是值得研究的。其中的一个特点较为显著：
+一般的应用可以使用[tcmalloc](http://goog-perftools.sourceforge.net/doc/tcmalloc.html)、[jemalloc](https://github.com/jemalloc/jemalloc)等成熟的内存分配方案，但这对于较为底层，关注性能长尾的应用是不够的。多线程框架广泛地通过传递对象的ownership来让问题异步化，如何让分配这些小对象的开销变的更小是值得研究的问题。其中的一个特点较为显著：
 
 - 大多数结构是等长的。
 

docs/en/client.md

@@ -96,6 +96,18 @@ Servers are directly written after list://, separated by comma. For example: "li
 
 Connect all servers under the domain, for example: http://www.baidu.com:80. Note: although Init() for connecting single server(2 parameters) accepts hostname as well, it only connects one server under the domain.
 
+### consul://\<service-name\>
+
+Get a list of servers with the specified service-name through consul. The default address of consul is localhost:8500, which can be modified by setting -consul\_agent\_addr in gflags. The connection timeout of consul is 200ms by default, which can be modified by -consul\_connect\_timeout\_ms. 
+
+By default, [stale](https://www.consul.io/api/index.html#consistency-modes) and passing(only servers with passing in statuses are returned) are added to [parameters of the consul request]((https://www.consul.io/api/health.html#parameters-2)), which can be modified by -consul\_url\_parameter in gflags.
+
+Except the first request to consul, the follow-up requests use the [long polling](https://www.consul.io/api/index.html#blocking-queries) feature. That is, the consul responds only when the server list is updated or the request times out. The timeout defaults to 60 seconds, which can be modified by -consul\_blocking\_query\_wait\_secs.
+
+If the server list returned by the consul does not follow [response format](https://www.consul.io/api/health.html#sample-response-2), or all servers in the list are filtered because the key fields such as the address and port are missing or cannot be parsed, the server list will not be updated and the consul service will be revisited after a period of time(default 500ms, can be modified by -consul\_retry\_interval\_ms）.
+
+If consul is not accessible, the naming service can be automatically downgraded to file naming service. This feature is turned off by default and can be turned on by setting -consul\_enable\_degrade\_to\_file\_naming\_service. After downgrading, in the directory specified by -consul\_file\_naming\_service\_dir, the file whose name is the service-name will be used. This file can be generated by the consul-template, which holds the latest server list before the consul is unavailable. The consul naming service is automatically restored when consul is restored.
+
 ### Naming Service Filter
 
 Users can filter servers got from the NamingService before pushing to LoadBalancer.

src/brpc/socket.cpp

@@ -651,6 +651,7 @@ int Socket::Create(const SocketOptions& options, SocketId* id) {
     }
     m->_last_writetime_us.store(cpuwide_now, butil::memory_order_relaxed);
     m->_unwritten_bytes.store(0, butil::memory_order_relaxed);
+    m->_options = options;
     CHECK(NULL == m->_write_head.load(butil::memory_order_relaxed));
     // Must be last one! Internal fields of this Socket may be access
     // just after calling ResetFileDescriptor.
@@ -662,7 +663,6 @@ int Socket::Create(const SocketOptions& options, SocketId* id) {
         return -1;
     }
     *id = m->_this_id;
-    m->_options = options;
     return 0;
 }
 
@@ -1733,7 +1733,7 @@ ssize_t Socket::DoWrite(WriteRequest* req) {
         }
     }
 
-    CHECK(ssl_state() == SSL_CONNECTED);
+    CHECK_EQ(SSL_CONNECTED, ssl_state());
     if (_conn) {
         // TODO: Separate SSL stuff from SocketConnection
         return _conn->CutMessageIntoSSLChannel(_ssl_session, data_list, ndata);
@@ -1772,6 +1772,10 @@ ssize_t Socket::DoWrite(WriteRequest* req) {
 
 int Socket::SSLHandshake(int fd, bool server_mode) {
     if (_options.ssl_ctx == NULL) {
+        if (server_mode) {
+            LOG(ERROR) << "Lack SSL configuration to handle SSL request";
+            return -1;
+        }
         return 0;
     }
 
@@ -1877,7 +1881,7 @@ ssize_t Socket::DoRead(size_t size_hint) {
         return _read_buf.append_from_file_descriptor(fd(), size_hint);
     }
 
-    CHECK(ssl_state() == SSL_CONNECTED);
+    CHECK_EQ(SSL_CONNECTED, ssl_state());
     int ssl_error = 0;
     ssize_t nr = _read_buf.append_from_SSL_channel(_ssl_session, &ssl_error, size_hint);
     switch (ssl_error) {

test/brpc_ssl_unittest.cpp

@@ -331,4 +331,6 @@ TEST_F(SSLTest, ssl_perf) {
     ASSERT_EQ(0, pthread_create(&spid, NULL, ssl_perf_server , serv_ssl));
     ASSERT_EQ(0, pthread_join(cpid, NULL));
     ASSERT_EQ(0, pthread_join(spid, NULL));
+    close(clifd);
+    close(servfd);
 }

test/butil_unittest_main.cpp

@@ -7,9 +7,7 @@
 #include "butil/logging.h"
 #include "multiprocess_func_list.h"
 
-// Disable coredumps by default to avoid generating a lot of coredumps
-// after running death tests.
-DEFINE_bool(disable_coredump, true, "Never core dump");
+DEFINE_bool(disable_coredump, false, "Never core dump");
 
 int main(int argc, char** argv) {
     butil::AtExitManager at_exit;

test/run_tests.sh

@@ -17,11 +17,19 @@ if [ $test_num -eq 0 ]; then
     >&2 echo "[runtest] Cannot find any tests"
     exit 1
 fi
+print_bt () {
+    COREFILE=$(find . -maxdepth 2 -name "core*" | head -n 1) # find core file
+    if [[ -f "$COREFILE" ]]; then
+        gdb -c "$COREFILE" $1 -ex "thread apply all bt" -ex "set pagination 0" -batch;
+    fi
+}
 if [ -z "$failed_test" ]; then
     >&2 echo "[runtest] $test_num succeeded"
 elif [ $test_num -gt 1 ]; then
+    print_bt $failed_test
     >&2 echo "[runtest] '$failed_test' failed, $((test_num-1)) succeeded"
 else
+    print_bt $failed_test
     >&2 echo "[runtest] '$failed_test' failed"
 fi
 exit $rc