client.md 60.7 KB
Newer Older
1 2
[中文版](../cn/client.md)

gejun's avatar
gejun committed
3 4
# Example

gejun's avatar
gejun committed
5
[client-side code](https://github.com/brpc/brpc/blob/master/example/echo_c++/client.cpp) of echo.
gejun's avatar
gejun committed
6 7 8 9

# Quick facts

- Channel.Init() is not thread-safe.
gejun's avatar
gejun committed
10 11 12
- Channel.CallMethod() is thread-safe and a Channel can be used by multiple threads simultaneously.
- Channel can be put on stack.
- Channel can be destructed just after sending asynchronous request.
gejun's avatar
gejun committed
13 14 15
- No class named brpc::Client.

# Channel
16
Client-side of RPC sends requests. It's called [Channel](https://github.com/brpc/brpc/blob/master/src/brpc/channel.h) rather than "Client" in brpc. A channel represents a communication line to one server or multiple servers, which can be used for calling services.
gejun's avatar
gejun committed
17

gejun's avatar
gejun committed
18
A Channel can be **shared by all threads** in the process. Yon don't need to create separate Channels for each thread, and you don't need to synchronize Channel.CallMethod with lock. However creation and destroying of Channel is **not** thread-safe,  make sure the channel is initialized and destroyed only by one thread.
gejun's avatar
gejun committed
19

gejun's avatar
gejun committed
20
Some RPC implementations have so-called "ClientManager", including configurations and resource management at the client-side, which is not needed by brpc. "thread-num", "connection-type" such parameters are either in brpc::ChannelOptions or global gflags. Advantages of doing so:
gejun's avatar
gejun committed
21

gejun's avatar
gejun committed
22 23
1. Convenience. You don't have to pass a "ClientManager" when the Channel is created, and you don't have to store the "ClientManager". Otherwise code has to pass "ClientManager" layer by layer, which is troublesome. gflags makes configurations of global behaviors easier.
2. Share resources. For example, servers and channels in brpc share background workers (of bthread).
gejun's avatar
gejun committed
24 25
3. Better management of Lifetime. Destructing a "ClientManager" is very error-prone, which is managed by brpc right now.

gejun's avatar
gejun committed
26
Like most classes, Channel must be **Init()**-ed before usage. Parameters take default values when `options` is NULL. If you want non-default values, code as follows:
gejun's avatar
gejun committed
27 28 29 30 31 32
```c++
brpc::ChannelOptions options;  // including default values
options.xxx = yyy;
...
channel.Init(..., &options);
```
gejun's avatar
gejun committed
33
Note that Channel neither modifies `options` nor accesses `options` after completion of Init(), thus options can be put on stack safely as in above code. Channel.options() gets options being used by the Channel.
gejun's avatar
gejun committed
34

gejun's avatar
gejun committed
35
Init() can connect one server or a cluster(multiple servers).
gejun's avatar
gejun committed
36

Ge Jun's avatar
Ge Jun committed
37
# Connect to a server
gejun's avatar
gejun committed
38 39 40 41 42 43 44

```c++
// Take default values when options is NULL.
int Init(EndPoint server_addr_and_port, const ChannelOptions* options);
int Init(const char* server_addr_and_port, const ChannelOptions* options);
int Init(const char* server_addr, int port, const ChannelOptions* options);
```
gejun's avatar
gejun committed
45
The server connected by these Init() has fixed address genrally. The creation does not need NamingService or LoadBalancer, being relatively light-weight.  The address could be a hostname, but don't frequently create Channels connecting to a hostname, which requires a DNS lookup taking at most 10 seconds. (default timeout of DNS lookup). Reuse them.
gejun's avatar
gejun committed
46 47 48

Valid "server_addr_and_port":
- 127.0.0.1:80
gejun's avatar
gejun committed
49
- www.foo.com:8765
gejun's avatar
gejun committed
50 51
- localhost:9000

52
Invalid "server_addr_and_port":
gejun's avatar
gejun committed
53
- 127.0.0.1:90000     # too large port
gejun's avatar
gejun committed
54
- 10.39.2.300:8000   # invalid IP
gejun's avatar
gejun committed
55

Ge Jun's avatar
Ge Jun committed
56
# Connect to a cluster
gejun's avatar
gejun committed
57 58 59 60 61 62

```c++
int Init(const char* naming_service_url,
         const char* load_balancer_name,
         const ChannelOptions* options);
```
63
Channels created by above Init() get server list from the NamingService specified by `naming_service_url` periodically or driven-by-events, and send request to one server chosen from the list according to the algorithm specified by `load_balancer_name` .
gejun's avatar
gejun committed
64

gejun's avatar
gejun committed
65
You **should not** create such channels ad-hocly each time before a RPC, because creation and destroying of such channels relate to many resources, say NamingService needs to be accessed once at creation otherwise server candidates are unknown. On the other hand, channels are able to be shared by multiple threads safely and has no need to be created frequently.
gejun's avatar
gejun committed
66

gejun's avatar
gejun committed
67
If `load_balancer_name` is NULL or empty, this Init() is just the one for connecting single server and `naming_service_url` should be "ip:port" or "host:port" of the server. Thus you can unify initialization of all channels with this Init(). For example, you can put values of `naming_service_url` and `load_balancer_name` in configuration file, and set `load_balancer_name` to empty for single server and a valid algorithm for a cluster.
gejun's avatar
gejun committed
68

gejun's avatar
gejun committed
69
## Naming Service
gejun's avatar
gejun committed
70

71
Naming service maps a name to a modifiable list of servers. It's positioned as follows at client-side:
gejun's avatar
gejun committed
72 73 74

![img](../images/ns.png)

gejun's avatar
gejun committed
75
With the help of naming service, the client remembers a name instead of every concrete server. When the servers are added or removed, only mapping in the naming service is changed, rather than telling every client that may access the cluster. This process is called "decoupling up and downstreams". Back to implementation details, the client does remember every server and will access NamingService periodically or be pushed with latest server list. The impl. has minimal impact on RPC latencies and very small pressure on the system providing naming service.
gejun's avatar
gejun committed
76 77

General form of `naming_service_url`  is "**protocol://service_name**".
gejun's avatar
gejun committed
78

gejun's avatar
gejun committed
79
### bns://\<bns-name\>
gejun's avatar
gejun committed
80

gejun's avatar
gejun committed
81
BNS is the most common naming service inside Baidu. In "bns://rdev.matrix.all", "bns" is protocol and "rdev.matrix.all" is service-name. A related gflag is -ns_access_interval: ![img](../images/ns_access_interval.png)
gejun's avatar
gejun committed
82

83
If the list in BNS is non-empty, but Channel says "no servers", the status bit of the machine in BNS is probably non-zero, which means the machine is unavailable and as a correspondence not added as server candidates of the Channel. Status bits can be checked by:
gejun's avatar
gejun committed
84 85 86

`get_instance_by_service [bns_node_name] -s`

gejun's avatar
gejun committed
87
### file://\<path\>
gejun's avatar
gejun committed
88

Ge Jun's avatar
Ge Jun committed
89 90 91 92 93 94 95 96 97 98 99 100
Servers are put in the file specified by `path`. For example, in "file://conf/machine_list", "conf/machine_list" is the file:
 * in which each line is address of a server. 
 * contents after \# are comments and ignored.
 * non-comment contents after addresses are tags, which are separated from addresses by one or more spaces, same address + different tags are treated as different instances.
 * brpc reloads the file when it's updated.
```
# This line is ignored
10.24.234.17 tag1  # a comment
10.24.234.17 tag2  # an instance different from the instance on last line
10.24.234.18
10.24.234.19
```
Ge Jun's avatar
Ge Jun committed
101 102 103 104

Pros: easy to modify, convenient for unittests.

Cons: need to update every client when the list changes, not suitable for online deployment.
gejun's avatar
gejun committed
105

gejun's avatar
gejun committed
106
### list://\<addr1\>,\<addr2\>...
gejun's avatar
gejun committed
107

gejun's avatar
gejun committed
108
Servers are directly written after list://, separated by comma. For example: "list://db-bce-81-3-186.db01:7000,m1-bce-44-67-72.m1:7000,cp01-rd-cos-006.cp01:7000" has 3 addresses.
Ge Jun's avatar
Ge Jun committed
109

Ge Jun's avatar
Ge Jun committed
110
Tags can be appended to addresses, separated with one or more spaces. Same address + different tags are treated as different instances.
Ge Jun's avatar
Ge Jun committed
111 112 113 114

Pros: directly configurable in CLI, convenient for unittests.

Cons: cannot be updated at runtime, not suitable for online deployment at all.
gejun's avatar
gejun committed
115

gejun's avatar
gejun committed
116
### http://\<url\>
gejun's avatar
gejun committed
117

Ge Jun's avatar
Ge Jun committed
118 119 120 121 122 123 124
Connect all servers under the domain, for example: http://www.baidu.com:80. 

Note: although Init() for connecting single server(2 parameters) accepts hostname as well, it only connects one server under the domain.

Pros: Versatility of DNS, useable both in private or public network.

Cons: limited by transmission formats of DNS, unable to implement notification mechanisms.
gejun's avatar
gejun committed
125

Ge Jun's avatar
Ge Jun committed
126 127 128 129
### https://\<url\>

Similar with "http" prefix besides that the connections will be encrypted with SSL.

130 131
### consul://\<service-name\>

132
Get a list of servers with the specified service-name through consul. The default address of consul is localhost:8500, which can be modified by setting -consul\_agent\_addr in gflags. The connection timeout of consul is 200ms by default, which can be modified by -consul\_connect\_timeout\_ms. 
133

134
By default, [stale](https://www.consul.io/api/index.html#consistency-modes) and passing(only servers with passing in statuses are returned) are added to [parameters of the consul request]((https://www.consul.io/api/health.html#parameters-2)), which can be modified by -consul\_url\_parameter in gflags.
135

136
Except the first request to consul, the follow-up requests use the [long polling](https://www.consul.io/api/index.html#blocking-queries) feature. That is, the consul responds only when the server list is updated or the request times out. The timeout defaults to 60 seconds, which can be modified by -consul\_blocking\_query\_wait\_secs.
137

138
If the server list returned by the consul does not follow [response format](https://www.consul.io/api/health.html#sample-response-2), or all servers in the list are filtered because the key fields such as the address and port are missing or cannot be parsed, the server list will not be updated and the consul service will be revisited after a period of time(default 500ms, can be modified by -consul\_retry\_interval\_ms).
139

140
If consul is not accessible, the naming service can be automatically downgraded to file naming service. This feature is turned off by default and can be turned on by setting -consul\_enable\_degrade\_to\_file\_naming\_service. After downgrading, in the directory specified by -consul\_file\_naming\_service\_dir, the file whose name is the service-name will be used. This file can be generated by the consul-template, which holds the latest server list before the consul is unavailable. The consul naming service is automatically restored when consul is restored.
141

Ge Jun's avatar
Ge Jun committed
142 143 144 145
### More naming services
User can extend to more naming services by implementing brpc::NamingService, check [this link](https://github.com/brpc/brpc/blob/master/docs/cn/load_balancing.md#%E5%91%BD%E5%90%8D%E6%9C%8D%E5%8A%A1) for details.

### The tag in naming service
Ge Jun's avatar
Ge Jun committed
146 147 148
Every address can be attached with a tag. The common implementation is that if there're spaces after the address, the content after the spaces is the tag.
Same address with different tag are treated as different instances which are interacted with separate connections. Users can use this feature to establish connections with remote servers in a more flexible way.
If you need weighted round-robin, you should consider using [wrr algorithm](#wrr) first rather than emulate "a coarse-grained version" with tags.
Ge Jun's avatar
Ge Jun committed
149 150 151

### VIP related issues
VIP is often the public IP of layer-4 load balancer, which proxies traffic to RS behide. When a client connects to the VIP, a connection is established to a chosen RS. When the client connection is broken, the connection to the RS is reset as well.
Ge Jun's avatar
Ge Jun committed
152

Ge Jun's avatar
Ge Jun committed
153
If one client establishes only one connection to the VIP("single" connection type in brpc), all traffic from the client lands on one RS. If number of clients are large enough, each RS should gets many connections and roughly balanced, at least from the cluster perspective. However, if clients are not large enough or workload from clients are very different, some RS may be overloaded. Another issue is that when multiple VIP are listed together, the traffic to them may not be proportional to the number of RS behide them.
Ge Jun's avatar
Ge Jun committed
154

Ge Jun's avatar
Ge Jun committed
155
One solution to these issues is to use "pooled" connection type, so that one client may create multiple connections to one VIP (roughly the max concurrency recently) to make traffic land on different RS. If more than one VIP are present, consider using [wrr load balancing](#wrr) to assign weights to different VIP, or add different tags to VIP to form more instances.
Ge Jun's avatar
Ge Jun committed
156

Ge Jun's avatar
Ge Jun committed
157
If higher performance is demanded, or number of connections is limited (in a large cluster), consider using single connection and attach same VIP with different tags to create different connections. Comparing to pooled connections, number of connections and overhead of syscalls are often lower, but if tags are not enough, RS hotspots may still present.
Ge Jun's avatar
Ge Jun committed
158

gejun's avatar
gejun committed
159
### Naming Service Filter
gejun's avatar
gejun committed
160

gejun's avatar
gejun committed
161
Users can filter servers got from the NamingService before pushing to LoadBalancer.
gejun's avatar
gejun committed
162 163 164

![img](../images/ns_filter.jpg)

165
Interface of the filter:
gejun's avatar
gejun committed
166
```c++
gejun's avatar
gejun committed
167 168 169 170 171 172 173
// naming_service_filter.h
class NamingServiceFilter {
public:
    // Return true to take this `server' as a candidate to issue RPC
    // Return false to filter it out
    virtual bool Accept(const ServerNode& server) const = 0;
};
174

gejun's avatar
gejun committed
175 176 177 178 179 180
// naming_service.h
struct ServerNode {
    butil::EndPoint addr;
    std::string tag;
};
```
gejun's avatar
gejun committed
181 182
The most common usage is filtering by server tags.

gejun's avatar
gejun committed
183
Customized filter is set to ChannelOptions to take effects. NULL by default means not filter.
gejun's avatar
gejun committed
184 185

```c++
gejun's avatar
gejun committed
186 187 188 189 190 191
class MyNamingServiceFilter : public brpc::NamingServiceFilter {
public:
    bool Accept(const brpc::ServerNode& server) const {
        return server.tag == "main";
    }
};
192

gejun's avatar
gejun committed
193 194 195 196 197 198 199 200 201 202
int main() {
    ...
    MyNamingServiceFilter my_filter;
    ...
    brpc::ChannelOptions options;
    options.ns_filter = &my_filter;
    ...
}
```

gejun's avatar
gejun committed
203
## Load Balancer
gejun's avatar
gejun committed
204

gejun's avatar
gejun committed
205
When there're more than one server to access, we need to divide the traffic. The process is called load balancing, which is positioned as follows at client-side.
gejun's avatar
gejun committed
206 207 208

![img](../images/lb.png)

209
The ideal algorithm is to make every request being processed in-time, and crash of any server makes minimal impact. However clients are not able to know delays or congestions happened at servers in realtime, and load balancing algorithms should be light-weight generally, users need to choose proper algorithms for their use cases. Algorithms provided by brpc (specified by `load_balancer_name`):
gejun's avatar
gejun committed
210 211 212

### rr

213
which is round robin. Always choose next server inside the list, next of the last server is the first one. No other settings. For example there're 3 servers: a,b,c, brpc will send requests to a, b, c, a, b, c, … and so on. Note that presumption of using this algorithm is the machine specs, network latencies, server loads are similar.
gejun's avatar
gejun committed
214

caidj's avatar
caidj committed
215 216 217 218
### wrr

which is weighted round robin. Choose the next server according to the configured weight. The chances a server is selected is consistent with its weight, and the algorithm can make each server selection scattered.

gejun's avatar
gejun committed
219 220
### random

gejun's avatar
gejun committed
221
Randomly choose one server from the list, no other settings. Similarly with round robin, the algorithm assumes that servers to access are similar.
gejun's avatar
gejun committed
222 223 224

### la

gejun's avatar
gejun committed
225
which is locality-aware. Perfer servers with lower latencies, until the latency is higher than others, no other settings. Check out [Locality-aware load balancing](lalb.md) for more details.
gejun's avatar
gejun committed
226 227 228

### c_murmurhash or c_md5

gejun's avatar
gejun committed
229
which is consistent hashing. Adding or removing servers does not make destinations of requests change as dramatically as in simple hashing. It's especially suitable for caching services.
gejun's avatar
gejun committed
230

gejun's avatar
gejun committed
231
Need to set Controller.set_request_code() before RPC otherwise the RPC will fail. request_code is often a 32-bit hash code of "key part" of the request, and the hashing algorithm does not need to be same with the one used by load balancer. Say `c_murmurhash`  can use md5 to compute request_code of the request as well.
gejun's avatar
gejun committed
232

233
[src/brpc/policy/hasher.h](https://github.com/brpc/brpc/blob/master/src/brpc/policy/hasher.h) includes common hash functions. If `std::string key` stands for key part of the request, controller.set_request_code(brpc::policy::MurmurHash32(key.data(), key.size())) sets request_code correctly.
gejun's avatar
gejun committed
234

gejun's avatar
gejun committed
235
Do distinguish "key" and "attributes" of the request. Don't compute request_code by full content of the request just for quick. Minor change in attributes may result in totally different hash code and change destination dramatically. Another cause is padding, for example: `struct Foo { int32_t a; int64_t b; }` has a 4-byte undefined gap between `a` and `b` on 64-bit machines, result of `hash(&foo, sizeof(foo))` is undefined. Fields need to be packed or serialized before hashing.
gejun's avatar
gejun committed
236

237
Check out [Consistent Hashing](consistent_hashing.md) for more details.
gejun's avatar
gejun committed
238

239 240
### Client-side throttling for recovery from cluster downtime

241
Cluster downtime refers to the state in which all servers in the cluster are unavailable. Due to the health check mechanism, when the cluster returns to normal, server will go online one by one. When a server is online, all traffic will be sent to it, which may cause the service to be overloaded again. If circuit breaker is enabled, server may be offline again before the other servers go online, and the cluster can never be recovered. As a solution, brpc provides a client-side throttling mechanism for recovery after cluster downtime. When no server is available in the cluster, the cluster enters recovery state. Assuming that the minimum number of servers that can serve all requests is min_working_instances, current number of servers available in the cluster is q, then in recovery state, the probability of client accepting the request is q/min_working_instances, otherwise it is discarded. If q remains unchanged for a period of time(hold_seconds), the traffic is resent to all available servers and leaves recovery state. Whether the request is rejected in recovery state is indicated by whether controller.ErrorCode() is equal to brpc::ERJECT, and the rejected request will not be retried by the framework.
242 243 244 245 246 247

This recovery mechanism requires the capabilities of downstream servers to be similar, so it is currently only valid for rr and random. The way to enable it is to add the values of min_working_instances and hold_seconds parameters after *load_balancer_name*, for example:
```c++
channel.Init("http://...", "random:min_working_instances=6 hold_seconds=10", &options);
```

gejun's avatar
gejun committed
248
## Health checking
gejun's avatar
gejun committed
249

gejun's avatar
gejun committed
250
Servers whose connections are lost are isolated temporarily to prevent them from being selected by LoadBalancer. brpc connects isolated servers periodically to test if they're healthy again. The interval is controlled by gflag -health_check_interval:
gejun's avatar
gejun committed
251 252 253

| Name                      | Value | Description                              | Defined At              |
| ------------------------- | ----- | ---------------------------------------- | ----------------------- |
254
| health_check_interval (R) | 3     | seconds between consecutive health-checkings | src/brpc/socket_map.cpp |
gejun's avatar
gejun committed
255

gejun's avatar
gejun committed
256
Once a server is connected, it resumes as a server candidate inside LoadBalancer. If a server is removed from NamingService during health-checking, brpc removes it from health-checking as well.
gejun's avatar
gejun committed
257

gejun's avatar
gejun committed
258
# Launch RPC
gejun's avatar
gejun committed
259

260
Generally, we don't use Channel.CallMethod directly, instead we call XXX_Stub generated by protobuf, which feels more like a "method call". The stub has few member fields, being suitable(and recommended) to be put on stack instead of new(). Surely the stub can be saved and re-used as well. Channel.CallMethod and stub are both **thread-safe** and accessible by multiple threads simultaneously. For example:
gejun's avatar
gejun committed
261 262 263 264
```c++
XXX_Stub stub(&channel);
stub.some_method(controller, request, response, done);
```
gejun's avatar
gejun committed
265
Or even:
gejun's avatar
gejun committed
266 267 268
```c++
XXX_Stub(&channel).some_method(controller, request, response, done);
```
Ge Jun's avatar
Ge Jun committed
269
A exception is http/h2 client, which is not related to protobuf much. Call CallMethod directly to make a http call, setting all parameters to NULL except for `Controller` and `done`, check [Access http/h2](http_client.md) for details.
gejun's avatar
gejun committed
270

gejun's avatar
gejun committed
271
## Synchronous call
gejun's avatar
gejun committed
272

273
CallMethod blocks until response from server is received or error occurred (including timedout).
gejun's avatar
gejun committed
274

gejun's avatar
gejun committed
275
response/controller in synchronous call will not be used by brpc again after CallMethod, they can be put on stack safely. Note: if request/response has many fields and being large on size, they'd better be allocated on heap.
gejun's avatar
gejun committed
276 277 278 279 280
```c++
MyRequest request;
MyResponse response;
brpc::Controller cntl;
XXX_Stub stub(&channel);
281

gejun's avatar
gejun committed
282 283 284
request.set_foo(...);
cntl.set_timeout_ms(...);
stub.some_method(&cntl, &request, &response, NULL);
ZX Xu's avatar
ZX Xu committed
285
if (cntl.Failed()) {
286
    // RPC failed. fields in response are undefined, don't use.
gejun's avatar
gejun committed
287
} else {
288
    // RPC succeeded, response has what we want.
gejun's avatar
gejun committed
289 290 291
}
```

gejun's avatar
gejun committed
292
## Asynchronous call
gejun's avatar
gejun committed
293

294
Pass a callback `done` to CallMethod, which resumes after sending request, rather than completion of RPC. When the response from server is received  or error occurred(including timedout), done->Run() is called. Post-processing code of the RPC should be put in done->Run() instead of after CallMethod.
gejun's avatar
gejun committed
295

gejun's avatar
gejun committed
296
Because end of CallMethod does not mean completion of RPC, response/controller may still be used by brpc or done->Run(). Generally they should be allocated on heap and deleted in done->Run(). If they're deleted too early, done->Run() may access invalid memory.
gejun's avatar
gejun committed
297

298
You can new these objects individually and create done by [NewCallback](#use-newcallback), or make response/controller be member of done and [new them together](#Inherit-google::protobuf::Closure). Former one is recommended.
gejun's avatar
gejun committed
299

300
**Request and Channel can be destroyed immediately after asynchronous CallMethod**, which is different from response/controller. Note that "immediately" means destruction of request/Channel can happen **after** CallMethod, not during CallMethod. Deleting a Channel just being used by another thread results in undefined behavior (crash at best).
gejun's avatar
gejun committed
301

gejun's avatar
gejun committed
302
### Use NewCallback
gejun's avatar
gejun committed
303 304
```c++
static void OnRPCDone(MyResponse* response, brpc::Controller* cntl) {
305
    // unique_ptr helps us to delete response/cntl automatically. unique_ptr in gcc 3.4 is an emulated version.
gejun's avatar
gejun committed
306 307 308
    std::unique_ptr<MyResponse> response_guard(response);
    std::unique_ptr<brpc::Controller> cntl_guard(cntl);
    if (cntl->Failed()) {
309
        // RPC failed. fields in response are undefined, don't use.
gejun's avatar
gejun committed
310
    } else {
gejun's avatar
gejun committed
311
        // RPC succeeded, response has what we want. Continue the post-processing.
gejun's avatar
gejun committed
312
    }
313
    // Closure created by NewCallback deletes itself at the end of Run.
gejun's avatar
gejun committed
314
}
315

gejun's avatar
gejun committed
316 317 318
MyResponse* response = new MyResponse;
brpc::Controller* cntl = new brpc::Controller;
MyService_Stub stub(&channel);
319

gejun's avatar
gejun committed
320 321 322 323 324
MyRequest request;  // you don't have to new request, even in an asynchronous call.
request.set_foo(...);
cntl->set_timeout_ms(...);
stub.some_method(cntl, &request, response, google::protobuf::NewCallback(OnRPCDone, response, cntl));
```
325
Since protobuf 3 changes NewCallback to private, brpc puts NewCallback in [src/brpc/callback.h](https://github.com/brpc/brpc/blob/master/src/brpc/callback.h) after r32035 (and adds more overloads). If your program has compilation issues with NewCallback, replace google::protobuf::NewCallback with brpc::NewCallback.
gejun's avatar
gejun committed
326

gejun's avatar
gejun committed
327
### Inherit google::protobuf::Closure
gejun's avatar
gejun committed
328

gejun's avatar
gejun committed
329
Drawback of using NewCallback is that you have to allocate memory on heap at least 3 times: response, controller, done. If profiler shows that the memory allocation is a hotspot, you can consider inheriting Closure by your own, and enclose response/controller as member fields. Doing so combines 3 new into one, but the code will be worse to read. Don't do this if memory allocation is not an issue.
gejun's avatar
gejun committed
330 331 332 333
```c++
class OnRPCDone: public google::protobuf::Closure {
public:
    void Run() {
gejun's avatar
gejun committed
334
        // unique_ptr helps us to delete response/cntl automatically. unique_ptr in gcc 3.4 is an emulated version.
gejun's avatar
gejun committed
335
        std::unique_ptr<OnRPCDone> self_guard(this);
336

gejun's avatar
gejun committed
337
        if (cntl->Failed()) {
gejun's avatar
gejun committed
338
            // RPC failed. fields in response are undefined, don't use.
gejun's avatar
gejun committed
339
        } else {
gejun's avatar
gejun committed
340
            // RPC succeeded, response has what we want. Continue the post-processing.
gejun's avatar
gejun committed
341 342
        }
    }
343

gejun's avatar
gejun committed
344 345 346
    MyResponse response;
    brpc::Controller cntl;
}
347

gejun's avatar
gejun committed
348 349
OnRPCDone* done = new OnRPCDone;
MyService_Stub stub(&channel);
350

gejun's avatar
gejun committed
351 352 353 354 355 356
MyRequest request;  // you don't have to new request, even in an asynchronous call.
request.set_foo(...);
done->cntl.set_timeout_ms(...);
stub.some_method(&done->cntl, &request, &done->response, done);
```

gejun's avatar
gejun committed
357
### What will happen when the callback is very complicated?
gejun's avatar
gejun committed
358

gejun's avatar
gejun committed
359
No special impact, the callback will run in separate bthread, without blocking other sessions. You can do all sorts of things in the callback.
gejun's avatar
gejun committed
360

gejun's avatar
gejun committed
361
### Does the callback run in the same thread that CallMethod runs?
gejun's avatar
gejun committed
362

gejun's avatar
gejun committed
363
The callback runs in a different bthread, even the RPC fails just after entering CallMethod. This avoids deadlock when the RPC is ongoing inside a lock(not recommended).
gejun's avatar
gejun committed
364

gejun's avatar
gejun committed
365
## Wait for completion of RPC
gejun's avatar
gejun committed
366
NOTE: [ParallelChannel](combo_channel.md#parallelchannel) is probably more convenient to  launch multiple RPCs in parallel.
gejun's avatar
gejun committed
367

gejun's avatar
gejun committed
368
Following code starts 2 asynchronous RPC and waits them to complete.
gejun's avatar
gejun committed
369 370 371 372 373 374 375 376 377 378
```c++
const brpc::CallId cid1 = controller1->call_id();
const brpc::CallId cid2 = controller2->call_id();
...
stub.method1(controller1, request1, response1, done1);
stub.method2(controller2, request2, response2, done2);
...
brpc::Join(cid1);
brpc::Join(cid2);
```
379
Call `Controller.call_id()` to get an id **before launching RPC**, join the id after the RPC.
gejun's avatar
gejun committed
380

381
Join() blocks until completion of RPC **and end of done->Run()**,  properties of Join:
gejun's avatar
gejun committed
382

gejun's avatar
gejun committed
383
- If the RPC is complete, Join() returns immediately.
384 385
- Multiple threads can Join() one id, all of them will be woken up.
- Synchronous RPC can be Join()-ed in another thread, although we rarely do this.
gejun's avatar
gejun committed
386

387
Join() was called JoinResponse() before, if you meet deprecated issues during compilation, rename to Join().
gejun's avatar
gejun committed
388

389
Calling `Join(controller->call_id())` after completion of RPC is **wrong**, do save call_id before RPC, otherwise the controller may be deleted by done at any time. The Join in following code is **wrong**.
gejun's avatar
gejun committed
390 391 392 393 394 395 396

```c++
static void on_rpc_done(Controller* controller, MyResponse* response) {
    ... Handle response ...
    delete controller;
    delete response;
}
397

gejun's avatar
gejun committed
398 399 400 401 402 403 404 405
Controller* controller1 = new Controller;
Controller* controller2 = new Controller;
MyResponse* response1 = new MyResponse;
MyResponse* response2 = new MyResponse;
...
stub.method1(controller1, &request1, response1, google::protobuf::NewCallback(on_rpc_done, controller1, response1));
stub.method2(controller2, &request2, response2, google::protobuf::NewCallback(on_rpc_done, controller2, response2));
...
gejun's avatar
gejun committed
406 407
brpc::Join(controller1->call_id());   // WRONG, controller1 may be deleted by on_rpc_done
brpc::Join(controller2->call_id());   // WRONG, controller2 may be deleted by on_rpc_done
gejun's avatar
gejun committed
408 409
```

gejun's avatar
gejun committed
410
## Semi-synchronous call
gejun's avatar
gejun committed
411

412
Join can be used for implementing "Semi-synchronous" call: blocks until multiple asynchronous calls to complete. Since the callsite blocks until completion of all RPC, controller/response can be put on stack safely.
gejun's avatar
gejun committed
413 414 415 416 417 418 419 420 421 422 423 424
```c++
brpc::Controller cntl1;
brpc::Controller cntl2;
MyResponse response1;
MyResponse response2;
...
stub1.method1(&cntl1, &request1, &response1, brpc::DoNothing());
stub2.method2(&cntl2, &request2, &response2, brpc::DoNothing());
...
brpc::Join(cntl1.call_id());
brpc::Join(cntl2.call_id());
```
425
brpc::DoNothing() gets a closure doing nothing, specifically for semi-synchronous calls. Its lifetime is managed by brpc.
gejun's avatar
gejun committed
426

gejun's avatar
gejun committed
427
Note that in above example, we access `controller.call_id()` after completion of RPC, which is safe right here, because DoNothing does not delete controller as in `on_rpc_done` in previous example.
gejun's avatar
gejun committed
428

gejun's avatar
gejun committed
429
## Cancel RPC
gejun's avatar
gejun committed
430

431
`brpc::StartCancel(call_id)` cancels corresponding RPC, call_id must be got from Controller.call_id() **before launching RPC**, race conditions may occur at any other time.
gejun's avatar
gejun committed
432

433
NOTE: it is `brpc::StartCancel(call_id)`, not `controller->StartCancel()`, which is forbidden and useless. The latter one is provided by protobuf by default and has serious race conditions on lifetime of controller.
gejun's avatar
gejun committed
434

gejun's avatar
gejun committed
435
As the name implies, RPC may not complete yet after calling StartCancel, you should not touch any field in Controller or delete any associated resources, they should be handled inside done->Run(). If you have to wait for completion of RPC in-place(not recommended), call Join(call_id).
gejun's avatar
gejun committed
436

437
Facts about StartCancel:
gejun's avatar
gejun committed
438

439 440 441 442
- call_id can be cancelled before CallMethod, the RPC will end immediately(and done will be called).
- call_id can be cancelled in another thread.
- Cancel an already-cancelled call_id has no effect. Inference: One call_id can be cancelled by multiple threads simultaneously, but only one of them takes effect.
- Cancel here is a client-only feature, **the server-side may not cancel the operation necessarily**, server cancelation is a separate feature.
gejun's avatar
gejun committed
443

gejun's avatar
gejun committed
444
## Get server-side address and port
gejun's avatar
gejun committed
445

gejun's avatar
gejun committed
446
remote_side() tells where request was sent to, the return type is [butil::EndPoint](https://github.com/brpc/brpc/blob/master/src/butil/endpoint.h), which includes an ipv4 address and port. Calling this method before completion of RPC is undefined.
gejun's avatar
gejun committed
447

448
How to print:
gejun's avatar
gejun committed
449 450 451 452
```c++
LOG(INFO) << "remote_side=" << cntl->remote_side();
printf("remote_side=%s\n", butil::endpoint2str(cntl->remote_side()).c_str());
```
gejun's avatar
gejun committed
453
## Get client-side address and port
gejun's avatar
gejun committed
454

gejun's avatar
gejun committed
455
local_side() gets address and port of the client-side sending RPC after r31384
gejun's avatar
gejun committed
456

457
How to print:
gejun's avatar
gejun committed
458
```c++
459
LOG(INFO) << "local_side=" << cntl->local_side();
gejun's avatar
gejun committed
460 461
printf("local_side=%s\n", butil::endpoint2str(cntl->local_side()).c_str());
```
gejun's avatar
gejun committed
462
## Should brpc::Controller be reused?
gejun's avatar
gejun committed
463

464
Not necessary to reuse deliberately.
gejun's avatar
gejun committed
465

466
Controller has miscellaneous fields, some of them are buffers that can be re-used by calling Reset().
gejun's avatar
gejun committed
467 468

In most use cases, constructing a Controller(snippet1) and re-using a Controller(snippet2) perform similarily.
gejun's avatar
gejun committed
469 470 471 472 473 474 475
```c++
// snippet1
for (int i = 0; i < n; ++i) {
    brpc::Controller controller;
    ...
    stub.CallSomething(..., &controller);
}
476

gejun's avatar
gejun committed
477 478 479 480 481 482 483 484
// snippet2
brpc::Controller controller;
for (int i = 0; i < n; ++i) {
    controller.Reset();
    ...
    stub.CallSomething(..., &controller);
}
```
485
If the Controller in snippet1 is new-ed on heap, snippet1 has extra cost of "heap allcation" and may be a little slower in some cases.
gejun's avatar
gejun committed
486

gejun's avatar
gejun committed
487
# Settings
gejun's avatar
gejun committed
488

489
Client-side settings has 3 parts:
gejun's avatar
gejun committed
490

491 492 493
- brpc::ChannelOptions: defined in [src/brpc/channel.h](https://github.com/brpc/brpc/blob/master/src/brpc/channel.h), for initializing Channel, becoming immutable once the initialization is done.
- brpc::Controller: defined in [src/brpc/controller.h](https://github.com/brpc/brpc/blob/master/src/brpc/controller.h), for overriding fields in brpc::ChannelOptions for some RPC according to contexts.
- global gflags: for tuning global behaviors, being unchanged generally. Read comments in [/flags](flags.md) before setting.
gejun's avatar
gejun committed
494

495
Controller contains data and options that request may not have. server and client share the same Controller class, but they may set different fields. Read comments in Controller carefully before using.
gejun's avatar
gejun committed
496

gejun's avatar
gejun committed
497
A Controller corresponds to a RPC. A Controller can be re-used by another RPC after Reset(), but a Controller can't be used by multiple RPC simultaneously, no matter the RPCs are started from one thread or not.
gejun's avatar
gejun committed
498

499 500 501 502
Properties of Controller:
1.  A Controller can only have one user. Without explicit statement, methods in Controller are **not** thread-safe by default.
2.  Due to the fact that Controller is not shared generally, there's no need to manage Controller by shared_ptr. If you do, something might goes wrong.
3.  Controller is constructed before RPC and destructed after RPC, some common patterns:
gejun's avatar
gejun committed
503
   - Put Controller on stack before synchronous RPC, be destructed when out of scope. Note that Controller of asynchronous RPC **must not** be put on stack, otherwise the RPC may still run when the Controller is being destructed and result in undefined behavior.
504
   - new Controller before asynchronous RPC, delete in done.
gejun's avatar
gejun committed
505

506 507 508 509
## Number of worker pthreads

There's **no** independent thread pool for client in brpc. All Channels and Servers share the same backing threads via [bthread](bthread.md).  Setting number of worker pthreads in Server works for Client as well if Server is in used. Or just specify the [gflag](flags.md) [-bthread_concurrency](brpc.baidu.com:8765/flags/bthread_concurrency) to set the global number of worker pthreads.

gejun's avatar
gejun committed
510
## Timeout
gejun's avatar
gejun committed
511

512
**ChannelOptions.timeout_ms** is timeout in milliseconds for all RPCs via the Channel, Controller.set_timeout_ms() overrides value for one RPC. Default value is 1 second, Maximum value is 2^31 (about 24 days), -1 means wait indefinitely for response or connection error.
gejun's avatar
gejun committed
513

514
**ChannelOptions.connect_timeout_ms** is the timeout in milliseconds for establishing connections of RPCs over the Channel, and -1 means no deadline. This value is limited to be not greater than timeout_ms. Note that this connection timeout is different from the one in TCP, generally this one is smaller.
gejun's avatar
gejun committed
515

516
NOTE1: timeout_ms in brpc is **deadline**, which means once it's reached, the RPC ends without more retries. As a comparison, other implementations may have session timeouts and deadline timeouts. Do distinguish them before porting to brpc.
gejun's avatar
gejun committed
517 518

NOTE2: error code of RPC timeout is **ERPCTIMEDOUT (1008) **, ETIMEDOUT is connection timeout and retriable.
gejun's avatar
gejun committed
519

gejun's avatar
gejun committed
520
## Retry
gejun's avatar
gejun committed
521

522
ChannelOptions.max_retry is maximum retrying count for all RPC via the channel, Controller.set_max_retry() overrides value for one RPC. Default value is 3. 0 means no retries.
gejun's avatar
gejun committed
523

gejun's avatar
gejun committed
524
Controller.retried_count() returns number of retries.
gejun's avatar
gejun committed
525

gejun's avatar
gejun committed
526
Controller.has_backup_request() tells if backup_request was sent.
gejun's avatar
gejun committed
527

528
**Servers tried before are not retried by best efforts**
gejun's avatar
gejun committed
529

530
Conditions for retrying (AND relations):
gejun's avatar
gejun committed
531
- Broken connection.
532 533 534
- Timeout is not reached.
- Has retrying quota. Controller.set_max_retry(0) or ChannelOptions.max_retry = 0 disables retries.
- The retry makes sense. If the RPC fails due to request(EREQUEST), no retry will be done because server is very likely to reject the request again, retrying makes no sense here.
gejun's avatar
gejun committed
535

gejun's avatar
gejun committed
536
### Broken connection
gejun's avatar
gejun committed
537

gejun's avatar
gejun committed
538
If the server does not respond and connection is good, retry is not triggered. If you need to send another request after some timeout, use backup request.
gejun's avatar
gejun committed
539

540
How it works: If response does not return within the timeout specified by backup_request_ms, send another request, take whatever the first returned. New request will be sent to a different server that never tried before by best efforts. NOTE: If backup_request_ms is greater than timeout_ms, backup request will never be sent. backup request consumes one retry. backup request does not imply a server-side cancellation.
gejun's avatar
gejun committed
541

542
ChannelOptions.backup_request_ms affects all RPC via the Channel, unit is milliseconds, Default value is -1(disabled), Controller.set_backup_request_ms() overrides value for one RPC.
gejun's avatar
gejun committed
543

gejun's avatar
gejun committed
544
### Timeout is not reached
gejun's avatar
gejun committed
545

546
RPC will be ended soon after the timeout.
gejun's avatar
gejun committed
547

gejun's avatar
gejun committed
548
### Has retrying quota
gejun's avatar
gejun committed
549

550
Controller.set_max_retry(0) or ChannelOptions.max_retry = 0 disables retries.
gejun's avatar
gejun committed
551

gejun's avatar
gejun committed
552
### The retry makes sense
gejun's avatar
gejun committed
553

554
If the RPC fails due to request(EREQUEST), no retry will be done because server is very likely to reject the request again, retrying makes no sense here.
gejun's avatar
gejun committed
555

Ge Jun's avatar
Ge Jun committed
556
Users can inherit [brpc::RetryPolicy](https://github.com/brpc/brpc/blob/master/src/brpc/retry_policy.h) to customize conditions of retrying. For example brpc does not retry for http/h2 related errors by default. If you want to retry for HTTP_STATUS_FORBIDDEN(403) in your app, you can do as follows:
gejun's avatar
gejun committed
557 558 559

```c++
#include <brpc/retry_policy.h>
560

gejun's avatar
gejun committed
561 562 563
class MyRetryPolicy : public brpc::RetryPolicy {
public:
    bool DoRetry(const brpc::Controller* cntl) const {
Ge Jun's avatar
Ge Jun committed
564
        if (cntl->ErrorCode() == brpc::EHTTP && // http/h2 error
gejun's avatar
gejun committed
565 566 567
            cntl->http_response().status_code() == brpc::HTTP_STATUS_FORBIDDEN) {
            return true;
        }
gejun's avatar
gejun committed
568
        // Leave other cases to brpc.
gejun's avatar
gejun committed
569 570 571 572
        return brpc::DefaultRetryPolicy()->DoRetry(cntl);
    }
};
...
573 574 575

// Assign the instance to ChannelOptions.retry_policy.
// NOTE: retry_policy must be kept valid during lifetime of Channel, and Channel does not retry_policy, so in most cases RetryPolicy should be created by singleton..
gejun's avatar
gejun committed
576 577 578 579 580 581
brpc::ChannelOptions options;
static MyRetryPolicy g_my_retry_policy;
options.retry_policy = &g_my_retry_policy;
...
```

gejun's avatar
gejun committed
582 583 584 585 586 587 588 589
Some tips:

- Get response of the RPC by cntl->response().
- RPC deadline represented by ERPCTIMEDOUT is never retried, even it's allowed by your derived RetryPolicy.

### Retrying should be conservative

Due to maintaining costs, even very large scale clusters are deployed with "just enough" instances to survive major defects, namely offline of one IDC, which is at most 1/2 of all machines. However aggressive retries may easily make pressures from all clients double or even tripple against servers, and make the whole cluster down: More and more requests stuck in buffers, because servers can't process them in-time. All requests have to wait for a very long time to be processed and finally gets timed out, as if the whole cluster is crashed. The default retrying policy is safe generally: unless the connection is broken, retries are rarely sent. However users are able to customize starting conditions for retries by inheriting RetryPolicy, which may turn retries to be "a storm". When you customized RetryPolicy, you need to carefully consider how clients and servers interact and design corresponding tests to verify that retries work as expected.
gejun's avatar
gejun committed
590

591 592 593 594
## Circuit breaker

Check out [circuit_breaker](../cn/circuit_breaker.md) for more details.

gejun's avatar
gejun committed
595
## Protocols
gejun's avatar
gejun committed
596

gejun's avatar
gejun committed
597
The default protocol used by Channel is baidu_std, which is changeable by setting ChannelOptions.protocol. The field accepts both enum and string.
gejun's avatar
gejun committed
598

gejun's avatar
gejun committed
599
 Supported protocols:
gejun's avatar
gejun committed
600

601
- PROTOCOL_BAIDU_STD or "baidu_std", which is [the standard binary protocol inside Baidu](baidu_std.md), using single connection by default.
Ge Jun's avatar
Ge Jun committed
602 603
- PROTOCOL_HTTP or "http", which is http/1.0 or http/1.1, using pooled connection by default (Keep-Alive).
  - Methods for accessing ordinary http services are listed in [Access http/h2](http_client.md).
Ge Jun's avatar
Ge Jun committed
604
  - Methods for accessing pb services by using http:json or http:proto are listed in [http/h2 derivatives](http_derivatives.md)
605
- PROTOCOL_H2 or ”h2", which is http/2, using single connection by default.
Ge Jun's avatar
Ge Jun committed
606
  - Methods for accessing ordinary h2 services are listed in [Access http/h2](http_client.md).
Ge Jun's avatar
Ge Jun committed
607 608
  - Methods for accessing pb services by using h2:json or h2:proto are listed in [http/h2 derivatives](http_derivatives.md)
- "h2:grpc", which is the protocol of [gRPC](https://grpc.io) and based on h2, using single connection by default, check out [h2:grpc](http_derivatives.md#h2grpc) for details.
Ge Jun's avatar
Ge Jun committed
609 610
- PROTOCOL_THRIFT or "thrift", which is the protocol of [apache thrift](https://thrift.apache.org), using pooled connection by default, check out [Access thrift](thrift.md) for details.
- PROTOCOL_MEMCACHE or "memcache", which is binary protocol of memcached, using **single connection** by default. Check out [Access memcached](memcache_client.md) for details.
Ge Jun's avatar
Ge Jun committed
611
- PROTOCOL_REDIS or "redis", which is protocol of redis 1.2+ (the one supported by hiredis), using **single connection** by default. Check out [Access Redis](redis_client.md) for details.
612 613
- PROTOCOL_HULU_PBRPC or "hulu_pbrpc", which is protocol of hulu-pbrpc, using single connection by default.
- PROTOCOL_NOVA_PBRPC or "nova_pbrpc",  which is protocol of Baidu ads union, using pooled connection by default.
gejun's avatar
gejun committed
614
- PROTOCOL_SOFA_PBRPC or "sofa_pbrpc", which is protocol of sofa-pbrpc, using single connection by default.
615 616 617
- PROTOCOL_PUBLIC_PBRPC or "public_pbrpc", which is protocol of public_pbrpc, using pooled connection by default.
- PROTOCOL_UBRPC_COMPACK or "ubrpc_compack", which is protocol of public/ubrpc, packing with compack, using pooled connection by default. check out [ubrpc (by protobuf)](ub_client.md) for details. A related protocol is PROTOCOL_UBRPC_MCPACK2 or ubrpc_mcpack2, packing with mcpack2.
- PROTOCOL_NSHEAD_CLIENT or "nshead_client", which is required by UBXXXRequest in baidu-rpc-ub, using pooled connection by default. Check out [Access UB](ub_client.md) for details.
gejun's avatar
gejun committed
618
- PROTOCOL_NSHEAD or "nshead", which is required by sending NsheadMessage, using pooled connection by default. Check out [nshead+blob](ub_client.md#nshead-blob) for details.
619 620
- PROTOCOL_NSHEAD_MCPACK or "nshead_mcpack", which is as the name implies, nshead + mcpack (parsed by protobuf via mcpack2pb), using pooled connection by default.
- PROTOCOL_ESP or "esp", for accessing services with esp protocol, using pooled connection by default.
gejun's avatar
gejun committed
621

gejun's avatar
gejun committed
622
## Connection Type
gejun's avatar
gejun committed
623

gejun's avatar
gejun committed
624
brpc supports following connection types:
gejun's avatar
gejun committed
625

626 627
- short connection: Established before each RPC, closed after completion. Since each RPC has to pay the overhead of establishing connection, this type is used for occasionally launched RPC, not frequently launched ones. No protocol use this type by default. Connections in http/1.0 are handled similarly as short connections.
- pooled connection: Pick an unused connection from a pool before each RPC, return after completion. One connection carries at most one request at the same time. One client may have multiple connections to one server. http/1.1 and the protocols using nshead use this type by default.
628
- single connection: all clients in one process has at most one connection to one server, one connection may carry multiple requests at the same time. The sequence of received responses does not need to be same as sending requests. This type is used by baidu_std, hulu_pbrpc, sofa_pbrpc by default.
gejun's avatar
gejun committed
629

630 631 632 633 634 635 636
|                                          | short connection                         | pooled connection                       | single connection                        |
| ---------------------------------------- | ---------------------------------------- | --------------------------------------- | ---------------------------------------- |
| long connection                          | no                                       | yes                                     | yes                                      |
| \#connection at server-side (from a client) | qps*latency ([little's law](https://en.wikipedia.org/wiki/Little%27s_law)) | qps*latency                             | 1                                        |
| peak qps                                 | bad, and limited by max number of ports  | medium                                  | high                                     |
| latency                                  | 1.5RTT(connect) + 1RTT + processing time | 1RTT + processing time                  | 1RTT + processing time                   |
| cpu usage                                | high, tcp connect for each RPC           | medium, every request needs a sys write | low, writes can be combined to reduce overhead. |
gejun's avatar
gejun committed
637

638
brpc chooses best connection type for the protocol by default, users generally have no need to change it. If you do, set ChannelOptions.connection_type to:
gejun's avatar
gejun committed
639

gejun's avatar
gejun committed
640
- CONNECTION_TYPE_SINGLE or "single" : single connection
gejun's avatar
gejun committed
641

642
- CONNECTION_TYPE_POOLED or "pooled": pooled connection. Max number of pooled connections from one client to one server is limited by -max_connection_pool_size. Note the number is not same as "max number of connections". New connections are always created when there's no idle ones in the pool; the returned connections are closed immediately when the pool already has max_connection_pool_size connections. Value of max_connection_pool_size should respect the concurrency, otherwise the connnections that can't be pooled are created and closed frequently which behaves similarly as short connections. If max_connection_pool_size is 0, the pool behaves just like fully short connections. 
gejun's avatar
gejun committed
643 644 645

  | Name                         | Value | Description                              | Defined At          |
  | ---------------------------- | ----- | ---------------------------------------- | ------------------- |
646
  | max_connection_pool_size (R) | 100   | Max number of pooled connections to a single endpoint | src/brpc/socket.cpp |
gejun's avatar
gejun committed
647

gejun's avatar
gejun committed
648
- CONNECTION_TYPE_SHORT or "short" : short connection
gejun's avatar
gejun committed
649

gejun's avatar
gejun committed
650
- "" (empty string) makes brpc chooses the default one.
gejun's avatar
gejun committed
651

652
brpc also supports [Streaming RPC](streaming_rpc.md) which is an application-level connection for transferring streaming data.
gejun's avatar
gejun committed
653

gejun's avatar
gejun committed
654
## Close idle connections in pools
gejun's avatar
gejun committed
655

gejun's avatar
gejun committed
656
If a connection has no read or write within the seconds specified by -idle_timeout_second, it's tagged as "idle", and will be closed automatically. Default value is 10 seconds. This feature is only effective to pooled connections. If -log_idle_connection_close is true, a log is printed before closing.
gejun's avatar
gejun committed
657 658 659 660 661 662

| Name                      | Value | Description                              | Defined At              |
| ------------------------- | ----- | ---------------------------------------- | ----------------------- |
| idle_timeout_second       | 10    | Pooled connections without data transmission for so many seconds will be closed. No effect for non-positive values | src/brpc/socket_map.cpp |
| log_idle_connection_close | false | Print log when an idle connection is closed | src/brpc/socket.cpp     |

gejun's avatar
gejun committed
663
## Defer connection close
gejun's avatar
gejun committed
664

665
Multiple channels may share a connection via referential counting. When a channel releases last reference of the connection, the connection will be closed. But in some scenarios, channels are created just before sending RPC and destroyed after completion, in which case connections are probably closed and re-open again frequently, as costly as short connections.
gejun's avatar
gejun committed
666

gejun's avatar
gejun committed
667
One solution is to cache channels commonly used by user, which avoids frequent creation and destroying of channels.  However brpc does not offer an utility for doing this right now, and it's not trivial for users to implement it correctly.
gejun's avatar
gejun committed
668

gejun's avatar
gejun committed
669
Another solution is setting gflag -defer_close_second
gejun's avatar
gejun committed
670 671 672 673 674

| Name               | Value | Description                              | Defined At              |
| ------------------ | ----- | ---------------------------------------- | ----------------------- |
| defer_close_second | 0     | Defer close of connections for so many seconds even if the connection is not used by anyone. Close immediately for non-positive values | src/brpc/socket_map.cpp |

675
After setting, connection is not closed immediately after last referential count, instead it will be closed after so many seconds. If a channel references the connection again during the wait, the connection resumes to normal. No matter how frequent channels are created, this flag limits the frequency of closing connections. Side effect of the flag is that file descriptors are not closed immediately after destroying of channels, if the flag is wrongly set to be large, number of active file descriptors in the process may be large as well.
gejun's avatar
gejun committed
676

677
## Buffer size of connections
gejun's avatar
gejun committed
678

679
-socket_recv_buffer_size sets receiving buffer size of all connections, -1 by default (not modified)
gejun's avatar
gejun committed
680

681
-socket_send_buffer_size sets sending buffer size of all connections, -1 by default (not modified)
gejun's avatar
gejun committed
682 683 684 685 686 687 688 689

| Name                    | Value | Description                              | Defined At          |
| ----------------------- | ----- | ---------------------------------------- | ------------------- |
| socket_recv_buffer_size | -1    | Set the recv buffer size of socket if this value is positive | src/brpc/socket.cpp |
| socket_send_buffer_size | -1    | Set send buffer size of sockets if this value is positive | src/brpc/socket.cpp |

## log_id

690
set_log_id() sets a 64-bit integral log_id, which is sent to the server-side along with the request, and often printed in server logs to associate different services accessed in a session. String-type log-id must be converted to 64-bit integer before setting.
gejun's avatar
gejun committed
691

692
## Attachment
gejun's avatar
gejun committed
693

694
baidu_std and hulu_pbrpc supports attachments which are sent along with messages and set by users to bypass serialization of protobuf. As a client, data set in Controller::request_attachment() will be received by server and response_attachment() contains attachment sent back by the server.
gejun's avatar
gejun committed
695

696 697
Attachment is not compressed by framework.

Ge Jun's avatar
Ge Jun committed
698
In http/h2, attachment corresponds to [message body](http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html), namely the data to post to server is stored in request_attachment().
gejun's avatar
gejun committed
699

old-bear's avatar
old-bear committed
700 701
## Turn on SSL

gejun's avatar
gejun committed
702 703
Update openssl to the latest version before turning on SSL, since older versions of openssl may have severe security problems and support less encryption algorithms, which is against with the purpose of using SSL. 
Set ChannelOptions.mutable_ssl_options() to enable SSL. Refer to [ssl_options.h](https://github.com/brpc/brpc/blob/master/src/brpc/ssl_options.h) for the detailed options. ChannelOptions.has_ssl_options() checks if ssl_options was set; ChannelOptions.ssl_options() returns const reference to the ssl_options.
old-bear's avatar
old-bear committed
704 705

```c++
gejun's avatar
gejun committed
706 707 708 709 710 711
// Enable client-side SSL and use default values.
options.mutable_ssl_options();

// Enable client-side SSL and customize values.
options.mutable_ssl_options()->ciphers_name = "...";
options.mutable_ssl_options()->sni_name = "...";
old-bear's avatar
old-bear committed
712 713
```

gejun's avatar
gejun committed
714
- Channels connecting to a single server or a cluster both support SSL (the initial implementation does not support cluster)
old-bear's avatar
old-bear committed
715
- After turning on SSL, all requests through this Channel will be encrypted. Users should create another Channel for non-SSL requests if needed.
gejun's avatar
gejun committed
716
- Accessibility improvements for HTTPS: Channel.Init recognizes https:// prefix and turns on SSL automatically; -http_verbose prints certificate information when SSL is on.
old-bear's avatar
old-bear committed
717

718
## Authentication
old-bear's avatar
old-bear committed
719

720 721 722
Generally there are 2 ways of authentication at the client side:

1. Request-based authentication: Each request carries authentication information. It's more flexible since the authentication information can contain fields based on this particular request. However, this leads to a performance loss due to the extra payload in each request.
zyearn's avatar
zyearn committed
723
2. Connection-based authentication: Once a TCP connection has been established, the client sends an authentication packet. After it has been verfied by the server, subsequent requests on this connection no longer needs authentication. Compared with the former, this method can only carry some static information such as local IP in the authentication packet. However, it has better performance especially under single connection / connection pool scenario.
724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741

It's very simple to implement the first method by just adding authentication data format into the request proto definition. Then send it as normal RPC in each request. To achieve the second one, brpc provides an interface for users to implement:

```c++
class Authenticator {
public:
    virtual ~Authenticator() {}

    // Implement this method to generate credential information
    // into `auth_str' which will be sent to `VerifyCredential'
    // at server side. This method will be called on client side.
    // Returns 0 on success, error code otherwise
    virtual int GenerateCredential(std::string* auth_str) const = 0;
};
```

When the user calls the RPC interface with a single connection to the same server, the framework guarantee that once the TCP connection has been established, the first request on the connection will contain the authentication string generated by `GenerateCredential`. Subsequent requests will not carried that string. The entire sending process is still highly concurrent since it won't wait for the authentication result. If the verification succeeds, all requests return without error. Otherwise, if the verification fails, generally the server will close the connection and those requests will receive the corresponding error.

gejun's avatar
gejun committed
742
Currently only those protocols support client authentication: [baidu_std](../cn/baidu_std.md) (default protocol), HTTP, hulu_pbrpc, ESP. For customized protocols, generally speaking, users could call the `Authenticator`'s interface to generate authentication string during the request packing process in order to support authentication.
gejun's avatar
gejun committed
743

744
## Reset
gejun's avatar
gejun committed
745

746
This method makes Controller back to the state as if it's just created.
gejun's avatar
gejun committed
747

748
Don't call Reset() during a RPC, which is undefined.
gejun's avatar
gejun committed
749

750
## Compression
gejun's avatar
gejun committed
751

752
set_request_compress_type() sets compress-type of the request, no compression by default.
gejun's avatar
gejun committed
753

754
NOTE: Attachment is not compressed by brpc.
gejun's avatar
gejun committed
755

Ge Jun's avatar
Ge Jun committed
756
Check out [compress request body](http_client#压缩request-body) to compress http/h2 body.
gejun's avatar
gejun committed
757

758
Supported compressions:
gejun's avatar
gejun committed
759

760 761 762
- brpc::CompressTypeSnappy : [snanpy](http://google.github.io/snappy/), compression and decompression are very fast, but compression ratio is low.
- brpc::CompressTypeGzip : [gzip](http://en.wikipedia.org/wiki/Gzip), significantly slower than snappy, with a higher compression ratio.
- brpc::CompressTypeZlib : [zlib](http://en.wikipedia.org/wiki/Zlib), 10%~20% faster than gzip but still significantly slower than snappy, with slightly better compression ratio than gzip.
gejun's avatar
gejun committed
763

764
Following table lists performance of different methods compressing and decompressing **data with a lot of duplications**, just for reference.
gejun's avatar
gejun committed
765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780

| Compress method | Compress size(B) | Compress time(us) | Decompress time(us) | Compress throughput(MB/s) | Decompress throughput(MB/s) | Compress ratio |
| --------------- | ---------------- | ----------------- | ------------------- | ------------------------- | --------------------------- | -------------- |
| Snappy          | 128              | 0.753114          | 0.890815            | 162.0875                  | 137.0322                    | 37.50%         |
| Gzip            | 10.85185         | 1.849199          | 11.2488             | 66.01252                  | 47.66%                      |                |
| Zlib            | 10.71955         | 1.66522           | 11.38763            | 73.30581                  | 38.28%                      |                |
| Snappy          | 1024             | 1.404812          | 1.374915            | 695.1555                  | 710.2713                    | 8.79%          |
| Gzip            | 16.97748         | 3.950946          | 57.52106            | 247.1718                  | 6.64%                       |                |
| Zlib            | 15.98913         | 3.06195           | 61.07665            | 318.9348                  | 5.47%                       |                |
| Snappy          | 16384            | 8.822967          | 9.865008            | 1770.946                  | 1583.881                    | 4.96%          |
| Gzip            | 160.8642         | 43.85911          | 97.13162            | 356.2544                  | 0.78%                       |                |
| Zlib            | 147.6828         | 29.06039          | 105.8011            | 537.6734                  | 0.71%                       |                |
| Snappy          | 32768            | 16.16362          | 19.43596            | 1933.354                  | 1607.844                    | 4.82%          |
| Gzip            | 229.7803         | 82.71903          | 135.9995            | 377.7849                  | 0.54%                       |                |
| Zlib            | 240.7464         | 54.44099          | 129.8046            | 574.0161                  | 0.50%                       |                |

781
Following table lists performance of different methods compressing and decompressing **data with very few duplications**, just for reference.
gejun's avatar
gejun committed
782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799

| Compress method | Compress size(B) | Compress time(us) | Decompress time(us) | Compress throughput(MB/s) | Decompress throughput(MB/s) | Compress ratio |
| --------------- | ---------------- | ----------------- | ------------------- | ------------------------- | --------------------------- | -------------- |
| Snappy          | 128              | 0.866002          | 0.718052            | 140.9584                  | 170.0021                    | 105.47%        |
| Gzip            | 15.89855         | 4.936242          | 7.678077            | 24.7294                   | 116.41%                     |                |
| Zlib            | 15.88757         | 4.793953          | 7.683384            | 25.46339                  | 107.03%                     |                |
| Snappy          | 1024             | 2.087972          | 1.06572             | 467.7087                  | 916.3403                    | 100.78%        |
| Gzip            | 32.54279         | 12.27744          | 30.00857            | 79.5412                   | 79.79%                      |                |
| Zlib            | 31.51397         | 11.2374           | 30.98824            | 86.90288                  | 78.61%                      |                |
| Snappy          | 16384            | 12.598            | 6.306592            | 1240.276                  | 2477.566                    | 100.06%        |
| Gzip            | 537.1803         | 129.7558          | 29.08707            | 120.4185                  | 75.32%                      |                |
| Zlib            | 519.5705         | 115.1463          | 30.07291            | 135.697                   | 75.24%                      |                |
| Snappy          | 32768            | 22.68531          | 12.39793            | 1377.543                  | 2520.582                    | 100.03%        |
| Gzip            | 1403.974         | 258.9239          | 22.25825            | 120.6919                  | 75.25%                      |                |
| Zlib            | 1370.201         | 230.3683          | 22.80687            | 135.6524                  | 75.21%                      |                |

# FAQ

800
### Q: Does brpc support unix domain socket?
gejun's avatar
gejun committed
801

802
No. Local TCP sockets performs just a little slower than unix domain socket since traffic over local TCP sockets bypasses network. Some scenarios where TCP sockets can't be used may require unix domain sockets. We may consider the capability in future.
gejun's avatar
gejun committed
803

804
### Q: Fail to connect to xx.xx.xx.xx:xxxx, Connection refused
gejun's avatar
gejun committed
805

806
The remote server does not serve any more (probably crashed).
gejun's avatar
gejun committed
807

808
### Q: often met Connection timedout to another IDC
gejun's avatar
gejun committed
809 810 811

![img](../images/connection_timedout.png)

812
The TCP connection is not established within connection_timeout_ms, you have to tweak options:
gejun's avatar
gejun committed
813 814 815 816 817 818 819 820 821

```c++
struct ChannelOptions {
    ...
    // Issue error when a connection is not established after so many
    // milliseconds. -1 means wait indefinitely.
    // Default: 200 (milliseconds)
    // Maximum: 0x7fffffff (roughly 30 days)
    int32_t connect_timeout_ms;
822

gejun's avatar
gejun committed
823 824 825 826 827 828 829 830 831
    // Max duration of RPC over this Channel. -1 means wait indefinitely.
    // Overridable by Controller.set_timeout_ms().
    // Default: 500 (milliseconds)
    // Maximum: 0x7fffffff (roughly 30 days)
    int32_t timeout_ms;
    ...
};
```

832
NOTE: Connection timeout is not RPC timeout, which is printed as "Reached timeout=...".
gejun's avatar
gejun committed
833

834
### Q: synchronous call is good, asynchronous call crashes
gejun's avatar
gejun committed
835

836
Check lifetime of Controller, Response and done. In asynchronous call, finish of CallMethod is not completion of RPC which is entering of done->Run(). So the objects should not deleted just after CallMethod, instead they should be delete in done->Run(). Generally you should allocate the objects on heap instead of putting them on stack. Check out [Asynchronous call](client.md#asynchronous-call) for details.
gejun's avatar
gejun committed
837

838
### Q: How to make requests be processed once and only once
gejun's avatar
gejun committed
839

840
This issue is not solved on RPC layer. When response returns and being successful, we know the RPC is processed at server-side. When response returns and being rejected, we know the RPC is not processed at server-side. But when response is not returned, server may or may not process the RPC. If we retry, same request may be processed twice at server-side. Generally RPC services with side effects must consider [idempotence](http://en.wikipedia.org/wiki/Idempotence) of the service, otherwise retries may make side effects be done more than once and result in unexpected behavior. Search services with only read often have no side effects (during a search), being idempotent natually. But storage services that need to write have to design versioning or serial-number mechanisms to reject side effects that already happen, to keep idempoent.
gejun's avatar
gejun committed
841

842
### Q: Invalid address=`bns://group.user-persona.dumi.nj03'
gejun's avatar
gejun committed
843
```
gejun's avatar
gejun committed
844
FATAL 04-07 20:00:03 7778 src/brpc/channel.cpp:123] Invalid address=`bns://group.user-persona.dumi.nj03'. You should use Init(naming_service_name, load_balancer_name, options) to access multiple servers.
gejun's avatar
gejun committed
845
```
846
Accessing servers under naming service needs the Init() with 3 parameters(the second param is `load_balancer_name`). The Init() here is with 2 parameters and treated by brpc as accessing single server, producing the error.
gejun's avatar
gejun committed
847

848
### Q: Both sides use protobuf, why can't they communicate with each other
gejun's avatar
gejun committed
849

850
**protocol != protobuf**. protobuf serializes one package and a message of a protocol may contain multiple packages along with extra lengths, checksums, magic numbers. The capability offered by brpc that "write code once and serve multiple protocols" is implemented by converting data from different protocols to unified API, not on protobuf layer.
gejun's avatar
gejun committed
851

852
### Q: Why C++ client/server may fail to talk to client/server in other languages
gejun's avatar
gejun committed
853

854
Check if the C++ version turns on compression (Controller::set_compress_type), Currently RPC impl. in other languages do not support compression yet.
gejun's avatar
gejun committed
855

856
# PS: Workflow at Client-side
gejun's avatar
gejun committed
857 858 859

![img](../images/client_side.png)

860 861 862 863 864 865 866 867 868 869 870 871 872 873 874
Steps:

1. Create a [bthread_id](https://github.com/brpc/brpc/blob/master/src/bthread/id.h) as correlation_id of current RPC.
2. According to how the Channel is initialized, choose a server from global [SocketMap](https://github.com/brpc/brpc/blob/master/src/brpc/socket_map.h) or [LoadBalancer](https://github.com/brpc/brpc/blob/master/src/brpc/load_balancer.h) as  destination of the request.
3. Choose a [Socket](https://github.com/brpc/brpc/blob/master/src/brpc/socket.h) according to connection type (single, pooled, short)
4. If authentication is turned on and the Socket is not authenticated yet, first request enters authenticating branch, other requests block until the branch writes authenticating information into the Socket. Server-side only verifies the first request.
5. According to protocol of the Channel, choose corresponding serialization callback to serialize request into [IOBuf](https://github.com/brpc/brpc/blob/master/src/butil/iobuf.h).
6. If timeout is set, setup timer. From this point on, avoid using Controller, since the timer may be triggered at anytime and calls user's callback for timeout, which may delete Controller.
7. Sending phase is completed. If error occurs at any step, Channel::HandleSendFailed is called.
8. Write IOBuf with serialized data into the Socket and add Channel::HandleSocketFailed into id_wait_list of the Socket. The callback will be called when the write is failed or connection is broken before completion of RPC.
9. In synchronous call, Join correlation_id; otherwise CallMethod() returns.
10. Send/receive messages to/from network.
11. After receiving response, get the correlation_id inside, find out associated Controller within O(1) time. The lookup does not need to lock a global hashmap, and scales well.
12. Parse response according to the protocol
13. Call Controller::OnRPCReturned, which may retry errorous RPC, or complete the RPC. Call user's done in asynchronous call. Destroy correlation_id and wakeup joining threads.