Considerations for Implementing Pipelining

These are the practical considerations that we have learned from implementing pipelining

Output Buffer Size

Should be set to about 1/2 of the TCP buffer sixe returned by getsockopt. This avoids that the network blocks too much on writing.

Client should Write with Patience and Read as Fast as Possible

As most servers read as fast/slow as they write, clients should not write too much to the network. I have found that it is useful to not pipeline more than 100 requests at a time. The client doesn't gain anything performance wise, and the mess from cleaning up a prematurely closed connection gets pretty bad.

Flush Delay

I use 30-50 ms, which is on the limit of human perception. Probably a better way would be to take into account the RTT of the connection.

Explicit Flush in the API

Is often needed as the application has a better knowledge of the page construction and hence when to flush

Turn off Nagle

When using pipelining and output buffering, it is often a good idea to turn off Nagle's algorithm.