Pipelining Problems in Apache 1.2b1

We have found what we believe to be two problems with Apache 1.2b1 server when using pipelined requests. This is a result of our current work on optimizing HTTP/1.1 using pipelining. Please read this preliminary paper for more details. We use xplot as the main tool for illustrating the TCP behavior.

Cache Validation Uses Too Many Packets

The apache server issues a flush after each generated response. When using GET requests, this is normally not a problem as most responses is longer than can be contained in a single TCP segment (often 1460 bytes on a LAN). However, when doing cache validation, the responses are of the same size as the request (200-250 bytes) and here it is causing bad behavior on the network. Instead, we suggest an output buffering scheme so that multiple responses can be put into the same TCP segment before transmitted to the client.

Losing Bytes in the Response

Another problem that we found is that Apache sometimes get confuses when a client pipelines request. The result is that 3 responses indeed are sent in the same TCP segment but a multiplum of 4 bytes are lost (4-8-12, etc). This causes the client to hang as it sends the requests but never catches the responses. I have included some xplots below to show the problem.

I should note that we can read multiple responses with the same TCP segments coming from Jigsaw without problems.

xplot set with no loss

This set is to illustrate the situation where we issue a GET request on the HTML document and 41 HEAD requests on inlined images. We get back the full HTML and 41 HEAD responses.

xplot set with loss

This set illustrates the sam scenario as above but you will find that one segment contains three HEAD responses and from that point in time it looses some bytes. The problem occurs just after sequence number 48,000.

Connection Closes after 5 Responses

Another problem that comes up when using pipelining is that the default limit of max 5 HTTP messages pr TCP connection is a severe limitation and puts heavy constraints on the performance of a server. Whereas it may be a good idea to keep a timeout on a connection, measuring it in HTTP messages is not a good idea as it doesn't reflect the load on the server. In our testing we effectively disabled the feature and we would recommend that this becomes the default behavior for HTTP/1.1 responses from Apache.

If you decide on a closing mechanim then it is very important that you only close the outgoing TCP connection as the client already may have sent large numbers of requests at the server. If both incoming and outgoing connection is closed then TCP will generate a reset and the response will get lost.