WD-http-cache-@@dateYYMMDD
WD-http-cache-@@dateYYMMDD
This is a W3C Working Draft for review by W3C members and other interested parties. It is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress". A list of current W3C working drafts can be found at: /TR
Note: since working drafts are subject to frequent change, you are advised to reference the above URL, rather than the URLs for working drafts themselves.
@@abstract
The basic principle behind caching is simple: Once you've figured out the answer to a question, keep it around in case you get asked again.
The principal web protocol, HTTP, is a simple request-response protocol [HTTP]. A proxy mechanism was introduced in [Ari9?]. A typical interaction looks like:
In addition to crossing firewalls, proxies are used to cache responses. A typical cache hit interaction looks like:
We can consider the HTTP request-response mechanism as a simple function computation. We might consider a function get that maps document addresses (URLs) to document contents:
get: URL -> Content
In the cache hit scenario, Client 1 is trying to compute get(a1) where a1 = "http://origin/path". It asks the proxy, which asks the origin server. The origin server returns the answer c1, where c1="xyz" [ignoring content-type]. The proxy records the answer get(a1)=c1 for later use, and returns c1 to Client 1. Client 2 asks the same question, and the proxy returns the same answer, c1.
Next we consider a relation between HTTP requests and responses (actually, its characteristic function):
http: Request, Response -> Bool
http(req, resp) is true whenever req and resp constitute a valid HTTP transaction.
Associated with each HTTP request is the address of some object, the method that the request is invoking, and some optional fields and content:
address: Request -> URL method: Request -> Method fields: Request -> Fieldscontent: Request -> Content
Associated with each HTTP response is a status code and optionally some fields and content:
status: Response -> Status fields: Response -> Fields content: Response -> Content
The proxy interaction can be captured in a simple rule: when a proxy receives a request from a client, if it conducts a valid transaction with an origin server where the method, address, fields, and content of its request are the same as the client's request, then a response to the client with the same status, fields, and content as the origin servers response completes a valid HTTP transaction [ignoring Forwarded: header, other proxy munging]. We might express this formally as:
proxy: Request, Response, Request, Response -> Bool proxy(c_p, p_c, p_o, o_p) == method(p_o) = method(c_p)/\ address(p_o) = address(c_p) /\ fields(p_o) = fields(c_p) /\ content(p_o) = content(c_p) /\ status(p_c) = status(o_p) /\ fields(p_c) = fields(o_p) /\ content(p_c) = content(o_p) http(p_o, o_p)
/\ proxy(c_p, p_c, p_o, o_p) -> http(c_p, p_c)
Each successful GET request give information about the get function. We might express this using the following http/get rule:
http(req, resp)/\ method(req) = GET
/\ status(resp) = 200
->
get(address(req)) = content(resp)
In the cache hit scenario above, this allows the proxy to conclude that the content of its response to the second client will be the same as the content of the first response from the fact that the addresses of the requests were the same.
In fact, things aren't quite so simple. Between the time c1 and c2 make their requests, the answer can change. The result of a GET operation depends on more than just the address of the object; it may depend on some state of the object that changes over time. For example, many web objects are implemented as files, and those files are revised from time to time.
The following scenario illustrates a problem with our simple caching model:
The proxy has clearly violated the HTTP protocol by giving Client 3 a stale response. This situation where two GET requests on the same object return different results can be expressed as:
http(req1, resp1) /\ http(req2, resp2) /\ address(req1) = address(req2) /\ method(req1) = method(req2) = GET /\ status(resp1) = status(resp2) = 200 /\ content(resp1) != content(resp2)
From the http/get inference rule above, we could conclude:
get(address(req1)) != get(address(req1))
which is absurd. The only rational conclusion is that the relationship between document addresses and their content is not a function at all, but merely a relation. The http/get rule as stated above is clearly not an accurate model.
In practice, HTTP proxies use a mechanism known as If-Modified-Since to prevent stale responses. A typical interaction using this mechanism looks like:
This relies on the fact that each HTTP response may have a last-modified date associated with it. The If-Modified-Since header associates a date with a request:
lm: Response -> Date ims: Request -> Date
To use the mechanism, a client must have made a successful GET request on some object. It makes a second GET request to the same address, setting the If-Modified-Since date to the Last-Modified date from the first response. If the server returns 304 Not Modified, then the client can infer that the content of the second response from the first response. Formally:
http(req1, resp1) /\ lm(resp1) = ims(req2)/\ method(req1) = method(req2) = GET /\ address(req1 = address(req2) /\ http(req2, resp2) /\ status(resp2) = 304 -> content(resp2) = content(resp1)
[@@ Note: no global time ordering: we only compare times for equality, not order.]
The proxy rule, as stated above, includes the same problem as the get/http rule: it doesn't account for time. It implies that if one valid transaction has occurred, and another request comes in with the same address etc., a response with the same status, fields, and content completes a valid transaction, no matter how much time has elapsed between the transactions.
Without appealing to a globally synchronized clock mechanism, the simplest way to correct the proxy rule is to introduce ordering of requests and responses:
< : Request, Request -> Bool < : Response, Response -> Bool< : Request, Response -> Bool
< : Response, Request -> Bool
We weaken the proxy rule by requiring the messages in the hypothesis to be ordered:
proxy(c_p, p_c, p_o, o_p) == c_p < p_o /\ o_p < p_c/\ method(p_o) = method(c_p)
/\ address(p_o) = address(c_p)
/\ fields(p_o) = fields(c_p)
/\ content(p_o) = content(c_p)
/\ status(p_c) = status(o_p)
/\ fields(p_c) = fields(o_p)
/\ content(p_c) = content(o_p)
The content in an HTTP GET response is typed, and the type depends on a number of factors, including the preferences given in the request. This leads to a problematic scenario where two clients make requests on the same object with different format preferences. The origin server returns different content, but the proxy fails to make the distinction:
A simple solution would be to weaken the If-Modified-Since rule by requiring that the fields of the requests be the same:
http(req1, resp1)/\ lm(resp1) = ims(req2)
/\ method(req1) = method(req2) = GET
/\ address(req1 = address(req2)
/\ fields(req1) = fields(req2)
/\ http(req2, resp2)
/\ status(resp2) = 304 ->
content(resp2) = content(resp1)
This puts the blame for the above protocol interaction on the proxy: Client 3's request was not a candidate for the If-Modified-Since mechanism.
In fact, the candidates for the If-Modified-Since mechanism would be reduced to a small minority. This would be unfortunate, since the vast majority of web objects do not offer variant formats, and in this majority of cases, the differences in the Accept fields have no impact on the protocol transaction.
This approach blames the origin server in the above scenario. Rather than selecting fewer requests as candidates for the If-Modified-Since mechanism, we select the responses. We introduce a distinction between responses that vary depending on request fields from those that do not:
vary: Response -> Bool fields(resp1) = fields(resp2)-> vary(resp1) = vary(resp2)
The If-Modified-Since specification becomes:
http(req1, resp1)/\ vary(resp1) = False
/\ lm(resp1) = ims(req2)
/\ method(req1) = method(req2) = GET
/\ address(req1 = address(req2)
/\ http(req2, resp2)
/\ status(resp2) = 304
->
content(resp2) = content(resp1)
The vary distinction is expressed in the HTTP URI: header. If the URI: header is absent, the response does not vary.
@@caching policies regarding vary=user-agent
The If-Modified-Since mechanism, as specified above, provides tremendous bandwidth savings, but provides little improvement in latency: every request involves a round-trip to the origin server. Only in the case of modified object state is the round-trip necessary.
The HTTP Expires mechanism is intended to alleviate these round trips by specifying a time until which they are not needed.
The specification of Expires appeals to a globally synchronized clock; the reliability of this mechanism is sensitive to clock skew and latency.
expires: Response -> Date < : Date, Date -> Bool
A client uses the Expires mechanism to eliminate some HTTP transactions entirely. To model GET requests involving no HTTP transactions, we revive the notion of the get function computation. But since the relation between an object address and its content is not functional, we model it as a relation between an address, some content, and the interval of time over which the binding between them lasts:
get: URL, Content, Date, Date -> Bool get(a, c, t1, t4) /\ t1 < t2 /\ t2 < t3 /\ t3 < t4 -> get(a, c, t2, t3)
The Last-Modified and Expires fields of a response define an interval in which the address in the request is bound to the content in the response:
http(req, resp)/\ vary(resp) = False
/\ method(req) = GET
/\ status(resp) = 200
->
get(address(req), content(resp),
lm(resp), expires(resp))
@@concrete scenario using expires
In order to provide an Expires header, an HTTP server must be able to predict when an object's state will change (or more importantly, when it will not change).
A strict approach would require HTTP clients (including caching proxies) to use expires(resp) = now as a default when no Expires header is provided.
In practice, HTTP caching proxy servers heuristically assign a future expiry time as a matter of local policy. For file-based web pages, this results in reduced latency at the cost of some stale responses. But this optimization can reduce the reliability of interactive applications to an unacceptable level.
The Proxy: no-cache mechanism allows users to deal with the problem of stale responses by exception. Requiring users to detect protocol faults is especially problematic in the case of interactive applications.
The Max-Age mechanism has been proposed [cite http-wg archives] as a more general approach to this situation. It allows clients to explictly state their freshness requirements. @@example user interface: integrity vs. performance slider.
Given the ability for clients to state their freshness requirements, it seems reasonable for caching proxies (and clients in general) to heuristically assign default expiry times. This balances efficient use of network with reliability.
@@rather than min-date = x, client writes min-date = current-time - x.
min-date: Request -> Date date : Response -> Date fields(resp1) = fields(resp2) -> date(resp1) = date(resp2) http(req1, resp1)/\ vary(resp1) = False /\ status(resp1) = 200
/\ date(resp1) > min-date(req2)
/\ method(req1) = method(req2) = GET
/\ address(req1) = address(req2) /\ status(resp2) = status(resp1) /\ fields(resp2) = fields(resp1) /\ content(resp2) = content(resp1)
->
http(req2, resp2)
@@ somtimes clients know that objects change slowly (*.gif, rfcNNN). This allows them to express/exploit that knowledge.
@@concrete scenario using max-age
@@caches must not violate authorization policies. Transactions involving authentication require special handling. Once it's established that the user is authorized, for cache hit purposes, we could include vary=user in URI: header requirements.
@@ explain about Larch and /\, etc.
The World Wide Web Consortium:
http://www.w3.org/