WD-http-pep-960220

PEP: An Extension Mechanism for HTTP/1.1

W3C Working Draft 20-Feb-96

This version:

http://www.w3.org/pub/WWW/TR/WD-tab WD-http-pep-960220.html

Previous version:

http://www.w3.org/pub/WWW/TR/WD-http-pep-951122 les-951122.html

Latest version:

http://www.w3.org/pub/WWW/TR/WD-http-pep.html

Author:

Rohit Khare <khare@w3.org>

This is a W3C Working Draft for review by W3C members and other interested parties. It is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress". A list of current W3C working drafts can be found at: http://www.w3.org/pub/WWW/TR

Note: since working drafts are subject to frequent change, you are advised to reference the above URL, rather than the URLs for working drafts themselves.

Abstract

Table of Contents

1. Introduction

2. Concepts

2.1 PEP Architecture
2.1.1 Naming

2.1.2 Addressing

2.1.3 Negotiation

2.1.4 Processing

2.2 Terminology

3. Operation

3.1 Scenarios

3.2 Interpretation

3.3 Negotiation

3.4 Relaying PEP Messages

3.5 Deployment Issues

3.5.1 Relationship to HTTP/1.1

3.5.2 Protocol Naming and Specification

4. Notation

4.1 Header Fields

4.2 Content Codings

4.3 Status Codes

5. Usage

5.1 Strength & Scope

5.2 Payments Negotiation

5.3 Security Processing

6. Security Considerations

6.1 Information Leakage

6.2 Trusting PEP Headers

6.3 Protocol Interaction Effects

6.4 Protocol Substitution Effects

6.5 Negotiation Scope

7. Development Path

8. Acknowledgements

9. References

10. Author's Address

Appendix A: Minimum Compliance

A.1 Operating as a Server

A.2 Operating as a Client

A.3 Operating as a Proxy

Appendix B: Implementation Model

1. Introduction

HTTP messages, like most applications of RFC 822 [7], can be extended with additional header fields. However, this provides no guidance to HTTP agents on whether to strip the header, or to act on a header, and if acted upon, in what order, and so on. Furthermore, multiple extensions may use conflicting header field names.

``Protocol extensions'' are a higher-level abstraction. They can specify any associated header lines and also provide guidance on each of the above decisions. PEP is an extension protocol for HTTP that captures such information about protocol extensions (hence `PEP': Protocol Extension Protocol).

Using PEP, HTTP agents can interoperate correctly with unknown protocol extensions and also negotiate a set of common protocol extensions. PEP brings to HTTP the extensibility lessons learned from ESMTP (extension naming) [15,17], IPv6 (unknown-option disposition) [8], and Telnet (option negotiation) [18].

2. Concepts

First, though, a brief introduction to the headers of PEP for HTTP/1.1. PEP uses the Protocol: and Protocol-Request: headers to indicate which protocol extensions the current message conforms to and which are requested for subsequent messages, respectively.

The Protocol: header indicates that the HTTP agent is acting in accordance with the given protocol extension in constructing the message, and the receiver, according to the strength and scope of the extension, may be required to understand the same protocol, or report an error.

The Protocol-Request: header indicates that the HTTP agent, according to the strength and scope of the extension, requires the receiver to conform to the given protocol extension in constructing replies or subsequent messages, or report an error.

In many cases, a protocol extension will correspond to a software module; for example, the "rot13" protocol is an agreement to shift all characters by 13 positions. Applying several extensions, in sequence, corresponds to a pipeline. To order each extension, PEP reuses a third header, Content-Encoding:.

2.1 PEP Architecture

Note that the PEP architecture is completely symmetric; none of the facilites below distinguish between ``client'' and ``server''.

2.1.1 Naming

PEP identifies protocol extensions in two contexts: describing a message which employs a protocol, and in describing an request to use a protocol. Each, in turn, is separated into a protocol name, and a configuration that describes how the protocol is being used.

The protocol name is a pointer to a specification of the protocol itself: the full URI of a resource describing the extension. This takes the thinking of ESMTP [15] -- encouraging proliferation of standardized, named extensions -- one step further, by using the web to dynamically register names.

Although URIs can be longer than traditional names, they have many advantages for this application. URIs imply registration of protocols by associating them with the owner of a domain name. It also eliminates the transition phase between experimental and standardized use, the so-called ``X- Problem.''

A protocol specification, in addition to the ``usual'' material defining the meaning and structure of its payload, can also specify well-known parameters that specify how the protocol is being used (e.g. modes, data sizes, etc). PEP identifies protocol instances and requests by combining the name, configuration, and extension-specific parameters.

2.1.2 Addressing

While HTTP is a host-to-host protocol, an HTTP transaction is not; it may be routed through proxies, caches, and gateways. PEP offers a mechanism to select which agents are involved in processing each protocol instance or request.

PEP uses a standard configuration attribute, "scope", to indicate which HTTP agents are required to pay attention to an instance or request. Scope has three values: connection, route, and origin. The concept is largely similar to option processing in IPv6 [8], particularly the importance of proper error reporting by agents which are in scope, but cannot handle the option/extension.

``Connection'' scope is addressed to the next HTTP agent; each protocol instance or request so labeled must be handled and then removed before passing the resulting message onward. The rationale is similar to the Connection: header in Section 10.9 of HTTP/1.1.

``Route'' scope is addressed to all HTTP agents in the transaction, to set up synchronous tunnel-type extensions.

``Origin'' scope is addressed exclusively to the opposite endpoint (the ``origin server'' or ``origin client''). No intermediate agents are allowed to act upon or modify such protocol instances or requests -- unless such an agent is explicitly authorized to act for the origin (e.g. a content-filtering firewall).

2.1.3 Negotiation

Before an HTTP transaction begins, none of the agents involved can be fully informed of the other agents' capabilities. PEP provides a framework for advertising capabilities and selecting interoperable sets of protocol extensions. Telnet option negotiation [18] is the direct inspiration for the symmetric negotiation model PEP uses. PEP, though, adds a twist, borrowed from S-HTTP [19] of controlling the process by explicitly encoding the "strength" of a request.

PEP negotiation describes protocol requests by name (Section 2.1.1) and by strength. This allows agents to explictly require, reject, or optionally accept particular protocol configurations.

The only source of asymmetry is that, in HTTP, the client always moves first. Once the client has listed a set of protocol requests, the server can choose, according to its own preferences, which protocol extensions it will apply, and which to request.

Note that the negotiation proceeds not just on the name, but on the offered parameters, akin to ``subnegotiation'' in Telnet. A particular protocol specification can specify how to reconcile requests and compute responses. For example, consider the simple-cipher extension, which defines one parameter, key-length. The client can offer one range, the server another, and the servers' response would be computed by choosing a value from the intersection.

Finally, agents may choose to respond not with the requested extension, but with an extension considered to be equivalent. This kind of compatibility test allows a generic request to be answered by a specific instance.

2.1.4 Processing

PEP is designed to accomodate multiple extensions of a single HTTP message. If an extension must be evaluated in a certain order, the protocol instance can define an "enc" attribute, whose value can be cited as part of the Content-Encoding: pipeline. Protocol extensions that are order-independent need not specify an "enc", and are evaluated in arbitrary order after those cited in the pipeline.

PEP does not offer a way to order negotiation requests (i.e. ``only accept A after B''), as [13] does.

2.2 Terminology

bag

In HTTP/1.1 and PEP, a named, unordered list according to the <bag> grammar (see Section 4.1). Attribute names can either be complete URIs, tokens defined in a protocol specification, or tokens defined by this document. Note: if a bag contains two sub-bags with the same attribute name, it is implementation-dependent whether they will be concatenated or one will be discarded.

connection

("conn") The scope which addresses the next HTTP agent to receive the message.

encoding

("enc") An encoding is a process that read or writes the message (header and/or body). The Content-Encoding: header is an ordered list of encodings to apply; in addition to the methods defined in HTTP/1.1, an encoding may refer to any protocol instance "enc" value.

HTTP agent

Any process that communicates according to HTTP. In particular, any process that communicates in HTTP/1.1 or later is expected to be PEP-compliant. ``Agent'' encompasses servers, proxies, caches, gateways, and clients.

module

Many protocols will imply complementary processing, which is implemented by a module. A module can be used as a stage of a processing pipeline. Note that a single module could implement several protocols, separately or in combination.

negotiation context

To remember what protocols are available between HTTP transactions, protocol offers can be stored into some per-session or per-request context. In particular, ``subsequent request'' can be defined as any client request message from the same negotiation context.

optional

("opt") A strength value indicating that the associated protocol is optional. As an attribute of an instance, it means that an agent may elide corresonding processing. As an attribute of an request, it means that a reply or subsequent request may be created in accordance with this protocol.

origin

("origin") The scope which addresses the opposite endpoint of an HTTP transaction. For a request, it means the origin server; for a reply message, it means the origin client (user). This scope includes proxies trusted to act on behalf of the intended origin.

parameters

("params") A list of values related to the particular instance or offer. Any sub-bags in the list should use specification-defined attributes, and may use differing syntax for negotiable values (in requests) and for application (in instances). Also referred to as ``parameter configuration''

protocol

A convention for communication between two or more parties relating the syntax, sequence and semantics of the communication between them.

protocol instance

A bag describing the protocol and parameter configuration to which the message conforms.

protocol name

The name of a protocol specification: a full URI [3,4] to a resource describing the extension. Content negotiation could be used to retrieve machine-readable formats from the same location: 822 headers, downloadable code, etc.

protocol request

A bag describing a protocol and parameter configuration that may or may not be required of the replying agent.

protocol specification

A human-readable document that describes a protocol defining a message, associated semantics, and possible compatibility with other specifications. The specification should be available by dereferencing the protocol name.

refused

("ref") A strength value indicating that the associate protocol should not be used in this negotiation context. Valid only as a request strength: a reply or subsequent request message must not be created in accordance with this protocol or any equivalent.

required

("req") A strength value indicating that the associated protocol is required. As an attribute of an instance: an agent must not elide corresponding processing. As an attribute of an request: a reply or subsequent request message must be created in accordance with this protocol or an equivalent.

route

("route") The scope which addresses every HTTP agent in a transaction from the current agent to the origin.

scope

("scope") In an HTTP transaction, the set of HTTP agents being addressed: the next hop ("conn"), the subsequent chain ("route"), or the endpoints ("origin").

strength

("str") As an attribute of an instance: whether the recipient may or may not elide processing according to the given protocol. As an attribute of a request: whether the offering party will require, refuse, or optionally accept the given protocol in a reply or subsequent request.

strip

To strip a protocol instance or offer, an agent must remove the bag describing it, and each of the header fields listed in that bag's header attribute.

3. Operation

This section is an operational guide to PEP. Section 4 includes a formal presentation of the syntax, status codes, and semantics.

When a PEP-capable HTTP agent receives a PEP-enhanced message, it will parse the various headers, store negotiation data away for later use, and decide which protocols to ``invoke'' and, if relaying the messsage, what data to strip from the message. This section covers each of these phases in detail.

3.1 Scenarios

The client can demand that the server initiate use of trackIt:

1. The client issues a request: "Protocol-Request: {http://pep.w3.org/trackIt {str req}}"
   1. The server can reply using trackIt: "Protocol: {http://pep.w3.org/trackIt {str req}}"
   2. The server can deny the request: "521 Protocol Extension Not Implemented"

1. The client issues a request using trackIt: "Protocol: {http://pep.w3.org/trackIt {str req}}"
   1. The server can continue the trackIt-enchanced conversation: "Protocol: {http://pep.w3.org/trackIt {str req}}"
   2. The server can reject trackIt: "422 Protocol Extension Refused"

   3. The server may not know trackIt at all: "421 Protocol Extension Unknown"

3.2 Interpretation

For each protocol instance and request, the response will depend on the strength and scope. Second, select only those which are ``in scope'' (for the origin, everything; otherwise only "conn" and "route").

Third, for each protocol instance and request, the agent must return the error codes below if it's required strength; error reporting is discretionary otherwise.

• If it refers to a protocol name whose specification cannot be found, Error 421, Protocol Extension Unknown.

• If it refers to a protocol name and parameters combination which is refused, Error 422, Protocol Extension Refused.

• If the parameters are not acceptable (e.g. not parseable, out-of-bounds, not acceptable by local implementation), Error 423, Bad Protocol Extension Parameters.

• If it has "conn" or "route" scope and was sent by an HTTP agent below version 1.1, Error 520, Protocol Extension Error. (see Section 3.5).

• If it refers to a protocol name whose implementation is not available (nor any equivalent), Error 521, Protocol Extension Not Implemented.

• If it refers to a protocol name and parameters combination that are correct, but not acceptable to the server, Error 522, Protocol Extension Parameters Not Acceptable. This might only be determined after the agent attempts processing.

Each protocol instance that ends up in scope, required (or optional but elected by the agent), must be evaluated. If it has an "enc" attribute cited in Content-Encoding:, it must be evaluated in that order, otherwise in any order after decoding is complete. Note that some extensions may not modify the message contents, so multiple extensions could be evaluated in parallel.

3.3 Negotiation

If there is no compabtible set, a server may be forced to reply with Error 520, Protocol Extension Error, akin to Error 406, None Acceptable, currently used for content negotiation.

3.4 Relaying PEP Messages

1. Select protocol instances that must be evaluated and do so (Section 3.2)

2. Before relaying, strip all instances and requests in "conn" scope.

3. Before relaying, optionally strip optional instances and requests in "route" scope. This is suggested if the agent cannot handle the particular protocol extension or configuration.

3.5 Deployment Issues

• Report error 420 if an HTTP message before 1.1 contains protocol offers or requests which are required in "conn" or "route" scope.

For experimental purposes, PEP-compatbility is equated with HTTP/1.1. To deploy PEP services to the installed base of HTTP/1.0, it is possible to design a local, trusted PEP HTTP/1.1 <--> HTTP/1.0 proxy.

3.5.1 Relationship to HTTP/1.1

HTTP/1.1 defines a number of new constructs that PEP relies upon and/or integrates with:

Wrapping

Options

This new method returns the methods and other properties of the specified URI, or of the entire server if the URI is "*". A server should reply with the various Protocol-Request: policies it supports for the server or the resource.

Tunneling

PEP can be used to set up a tunnel (e.g. a secure channel protocol). Note that any HTTP agent acting as a tunnel in a transaction, by definition, cannot act upon PEP directives in the encapsulated traffic.

Chunked Transfer Encoding

This transfer encoding allows agents to manipulate streaming, unknown-length data. Protocol extensions adapted to streaming will operate cleanly above this layer, but some extension modules may force PEP-aware agents to buffer the entire data stream before completing processing.

3.5.2 Protocol Naming and Specification

PEP, unlike many Internet applications, does not cite a central registration authority: protocol extensions are simply URIs. This allows anyone who can control a long-lived URI to create a protocol without any further registration. There are several potentially useful properties of this scheme:

Appropriate Authority

This scheme was chosen to make the process of creating and disseminating useful protocols as straightforward as possible. It avoids the problems inherent with central registration including editorial decisions, assurance of support, trademark conflict, international naming and legal variances, and so on.

The application designer can independently choose the appropriate authority for each extension. It is expected that Web developers, industry consortia, and possibly IANA will each become authorities for various extensions. Note that the compatibility clause in Section 2.1.3 makes cross-endorsement of compatible extensions possible.

Appropriate Lifetime

Protocols, once specified, should be forward-compatible. A protocol can be extended (e.g. new attributes), but designers should not redefine existing syntax or semantics. If incompatible additions or changes to a particular protocol are needed, a new protocol should be created. Protocols can easily use naming conventions that identify revisions. For example, the URI of a protocol can include the version number of the protocol, such as http://pep.w3.org/trackIT/v1.0 or use a {ver 1.0} parameter.

Protocols should be long-lasting. This means that the URI in a protocol name should be reliably resolvable for the foreseeable future. Designers should consider the longetivity of the server and domain when specifying a URI; it will be embedded in PEP software modules and compatibility expressions. One technique is to create a unique host name for all protocol names within an organization, such as http://pep.w3.org/.

Note that this does not mean that particular extensions should be supported indefinitely. The role of PEP is to catalyze just such experimentation and ``creative destruction.'' Nevertheless, to make PEP-enhanced HTTP messages meaningful, even in the future, protocol definitions should be immutable and available.

Packages

Future clients and servers that support many protocol extensions could package their default settings into a single protocol. `Packages' are protocols that consist only of a list of protocol requests. Thus, http://pep.w3.org/LMB9.0b7 would be shorthand for the list of extensions supported by W3C's Line Mode Browser, version 9.0 beta 7.

Similarly, some large-scale PEP applications, such as W3C's Security Extension Architecture [14], will declare "top-level" protocols, so agents can simply request http://pep.w3.org/SEA to indicate an understanding of the entire SEA hierarchy.

Protocol Specification Content

A protocol name is a URI that resolves into a human-readable text description of the extension, ideally a specification. If the scheme is HTTP, then the media types of "text/plain" or "text/html" must be human-readable. Any other media type can be used for machine-readable purposes.

For example, the description of a `package' is nothing more than a series of headers, so its readable representation would be "message/rfc822". Downloadable code could be made available in several formats. A syntax could also be developed to encode metainformation such as compatibility assertions.

4. Notation

4.1 Header Fields

       /* Added to General Header rule, Sec 4.3 of HTTP/1.1 */
       Protocol         = "Protocol" ":" 1#bag
       Protocol-Request = "Protocol-Request" ":" 1#bag
    
       /* Following rules are copied from Section 2.2 of HTTP/1.1 */    
       bag              = "{" bagname 1*LWS *bagitem "}"
       bagname          = token ¦ URI
       bagitem          = bag ¦ token ¦ quoted-string
       
       word             = token ¦ quoted-string
       token            = 1*<any CHAR except CTLs or tspecials>
       tspecials        = "(" ¦ ")" ¦ "<" ¦ ">" ¦ "@"
                        ¦ "," ¦ ";" ¦ ":" ¦ "\" ¦ <">
                        ¦ "/" ¦ "[" ¦ "]" ¦ "?" ¦ "="
                        ¦ "{" ¦ "}" ¦ SP  ¦ HT

       quoted-string    = ( <"> *(qdtext) <"> )
       qdtext           = <any CHAR except <"> and CTLs but including LWS>

{<protocol-name> {scope (origin ¦ conn ¦ route)} {str (opt ¦ req ¦ ref)} {enc <token>} {headers *<token>} {params ...}}

protocol-name

The well-known protocol extension URI.

scope

scope is one of "conn", "route", or "origin" ; the default is "origin".

str

strength is one of "req", "ref" or "opt" ; the default is "opt". ["ref" only allowed on requests]

enc

encoding is a unique token ; the default is undefined [only allowed on instances]

headers

A list of associated headers; the default is the empty set.

params

A list of parameters configuring the instance or request, according to the protocol specification; the default is an empty list.

4.2 Content Codings

4.3 Status Codes

PEP defines several new status codes for HTTP replies. Note that the HTTP/1.0 specification [5] states in Section 6.1.1:

The first digit of the Status-Code defines the class of response. The last two digits do not have any categorization role.

200 Class

220 Uses Protocol Extensions

400 Class

420 Bad Protocol Extension Request

421 Protocol Extension Unknown

422 Protocol Extension Refused

423 Bad Protocol Extension Parameters

500 Class

520 Protocol Extension Error

521 Protocol Extension Not Implemented

522 Protocol Extension Parameters Not Acceptable

5. Usage Examples

5.1 Strength & Scope

PICS Labels

(origin, opt) The only parties that have to agree to transmit PICS labels [16] are the endpoints, and compliance is optional.

rot-13

(origin, req) The only parties that have to agree to transmit rot-13 data labels are the endpoints, but compliance is required.

keep-alive

(conn, req ¦ opt) The only parties that have to agree to keep alive an HTTP connection are the immediate two hosts, but compliance may be required. [12]

SSL tunnel

(route, req) To set up a secure channel, every agent on the path is required to cooperate.

5.2 Payments Negotiation

This example uses currently fictitious domain names MyLocalBank.com, CreditCentral.com, and CashMachine.com as examples of how different financial institutions can participate in a single electronic payment framework.

At any point, a wallet or cash register can emit a protocol request, in several currencies and account types. It can also refuse an option, particularly useful as a feedback mechanism. Here, the client specifically offers to use MyLocalBank's DebitCard2.1 instrument, but also asks the merchant to list other available options:

GET /Catalog HTTP/1.1
Protocol-Request: {http://pep.w3.org/Payment}, \
   {http://MyLocalBank.com/DebitCard2.1 {params {acct Checking}}}

220 Uses Protocol Extensions
Content-Type: text/html
Protocol-Request: {http://www.CreditCentral.com/TypeF},\
   {http://pep.CashMachine.com/e$ {params {cost {USD} {JPY}}}},\
   {http://MyLocalBank.com/DebitCard2.1 {str ref}},

Later, the merchant might show the contents of the shopping cart to the client with a ``Pay'' button. The cart page will provide very specific parameters about the purchase for each system. At this point, the merchant wants to force payment info to be sent, an application of a protocol rather than a request for one:

GET /MyCart HTTP/1.1

220 Uses Protocol Extensions
Content-Type: text/html
Protocol-Request: {http://pep.w3.org/Payment {str req}}, \
    {http://www.CreditCentral.com/TypeF \
    		{params {cost {CaD 15.07} {USD 11.42}} {merchant 69}}},\
    {http://pep.CashMachine.com/e$ \
            {params {cost {USD 11.00} {JPY 1200.}}}}

POST /CheckoutRegister HTTP/1.1
Protocol: {http://pep.CashMachine.com/e$ \
    {params {cost {USD 11.00}} {acct 73} {persona "Anonymous"}}}

200 OK
Content-Type: application/x-e$
...

This is an elementary example of this system; further interactions and negotiation would highlight the flexibility and extensibility of PEP.

5.3 Security Processing

  WRAPPED * HTTP/1.1
  Protocol: {http://pep.w3.org/SEA/Signature/RSA-MD5 \
     {params {key-id RSA://khare@w3.org/hexhexhex} {sig hexhexhex}}}
  Protocol-Request: {http://pep.w3.org/SEA/Key-Exchange {str req}}
  Protocol-Request: {http://pep.w3.org/SEA/Encryption {str req}}

+ GET /MyProfile HTTP/1.1
+ ...

  220 Uses Protocol Extensions
  Protocol: {http://pep.w3.org/SEA/Signature/RSA-MD5 \
     {params {key-id RSA://server@foo.org/hexhexhex} {sig hexhexhex}}}
  Protocol: {http://pep.w3.org/SEA/Key-Exchange/RSA \
     {params {key-id RSA://khare@w3.org/hexhexhex} \
             {session-key hexhexhex} {enc key-exch3}} {str req}}
  Protocol: {http://pep.w3.org/SEA/Encryption/IDEA \
     {params {message-key "key-exch3"} {enc enc7}} {str req}}
  Content-Type: text/html
  Content-Encoding: key-exch3, enc7, gzip

+ encrypted(compressed(HTML)

In particular, the server's reply indicates that first, the recipient is required to evaluate the RSA key exchange module, which generates the "key-exch3" message key. The next stage of the pipeline is IDEA decryption, followed by Gzip decompression. At this point, any protocols not cited in Content-Encoding: can be executed; in this case, verifying the server's signature.

In security applications, it is essential that authors be able to make assertions such as ``If you fetch your account records from Foo's server, be sure to demand a signature.'' Since that policy corresponds to a PEP Protocol-Request: header, it can be directly embedded in to HTML as just one more HTTP metainformation header. See [14] for more details on PEP integration with HTML.

6. Security Considerations

Separately, PEP encourages a plug-in software architecture for HTTP agents. There are manifold risks to executing untrusted or marginally-trusted code, especially if sensitive data is passed into such modules. These are not PEP-specific risks, but are of importance to any implementor.

6.1 Information Leakage

Typically, a PEP-compliant implementation will read Content-Encoding: and create a processing pipeline for each module.

``The information passing between processing stages should be considered sensitive.''

For example, one module may compute a shared session key, and pass it inband (in the clear) to the next stage, an encipherment protocol that protects the key.

Implementations should carefully protect intermediate data flows. Consider: controlling access to pipe endpoints, avoiding writing to disk, and wiping clean all memory buffers after use.

Implementors should be particularly careful on platforms that do not provide secure interprocess communication.

Finally, protocol-extension designers may wish to specify that implementations should handle several protocol instances with a single module; in the example above, a single module that generated the session key and the ciphertext.

6.2 Trusting PEP Headers

HTTP messages travel in the clear; messages using PEP are no different.

``Do not trust the integrity of PEP headers without proof. ''

Unless PEP headers are being read from an authenticated channel or a wrapped, signed or encrypted message, PEP headers are unreliable. An attacker can modify, remove, or add protocol offers, instances, and encoding order.

The consequences of such a man-in-the-middle attack include denial-of-service, since two parties that actually have facilities to communicate can each end up believing that the other does not.

In general, the risks here can be limited if implementors apply reasonable sanity checks: e.g. don't send sensitive data in the clear, scrutinize the order and plausibility of the modules to be run, and so on.

6.3 Protocol Interaction Effects

Protocol extension designers must be very careful about interactions with other protocols.

``Protocol extensions considered safe individually can be dangerous in combination or reordered.''

Separate protocols may also interpret data in conflicting ways, or offer contradictory modes of operation. Repeated application may also be an error, e.g. rot-13(rot-13(text)).

6.4 Protocol Substitution Effects

The language of the PEP specification is carefully formulated to allow agents to reply using protocol extensions that an agent believes to be equivalent.

``Any protocol request may be satisfied using a different protocol the originator believes to be interchangeable.''

6.5 Negotiation Scope

HTTP is a stateless protocol, and PEP does not modify that. As a result, agents cannot rely on negotiating within a fixed context; each request may be considered anew.

``PEP for HTTP/1.1 cannot require all agents to maintain common beliefs about capabilities.''

For example, if a server refuses clear text POST to a certain URI, there is no basis to assume that a client will not, in fact, attempt to POST clear text.

7. Development Path

The W3 Consortium is actively pursuing PEP research and deployment. Reference implementations will be freely available from W3C, as will protocol extension modules for a wide variety of applications, including PICS [16]. W3C's Security and Payments Working Groups have been involved with PEP since July 1995, and are developing PEP-based solutions.

Please contact khare@w3.org with any questions, comments, or concerns in this area.

8. Acknowledgements

This specification makes heavy use of the grammar, constructs, and style of HTTP/1.0 and HTTP/1.1. Thanks to Roy T. Fielding for his work on those documents, and for his input to PEP.

After the first draft was released, several people came forward with excellent suggestions, but Paul Hoffman of the Internet Mail Consortium in particular put considerable time and effort into helping refine the PEP model and clean up the text of this document.

The W3 Consortium technical staff at MIT have put significant design effort into this proposal: Tim Berners-Lee, Dan Connolly, Jim Gettys, Phillip Hallam-Baker, Jim Miller, Henrik Frystyk Nielsen, and Dave Raggett.

Allan Schiffman of Terisa Systems helped clarify the logic and power of PEP security. Finally, credit is due to Dave Kristol, whose original ``A Proposed Extension Mechanism for HTTP'' Internet Draft directly led to the PEP proposal presented here.

9. References

[1]

M. Abadi and R. Needham. "Prudent Engineering Practice for Cryptographic Protocols." Digital Systems Research Center: Report 125, Digital, June 1994.

[2]

R. Anderson and R. Needham. "Robustness principles for public key protocols." ftp://ftp.cl.cam.ac.uk/users/rja14/robustness.ps.Z [in proceedings of Crypto '95], Cambridge University Computer Laboratory, 1995.

[3]

Berners-Lee, T., "Universal Resource Identifiers in WWW: A Unifying Syntax for the Expression of Names and Addresses of Objects on the Network as used in the World-Wide Web", RFC 1630, CERN, June 1994.

[4]

Berners-Lee, T., Masinter, L., and M. McCahill, Editors, "Uniform Resource Locators (URL)", RFC 1738, CERN, Xerox Corporation, University of Minnesota, December 1994.

[5]

T. Berners-Lee, R. Fielding, and H. Frystyk Nielsen, "Hypertext Transfer Protocol -- HTTP/1.0". Internet Draft W3 Consortium/MIT, UC Irvine, W3 Consortium/MIT, October 1995 (Work in Progress).

[6]

T. Berners-Lee, R. Fielding, and H. Frystyk Nielsen, "Hypertext Transfer Protocol -- HTTP/1.1". Internet Draft W3 Consortium/MIT, UC Irvine, W3 Consortium/MIT, November 1995 (Work in Progress).

[7]

D. H. Crocker. "Standard for the Format of ARPA Internet Text Messages." STD 11, RFC 822, UDEL, August 1982.

[8]

S. Deering, R. Hinden, Editors, "Internet Protocol, Version 6 (IPv6) Specification", Internet Draft, June 1995 (Work in Progress).

[9]

D. Eastlake, B. Boesch "Universal Payment Preamble", Internet Draft CyberCash, 6 November 1995 (Work in Progress).

[10]

D. Eastlake, "Application Level Internet Payment Syntax", Internet Draft CyberCash, 29 January 1996 (Work in Progress).

[11]

R. Fielding. "Relative Uniform Resource Locators." RFC 1808 , UC Irvine, June 1995.

[12]

A. Hopmann, "HTTP Session Extension", Internet Draft, July 1995 (Work in Progress).

[13]

D. Kristol, "A Proposed Extension Mechanism for HTTP", Internet Draft, January 1995 (Work in Progress, Expired).

[14]

R. Khare. "SEA: A Security Extension Architecture for HTTP/1.x." W3C Working Draft , W3 Consortium, February 1996 (Work In Progress).

[15]

J. Klensin, N. Freed, M. Rose, E. Stefferud, and D. Crocker. "SMTP Service Extensions." RFC 1869. MCI, Innosoft, Dover Beach Consulting, Network Management Associates, Brandenburg Consulting, November 1995.

[16]

J. Miller. "Label Syntax and Communication Protocols." Internet Draft , W3 Consortium/PICS, November 1995 (Work In Progress).

[17]

J. Postel. "Simple Mail Transfer Protocol." STD 10, USC/ISI, August 1982.

[18]

J. Postel, J. Reynolds, "Telnet Protocol specification." STD 8, RFC 854, USC/ISI, May 1983

[19]

E. Rescorla, and A. Schiffman, "The Secure Hypertext Transfer Protocol", Internet Draft, July 1995 (Work in Progress).

10. Author's Address

Technical Staff, W3 Consortium

MIT Laboratory for Computer Science

545 Technology Square

Cambridge, MA 02139, U.S.A.

Tel: +1 (617) 253 5884

Fax: +1 (617) 258 8682

Email: khare@w3.org

Web: http://www.w3.org/People/Khare

Appendices

These appendices are provided for informational purposes only -- they do not form a part of the PEP specification.

Appendix A: Minimum Compliance

There is one subtle detail to mention with respect to all three contexts: handling PEP messages relayed through non-PEP-compliant agents (Section 3.5). The rules below handle this case, but systems that actually implement one or more PEP extensions will have to detect separately when a requirement was actually directed at a previous agent, not the current agent.

It should be noted that parsing the <bag> data structure is a simple affair (< 200 lines of C, for example). Error reporting is not much more difficult.

A.1 Operating as a Server

1. Parse PEP Headers. Parse the bags in the Protocol: and Request-Protocol: headers. The only fields that need to be kept are the protocol name, scope, and strength, and whether the bag represents an instance or request. Return Error 420 if there are any difficulties at this stage.

2. Return Error 521. For each request or instance, if the strength is req, return an Error 521. Explanatory text could be generated for each error raised.

3. Proceed. If there are no required instances, the message can be understood without further processing -- simply ignore unknown encoding tokens in Content-Encoding:.

A.2 Operating as a Client

1. Parse PEP Headers. Parse the bags in the Protocol: header; The only fields that need to be kept are the protocol name, scope, strength, and encoding. Raise a parsing error if there are any difficulties at this stage.

2. Report Unknown Extensions. For each instance, if the strength is req, notify the user that the message cannot be handled, or discard the message. Explanatory text could be generated for each error raised. Additionally, it is useful to flag required requests, since the inability to satisfy such demands could cause the failure of subsequent request messages.

3. Proceed. If there are no required instances, the message can be understood without further processing -- simply ignore unknown encoding tokens in Content-Encoding:.

A.3 Operating as a Proxy

1. Parse PEP Headers. Parse the bags in the Protocol: and Request-Protocol: headers. The only fields that need to be kept are the protocol name, scope, and strength, and whether the bag represents an instance or request. Return Error 420 to the client if there are any difficulties at this stage -- there is no way to return an error upstream to the server if a reply message could not be parsed.

2. Return Error 521. For each request or instance, if the scope is conn or route and the strength is req, return an Error 521 to the client. Explanatory text could be generated for each error raised, as well as indicating if the fault was generated by the proxy or a later server.

3. Strip conn. For each offer or instance, if the scope is conn, strip that bag and any associated headers.

4. Proceed. If there are no required instances in scope, the message can be relayed without further processing.

In addition, a caching proxy should be aware of the following policies:

• Don't cache 220 responses. Documents retreived that use PEP should be presumed noncacheable unless the Cache-control: information says otherwise. For example, a merchant per Section 5.2 may want the /Catalog page cached along with its default set of payment instruments, but not the later, specific-cost pages. This is a conservative policy recommendation; see the discussion in Section 4.3.
• Don't answer PEP-enhanced requests from the cache. If a request uses a PEP enhancement clearly addressed to the origin or other servers (i.e. any scope other than conn), it is likely incorrect to return an answer from the cache.

Appendix B: Implementation Model

This appendix describes a formal model of how an HTTP agent works with PEP messages, in any of the roles just described.

An agent A has a set L of protocol modules: L.encoders and L.decoders. Each module l registers the protocol extensions e it implements, yielding a set E of protocol extensions that can be generated (E.encoders) and accepted (E.decoders). For any extension, e, E.compatible(e) is a set of extensions registered as compatible with e.

An agent A has a set R of resources it knows of. A resource has a set of URIs (r.uri), an entity body (r.entity), and a policy set (r.policy), in addition to the usual HTTP message attributes. A policy is a list of Protocol-Request: lines that A believes must be satisfied to access or transmit r.

Any message m between two agents includes the resource (m.r), an on-the-wire entity body (m.entity), a list of extensions applied (m.extensions), an order (m.encoding), and a policy (m.policy). Each extension or policy entry has strength, scope, etc attributes.

An agent A can maintain a negotiation context c, which exists until the next message is sent to the same agent, c.agent, that it was created for. A context holds c.agent's policy in c.policy. [On a client, this might be more effectively indexed by c.r, the resource fetched]

An agent A functions in a fixed scope, A.scope. For an origin client or server, it is {conn, route, origin}; for a proxy, it is only {conn, route}.

A PEP-enhanced message m can be received correctly with the following steps:

1. Check the policy. The message must be acceptable according to A's policy. The magic predicate isAcceptable(E, m.r.policy, m.extensions, m.encoding) checks whether the order and configuration of the extensions applied to m satisfy A's policy for the resource m is addressed to.

2. Run the pipeline. For each encoding g in m.encoding, execute some e'.decoder from E.decoders.compatible(m.extensions.enc = g) if e'.scope = A.scope and e'.strength = req.

3. Run the rest. For each extension e' where e'.encoding is undefined or not in m.encoding, execute some e'.decoder from E.decoders.compatible(m.extensions.enc = g) if e'.scope = A.scope and e'.strength = req.

4. Store the policy. Create some context c with c.agent = m.sender and c.policy = m.policy.

After this, A generates a reply. Either the user clicks on a link from that resource (client), a response is generated (server), or a reply is relayed back (proxy). The original HTTP response m for context m.c is modified into m' according to:

1. Advertise the policy. The message should express the resource's policy: m'.policy = m.r.policy.

2. Extend the message. The message must be extended to satisfy the resource's and the context policy (i.e. the originator). There is a magic predicate extendMessage(E, m.r.policy, m.c.policy) that specifies m'.extensions and m'.encoding. m' must then actually be encoded according to that pipeline, etc.

In summary:

For each e, the recipient may/must apply e'.decode for some e' in e.compatible, according to the order stated by m.pipeline, if applicable. Furthermore, any response in the same context as this message must conform to m.policy.

There are other, genuine implementation details, such as the life and scope of negotiation contexts -- at what point does A forget what it knows about r? A client's reply is indeterminate -- if r points to a resource on some other server, does a subsequent request abide by r.c.policy? It depends on the definition of context -- the PEP requirement is only that a reload of the same resource is a 'subsequent request'. If the other request is insufficiently enhanced, the only cost is an extra round-trip to be told the policy.

The software architecture for calculating extendMessage() and isAcceptable() based only on callbacks to each module is an open challenge. In particular, implementations will encounter nondeterministic interactions in negotiating solutions to extendMessage().