Disclaimer: This document strictly represents my own opinion and not necessarily those of the W3C.
Status: This document is in draft mode - it proposes a new endpoint model where endpoints are first class objects. Comments are welcome!
The term "endpoint" is rather overloaded but in this context, I use the following definition of endpoints for unicast services on the Internet:
Definition: An endpoint is the contact point through which clients can communicate with a service.
Not all endpoints are connected to a service but all services are connected to at least one endpoint. The relationship between an endpoint and a service is similar to that between a phone number and a subscriber answering calls to that phone number.
The traditional model for endpoints in the Internet (not considering non-IP based nor non-unicast protocols) describes an endpoint as a triple consisting of the parameters:
(hostname, transport, port number)
where
As an example, the endpoint (www.w3.org, TCP, 80) defines the endpoint used by the W3C HTTP servers which clients can connect to in order to access the W3C Web space.
In order for a client to communicate with a service, it must resolve the endpoint used by that service. In the example of (www.w3.org, TCP, 80) if we for a moment don't consider proxies and firewalls, the client has to perform the following steps:
Note, that it is a property of the transport whether the operation of establishing a transport connection with a service can happen simultaneously with communicating with the service. In a non-connection oriented transport like UDP, the scope of the "connection" is a single transport packet which means that the communication and establishement has to happen simultaneously. In a connection oriented transport like TCP, the two operations are performed separately.
If there is a service picking up the transport connection the next problem is to determine how to communicate with that particular service. Again, note the destinction between resolving an endpoint and communicating with the service answering at that particular endpoint. If the port number is a well-known port (in the range 0-1023), then the service is assigned by the IANA registry under protocol numbers. If the port number is a registered port (1024-49151), then the service may be registered by IANA but is not owned by IANA . The remaining set of ports are private or ephemeral ports (49152-65535) assigned dynamically without any conflict avoidance mechanism.
Traditional places where a client can learn about the capabilities of
endpoints not registered in IANA are port mappers, URIs (using the
":port" part of the URI) and plain old out-of-band knowledge like
word of mouth etc. Although not a requirement, most common URL schemes like
http:, ftp, and mailto: use the current
endpoint architecture to achieve global uniqueness even though the endpoints
themselves in fact are not considered first class objects - it is
only the resource identified by the URI that is a first class object. In
effect, there is no URI for the W3C Web service, only for resources exported
by that service.
In general, proxies are described using the same endpoint model and do not change the resolver mechanism above except that they add an extra layer of indirection in what we can call the resolution process of a URI. That is, the three steps above are repeated for each additional proxy inserted into the system. As for what we call origin services, proxy services are not considered first class objects - only the resources that they proxy.
The model outlined above has been extremely successful and has served its purpose well. However, the model is being stretched in various dimensions which we will outline in this section. Two major reasons for this is the increasingly decentralized evolution of services on the Web and the Internet in general and that more and more services can use multiple endpoints and pick between these endpoints dynamically.
In the following, we describe the limitations and problems in the endpoint triple in reverse as this is in fact symptomatic of the way existing endpoints are being stretched.
Traditionally, well-known port numbers have been used to identify rather large application level protocols like FTP, SMTP, NNTP, HTTP, etc. Such protocols used to be defined by a relatively small group of people, typically within an IETF working group. The large granularity of these protocols is emphasized by the small number of well-known port numbers in use on the Internet today and the fact that port 80 alone accounts for between 60-70% of the Internet traffic.
Over the last 5 years or so, the granularity of application layer services have decreased significantly from the traditional application protocols to much finer grained differences in services that change over time. A major reason for this is the increasing amount of reusability of existing features provided by already deployed application layer protocols. HTTP, for example, is increasingly being used as a "transport" for new services rather than a service in itself.
HTTP happens to be somewhat suited for this use as it has a (although implicit) layered model but it also has serious drawbacks that has been described in "HTTP-NG Overview Problem Statement, Requirements, and Solution Outline" and other places. Regardless of the suitability of HTTP for this use, the development is interesting in itself: What has happened with HTTP is that it has propagated from being designed by a small group of people to becoming a "common good" used in an increasingly distributed and localized environment with no direct coordination or supervision. There are no signs that this trend will slow down, on the contrary, it is likely that it will spread to other application services as well as transports.
Protocol version numbers are inherently unable to accommodate this trend as they rely on a central registry defining the semantics of what a version number means and does not provide the required level of granularity for handling incremental, distributed, extensions.
The HTTP Extension Framework provides a mechanism for handling the declaration of distributed, independent extensions but does not provide a mechanism for determining the characteristics of a service including what types of resources it is exporting, what transports are available, local access points etc.
The lack of granularity in port numbers does not only affect communication between endpoints but also intermediaries like firewalls and proxies that has an increasingly hard time dealing with the highly differentiated set of services provided by non-differentiated endpoints. This is especially the case if these services include tunnelling which certainly is true for HTTP.
HTTP has the notion of a special address named "*" which can
be used in an OPTIONS HTTP request for addressing the server as an
independent entity. The problem with this solution is that the "*" identifier
is a function of the HTTP OPTIONS method and can not be expressed
as a first class object in URI space by itself. Furthermore, the options have
to exchanged and re-negotiated upon each new transport level connection. Other
protocols like NNTP and SMTP send rather limited information in a
Hello type response which contains little information about what kind
of service is available at that server.
In RPC type services, a portmapper is often used as a mechanism for binding services to ephemeral ports. A portmapper is a kind of registrar running on a well-known port keeping track of the correspondence between ports and services on a machine, and provides a standard way for a client to look up the (ephemeral) port number of any remote procedure call program supported by the server.
In the SunRPC portmapper service (port 111) a service is identified by its RPC program number (administered by a central authority, rpc@sun.com), a version number, and the transport address where it may be reached. The transport address, in turn, consists of a network address and a transport selector which can either be a TCP or a UDP port number. That is, the administration of the services controlled by the portmapper is no more decentralized or flexible than services registered on well-known ports.
A major problem with the existing endpoint model is that it provides no mechanism for graceful deployment of new transports, has no mechanism for dynamically negotiating between alternative transports, and does not contain the concept of stacking transports on top of each other.
For example, a single resource may be available through different endpoints by the service serving the resource. These endpoints may or may not be compatible: HTTP/0.9, HTTP/1.0, and HTTP/1.1 are backwards compatible protocols but HTTP running on top of SSL is not although it is in fact using HTTP as part of the transport.
IANA currently registers separate well-known port numbers for SSL used as a transport together with HTTP, NNTP, FTP, IIOP, IMAP4, LDAP, telnet, IRC, POP3 and several other less known protocols. These port numbers are of course different from the equivalent services running on top of TCP and when the next transport comes along, the number will multiply again.
In order to differentiate between the SSL versions of these services (which is really the same services using different endpoints), people have been inventing new URI name spaces. Although it often is claimed that ensuring the stability of URIs is a social engineering task, evolution of transports is a technical problem in the current infrastructure that must be solved in order to allow for stable URIs.
In order to obtain a resonable degree of reliability and to maintain acceptable response times, services are often mirrored through out the Internet, a typical example is the mirroring of IETF RFC's, another is software distribution sites, etc. Mirroring and load balancing is typically handled manually without any machine understandable relationship between the participating hosts.
In some cases, DNS round robin mechanisms on multi-homed hosts have been used to provide mirroring and load balancing capabilities between multiple physical machines handling the load of a single service. For example, the W3C Web site is made available through servers in the US, Europe, and Japan by generating the DNS response as a function of the AS number of the requestor.
The problem is that DNS can handle this on a per host basis but the granularity is not fine enough to deal efficiently with hotspots etc. for two reasons:
@@@In other cases, TCP routing is used to perform load balancing - check IBM paper on this issue@@@
@@@Proxying and redirection supported by at least DNS and HTTP.@@@
The limitation of host names also causes problems in application layer protocols: There is currently no mechanism in HTTP proxies, for example, to describe which resources they are proxying and which they are not. There have been attempts of providing this service in HTTP in the form of proxy redirections but because the service is not a first class object, the trust mechanism for dealing with these types of redirections is very weak.
The fundamental idea behind this proposal is to extend the definition of an endpoint from a triple to a first class object referenced by a URI. We call this new definition of an endpoint for a service point. The advantages of using a full URI to identify endpoints instead of the existing (hostname, transport, port number) triples can help solve most of the current shortcomings dealing with evolution while still taking advantage of a minimum of shared knowledge in order to achive interoperability.
By making services first class objects, it becomes possible to talk about a service and to describe its trust relationships, security, etc. as if it were any other resource ideintified by a URI. Furthermore, if the URI can be dereferenced, it may provide information describing:
Because of normal evolution, the description of a service point can change over time accomodating new transports, delocation, access policies, etc. The association between the service point and its description might be made available in different representations ranging from
By using service points, evolution can happen without changing the name of the service point itself. It is important to note that it is not a requirement that service points can be dereferenced nor that applications must do so. The URIs identifying the service point can be handled strictly as a name.
If the description of a service endpoint is formalized using a datamodel like for example provided by RDF, the description can be made recursive to contain separate layers of descriptions:
Depending on the capabilities of the mechanism used to deference the service point, the result of such an operation may be cachable and may be served to other clients accessing that service point. This is for example the case if DNS or HTTP were used for either discovery or dereferencing of service points.
@@@Allows for more superior filtering in firewalls instead of existing port number filtering, now it can be done on a per URI bases@@@
@@@Security and trust mechanisms developed for other resources identified by URIs can immediately be applied to services.@@@