Disclaimer: This document strictly represents my own opinion and not necessarily those of the W3C.
Status: This document is in draft mode - it proposes a new endpoint model where endpoints are first class objects. Comments are welcome!
The term "endpoint" is rather overloaded but in this context, I use the following definition of endpoints for unicast services on the Internet:
Definition: An endpoint is the contact point through which clients can obtain access to a service provided by a server.
The traditional model for endpoints in the Internet (not considering non-IP based nor non-unicast protocols) describes an endpoint as a triple consisting of the parameters:
(hostname, transport, port number)
where
As an example, the endpoint (www.w3.org, TCP, 80) defines the endpoint used by the W3C HTTP servers which in this particular example consists of several independent homes geographically distributed accross the world.
The question is then - how does a client know how to contact this particular endpoint? If we for a moment don't consider proxies and firewalls, the client has to perform the following steps before it can start accessing the service:
At this point the client has a transport link to the service but it doesn't know what the service is nor now to communicate with it. Note, that I am not talking about accessing a particular resource at this point - all that has happened so far is that we have contacted the endpoint.
If the port number is a so-called well-known port, then the service is registered in the IANA registry under port numbers but if not then the client must get the knowledge from somewhere else. Traditional places where a client can learn about the capabilities of service endpoint not registered in IANA are port mappers, URIs and plain old out-of-band knowledge like word of mouth etc.
Proxies do not in general change this model but they add an extra layer of indirection in what we can call the resolution process - that is, the three steps above are repeated for each additional procy inserted into the system.
This model has been extremely successful in the past but is beginning to show significant signs of being stretched in various dimensions. If it was just a question of enhancing the current registration mechanism, then there is no particular good reason to attempting to apply a technical solution because in that case it would primarily be a social problem of redefining the registration mechanism.
However, I don't think this is in fact not only a social problem but indeed an architectural problem that affects the evolution of the Internet and its services. In the following, I will try and clarify this view and propose a generalization of the existing endpoint model which can better accommodate the increasingly decentralized evolutionary nature of the Internet.
Care has been taken to ensure that the generalized model is a superset of the existing model in order to ease incremental deployment on the Internet.
Lets walk through the triple model described above in reverse order as this is symptomatic of the way existing endpoints are being stretched and have a look at the current problems.
Traditionally, well-known port numbers have been used to identify rather large application level protocols like FTP, SMTP, NNTP, HTTP, etc. Such protocols used to be defined by a relatively small group of people, typically within an IETF working group. The large granularity of these protocols is emphasized by the small number of well-known port numbers in use on the Internet today and the fact that port 80 alone accounts for between 60-70% of the Internet traffic.
Over the last 5 years or so, the granularity of application layer services have decreased significantly from the traditional application protocols to much finer grained differences in services that changes over time. A major reason for this is the increasing amount of reusability of existing features provided by already deployed application layer protocols. HTTP, for example, is increasingly being used as a "transport" for new services rather than a service in itself.
HTTP happens to be reasonably well suited for this use as it has a (although implicit) layered model but it also has serious drawbacks that has been described in "HTTP-NG Overview Problem Statement, Requirements, and Solution Outline". Regardless of the suitability of HTTP for this use, the development is interesting in itself: What has happened with HTTP is that it has propagated from being designed by a small group of people to becoming a "common good" used in an increasingly distributed and localized environment with no direct coordination or supervision. There are no signs that this trend will slow down, on the contrary, it is likely that it will spread to other application services as well as transports.
Protocol version numbers are inherently unable to accommodate this trend as they rely on a central registry defining the semantics of what a version number means and does not provide the required level of granularity for handling incremental extensions.
The lack of granularity in port numbers does not only affect communication between endpoints but also intermediaries like firewalls and proxies that has an increasingly hard time dealing with the highly differentiated set of services provided by non-differentiated endpoints. This is especially the case if these services includes tunnelling which certainly is true for HTTP.
Some protocols like HTTP has the notion of a special address named
"*" which can be used in an OPTIONS HTTP request for
addressing the server as an independent entity. The problem with this
solution, however, is that this identifier is not a first class object and has
to be renegotiated upon each new transport level connection. Other protocols
like NNTP and SMTP send rather limited information in a hello type
response which contains little information about what kind of service is
available at that server.
Services like the SunRPC portmapper solves a part of the problem as it provides a service mapping a well-known port into a specific service which can be registered dynamically on an arbitrary port. However, the problem is that portmapper depends on central knowledge of services and the relationship between version numbers of these services.
A major problem with the existing endpoint model is that it provides no mechanism for graceful deployment of new transports or services. For example, a single resource may be available through different endpoints by the party serving the resource. These endpoints may or may not be compatible: HTTP/0.9, HTTP/1.0, and HTTP/1.1 are backwards compatible protocols but HTTP running on top of SSL is not although it is in fact using HTTP as part of the endpoint definition.
As a result people have been inventing new URI name spaces to accommodate the need for describing differentiated endpoints dynamically. Although ensuring the stability of URIs to a high degree is a social engineering task, it is as important that the Web infrastructure supports evolution of transports.
In order to obtain a high degree of reliability and to maintain reasonable response times, services are often mirrored through out the Internet, a typical example is the mirroring of IETF RFC's, another is software distribution sites. Mirroring and load balancing is typically handled manually without any machine understandable relationship between the participating hosts.
In some cases, DNS round robin has been used to provide mirroring and load balancing capabilities. The problem is that DNS can handle this on a per host basis but the granularity is not fine enough to deal efficiently with hot spots etc. for two reasons:
@@@In other cases, TCP routing is used to perform load balancing @@@check IBM paper@@@. The problem with this mechanism is that it can not be automated.
The limitation of host names also causes problem in application layer protocols: There is currently no mechanism in HTTP proxies, for example, to describe which resources they are proxying and which they are not. There have been attempts of providing this service in HTTP in the form of proxy redirections but because the service is not a first class object, the trust mechanism for dealing with these types of redirections is very weak.
The problem with the triple definition of endpoints above is that nobody write
The brilliant idea about the URI space is that it institutionalized a simple syntax for how to describe endpoints as well as for
http://www.w3.org
the first time provided a homogenous, human readable representation
@@@
However, URIs were in fact very conservative in that they did not attempt to redefine the current endpoint architecture but rather provide a syntax that combined the endpoint definition with a resource identifier provided through that endpoint.
@@@Why not just use completly self-describing messages every time?@@@
The fundamental idea behind this proposal is to extend the definition of an endpoint from a triple to a first class object referenced by a URI. We simply call this new definition of an endpoint for a service. As will be discussed, the advantages of using URIs to identify endpoints instead of (hostname, transport, port number) triples are significant and can help solve most of the current shortcomings outlined above.
By making services first class object, it becomes possible to talk about a service as any other resource identified by a URI. This means that it can be derefenced to determine its capabilities and composition without necessarily knowing anything about the service beforehand:
Depending on the capabilities of the mechanism used to deference the service, the result of such a query may be cachable and can be served to other clients querying that service. This is for example the case if DNS or HTTP were used for either discovery or dereferencing of services.
However it is important to note that it is not a requirement that service can be dereferenced nor that applications must do so. The URIs identifying the service can be handled strictly as a name; the only strict requirement are that:
Service providers must take special care not to distribute conflicting specifications that reference the same name. Even when an extension specification is made available at the address of the URI, care must be taken that the specification made available at that address does not change over time. One agent may associate the identifier with the old semantics, while another might associate it with the new semantics.
The association between the service identifier and the specification might be made by distributing a specification, which references the extension identifier. The extension definition may be made available in different representations ranging from
Note that this is different from a "core" protocol that can act as a platform for all the services that people would like to deploy on the Internet. While this is a worth while goal and should be pursued, it may not be the only way to define a flexible framework for handling the increasing number of services.
@@@Link to HTTP extension framework, HTTP-NG, and Applcore
@@@Allows for more superior filtering in firewalls instead of existing port number filtering,
@@@Security and trust mechanisms developed for other resources identified by URIs can immediately be applied to services.