Techniques for specifying services

From Web of Things Community Group

Machine Interpretable descriptions

This section covers the means to support machine interpretable descriptions of services. The starting point is the idea of using globally unique names for things and relationships between them. The names are expressed as URIs, e.g. HTTP URLs such as: http://purl.org/dc/terms/abstract which is a term for a human readable abstract. This approach was first popularised in the following Scientific American article:

RDF

W3C's Resource Description Framework (RDF) is based upon labeled relationships in the form of a three place tuple, or triple:

  • <subject, verb, object>

Where the subject, verb (the label), and the object are all named with HTTP URLs. RDF was originally proposed with an XML serialization, but other formats have since become widely used, e.g. Turtle, N3 and more recently JSON-LD.

The following link is to the W3C page describing the relationships between the suite of specifications covering RDF. The Wikipedia article provides links to additional resources and tools relating to RDF.

Linked Data

RDF forms the basis for the Web of linked data. This is formed by the collection of servers that host linked data. The number of sites has risen dramatically in the last few years, mirroring the growth of the Web in the 1990's. The following illustration of the Web of linked data is from the The linking open data cloud diagram from September 2011.

The following table gives the corresponding number and size of datasets by domain, and is taken from State of the LOD Cloud document

Domain Data Sets Triples Percent RDF Links Percent
Media 25 1,841,852,061 5.82 50,440,705 10.01
Geographic 31 6,145,532,484 19.43 35,812,328 7.11
Government 49 13,315,009,400 42.09 19,343,519 3.84
Publications 87 2,950,720,693 9.33 139,925,218 27.76
Cross-domain 41 4,184,635,715 13.23 63,183,065 12.54
Life sciences 41 3,036,336,004 9.60 191,844,090 38.06
User generated 20 134,127,413 0.42 3,449,143 0.68
295 31,634,213,770 503,998,829


For small data sets, it is possible to use simple scripts for custom queries. For large data sets, you can use the SPARQL query language. Here is an example for a query that asks "What are all the country capitals in Africa?":

PREFIX abc: <http://example.com/exampleOntology#>
SELECT ?capital ?country
WHERE {
  ?x abc:cityname ?capital ;
     abc:isCapitalOf ?y .
  ?y abc:countryname ?country ;
     abc:isInContinent abc:Africa .
}

The W3C Linked Data Platform Working Group is developing standards for HTTP-based (RESTful) application integration patterns using read/write Linked Data, see:

Websites can add structured data in RDF to HTML pages for smarter support by search engines. The approach is called Rich Structured Data Markup for Web Documents, or more simply RDFa. For an introduction to using RDFa, see:

The following book by Tom Heath and Christiam Bizer provides an extended introduction to linked data:

Some relevant vocabularies

The marketplaces for the Web of Things relates to the Web of linked data in two ways:

  • Services that generate linked data, e.g. from sensors and interpretations thereof
  • Service descriptions expressed using linked data

RDF can be applied to data modelling. The starting point is the RDF Schema (RDFS), which defines terms for resource, class and literal. The Web ontology language (OWL) adds a richer set of modelling terms. SKOS is a standalone vocabulary intended for modelling thesauri in RDF.

Further work is needed to survey all of the potentially relevant data vocabularies.

A few common vocabularies include:

The Linked Open Vocabularies (LOV)] dataset covers the RDFS vocabularies or OWL ontologies for and used by datasets in the Linked Data Cloud. A similar collection is available at:

Note: the work the Semantic Sensor Net Ontology was developed by the W3C Semantic Sensor Network Incubator Group. This work has now been taken over by the Semantic Sensor Networks Community Group.

The main search engines (Bing, Google and Yahoo! jointly started Schema.org in June 2011 to collect terms that webmasters can use to markup their sites to improve the display of search results. The schems.rdfs.org site provides access to the schema.org terms in various formats including RDF/Turtle, RDF/XML and RDF/NTriples. They also maintain mappings from Web Data vocabularies such as the DBpedia ontology to Schema.org terms. This should also be applicable to search across open marketplaces for the Web of Things.

Note: DBpedia is crowd-sourced community effort to extract structured information from Wikipedia and make this information available on the Web. DBpedia allows you to ask sophisticated queries against Wikipedia, and to link the different data sets on the Web to Wikipedia data.

Defining Service data sources and sinks

This section covers techniques that can be used to specify the protocol or API for accessing services for the Web of Things. This includes the use of WebIDL for APIs, XML and JSON for structured data, Web Services and RESTful HTTP.

Web IDL

Web IDL is an interface definition language that is commonly used in W3C specifications for JavaScript APIs exposed by web browsers. It includes a suite of pre-defined data types including numbers, strings, arrays, as well as support for enumerations, unions and call backs. The Web IDL specification includes a binding to JavaScript (formally known as ECMAScript), but bindings to other languages are possible.

Here is an example of a Web IDL fragment:

exception GraphicsException {
  DOMString reason;
};

interface Paint { };

interface SolidColor : Paint {
  attribute float red;
  attribute float green;
  attribute float blue;
};

interface Pattern : Paint {
  attribute DOMString imageURL;
};

[Constructor]
interface GraphicalWindow {
  readonly attribute unsigned long width;
  readonly attribute unsigned long height;

  attribute Paint currentPaint;

  void drawRectangle(float x, float y, float width, float height);

  void drawText(float x, float y, DOMString text);
};

XML

The Extensible Markup Language (XML) is a simple and flexible text format derived from SGML (ISO 8879). Angle brackets are used to introduce mark-up tags into a text stream. The permitted structure of an XML document can be constrained to match a formal grammar, e.g. using the XML Schema language. XML is frequently used for representing structured data and as such can be used for messages as part of protocol used for accessing a service. XML is defined by a suite of standards produced by the W3C.

Here is a small example of a part of an XML document:

<part number="1976">
  <name>Windscreen Wiper</name>
  <description>The Windscreen wiper
    automatically removes rain
    from your windscreen, if it
    should happen to splash there.
    It has a rubber <ref part="1977">blade</ref>
    which can be ordered separately
    if you need to replace it.
  </description>
</part>

JSON

The JavaScript Object Notation (JSON) is a popular lightweight syntax for data exchange. It includes collections of name/value pairs, and a set of built-in data types, e.g. numbers, booleans, strings and arrays. JSON can be readily parsed into the native data types for languages such as JavaScript and Java.

Here is an example:

{"menu":
  {
    "id": "file",
    "value": "File",
    "popup":
    {
      "menuitem": [
        {"value": "New", "onclick": "CreateNewDoc()"},
        {"value": "Open", "onclick": "OpenDoc()"},
        {"value": "Close", "onclick": "CloseDoc()"}
      ]
    }
  }
}

JSON has been extended to support pointers, linked data and schemas:

JSON Pointer

JSON Pointer defines a syntax for identifying a specific value within a JSON document.

The following examples illustrate the use of JSON pointers in URI fragments for a JSON text document located at http://example.com/example.json

{
  "foo": {
    "bar": [ "element0", "element1" ],
    "inner object": {
      "baz": "qux"
    }
  }
}

http://example.com/example.json#
   Resolves to the object value at the root of the JSON text
   document.

http://example.com/example.json#/foo
   Resolves to the object value of the "foo" member in the root object.

http://example.com/example.json#/foo/inner%20object/baz
   Resolves to the string value "qux", which is the value of the
   "baz" member in the "inner object" member in the "foo" member in
   the root object.

http://example.com/example.json#/foo/bar/0
   Resolves to the string value "element0", which is the first value
   in the "bar" array in the "foo" member in the root object.

with thanks for the examples to Paul C. Bryan.

JSON-RPC

JSON-RPC is a framework for remote procedure calls expressed using JSON. An identifier is used to associate each response with its request. JSON-RPC is often used in conjunction with WebSockets.

Here is an example for a service that adds one to the integer passed to it:

--> { "method": "increment", "params": [65], "id": 7}
<-- { "result": 66, "error": null, "id": 7}

JSON Linked Data

JSON-LD is a set of conventions for representing linked data in JSON. Linked data provides a way to create a network of standards-based, machine-readable data across Web sites. For the Web of Things JSON-LD offers a way of exchanging rich metadata about things and services based upon them.

Here is a small example:

{
  "http://schema.org/name": "Manu Sporny",
  "http://schema.org/url": { "@id": "http://manu.sporny.org/" },
  "http://schema.org/image": { "@id": "http://manu.sporny.org/images/manu.png" }
}

This is equivalent to the following JSON:

{
  "name": "Manu Sporny",
  "homepage": "http://manu.sporny.org/",
  "image": "http://manu.sporny.org/images/manu.png"
}

JSON-LD uses IRIs to identify the name space for each name. @id is a predefined term that indicates that the value is an IRI (essentially a variant of web addresses that allows for non-western characters).

JSON Schema

JSON Schema can be used to define the structure of JSON data for the purposes of validation, documentation, and interaction control.

Here is a small example of a schema:

{
  "title": "Example Schema",
  "type": "object",
  "properties": {
    "givenName": {
      "type": "string"
    },
    "familyName": {
      "type": "string"
    },
    "age": {
      "description": "Age in years",
      "type": "integer",
      "minimum": 0
    }
  },
  "required": ["firstName", "lastName"]
}

and here is an example of a JSON document that conforms to it:

{
  "givenName": "John,
  "familyName": "Smith",
  "age": 42
}

Web Services

This term generally refers the the use of XML for services based upon the Simple Object Access Protocol (SOAP). Services can be specified using the Web Services Description Language (WSDL).

The W3C Web Services Activity page provides links to a suite of Web Services specifications.

Here is an example message:

POST /InStock HTTP/1.1
Host: www.example.org
Content-Type: application/soap+xml; charset=utf-8
Content-Length: 299
SOAPAction: "http://www.w3.org/2003/05/soap-envelope"
 
<?xml version="1.0"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Header>
  </soap:Header>
  <soap:Body>
    <m:GetStockPrice xmlns:m="http://www.example.org/stock">
      <m:StockName>IBM</m:StockName>
    </m:GetStockPrice>
  </soap:Body>
</soap:Envelope>

Some related standards:

  • WSDL -- Web Services Description Language
  • SOAP -- Simple Object Access Protocol
  • UDDI -- Universal Description, Discovery and Integration

Web Services have been widely used for enterprise applications, but in recent years its popularity on the public Internet has declined in favour of lighter weight alternatives involving RESTful HTTP together with JSON.

RESTful HTTP

HTTP provides a popular means for accessing services. In some cases, a device can integrate sensors, actuators and an HTTP server, or a device can act as a gateway, e.g. using ZigBee to communicate with sensors, and exposing their data via HTTP. In many cases, devices will be isolated behind security firewalls. In this situation, services can be exposed by servers in the cloud that hide the details of the communication with the sensors and actuators.

HTTP is a request/response protocol that can be used to implement remote procedure calls. The term representational state transfer (REST) was introduced and defined in 2000 by Roy Fielding, one of the principal authors of the HTTP specification. REST is essentially an architectural style for using HTTP in a scalable way.

The request specifies an HTTP method, and the appropriate choice of method depends on the nature of the service that is invoked. Here are the methods used for RESTful services:

GET
Is used for services that are side effect free, as this allows the response to be cached.
POST
Is used for services that aren't side effect free, e.g. operating an actuator.
PUT
Is used to store a specified resource, e.g. saving an image passed in the request.
PATCH
Is used to make a partial modification to the specified resource.
DELETE
Is used to delete the specified resource.

PUT and DELETE are idempotent, i.e. multiple identical requests should have the same effect as a single request.

HTTP requests and responses consist of a set of named headers and a body. If the body is present, its type is indicated by the Content-Type header. Responses also include a status code, e.g. "200 Okay" or "404 Not found". To specify HTTP based services, it is therefore necessary to define the data passed in the request, the method used, the data returned in the response and the response code.

Some services will result in asynchronous notifications or events. There are several approaches to implementing this with HTTP. One is to pass an HTTP URL to the service for where to deliver the notification. Another is to poll the service (using POST) to see what new notifications are available. A refinement is to have an extended HTTP response where the connection to the client is kept open, and the body is sent as a sequence of messages. The first approach won't work if the recipient for notifications is hidden behind a firewall, as in the firewall will block the incoming connection unless the firewall has been specifically configured to allow it. The latter two approaches are often used for pushing notifications to web page scripts executing in browsers. Service definitions need to specify the details for the approach used for notifications.