Requirements for open markets of services
A flexible and healthy ecosystem for the Web of Things will involve many stakeholders and many roles, for example:
- Designers and manufacturers of sensors, actuators, gateways and security firewalls
- Infrastructure providers for communication technologies
- Providers of cloud hardware and software
- Companies that host markets of services
- Consumers who purchase devices and register them with the Web of Things
- Designers of applications and services, including composite services
- Companies that provide generic or specific services for search & discovery
- Companies that support payments, including currency conversion, and trust brokering
- Security specialists that provide monitoring tools to detect security breaches
Understanding the needs of the different roles will help to clarify the requirements, and in turn, where new standards are needed, and where existing standards are adequate.
Overview of Marketplaces
Consumers are familiar with the proprietary marketplaces for native apps for Apple's iPhone and iPad, and likewise for Android. This suggests the potential for marketplaces for apps and services for the Web of Things.
Stakeholders
The requirements follow from the needs of the stakeholders.
Marketplace Operators
Companies that operate the marketplace.
Cloud Server Providers
Companies that own the hardware that the marketplace platform is built upon.
Businesses
Businesses will want to monitor the apps and services they provide, to place advertisements and monitor their effectiveness, using a variety of analytic tools.
Device Vendors
Companies that provide the sensors, actuators, gateways and other devices that form the basis for the Web of Things.
End Users
This includes people at home, on the move or at work. Imagine that you want to set up a home automation system. You sign into the marketplace and look for reviews of devices and applications that match your needs. You then purchase the devices online or from a bricks and mortar store. You identify the applications you need and install them from the marketplace. You then need to run the application to configure the devices you're using. Some days later, you receive a notification that your application and the services it uses have been updated (to add new features or to fix bugs).
End users will want ways to manage their apps and services, to carry out searches, read reviews, and to arrange payments. Users may also wish to create their own services using high level tools created by developers and operated by businesses.
Developers
Developers will need tools to support the creation and evolution of the apps and services they sell. This suggests the need for ways to create and manage development projects, and a suite of tools for editing, testing and debugging. This includes syntax colouring for editing scripts and structured data, and diagram based editors for service compositions as a graph of pipes and nodes, see e.g. Yahoo Pipes:
Debugging of scripts, that are running in the cloud, requires an API for setting breakpoints, single stepping, accessing named variables etc. Other requirements include the means for developers to search the marketplace for other services that they can use when defining their own services.
Functional Requirements
Identity and Accounts
End-users, businesses and developers will need to identify themselves to marketplaces. It is reasonable to expect that people are required to set up accounts and a means of authenticating themselves.
The Web has largely been based upon user-name and password credentials for authentication. This is known to be insecure as users find it difficult to remember strong passwords, and instead tend to use a weak, but easy to remember password across many websites. Email addresses are common for identifying people. This is bad for privacy as it makes it easy to aggregate information about people's online habits across websites.
Single sign-on schemes avoid the need for users to remember credentials for each website. Examples include:
OpenID
This allows you to sign in with a relying website using an existing account with a popular website such as Google or Yahoo! which plays the role of identity provider. The relying website trusts the identity provider when it comes to authenticating your identity.
Mozilla Persona and BrowserID
This uses your email address as an identity. The browser uses your email address to generate public key pair. The identity provider verifies that you own this email address and generates a digital certificate for this email address and public key. Your email address can then be used to generate a cryptographic proof to the relying website that you are the owner of that email address. Unlike OpenID, there is no need to contact the identity provider when logging into a website. BrowserID is thus said to be more privacy friendly.
- Does BrowserID provide a different identity for each relying website, unlike OpenID where the identity is shared across all websites using the same identity provider?
FIDO Alliance
This approach generates a public key pair for each relying website when registering an account with the site. This key pair is used as a proof of identity on on subsequent occasions. The proof is only generated after the user has proved her presence with a second factor. This could be a physical token (e.g. a usb dongle) or a biometric such as a finger print.
Search
Users, businesses and developers will want to able to search the marketplace. Generic search is based upon text queries and looks for apps and services or service definitions that match the terms present in the query. This can be refined by also looking for terms that are often found together with the given terms, or which are equivalent in meaning, or selected based upon common spelling errors. The search results can be sorted based upon the closeness of the match, the popularity of a given result (e.g. how many people use that service, and its rating by reviewers), and contextual information based upon who is making the query (personalized search). Search engines can also propose results or search topics that may be of interest based upon the aggregated behaviour of other users that are deemed to be similar to the current user in some respect.
Specific search is kind of service that "knows" about a specific search domain or user intent. A generic search engine could use heuristics to identify user intent from the search query. The search engine maintains a registry of specific search services for particular classes of intent. Each intent is associated with an API for invoking the specific search service and obtaining its results. An example is where a user asks for flights to Paris next Tuesday. The search engine then invokes search services passing the destination, the origin (based upon knowledge of the user's location), and the departure date. The results are concise offers for flights and prices. Specific search essentially saves users time and effort by carrying out a domain specific task. This needs to be done very quickly to provide the results back to the search engine for incorporation with other results.
Specific search services are a specialized class of services for the marketplace. The results can include actions, that if selected by the user, carries out further tasks that may take more time than the limits imposed by search engines. This could involve the dynamic creation of new services of a temporary nature.
Social requirements
A successful marketplace will require rich social interaction. This includes the means to publish, read and respond to reviews, to rate people, apps and services, to interact with like minded people interested in particular aspects relating to the marketplace, and to forge communities around particular topics. Today's proprietary marketplaces are by comparison rather sterile with limited scope and a lack of opportunities for enrichment by an ecosystem of third party services.
For social interaction, it is important to keep track of people's status using presence information, and to permit an exchange of chat messages. Some relevant approaches include:
- Extensible Messaging and Presence Protocol XMPP
- Short message services, e.g. Twitter
- Audio and Video chat, e.g. Gmail
- Personal message boards, e.g. Facebook walls, where friends can post their thoughts, views, or criticisms for everyone to see.
Payments
The marketplace needs to provide a means for supporting payments, e.g. for one off, per user or subscriptions. There are many proprietary solutions, but open standards for online payments have still to be developed. W3C is planning a workshop on Web payments for March 2014.
Policies and Capability Tokens
The marketplace will need to handle access control, privacy policies, and provenance. See further details.
Updates
There needs to be a means for automatically distributing security updates to applications and service definitions. It should also be possible to encourage users to install updates for functional improvements.
Data Formats and Protocols
Simple sensors provide basic data formats such as numbers (e.g. a thermometer) and booleans (e.g. a sensor for whether a window is open or closed). More generally, you can have structured data with named fields, arrays and so forth. For audio and video, there is often a need for streaming data. Some relevant approaches include:
- SIP, RTP and RTSP for real time streams
- MPEG-DASH for adaptation to changing network conditions
- WebRTC for peer to peer media streams
Security and Malware
A thriving marketplace will be a target for criminal activity. Countermeasures include security audits of new applications and services, and mechanisms for monitoring for unusual behaviour.
The W3C Web Application Security Working Group is developing content security policies whereby web pages can constrain the APIs that the web page scripts can use. This acts as a defense against attacks, both reducing the size of the attack surface and through enabling the browser to detect attempts to breach the constraints. A similar approach could be used for the scripts used by services to define their behaviour. For more details, see:
A further consideration is to allow services to access native crypto algorithms rather than implementing their own with a consequent risk of security flaws. The W3C WebCrypto Working Group is relevant to this, see:
Applications
Applications essentially run on a device with a user interface, e.g. a notebook computer, a smart phone, tablet or connected TV. Applications can be locally installed onto the device or hosted in the cloud. People will want a means to manage the applications they use, e.g. to search for interesting apps, to read reviews and comparisons, to install apps, and to uninstall them. There also needs to be a means to apply security updates, preferably automatically. Some apps will be free, perhaps funded by an advertising model, whilst others may require a one off payment, a per use payment, or a paid subscription.
Services
Services include metadata and documentation, bindings for specific instances along with the corresponding data, scripts that define the service's behaviour, and additional resources such as graphical icons for use in depicting the service, e.g. when displaying search results, by management tools, or by developer tools. The marketplace could support scripting languages such as JavaScript and Python for developers to define the service's behaviour. Node.js is an event-driven server-side JavaScript environment based on the V8 engine, with a thriving community of developers.
There are many tools for code verification, online editors, JS on Java, etc.
- ST-JS -- Strongly typed JavaScript
- JSLint -- Static code analysis tool for JavaScript
- altJS -- Web coding beyond JavaScript
- Rhino -- open-source implementation of JavaScript written entirely in Java
- Orion -- Open Source Platform For Cloud Based Development
- Ace -- Embeddable code editor written in JavaScript
Service classes versus service instances
There is a clear distinction between developing a class of service and instantiating it in a particular context. For example, a company could specialize in developing services for home security. These can be purchased and bound to particular home security sensors and alarms. The Web of Things platform needs to provide the means for developers to create, market and sell service definitions that others can then instantiate as appropriate to their needs. The marketplace should allow search for both service classes and service instances.
Data flow through services
When services are composed, one service consumes data provided by another. Composite services can be driven lazily on demand by invocations of the service, or eagerly as data is pushed to them by the services they in turn depend on. There is a requirement for being able to securely store data in the cloud to support this. This may be limited for services which produce large amounts of data. The ability to look back in time is important for services that infer events probabilistically on the basis of noisy data. It can also be important for forecasting future data patterns, e.g. traffic flows through a city. Data mining algorithms are applicable to streams of data that are too large to be conveniently stored.
Service descriptions
What are the different kinds of metadata needed for services? Some examples include:
- Information about the developer of a class of service
- Information about the owner of an instance of a service
- The human readable name of the service, and a graphical icon
- A human readable introduction to the service
- Longer, in depth documentation on how to use the service
- Machine readable definitions of the service data sources and sinks
- Machine readable descriptions of the service
- Resource constraints, e.g. how much data can be cached by the run-time platform
- Access control and related kinds of policies
The human readable aspects should be localizable for different languages and cultures.
Services with a UI
In principle, services could expose a user interface rather than a data API. This would be attractive to application developers who could embed such services within their applications. This implies the need for services to be able to include UI resources such as markup, scripts, images, and style sheets.
When it comes to configuring services, and monitoring their use, etc. there is a need for applications that provide the user interface. These applications form an essential part of the service when viewed as a product that can be marketed to end users who are expecting a complete solution for their needs. This is justified further in this [[user story.
Marketplace APIs
The marketplace can use internal APIs to support server side generation of web pages for each kind of stakeholder. Another possibility is to expose APIs that can be invoked from clients, e.g. web page scripts. This could be of advantage when it comes to enabling third parties to compete in providing user interfaces to the marketplace. The benefits would include greater personalization and support across a wider range of devices.
User APIs
These are APIs exposed by the marketplace to registered users (end-users, developers, etc.). This includes the ability to manage their account details, to manage the apps and services they have purchased, and to search for additional apps and services.
Business APIs
These can be used to publish services, configure advertisements, and to manage analytics tools, etc.
Run-time APIs
These are APIs exposed by the marketplace to monitor and manage running services.
Service APIs
These are the APIs that the platform provides for use by services, e.g.to
- Get data that other services connected to this service's data sinks
- Push data to other services (or apps) connected to this service's sources
- Use Web protocols to communicate with sensors, actuators and gateways
- Access the service's private data storage
- Access the service's metadata
- Invoke library functions provided by the platform, e.g. for crypto algorithms
- Invoke special hardware subsystems, e.g. graphics processors
Development APIs
By defining an open interface to the developer backend, third parties can innovate with different approaches to the front end for integrated development environments. One approach is as a locally installed IDE, e.g. based upon Eclipse. Another idea is a browser based IDE. A further idea is to allow for live editing, analogous to google docs, but for software projects. This requires coordination across the clients participating in a session for a particular development project. Coordination can be cloud based, or client based. The latter can use a means to elect a master that determines what proposed changes are accepted. This involves three way merge operations on sequences of mutations on structured information (e.g. scripts, JSON, or XML).