Publishing Open Government Data

W3C Working Draft 8 September 2009

This version:
Latest version:
Daniel Bennett <> (Invited Expert)
Adam Harvey (Invited Expert)

Please refer to the glossary for this document.


Every day, governments and government agencies publish more data on the Internet. Sharing this data enables greater transparency; delivers more efficient public services; and encourages greater public and commercial use and re-use of government information. Some governments have even created catalogs or portals (such as data.gov) to make it easy for the public to find and use this data.

Although the reasons may vary, the logistics and practicalities of opening government data are the same. To help governments open and share their data, the W3C eGov Interest Group has developed the following guidelines. These straightforward steps emphasize standards and methodologies to encourage publication of government data, allowing the public to use this data in new and innovative ways.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

The W3C eGovernment Interest Groupis pleased to offer this First Public Working Draft which briefly explains how to publish government data on the Web, based on the our gathered experience. We encourage anyone with additional experience to join the group or send written comment to help with the next version.

The W3C eGovernment Interest Group views this as an evolving document and seeks public feedback on this Working Draft. Please send your comments to public-egov-ig@w3.org (archives). If possible, please offer specific changes to the text that would address your concern. You may also wish to check the Wiki Version of this document and see if the relevant text has already been updated.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. The group does not expect this document to become a W3C Recommendation. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

Straightforward Steps to Publish Government Data

Step 1: The quickest and easiest way to make data available on the Internet is to publish the data in its raw form (e.g., an XML file of polling data from past elections). However, the data should be well-structured. Structure allows others to successfully make automated use of the data. Well-known formats or structures include XML, RDF and CSV. Formats that only allow the data to be seen, rather than extracted (for example, pictures of the data), are not useful and should be avoided.

Step 2: Create an online catalog of the raw data (complete with documentation) so people can discover what has been posted.

These raw datasets should be reliably structured and documented, otherwise their usefulness is negligible.  Most governments already have mechanisms in place to create and store data (e.g., Excel, Word, and other software-specific file formats).

Posting raw data, with an online catalog, is a great starting point, and reflects the next-step evolution of the Internet - "website as fileserver".

Step 3: Make the data both human- and machine-readable:

These steps will help the public to easily find, use, cite and understand the data. The data catalog should explain any rules or regulations that must be followed in the use of the dataset. Also, the data catalog itself is considered "data" and should be published as structured data, so that third parties can extract data about the datasets. Thoroughly document the parts of the web page, using valid XHTML, and choose easily patterned and discoverable URLs for the pages. Also syndicate the data for the catalog (using formats such as RSS) to quickly and easily advertise new datasets upon publication.

How to Publich Government Data, both for people and machines

Open Data Gives Real Power to the People


The ability to identify things by a URI/URL is the foundation of the Internet. Best practices for "open government data" depend on applying the architecture described in Architecture of the World Wide Web, Volume 1 [WEBARCH]. If you give permanent, patterned and/or discoverable URI/URLs to your data, processes and people will be able to find and use it more easily. URI/URLs can be used in databases and metadata as universal, unique identifiers (e.g. by appending a serial number or other internal naming system to a domain: http://www.example.gov/objects/optional-hierarchy/serial12345678.html).

Use internal IDs to identify specific data for machine re-use (e.g., http://www.census.gov/main/www/popclock.html). By using URIs, and incorporating internal links inside large files or documents, people can authoritatively cite the information on their own websites.


Without documentation, data is not very useful. When possible, you should use industry standards such as those based on XML/RDF, as they tend to be self-documenting. Create a web page with a plain language description of the dataset to help search engines find the data, so people can use it. The description should explain how to find the pages and/or files, and describe the contents of the files. At a minimum, document the title, description, publication date and authoritative source for the data. Use clear language, and link to related information to help put the data into context. When possible, also fully document the description of each datum and the data standard and/or data validation (e.g., if datum is a temperature, also explain the related standard, such as Celsius, Fahrenheit or Kelvin). As standards emerge for describing the datasets, take advantage of them. Document search tools and ReSTful methods of obtaining the data when applicable.


Preserving the data we publish is a top concern for governments. How can we ensure that data is findable and can be referenced for as long as people need it?  People may continue to cite older data at the original URI decades after it was initially published.

To maintain the history of a dataset, think about how to integrate new URIs for upgraded and new datasets, and structure your URIs accordingly.  Use versioning of datasets so people can cite and link to present and past versions. New and upgraded datasets can refer back to original datasets.  If needed, provide tools to translate older data.  Carefully document the changes between versions, and embed the version number/indicator within the data if possible. Consider the file format carefully, and opt for open standards that are not software specific.

See the References section for more information on standards and best practices for data preservation.

Expose Interfaces

If you really want to help people discover and explore the data you are publishing, there are some useful W3C standards which can help.  It is possible to make data human-readable by using XSLT for XML- and RDF-based formats. Make web pages machine-readable by embedding semantic information in the HTML, using RDFa and/or Microformats. (Note that search engines can take advantage of embedded semantic information, helping more people to find your data.) If it is too difficult to append XSL or CSS styles to old data, it is still possible to document how to transform the data with a specific or generic XSLT engines (for example, see http://www.xmldatasets.net).

External parties can create new and exciting interfaces that may not be obvious to the data publishers. For that reason, do not compromise the integrity of the data to create flashy interfaces. If you must create an interface, then publish the data separate from the interface, and ensure external parties have direct access to the raw data, so they can build their own interfaces if they wish.

Additionally, since all web documents using (X)HTML, XML and RDF can be used as an object database or ReSTful API, the public can create software, Web applications and mash-ups that use the datasets, and link back to the authoritative version on the government web site. Two W3C standard languages that make that possible are XQuery and SPARQL.  Once your data is published, governments themselves can also create a user-friendly, transparent interface that allows users to play with and refine the information in which they are interested.

Create Standard Names/URIs for All Government Objects

Having a unique identifier for every registered company or school (object) is as important as having information about those companies or schools. In addition to using open industry standards, an effort should be made to catalog all reference-able government entities, officers and objects.  The URIs can then be used within all of the data. This aids in discoverability, improves metadata, and ensures authenticity.

Choosing What Data to Publish

All data that can be shared with the public should be opened for public dissemination. Data should be published in compliance with applicable laws and regulations, and only after addressing issues of security and privacy.

First, publish all data that is already available in other formats (such as print), or already being collected and shared with the public in other ways.  Publish both highly structured data and text documents. Publish laws, regulations and codes, and public documents related to government agency mission, vision, values and goals.

There are many standards for such documents (such as AIIM's emerging Strategy Markup Language (StratML), XBRL and others). Use of standards enables stakeholders to easily discover and provide feedback on items of interest to them. Creating and publishing new documents in various standard formats is easier than fixing after the fact.

As you create your documentation for the data, include and reference XML Schemas, DTDs and taxonomies. Explain how to discover all of the files/documents in a collection. Use syndication standards and tools to announce additions or changes. Provide complete documentation to enable automatically-generated data dictionaries and provide query services, which will make it easier for the public to find your documents and datasets.

Choosing the Right Format for the Data

There are many different data formats, but which will work best with your data?  The primary format for human-readable data is (X)HTML.

Raw data is more likely to be produced using formats customized to the specific data, the tools used, or industry standards. The W3C has pioneered XML and RDF, which allow for excellent manipulation and standardized tool sets. RDF and XML files can be accessed like databases, using SPARQL, XQuery, JavaScript and many other computer languages. When possible, use established open standards, and tools that allow easy and efficient production and publishing of the data. See the References section for a list of current tools. Also keep in mind the power of linked data.

Restrictions on the Use of the Data

Once you've made your data available, be sure to document clearly any legal or regulatory restrictions on the use of the data. In many cases, there are defined standards to insert copyright and/or licensing information right into the data (see metadata standards, such as Dublin Core). It is very important to be clear regarding the expected uses, credits, limitations and responsibilities that particular parties have in regard to working with and providing the data.

Learn More

We still have much to learn about the best ways for governments to open up their data. The W3C eGov Interest Group plans to publish relevant use cases to illustrate working examples of current thinking and best practices. The W3C technologies and approaches described in this document can be implemented quickly, and often with relatively little cost. If you want to learn more, there are a number of communities that can help. If you are a government agency, think about joining the W3C and getting involved in e-Government activities. You can connect with people from other governments who are facing similar issues and sharing their experiences. Even if you don't join the W3C, please let us know how you are getting on. We can help.


Improving Access to Government through Better Use of the Web, S. Accar, J. Alonso, K. Novak, Editors, W3C Group Note, 12 May 2009.
Directive 2003/98/EC on the Re-Use of Public Sector Information, European Parliament and Council, 17 November 2003.
Open Data is Civic Capital: Best Practices for "Open Government Data", J. Tauberer, 20 July 2009
Design Issues: Putting Government Data Online, T. Berners-Lee.
Design Issues: Linked Data, T. Berners-Lee.
How to Publish Linked Data on the Web, C. Bizer, R. Cyganiak, T. Heath, 27 July 2007.
Memorandum for the Heads of Executive Departments and Agencies on Transparency and Open Government, B. Obama.
Intergovernmental Solutions Newsletter: Transparency and Open Government, GSA Office of Citizen Services and Communications, Spring 2009 Issue.
Architecture of the World Wide Web, Volume One, I. Jacobs, N. Walsh, Editors, W3C Recommendation, 15 December 2005.
The Self-Describing Web, N. Walsh, Editor, W3C TAG Finding, 7 February 2009.