W3C Wiki - User contributions [en]

ConverterToRdf

Rcygania2 — Thu, 07 Feb 2013 20:20:20 GMT

Rcygania2: clean up info around CSV/TSV/Excel/spreadsheets; add SDMX

__NOTOC__
A Converter to RDF is a tool which converts application data from an application-specific format into RDF for use with RDF tools and integration with other data. Converters may be part of a one-time migration effort, or part of a running system which provides a semantic web view of a given application. See also: [[RDFImportersAndAdapters]]

__TOC__

Please add converters as you make them or hear of them.

== Formats ==

in alphabetical order:

=== [[BibTex]] ===

[[BibTex]] is the format for bibliographic references in TeX.

* [http://data.bibbase.org BibBase] transforms BibTeX files (given in a URL) into Linked Data with RDF/XML output support.
** Uses a custom [http://purl.org/bibbase/ontology BibTeX ontology] but provides a table of mappings to other ontologies
** Also provides [http://bibbase.org HTML interface] and RSS feed
* [http://simile.mit.edu/repository/RDFizers/bibtex2rdf/ bibtex2rdf] transforms BibTEX files into RDF/XML. (Simile)
* [http://www.l3s.de/~siberski/bibtex2rdf/ bibtex2rdf] - A configurable BibTeX to RDF Converter by Wolf Siberski.
* [http://www.cs.vu.nl/%7Emcaklein/bib2rdf/ An online service] set up at the Vrije Universiteit in Amsterdam, the Netherlands, following the [http://ontoweb.aifb.uni-karlsruhe.de/ OntoWeb portal] vocabulary. The [http://www.cs.vu.nl/%7Emcaklein/bib2rdf/bib2rdf perl source] can also be downloaded.
* [http://www.aifb.uni-karlsruhe.de/WBS/pha/bib/index.html Java BibTeX-To-RDF Converter] based on the [http://ontobroker.semanticweb.org/ontos/swrc.html SWRC] terminology.

=== Bittorrent ===

* http://www.inf.unideb.hu/~jeszy/rdfizers is alas now 404 (in 2007). This was a link from RDFizers but may be incorrect.

=== CSV (Comma-Separated Values) ===

See also: [[#Flat files|Flat Files]] and [[#TSV|TSV]]

* An [http://lab.linkeddata.deri.ie/2010/grefine-rdf-extension/ RDF Extension] is available for [http://code.google.com/p/google-refine/ Google Refine]. It can convert Excel, CSV, and other tabular data to RDF. The schema mapping can be defined in a graphical UI.
* [http://rdf123.umbc.edu/ RDF123] has Windows and Linux applications to download, a Java application and servlet.
* [http://xlwrap.sourceforge.net XLWrap] wraps spreadsheets (including cross tables) to arbitrary RDF graphs; supports Excel/OpenDocument/CSV streamed processing, local/HTTP loading, expressions similar to Excel/OpenOffice Calc, custom functions, usage via API or SPARQL endpoint
* [http://purl.org/twc/id/software/csv2rdf4lod csv2rdf4lod] uses declarative RDF enhancement parameters to specify how to transform tabular data into well-structured, well-connected RDF. The tool uses identifiers for ''source'' organization, ''dataset'', and ''version'' to establish default namespaces for all URIs created and provides VoID and provenance metadata as part of the conversion output.
* [http://github.com/cygri/tarql Tarql] is a command-line application that converts CSV to RDF with a user-defined mapping. The mapping is written in standard SPARQL 1.1.

=== Debian ===

The package information in Debian and similar systems (Ubuntu, Fink, etc), with its general usefulness and its graph-like nature, is a clear candidate for conversion to RDF.

See [http://blog.drinsama.de/erich/en/xml/2007011204-rdf-representation-of-packages VitaVoni blog] about this.

* [http://www.w3.org/2000/10/swap/util/fink2n3.py finkn3.py] Takes Fink (OS-X port of Debian packaging) dependencies and converts to to RDF/N3. (SWAP) No idea whether this would be a quick hack to export debian data.
* [http://github.com/nbarrientos/steamy STEAMY] converts Debian packages to RDF.

=== Email (RFC822 headers) ===

* [http://simile.mit.edu/repository/RDFizers/email2rdf/ email2rdf ] transforms email mbox files into RDF/XML. (Simile)
* [http://www.w3.org/2000/04/maillog2rdf/aboutMsg.py aboutMsg.py] converts email metadata to RDF. (SWAP)
* [http://swaml.berlios.de/ SWAML] transforms a mailing list into RDF/XML and XHTML+RDFa using [[SIOC]].
** And [http://linkedmarkmail.wikier.org/ LinkedMarkMail] live transforms into RDF/XML the mailing lists' archives indexed by [http://markmail.org/ MarkMail].
* [http://search.cpan.org/dist/Email-MIME-XMTP/ Email::MIME::XMTP] Perl extension to read and write [http://www.openhealth.org/xmtp/ XMTP]
* [http://aperture.sourceforge.net aperture.sf.net] IMAP crawler

There are others in this vein which run over IMAP or mailbox files.@@

=== Excel ===

* Cambridge Semantics' [http://www.cambridgesemantics.com/products/anzo_for_excel Anzo for Excel] extracts RDF data from Excel spreadsheets while keeping the spreadsheet in-sync with the underlying data as things change
* [http://xlwrap.sourceforge.net XLWrap] wraps spreadsheets (including cross tables) to arbitrary RDF graphs; supports Excel/OpenDocument/CSV streamed processing, local/HTTP loading, expressions similar to Excel/OpenOffice Calc, custom functions, usage via API or SPARQL endpoint
* [http://www.topbraidcomposer.com TopBraid Composer] can convert Excel spreadsheets into instances of an RDF schema.
* [http://github.com/Data2Semantics/TabLinker TabLinker] can convert non-standard Excel spreadsheets to the Data Cube vocabulary, e.g. Excel files that contain hierarchical information in row and column headers etc.
* [http://oeg-dev.dia.fi.upm.es/nor2o/ NOR2O] can convert excel to Scovo and Data Cube Vocabulary.
* [http://www.mindswap.org/%7Erreck/excel2rdf.shtml Esxcel2rdf] is a Microsoft Windows program (exe) that converts Excel files into valid RDF. It has been tested on Windows 98, and Windows 2000 Professional. ([[MindSwap]]) Export can be done via comma- or tab- separated values. See Flat Files above.
* [http://aperture.sourceforge.net aperture.sf.net] includes Java crawler for Excel and open document. Does only extract plaintext and basic metadata, though.
* [http://www.tao-project.eu/researchanddevelopment/demosanddownloads/RDBToOnto.html RDBToOnto], see description below under SQL section.

=== EXIF ===

See JPEG.

=== File Systems ===

* [http://www.cs.univie.ac.at/publication.php?pid=5750 TripFS] exposes an entire file system as linked data, tracks changes, and links files to external data sources.

=== Flickr data ===

* Dave Becketts [http://librdf.org/flickcurl/ flickurl] library can access Flickr information (including machine tags) and convert it to RDF

=== Flat files ===

See also: [[#CSV|CSV]] and [[#TSV|TSV]]

* [http://simile.mit.edu/repository/RDFizers/flat2rdf/ flat2rdf] converts classic unix text database files, like /etc/passwd, into RDF/N3 (Simile)

=== GPS ===

* [http://www.hackdiary.com/archives/000040.html garmin2rdf.py] Reads a Garmin GOPS receiver, dumping the contents in RDF/XML. (Matt Biddulph)
* [http://www.w3.org/2000/10/swap/pim/fromGarmin.py fromGarmin.py] Downloads GPS data from a Garmin on a serial link to an RDF/N3 file. (SWAP)

=== iCalendar ===

iCalendar is an IETF standard for calendar (event and to-do list) data.
Icalendar files typically are stored with a .ics extension.

* [http://www.w3.org/2002/12/cal/fromIcal.py fromIcal.py] converts iCalendar form to RDF
* [http://www.w3.org/2000/10/swap/pim/toIcal.py toIcal.py] converts RDF back into iCalendar.
* [http://aperture.sourceforge.net aperture.sf.net] java converter for iCalendar included
* [http://torrez.us/ics2rdf/] iCal to RDF Service

=== Java bytecode ===

* [http://simile.mit.edu/repository/RDFizers/java2rdf/ java2rdf] scans [http://java.sun.com/ java] bytecode for method calls and creates a description of the dependencies between classes and the package/archive encoded in RDF/N3. (Simile)

=== Javadoc ===

* [http://simile.mit.edu/repository/RDFizers/javadoc2rdf/ javadoc2rdf] is a doclet that makes javadoc output metadata about your code (structure of the classes, methods, comments, etc.) encoded in RDF/N3. (Simile)

=== Issue tracking: [http://www.atlassian.com/software/jira/ Jira] ===

* [http://simile.mit.edu/repository/RDFizers/jira2rdf/ jira2rdf] transforms Atlassian Jira's events about bug reports and issue tracking into RDF/N3.

=== JPEG ===

The metadata within JPEG photo is encoded in the EXIF standard.

* [http://simile.mit.edu/repository/RDFizers/jpeg2rdf/ jpeg2rdf] scans a folder for JPEG files, parses the EXIF and IPCT metadata found in those files and dumps an RDF/N3 representation of it into a file. (Simile)
* [http://www.w3.org/2000/10/swap/pim/jhead/ An adapted version of jhead] extracts RDF data form the EXIT encoded in JPEG files within a directory. Generates RDF/N3. (SWAP)

=== LDIF ===

This is format used for contact information in LDAP server system. It is for example exported by Thunderbird's address-book.

* [http://www.w3.org/2000/10/swap/pim/ldif2n3.py ldif2n3.py] Very incomplete, but useful. Generates foaf. Hides email addresses by hashing in the FOAF style if -m command flag is given. (SWAP)

=== Makefile ===

The unix Makefile syntax expresses dependencies between files in a software build.

* [http://www.w3.org/2000/10/swap/util/make2n3.py make2n3.py] Convert the makefiles in several directories in RDF and merge them to get the big picture. (SWAP)

=== MARC ===

* [http://simile.mit.edu/repository/RDFizers/marcmods2rdf/ marcmods2rdf]
transforms [http://www.loc.gov/marc/ MARC] records from Z39.2 format into [http://www.loc.gov/standards/mods/ MODS] and then from MODS to an RDF representation of MODS.
* [http://www.oeg-upm.net/index.php/en/technologies/228-marimba MARiMbA] is a command-line tool, designed with librarians in mind, to transform MARC (MAchine-Readable Cataloging) records to RDF, following Linked Data best practices.

=== Meteographical ===
* [http://inamidst.com/sw/meteo/ Meteo] is UK weather forecast data in RDF, extracted from NOAA's public domain GRIB files. Example: [http://inamidst.com/sw/meteo/rdf/London London].

=== Microformats ===
* [http://www/incubator.apache.org/any23/ Apache Any23] is a Java library web service and command line tool for parsing multiple document formats and extracting structured data in RDF format from a variety of Web documents. It is used by [http://sindice.com/ Sindice.com]. The microformats support is [http://sindice.com/developers/microformat detailed in the Sindice.com documentation].

=== Multimedia ===

Following the [http://en.wikipedia.org/wiki/Don't_repeat_yourself DRY principle], a pointer to tools in the realm of multimedia (origin: [http://www.w3.org/2005/Incubator/mmsem MMSEM-XG]):

* [http://www.w3.org/2005/Incubator/mmsem/wiki/Tools_and_Resources Multimedia Semantics Tools]

=== OAI-PMH ===

* [http://simile.mit.edu/repository/RDFizers/oai2rdf/ oai2rdf] harvests an [http://www.openarchives.org/OAI/openarchivesprotocol.html OAI-PMH] repository and transforms the captured metadata in an RDF representation thru pluggable XSLT stylesheets.

=== Outlook ===

Microsoft Outlook contains contact and event data, and so on in a proprietary format.

* [http://www.w3.org/2000/10/swap/pim/lookout.py Lookout.py] convers the Microsoft Outlook calendar and address format into RDF. (SWAP)
* [http://aperture.sourceforge.net aperture.sf.net] includes Java crawler for MS Outlook

=== Open Financial Exchange (OFX) ===

[http://www.ofx.net/ OFX] is the format for downloaded bank statements and other financial information.
There are various levels of OFX, the early ones being HTTP headers followed by SGML, the later ones being HTTP-like headers followed by XML.

* [http://www.w3.org/2000/10/swap/pim/financial/OFX-to-n3.py OFX-to-n3.y] converts OFX format to RDF/N3. The conversion is only syntactic. The OFX modeling is pretty well thought out, so taking it as defining an RDF ontology seems to make sense. Rules can then be used to define mapping into your favorite ontology.

=== Open [[CourseWare]] ===

* [http://simile.mit.edu/repository/RDFizers/ocw2rdf/ ocw2rdf] harvests metadata from the MIT [http://ocw.mit.edu/ OpenCourseWare] web site and transforms it into an RDF representation of [http://meta.wikimedia.org/wiki/IEEE_LOM IEEE LOM].

=== Palm OS ===

* [http://dev.w3.org/cvsweb/2001/palmagent Palmagent] converts the calendar format of PalmOS into RDF. (SWAP)

=== plist ===

The Apple OS-X property list (.plist) filetype is an XML fromat for arbitrary structured data.
Numeric keys are used as local IDs. OS X applications store many kinds uf data in these files, including configuration data, iPhoto almum and photo data, iTunes metadata, and so on.

To convert plists well, added information is necessary, such as a namespace for the properties.

[http://dev.w3.org/cvsweb/2000/10/swap/util/plist2rdf.xsl plist2rdf.xsl] is an XSLT script to convert a plist file into RDF/XML. It does not add namespaces to the exported data.

=== Quicken Interchange Format (QIF) ===

* [http://www.w3.org/2000/10/swap/pim/qif2n3.py qif2n3.py] Takes Quicken interchange format and converts to to RDF/N3. (SWAP)

=== Quick and Dirty CSV to RDF Converter (QUIDICRC) ===

* [http://www.mindswap.org/~anant/quidicrc/ quidicrc] A perl script for rapidly transferring csv to RDF with some translation in the middle. (not actively being maintained, available open source -- SWAP)

=== Random ===

Seriously.

* [http://simile.mit.edu/repository/RDFizers/random2rdf/ random2rdf] generates synthetic random graphs encoded in RDF/N3.

=== SDMX ===

[http://sdmx.org/ SDMX] is an XML-based exchange format for statistical data and metadata, used by major statistics-producing organizations such as Eurostat, the World Bank, OECD, and the IMF.

* [https://github.com/csarven/sdmx-to-qb SDMX to QB] is an XSLT-based converter that turns SDMX data sets and data structure definitions into RDF, using the Data Cube Vocabulary.

=== Spreadsheet ===

See [[#CSV]] and [[#Excel]].

=== SQL ===

SQL databases are rich stores of relational data ideal for export as RDF.
Conference tracks and many papers cover this subject from different angles. See also: [[RdfAndSql]]

* [http://d2rq.org/ D2RQ] provides a mapping from a SQL server (tested with several brands), producing both linked virtual RDF data files and a SPARQL service. Uses a configuration file in Turtle. (DERI and FU Berlin)
* [http://www.w3.org/2000/10/swap/dbork/dbview.py dbview.py] provides a mapping from a SQL server (tested with mySQL), producing linked virtual RDF data files. Uses a configuration file in N3. (SWAP)
* [[VirtuosoUniversalServer|OpenLink Virtuoso]]'s [http://virtuoso.openlinksw.com/wiki/main/Main/VOSSQLRDF declarative N3/Turtle based Metaschema Language] enables the creation of RDF Instance Data for associated RDF Ontologies via RDF VIEWs of ODBC, JDBC, ADO.NET, and OLE-DB accessible SQL Data. It is important to note that these VIEWs also apply to Native Virtuoso Data and/or Heterogeneous Data from other Web Services, HTTP/WebDAV, NNTP, and other Data Sources known to Virtuoso. This is an enhancement of the traditional SQL VIEW concept than enables multiple use of the same base SQL Data from a variety of data access points.
* [http://triplify.org Triplify] is a small plugin for Web applications, which reveals the semantic structures encoded in relational databases by making database content available as RDF, JSON or Linked Data.
* [http://www.tao-project.eu/researchanddevelopment/demosanddownloads/RDBToOnto.html RDBToOnto] is a full-fledged conversion tool that can produce accurate RDF/OWL models from various types of relational databases and Excel spreadsheets. The conversion is fully automated while various parameters can be set through the user interface to refine the resulting models (e.g., derivation of rich class hierarchies, proper naming of instances, database optimization before conversion, etc).
* [https://github.com/jpcik/morph morph] or [https://github.com/boricles/morph morph] implement [http://www.w3.org/2001/sw/rdb2rdf/r2rml/ R2RML] and perform a transformation from RDB to RDF.

Some RDF Triple stores are implemented using SQL databases, but that is not covered here.

=== Subversion ===

[http://subversion.tigris.org/ Subversion] is a code-management system.

* [http://simile.mit.edu/repository/RDFizers/svn2rdf/ svn2rdf] A pair of scripts; one can be used in a post-commit subversion hook to generate RDF/N3 with each commit, the other on a working copy. (Simile)

=== TSV (Tab-Separated Values) ===

See also: [[#Flat files|Flat Files]] and [[#CSV|CSV]]

* [http://www.w3.org/2000/10/swap/tab2n3.py tab2n3.py] Takes Tab-separated text (as typically output by all kinds of things including Microsoft Output and Spreadsheets) and converts it to N3, using the column headings to generate property URIs. (SWAP)
* [http://www.topbraidcomposer.com TopBraid Composer] can convert tab-separated spreadsheet files into an RDF/OWL class with corresponding properties and instances.
* XLWrap, [http://xlwrap.sourceforge.net] wraps CSV files (and spreadsheets) to arbitrary RDF graphs; supports local/HTTP loading, expressions similar to Excel/OpenOffice Calc, custom functions, usage via API or SPARQL endpoint

=== Talis SW Format Converter ===

* [http://convert.test.talis.com/ Talis' converter], convert from various format to various formats (including RDF->RDF with various serializations, RDF->HTML, etc)

=== UML ===

* [http://www.topbraidcomposer.com TopBraid Composer] can convert UML Class Diagrams (XMI format) into RDF/OWL models.
* [http://eulergui.sourceforge.net/ EulerGUI] is a lightweight IDE that translates on the fly UML and eCore XMI into N3. Moreover there are N3 rules to convert UML to OWL.

=== VCARD, Addressbook, … ===

VACRD is a standard for interchange of contact data, such as business cards and address books.

[http://www.w3.org/TR/vcard-rdf "Representing vCard Objects in RDF/XML"] is a W3C note defining an [http://www.w3.org/2006/vcard/ns ontology] for VCARD. FOAF is widely used ontology covering some of the domain.

* [http://www.holygoat.co.uk/applications/address-book-foaf/projects/ab/ab.py code to convert your Apple Addressbook into FOAF file] (Richard Newman)
* [http://people.no-distance.net/ol/software/ab-foaf/ ab-foaf] does the same.
* [http://search.cpan.org/dist/XML-FOAFKnows-FromvCard/ XML::FOAFKnows::FromvCard], Perl extension to create FOAF dumps from vCards. Does not attempt to create a full model, just foaf:knows. It also has some privacy features. In addition to the module, which conforms with the Formatter API specification, comes with a command-line tool.

=== Weather ===

* [http://simile.mit.edu/repository/RDFizers/weather2rdf/ weather2rdf] Given a US city or ZIP code, retrieves weather report data from weather.com and returns it in RDF. (Simile)

=== XML ===

* '''GRDDL:''' Any XML files can be marked up with pointers to XSLT files which convert them to RDF. The standard for this is [http://www.w3.org/TR/grddl/ GRDDL]. A GRDDL pointer can even be put in an XML schema, so that automatically all XML documents written to that schema will have a defined RDF mapping which any GRDDL-aware processor will benefit from. Several XSLT conversion transformations can be found linked from [[MicroModels]]
* [http://kwarc.info/projects/krextor/ Krextor] is a framework for extracting RDF in various notations from various XML languages and can easily be extended for additional input languages. Support for RDFa and some mathematical markup languages is built in. The implementation is done in XSLT, with a command-line frontend and a Java wrapper.
* [http://www.topbraidcomposer.com TopBraid Composer] can convert XML Schema (and their XML instance files) into RDF/OWL models.
* Rhizomik [http://rhizomik.net/redefer/ ReDeFer] includes XSD2OWL and XML2RDF plus MPEG-7 to RDF (all XSLT-based)
* '''XHTML:''' Convert ''existing'' pages to RDF. For example, see [[HtmlToRdf]].
* [http://www.dblab.ntua.gr/~bikakis/SPARQL2XQuery.html SPARQL2XQuery] The SPARQL2XQuery Framework provides mechanisms for: (a) Query translation (SPARQL to XQuery) (b) Mapping specification & generation (Ontology to XML Schema) (c) Schema transformation (XML Schema to OWL) and (d) Data Transformation (XML to RDF and vice versa)

=== XMP ===

[http://www.adobe.com/products/xmp/ XMP] is an Adobe-sponsored specification for putting RDF metadata in virtually any form of file, including binary formats. XMP metadata is RDF data in fact, but it has to be extracted from the file.

* [http://www.inf.unideb.hu/~jeszy/xmp/ xmpextractor] extracts XMP data. ([http://www.inf.unideb.hu/~jeszy/ Jeszenszky Péter])
* [http://dev.w3.org/cvsweb/~checkout~/2004/PythonLib-IH/xmp.py A python script to extract XMP]. There is also a service to do that on-line, see [http://www.ivan-herman.net/WebLog/WorkRelated/SemanticWeb/xmpextract.html separate page]

== Frameworks ==

The following are general tools which provide conversion from many formats.

=== AnnoCultor ===

[http://annocultor.eu/ AnnoCultor] was built during several years of practical work on porting various datasets to RDF. It allows converting data from the following data sources:
* databases via SQL and JDBC;
* XML files, also in batch;
* RDF files,
* Solr servers,
* custom formats, via format-specific parsers written in Java.

AnnoCultor is specifically suited for the situations where XSLT is not sufficient.

It comes with built-in converters for Geonames and Getty vocabularies (AAT, ULAN, TGN), that are ready to use.
Several additional specific converters illustrate advanced use: converters for collections of Louvre and Joconde,
Institute Collection Netherlands, Dutch Museum of Asian Ceramics, Tropenmuseum Amsterdam.

As part of conversion, AnnoCultor can semantically tag (enrich) data with links to various vocabularies, with advanced customised disambiguation and term processing possibilities.
These vocabularies should be represented in RDF or SKOS to be imported via SPARQL queries.
AnnoCultor comes with built-in tagging with Geonames and a custom time ontology.

AnnoCultor is written in Java, but conversion rules are written in XML. They are extendible with either small Java snippets, or custom rules implementions in Java.
AnnoCultor has been practically used with datasets ranging from a few records to more than ten millions, containing up to dozens fields each.

=== Apache Any23 ===

[http://incubator.apache.org/any23/ Apache Any23] is a Java library web service and command line tool for parsing multiple document formats and extracting structured data in RDF format from a variety of Web documents. Currently it supports the following input formats:

* RDF/XML, Turtle, Notation 3
* RDFa
* Microformats: Adr, Geo, hCalendar, hCard, hListing, hResume, hReview, License, XFN and Species

Apache Any23 is used in major Web of Data applications such as [http://sindice.com/ sindice.com] and [http://sig.ma/ sig.ma].

=== Aperture ===

* [http://aperture.sourceforge.net/ Aperture] is a project written in Java gathering RDF extractors for many formats, mentioned in the list above.
Aperture supports crawling, making it not a converter but a framework to crawl updates of data (like rsync).

=== [[PiggyBank]] ===

* [http://simile.mit.edu/piggy-bank/ Piggy-bank] is a [http://simile.mit.edu/ Simile] project which allows the Firefox-based clent to automatically load "[http://simile.mit.edu/RDFizers/ RDFizers]", javascript-based converters to RDF.
Piggy-bank associates given scarping scripts with given web sites. (How?)

=== Triplr ===

[http://triplr.org/ Triplr] is a general “Stuff in, triples out” system by Dave Beckett. Triplr handles GRDDL, RSS, Atom, and other formats.

=== Virtuoso Sponger ===

[[OpenLinkSoftware|OpenLink Software]] via the "[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger Sponger]" component of [[VirtuosoUniversalServer|Virtuoso]]'s SPARQL Processor and Proxy Web Service (used by default by [[OpenLinkDataExplorer| OpenLink Data Explorer]]) provides RDFization for:
* RDFa
* GRDDL
* Amazon Web Services
* eBay Web Services
* Freebase Web Services
* Facebook Web Services
* Yahoo! Finance
* XBRL Instance documents
* DOI (includes a custom resolver for HTTP)
* OAI
* RSS/Atom Feeds
* Digital Music Files (various formats via ID3 Tags)
* Image Files
* vCard
* iCalendar
* Microformats - hCard, hCalendar
* HR-XML Resumes
* Flickr
* Del.icio.us
* Bugzilla
* ODBC or JDBC accessible SQL Data
* [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSources Many others]

= Notes =

Historically, this list was made from a lists of [http://simile.mit.edu/RDFizers/ RDFizers] and
[http://www.w3.org/2002/Talks/09-lcs-sweb-tbl/slide31-1.html SWAP converters].
It has grown significantly from community input since then.

This should be in a data format like Semantic Media Wiki or in N3 -- TimBL

> Would there an advantage to have this kind of list in an RDF file specifically to make queries on it. Maybe if we add a format on how to declare it here, we could create a converter to RDF. -- [[KarlDubost]]

> The task force [http://esw.w3.org/topic/SweoIG/TaskForces/InfoGathering InfoGathering] from SWEO works on such a vocabulary, if you want to rewrite this list using this vocab, look here: [http://esw.w3.org/topic/SweoIG/TaskForces/InfoGathering/DataVocabulary DataVocabulary] or contact me -- [[LeoSauermann]] on 22.1.2007
----

----
[[Category:SwTools]] [[Category:SwTools]]

TPAC2012

Rcygania2 — Wed, 31 Oct 2012 14:28:35 GMT

Rcygania2: /* Session Grid */

<div class="h-event vevent">
[http://www.w3.org/2012/10/TPAC/ TPAC 2012] (a [[Events|W3C event]]) takes place 29 Oct to 2 Nov 2012 in Lyon, France.
</div>

<div class="h-event vevent">
On Wednesday, 31 October, we hold a "Plenary Day." As we did for [[TPAC2011]], we will organize most of the day as "camp-style" breakout sessions. The [[TPAC2012-Committee]] is [http://www.w3.org/wiki/TPAC2012-Planning planning] the day. We invite you to add to or comment on [[TPAC2012/SessionIdeas]] . The people at the meeting that day will build the agenda, drawing from ideas socialized in advance and new ideas proposed the day of the meeting. 
</div>

Questions? See the [http://www.w3.org/wiki/TPAC2012/FAQ FAQ].

==Plenary Day Schedule==

Draft schedule for Wednesday, 31-October 2012. "*" means "in plenary"

* 08:30-09:00: W3C Update*, Jeff Jaffe ([http://www.w3.org/2012/Talks/jj-tpac2012-plenary.pptx slides])
* 09:00-09:30: New and upcoming work* (wgs, cgs, bgs, workshops).
** Web Performance Workshop, Philippe Le Hégaret ([http://www.w3.org/2012/Talks/1031-webperf-plh/ slides])
** SVG 2, WOFF 2, Chris Lilley ([http://www.w3.org/Talks/2012/ChrisLilley-TPAC-NewWork/cover-basic.svg slides])
** Pointer Events, Doug Schepers
** Web Platform Docs, Doug Schepers
** Publishing, Ivan Herman (no slides)
** TAG, Jeni Tennison ([http://www.w3.org/2012/Talks/jt-tpac2012-plenary/assets/fallback/index.html slides])
* 09:30-10:30: Agenda building*, Tantek Çelik and Ian Jacobs. See [http://www.w3.org/wiki/TPAC2012/FAQ#What_are_the_breakout_room_sizes.3F room availability]
* 10:30-11:00: Break
* 11:00-11:50: Breakouts
* 12:00-13:20: Lunch
* 13:30-14:20: Breakouts
* 14:30-15:20: Breakouts
* 15:20-16:00: Break
* 16:00-16:50: Breakouts
* 17:00-17:30: [http://www.w3.org/wiki/TPAC2012/FAQ#How_do_we_share_our_breakout_discussion_with_others.3F Sharing results]* (Ian Jacobs to chair)
* 17:30-17:40: Wrap-up*, thanks, TPAC 2013, Jeff Jaffe
* 18h30-21h30: Dinner

===Ideas===

<ul class="show_items">
* PGP key signing area (e.g., before dinner)?
* Ask groups for upcoming meeting info for calendar.
</ul>

== Session Grid ==

{| border="1" cellpadding="5" cellspacing="0" width="80%"
|-
! time
! PASTEUR
! RHONE 1
! RHONE 2
! RHONE 3
! RHONE 4
! SAINT CLAIR 1
! SAINT CLAIR 2
! SAINT CLAIR 3a
! SAINT CLAIR 3b
! SAINT CLAIR 4
|-
! 11:00-11:50
| [http://www.w3.org/wiki/index.php?title=Digital_Publishing Digital Publishing] (Ivan Herman) [http://www.w3.org/2012/10/31-digpub-irc #digpub]
| [[TPAC2012/Community Groups]] (Ian Jacobs) [http://www.w3.org/2012/10/31-community-irc #community]
| URL (Anne van Kesteren), [http://www.w3.org/2012/10/31-urlstandard-irc #urlstandard]
| | [http://www.w3.org/wiki/index.php?title=Social_Web Social Web] (Ann Bassetti, Wendy Seltzer) [http://www.w3.org/2012/10/31-social-irc #social]
| Stereoscope/3D Web (Dong-Yong Lee) [http://www.w3.org/2012/10/31-3dweb-irc #3dweb]
| Web Quality BP (Elie Sloem, Aurelia Levy) #
| [[TPAC2012/OfflineApps Offline Apps]] (Ashok Malhotra) [http://www.w3.org/2012/10/31-offline-irc #offline]
| Testing (Philippe Le Hégaret) #testing [http://www.w3.org/2012/10/31-testing-minutes.html IRC log/minutes]
| Web Intents (Claes) [http://www.w3.org/2012/10/31-intents-irc #intents]. See wiki [http://www.w3.org/wiki/index.php?title=TPAC2012/session-WebIntents-local-services Session on Web Intents and Web Intents for local services].
| .
|-
! 13:30-14:20
| Responsive Images (Marcos Caceres), [http://www.w3.org/2012/10/31-respimg-irc #respimg]
| Extended DRM (Toru and Kiyoshi), [http://www.w3.org/2012/10/31-edrm-irc #edrm]
| Digital Marketing (Karen Myers), [http://www.w3.org/2012/10/31-digitalm-irc #digitalm]
| Identity and Privacy on the Web (Henry Story), [http://www.w3.org/2012/10/31-identity-irc #identity]
| Restyling W3C Specs: boilerplate content and styling (fantasai, darobin, divya), [http://www.w3.org/2012/10/31-restyle-minutes #restyle]
| Intro of Sys Apps WG (Wonsuk), [http://www.w3.org/2012/10/31-sysapps-irc #sysapps]
| [[TPAC2012/agile W3C Process Agility]] (Steve Zilles), [http://www.w3.org/2012/10/31-agile-irc #agile]
| [http://adobe.github.com/web-platform/presentations/testtwf-tpac2012/#/ Test the Web Forward] (Rebecca Hauck, Alan Stearns), [http://www.w3.org/2012/10/31-testtwf-irc #testtwf]
| XML Memory/Change tracking (Steven Pemberton), [http://www.w3.org/2012/10/31-xmlmemory-minutes #xmlmemory]
| Linked Data/Gov Publishing (Bernadette Hyland) [http://www.w3.org/2012/10/31-linkeddata-irc #linkeddata]
|-
! 14:30-15:20
| How to be a good chair (Charles McCathieNevile), [http://www.w3.org/2012/10/31-chair-irc #chair]
| Performance in the real world (Paul Bakaus, Tobie Langel), [http://www.w3.org/2012/10/31-webperf-irc #webperf]
| [http://www.w3.org/2012/10/31-ld-dev-minutes.html Linked Data for Web Developers] (Francois Daoust), [http://www.w3.org/2012/10/31-ld-dev-irc #ld-dev]
| Browser Fingerprinting a lost cause? (Brad Hill), [http://www.w3.org/2012/10/31-fingerprint-irc #fingerprint]
| Social Web Workshop early planning (Ann Bassetti), [http://www.w3.org/2012/10/31-socwork-irc #socwork]
| APIs for trusted web apps (Dave Raggett), [http://www.w3.org/2012/10/31-sysapps-irc #sysapps]
| Web Platform Docs (Doug Schepers), [http://www.w3.org/2012/10/31-webplatform-irc #webplatform]
| Test Infrastructure (JGraham), [http://www.w3.org/2012/10/31-github-irc #github]
| Media and Disaster (Yosuke Funahashi), [http://www.w3.org/2012/10/31-disaster-irc #disaster]
| IndieUI (Rich Schwerdtfeger, Janina Sajka), [http://www.w3.org/2012/10/31-indie-ui-irc #indie-ui]
|-
! 16:00-16:50
| [http://www.w3.org/wiki/TPAC2012/SessionIdeas#Making_the_multilingual_web_work Multilingual Web] (Felix Sasaki see [http://www.w3.org/wiki/File:Mlw-tpac2012-slides-fsasaki.pdf session slides]), [http://www.w3.org/2012/10/31-mlw-irc #mlw]
| [http://www.w3.org/wiki/TPAC2012/SessionIdeas#End-to-end_W3C_APIs End to end W3C APIs] (Alexandre Morgaut), [http://www.w3.org/2012/10/31-jseverywhere-irc #jseverywhere]
| [[TPAC2012/Modern Guide]] (Ian Jacobs), [http://www.w3.org/2012/10/31-guide-irc #guide]
| Catch up on DNT (Nick Doty), [http://www.w3.org/2012/10/31-dnt-irc #dnt]
| [http://www.w3.org/wiki/TPAC2012/tvapis TV APIs] (JC Verdie), [http://www.w3.org/2012/10/31-tvapis-irc #tvapis]
| Smarter WebApps for Smarter Phones (Bryan Sullivan), [http://www.w3.org/2012/10/31-smarterwebapps-irc #smarterwebapps]
| Declarative 3D as Polyfill (Johannes Behr), [http://www.w3.org/2012/10/31-dec3d-irc #dec3d]
| Gov Open Data (Hadley Beeman), [http://www.w3.org/2012/10/31-opendata-irc #opendata]
| Future of W3C publishing process (Philippe Le Hégaret), [http://www.w3.org/2012/10/31-tr-irc #tr]
| Speech and HTML (Debbie Dahl), [http://www.w3.org/2012/10/31-speech-irc #speech]
|}

OTHER SESSIONS OFF THE GRID:

* 16:00-16:50 [http://microformats.org/wiki/microformats2 microformats2] (Tantek Celik, Ted O'Connor), Forum 2, [irc://irc.freenode.net/microformats #microformats]

== Feedback ==
Please feel free to provide feedback on TPAC 2012:

<ul class="show_items">
* [https://www.w3.org/2002/09/wbs/35125/tpac2012-feedback/ Official TPAC 2012 feedback form]
* [[TPAC2012/feedback|TPAC2012 feedback wiki page]]
</ul>

TPAC2012/Community Groups

Rcygania2 — Wed, 31 Oct 2012 10:10:20 GMT

Rcygania2:

Community and Business Groups. A breakout session during Plenary Day at TPAC 2012.

* [http://www.w3.org/2012/Talks/ij-tpac2012-plenary/ Ian Jacobs' slides]

== Minutes ==

RdfAndSql

Rcygania2 — Tue, 03 Apr 2012 10:25:30 GMT

Rcygania2: /* Proprietary APIs and query languages */

__NOTOC__
While SQL databases are sometimes used for [[StoringRDF]] (a la [[LargeTripleStores|Large Triple Stores]]) it's also quite interesting to take [[Rdb2RdfXG|SQL stores that were not designed with RDF in mind and export/expose them as RDF]]. With SPARQL, the connection becomes even more interesting.

Some implementations map the database contents to an RDF vocabulary that is created automatically from the database schema. Others require a manual mapping of tables and columns to RDF properties and classes, but support use of existing vocabularies without an external rules engine.

== SPARQL-based ==

* [http://d2rq.org/d2r-server D2R Server] - SPARQL and [[LinkedData]] over HTTP; automatic, highly customizable mapping. Part of D2RQ (see below).
* [http://jena.sourceforge.net/SquirrelRDF/ SquirrelRDF] - SPARQL over API or HTTP, automatic or manual mapping. SquirrelRDF exports LDAP as well as SQL via SPARQL. See [[AdapterArchitecture]] for more along those lines.
* [http://www.w3.org/2005/05/22-SPARQL-MySQL/XTech SPASQL] - SPARQL over MySQL, automatic mapping
* [http://sourceforge.net/projects/rdquery RDQuery] - RDQL and SPARQL (?) over GUI, automatic mapping (same as Relational.OWL)
* [http://esw.w3.org/topic/DartGrid DartGrid] - is a Semantic Grid system, includes a SPARQL rewriting component using Datalog-like rules (more details in a [http://ccnt.zju.edu.cn/projects/dartgrid/files/presentation/%5B05-11-29-SKG%5D.ppt PPT presentation] and a [http://ccnt.zju.edu.cn/projects/dartgrid/files/publication/Huajun-ICDE2006.pdf Conference paper]
* [[VirtuosoUniversalServer|Virtuoso Universal Server]] - [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSSQL2RDF SPARQL-based Declarative Metaschema Language] for exposing any Virtuoso-housed data, as well as HTTP-, ODBC-, JDBC-, and otherwise-accessible SQL and XML data sources as URI Dereferenceable [[LinkedData|Linked RDF Instance Data]]. RDFized data may be retrieved as RDF/XML, JSON, N3, etc.
* [http://moustaki.org/p2r/ P2R] - provides dynamic access to Prolog knowledge bases, which may wrap SQL queries, calls to web services, XML parsing, etc.
* [http://www.cambridgesemantics.com/products/anzo_data_collaboration_server Anzo Data Collaboration Server] - from Cambridge Semantics, uses SPARQL to do integrated queries of RDBMSes in addition RDF stores, LDAP directories, etc. Also pushes RDF updates to SQL inserts&updates when possible.

== SPARQL Across Federated Sources ==

'''The Semantic Discovery System''': Provides the functionality to rapidly build solutions for non technical Users to create and execute Ad Hoc queries using the network Graph User Interface (SPARQL to SQL is auto generated). Integrates and interconnects ALL data silo types - providing a virtual Semantic Web interface to all RDBMS's, Web Services, Excel Spreadsheets, and any Hybrid File Systems.
* [http://www.semanticdiscoverysystems.com/ Semantic Discovery Systems (Main Site)]
* [http://www.meaning2go.com/ Semantic Discovery Systems ("Trailer"/Summary Web site)]

== RDF/XML-based ==

* [http://www.w3.org/2000/10/swap/dbork/dbview.py dbview] - Browsable RDF/XML ([[LinkedData]]) over HTTP, manual mapping (?)
* [http://www.dbs.cs.uni-duesseldorf.de/RDF/ Relational.OWL] - RDF/XML dump, automatic mapping
* [http://www.wiwiss.fu-berlin.de/suhl/bizer/d2rmap/D2Rmap.htm D2R Map] - RDF/XML dump, manual mapping
* [http://metamorphoses.sourceforge.net/ METAmorphoses] - Produces RDF/XML through a template language that selects a subgraph. Includes servlet for publishing the RDF/XML as [[LinkedData]]. Manual mapping.
* [http://triplify.org/Overview Triplify] is a small plug-in for Web applications, which reveals the semantic structures encoded in relational databases by making database content available as RDF, JSON or Linked Data.
* [[VirtuosoUniversalServer|Virtuoso Universal Server]] (in SPARQL-based section above) facilitates retrieval of RDBMS-hosted data in RDF Linked Data form, including RDF/XML serialization.

* [http://www.dblab.ntua.gr/~bikakis/SPARQL2XQuery.html SPARQL2XQuery] - Querying XML Data with SPARQL.

== Proprietary APIs and query languages ==

* [http://d2rq.org/ D2RQ] - Java library, provides access to relational data from the command line, through SPARQL, and through the Jena API. Automatic, highly customizable mapping.
* [http://www.w3.org/2004/04/30-RDF-RDB-access/ FeDeRate] - Algae query language, manual mapping (?)
* [http://www.cs.man.ac.uk/~ocorcho/documents/SWDB2004_BarrasaEtAl.pdf R2O] - proprietary query language, manual mapping
* [http://jena.hpl.hp.com/juc2006/proceedings/wilkinson/paper.pdf Jena property tables] - Jena API, manual mapping
* [[VirtuosoUniversalServer|Virtuoso Universal Server]] - (in SPARQL-based section above) uses [[SPASQL|SPASQL]] to implement its own [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSSQL2RDF Declarative Metaschema Language] for exposing ODBC- or JDBC-accessible SQL data sources as URI Dereferenceable [[LinkedData|Linked RDF Instance Data]]. RDFized data may be retrieved as RDF/XML, JSON, N3, etc.

== Other Related Dynamic RDF->SQL Mappings ==
* [https://svn.rdflib.net/trunk/rdflib/store/FOPLRelationalModel/ FOPLRelationalModel] - Models a target relational [http://copia.ogbuji.net/files/N3RelationalModel.xml model] as a set of objects which dynamically generate optimized SQL queries (intersections, unions, etc..) from Basic Triple Patterns

== Benchmarking RDB-to-RDF Tools ==

* latest info is found in [[RdfStoreBenchmarking]]
* Martin Svihala, Ivan Jelinek: [http://metamorphoses.sourceforge.net/Papers/pdf/msvihla_dexa2007.pdf Benchmarking RDF Production Tools] Paper comparing the performance of relational database to RDF mapping tools (METAmorphoses, D2RQ, SquirrelRDF) with native RDF stores (Jena, Sesame)

== Some History ==

* TimBL wrote a [http://www.w3.org/DesignIssues/RDB-RDF.html design note on Relational Databases on the Semantic Web] in 1998.
* As a follow-up to some [http://dig.csail.mit.edu/breadcrumbs/node/140 semweb dev track stuff at WWW2006 (e.g., D2R Server talk)], a few of us ([[DanConnolly]], ericP, EliasT, TimBL, AndyS, chimezie) got together for an [http://lists.w3.org/Archives/Public/public-sparql-dev/2006AprJun/0016.html IRC/phone meeting 8 Jun], where we agreed to use [http://lists.w3.org/Archives/Public/public-sparql-dev/ sparql-dev] and this wiki topic.
* Chris B. and a few others had a [http://lists.w3.org/Archives/Public/public-sparql-dev/2006AprJun/0020.html lunch discussion at ESWC] a few days later.
* Note the [http://www.w3.org/2007/03/RdfRDB/ W3C Workshop on RDF Access to Relational Databases] 25 to 26 October 2007 Cambridge, MA, USA.

== Related ==
* [[DynamicRDFizers|List of Dynamic RDFizers]] for RDFization of non/less-structured sources.

----
[[Category:SwTools]] [[RdfRdbMappingExamples]]

RdfAndSql

Rcygania2 — Tue, 03 Apr 2012 10:24:07 GMT

Rcygania2: Update D2R Server link

__NOTOC__
While SQL databases are sometimes used for [[StoringRDF]] (a la [[LargeTripleStores|Large Triple Stores]]) it's also quite interesting to take [[Rdb2RdfXG|SQL stores that were not designed with RDF in mind and export/expose them as RDF]]. With SPARQL, the connection becomes even more interesting.

Some implementations map the database contents to an RDF vocabulary that is created automatically from the database schema. Others require a manual mapping of tables and columns to RDF properties and classes, but support use of existing vocabularies without an external rules engine.

== SPARQL-based ==

* [http://d2rq.org/d2r-server D2R Server] - SPARQL and [[LinkedData]] over HTTP; automatic, highly customizable mapping. Part of D2RQ (see below).
* [http://jena.sourceforge.net/SquirrelRDF/ SquirrelRDF] - SPARQL over API or HTTP, automatic or manual mapping. SquirrelRDF exports LDAP as well as SQL via SPARQL. See [[AdapterArchitecture]] for more along those lines.
* [http://www.w3.org/2005/05/22-SPARQL-MySQL/XTech SPASQL] - SPARQL over MySQL, automatic mapping
* [http://sourceforge.net/projects/rdquery RDQuery] - RDQL and SPARQL (?) over GUI, automatic mapping (same as Relational.OWL)
* [http://esw.w3.org/topic/DartGrid DartGrid] - is a Semantic Grid system, includes a SPARQL rewriting component using Datalog-like rules (more details in a [http://ccnt.zju.edu.cn/projects/dartgrid/files/presentation/%5B05-11-29-SKG%5D.ppt PPT presentation] and a [http://ccnt.zju.edu.cn/projects/dartgrid/files/publication/Huajun-ICDE2006.pdf Conference paper]
* [[VirtuosoUniversalServer|Virtuoso Universal Server]] - [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSSQL2RDF SPARQL-based Declarative Metaschema Language] for exposing any Virtuoso-housed data, as well as HTTP-, ODBC-, JDBC-, and otherwise-accessible SQL and XML data sources as URI Dereferenceable [[LinkedData|Linked RDF Instance Data]]. RDFized data may be retrieved as RDF/XML, JSON, N3, etc.
* [http://moustaki.org/p2r/ P2R] - provides dynamic access to Prolog knowledge bases, which may wrap SQL queries, calls to web services, XML parsing, etc.
* [http://www.cambridgesemantics.com/products/anzo_data_collaboration_server Anzo Data Collaboration Server] - from Cambridge Semantics, uses SPARQL to do integrated queries of RDBMSes in addition RDF stores, LDAP directories, etc. Also pushes RDF updates to SQL inserts&updates when possible.

== SPARQL Across Federated Sources ==

'''The Semantic Discovery System''': Provides the functionality to rapidly build solutions for non technical Users to create and execute Ad Hoc queries using the network Graph User Interface (SPARQL to SQL is auto generated). Integrates and interconnects ALL data silo types - providing a virtual Semantic Web interface to all RDBMS's, Web Services, Excel Spreadsheets, and any Hybrid File Systems.
* [http://www.semanticdiscoverysystems.com/ Semantic Discovery Systems (Main Site)]
* [http://www.meaning2go.com/ Semantic Discovery Systems ("Trailer"/Summary Web site)]

== RDF/XML-based ==

* [http://www.w3.org/2000/10/swap/dbork/dbview.py dbview] - Browsable RDF/XML ([[LinkedData]]) over HTTP, manual mapping (?)
* [http://www.dbs.cs.uni-duesseldorf.de/RDF/ Relational.OWL] - RDF/XML dump, automatic mapping
* [http://www.wiwiss.fu-berlin.de/suhl/bizer/d2rmap/D2Rmap.htm D2R Map] - RDF/XML dump, manual mapping
* [http://metamorphoses.sourceforge.net/ METAmorphoses] - Produces RDF/XML through a template language that selects a subgraph. Includes servlet for publishing the RDF/XML as [[LinkedData]]. Manual mapping.
* [http://triplify.org/Overview Triplify] is a small plug-in for Web applications, which reveals the semantic structures encoded in relational databases by making database content available as RDF, JSON or Linked Data.
* [[VirtuosoUniversalServer|Virtuoso Universal Server]] (in SPARQL-based section above) facilitates retrieval of RDBMS-hosted data in RDF Linked Data form, including RDF/XML serialization.

* [http://www.dblab.ntua.gr/~bikakis/SPARQL2XQuery.html SPARQL2XQuery] - Querying XML Data with SPARQL.

== Proprietary APIs and query languages ==

* [http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/ D2RQ] - Java library, provides access to relational data through the Jena API, Sesame API, SPARQL, RDQL. Automatic, highly customizable mapping.
* [http://www.w3.org/2004/04/30-RDF-RDB-access/ FeDeRate] - Algae query language, manual mapping (?)
* [http://www.cs.man.ac.uk/~ocorcho/documents/SWDB2004_BarrasaEtAl.pdf R2O] - proprietary query language, manual mapping
* [http://jena.hpl.hp.com/juc2006/proceedings/wilkinson/paper.pdf Jena property tables] - Jena API, manual mapping
* [[VirtuosoUniversalServer|Virtuoso Universal Server]] - (in SPARQL-based section above) uses [[SPASQL|SPASQL]] to implement its own [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSSQL2RDF Declarative Metaschema Language] for exposing ODBC- or JDBC-accessible SQL data sources as URI Dereferenceable [[LinkedData|Linked RDF Instance Data]]. RDFized data may be retrieved as RDF/XML, JSON, N3, etc.

== Other Related Dynamic RDF->SQL Mappings ==
* [https://svn.rdflib.net/trunk/rdflib/store/FOPLRelationalModel/ FOPLRelationalModel] - Models a target relational [http://copia.ogbuji.net/files/N3RelationalModel.xml model] as a set of objects which dynamically generate optimized SQL queries (intersections, unions, etc..) from Basic Triple Patterns

== Benchmarking RDB-to-RDF Tools ==

* latest info is found in [[RdfStoreBenchmarking]]
* Martin Svihala, Ivan Jelinek: [http://metamorphoses.sourceforge.net/Papers/pdf/msvihla_dexa2007.pdf Benchmarking RDF Production Tools] Paper comparing the performance of relational database to RDF mapping tools (METAmorphoses, D2RQ, SquirrelRDF) with native RDF stores (Jena, Sesame)

== Some History ==

* TimBL wrote a [http://www.w3.org/DesignIssues/RDB-RDF.html design note on Relational Databases on the Semantic Web] in 1998.
* As a follow-up to some [http://dig.csail.mit.edu/breadcrumbs/node/140 semweb dev track stuff at WWW2006 (e.g., D2R Server talk)], a few of us ([[DanConnolly]], ericP, EliasT, TimBL, AndyS, chimezie) got together for an [http://lists.w3.org/Archives/Public/public-sparql-dev/2006AprJun/0016.html IRC/phone meeting 8 Jun], where we agreed to use [http://lists.w3.org/Archives/Public/public-sparql-dev/ sparql-dev] and this wiki topic.
* Chris B. and a few others had a [http://lists.w3.org/Archives/Public/public-sparql-dev/2006AprJun/0020.html lunch discussion at ESWC] a few days later.
* Note the [http://www.w3.org/2007/03/RdfRDB/ W3C Workshop on RDF Access to Relational Databases] 25 to 26 October 2007 Cambridge, MA, USA.

== Related ==
* [[DynamicRDFizers|List of Dynamic RDFizers]] for RDFization of non/less-structured sources.

----
[[Category:SwTools]] [[RdfRdbMappingExamples]]

ConverterToRdf

Rcygania2 — Tue, 03 Apr 2012 10:22:55 GMT

Rcygania2: update D2R Server/D2RQ link

__NOTOC__
A Converter to RDF is a tool which converts application data from an application-specific format into RDF for use with RDF tools and integration with other data. Converters may be part of a one-time migration effort, or part of a running system which provides a semantic web view of a given application. See also: [[RDFImportersAndAdapters]]

Please add converters as you make them or hear of them.

== Formats ==

in alphabetical order:

=== [[BibTex]] ===

[[BibTex]] is the format for bibliographic references in TeX.

* [http://data.bibbase.org BibBase] transforms BibTeX files (given in a URL) into Linked Data with RDF/XML output support.
** Uses a custom [http://purl.org/bibbase/ontology BibTeX ontology] but provides a table of mappings to other ontologies
** Also provides [http://bibbase.org HTML interface] and RSS feed
* [http://simile.mit.edu/repository/RDFizers/bibtex2rdf/ bibtex2rdf] transforms BibTEX files into RDF/XML. (Simile)
* [http://www.l3s.de/~siberski/bibtex2rdf/ bibtex2rdf] - A configurable BibTeX to RDF Converter by Wolf Siberski.
* [http://www.cs.vu.nl/%7Emcaklein/bib2rdf/ An online service] set up at the Vrije Universiteit in Amsterdam, the Netherlands, following the [http://ontoweb.aifb.uni-karlsruhe.de/ OntoWeb portal] vocabulary. The [http://www.cs.vu.nl/%7Emcaklein/bib2rdf/bib2rdf perl source] can also be downloaded.
* [http://www.aifb.uni-karlsruhe.de/WBS/pha/bib/index.html Java BibTeX-To-RDF Converter] based on the [http://ontobroker.semanticweb.org/ontos/swrc.html SWRC] terminology.

=== Bittorrent ===

* http://www.inf.unideb.hu/~jeszy/rdfizers is alas now 404 (in 2007). This was a link from RDFizers but may be incorrect.

=== Debian ===

The package information in Debian and similar systems (Ubuntu, Fink, etc), with its general usefulness and its graph-like nature, is a clear candidate for conversion to RDF.

See [http://blog.drinsama.de/erich/en/xml/2007011204-rdf-representation-of-packages VitaVoni blog] about this.

* [http://www.w3.org/2000/10/swap/util/fink2n3.py finkn3.py] Takes Fink (OS-X port of Debian packaging) dependencies and converts to to RDF/N3. (SWAP) No idea whether this would be a quick hack to export debian data.
* [http://github.com/nbarrientos/steamy STEAMY] converts Debian packages to RDF.

=== Email (RFC822 headers) ===

* [http://simile.mit.edu/repository/RDFizers/email2rdf/ email2rdf ] transforms email mbox files into RDF/XML. (Simile)
* [http://www.w3.org/2000/04/maillog2rdf/aboutMsg.py aboutMsg.py] converts email metadata to RDF. (SWAP)
* [http://swaml.berlios.de/ SWAML] transforms a mailing list into RDF/XML and XHTML+RDFa using [[SIOC]].
** And [http://linkedmarkmail.wikier.org/ LinkedMarkMail] live transforms into RDF/XML the mailing lists' archives indexed by [http://markmail.org/ MarkMail].
* [http://search.cpan.org/dist/Email-MIME-XMTP/ Email::MIME::XMTP] Perl extension to read and write [http://www.openhealth.org/xmtp/ XMTP]
* [http://aperture.sourceforge.net aperture.sf.net] IMAP crawler

There are others in this vein which run over IMAP or mailbox files.@@

=== Excel ===

* Cambridge Semantics' [http://www.cambridgesemantics.com/products/anzo_for_excel Anzo for Excel] extracts RDF data from Excel spreadsheets while keeping the spreadsheet in-sync with the underlying data as things change
* [http://www.topbraidcomposer.com TopBraid Composer] can convert Excel spreadsheets into instances of an RDF schema.
* XLWrap, [http://xlwrap.sourceforge.net] wraps spreadsheets (including cross tables) to arbitrary RDF graphs; supports Excel/OpenDocument/CSV streamed processing, local/HTTP loading, expressions similar to Excel/OpenOffice Calc, custom functions, usage via API or SPARQL endpoint
* [http://www.tao-project.eu/researchanddevelopment/demosanddownloads/RDBToOnto.html RDBToOnto], see description below under SQL section.
* [http://github.com/Data2Semantics/TabLinker TabLinker] can convert non-standard Excel spreadsheets to the Data Cube vocabulary, e.g. Excel files that contain hierarchical information in row and column headers etc.

=== EXIF ===

See JPEG.

=== File Systems ===

* [http://www.cs.univie.ac.at/publication.php?pid=5750 TripFS] exposes an entire file system as linked data, tracks changes, and links files to external data sources.

=== Flickr data ===

* Dave Becketts [http://librdf.org/flickcurl/ flickurl] library can access Flickr information (including machine tags) and convert it to RDF

=== Flat files ===

Unix systems store data (such as /etc/passwd) in flat files with comma separation.

* [http://simile.mit.edu/repository/RDFizers/flat2rdf/ flat2rdf] converts classic unix text database files, like /etc/passwd, into RDF/N3 (Simile)
* [http://www.w3.org/2000/10/swap/tab2n3.py tab2n3.py] Takes Tab-separated text (as typically output by all kinds of things including Microsoft Output and Spreadsheets) and converts it to N3, using the column headings to generate property URIs. (SWAP)
* [http://www.topbraidcomposer.com TopBraid Composer] can convert tab-separated spreadsheet files into an RDF/OWL class with corresponding properties and instances.
* XLWrap, [http://xlwrap.sourceforge.net] wraps CSV files (and spreadsheets) to arbitrary RDF graphs; supports local/HTTP loading, expressions similar to Excel/OpenOffice Calc, custom functions, usage via API or SPARQL endpoint

=== GPS ===

* [http://www.hackdiary.com/archives/000040.html garmin2rdf.py] Reads a Garmin GOPS receiver, dumping the contents in RDF/XML. (Matt Biddulph)
* [http://www.w3.org/2000/10/swap/pim/fromGarmin.py fromGarmin.py] Downloads GPS data from a Garmin on a serial link to an RDF/N3 file. (SWAP)

=== iCalendar ===

iCalendar is an IETF standard for calendar (event and to-do list) data.
Icalendar files typically are stored with a .ics extension.

* [http://www.w3.org/2002/12/cal/fromIcal.py fromIcal.py] converts iCalendar form to RDF
* [http://www.w3.org/2000/10/swap/pim/toIcal.py toIcal.py] converts RDF back into iCalendar.
* [http://aperture.sourceforge.net aperture.sf.net] java converter for iCalendar included
* [http://torrez.us/ics2rdf/] iCal to RDF Service

=== Java bytecode ===

* [http://simile.mit.edu/repository/RDFizers/java2rdf/ java2rdf] scans [http://java.sun.com/ java] bytecode for method calls and creates a description of the dependencies between classes and the package/archive encoded in RDF/N3. (Simile)

=== Javadoc ===

* [http://simile.mit.edu/repository/RDFizers/javadoc2rdf/ javadoc2rdf] is a doclet that makes javadoc output metadata about your code (structure of the classes, methods, comments, etc.) encoded in RDF/N3. (Simile)

=== Issue tracking: [http://www.atlassian.com/software/jira/ Jira] ===

* [http://simile.mit.edu/repository/RDFizers/jira2rdf/ jira2rdf] transforms Atlassian Jira's events about bug reports and issue tracking into RDF/N3.

=== JPEG ===

The metadata within JPEG photo is encoded in the EXIF standard.

* [http://simile.mit.edu/repository/RDFizers/jpeg2rdf/ jpeg2rdf] scans a folder for JPEG files, parses the EXIF and IPCT metadata found in those files and dumps an RDF/N3 representation of it into a file. (Simile)
* [http://www.w3.org/2000/10/swap/pim/jhead/ An adapted version of jhead] extracts RDF data form the EXIT encoded in JPEG files within a directory. Generates RDF/N3. (SWAP)

=== LDIF ===

This is format used for contact information in LDAP server system. It is for example exported by Thunderbird's address-book.

* [http://www.w3.org/2000/10/swap/pim/ldif2n3.py ldif2n3.py] Very incomplete, but useful. Generates foaf. Hides email addresses by hashing in the FOAF style if -m command flag is given. (SWAP)

=== Makefile ===

The unix Makefile syntax expresses dependencies between files in a software build.

* [http://www.w3.org/2000/10/swap/util/make2n3.py make2n3.py] Convert the makefiles in several directories in RDF and merge them to get the big picture. (SWAP)

=== MARC ===

* [http://simile.mit.edu/repository/RDFizers/marcmods2rdf/ marcmods2rdf]
transforms [http://www.loc.gov/marc/ MARC] records from Z39.2 format into [http://www.loc.gov/standards/mods/ MODS] and then from MODS to an RDF representation of MODS.

=== Meteographical ===
* [http://inamidst.com/sw/meteo/ Meteo] is UK weather forecast data in RDF, extracted from NOAA's public domain GRIB files. Example: [http://inamidst.com/sw/meteo/rdf/London London].

=== Microformats ===
* [http://www/incubator.apache.org/any23/ Apache Any23] is a Java library web service and command line tool for parsing multiple document formats and extracting structured data in RDF format from a variety of Web documents. It is used by [http://sindice.com/ Sindice.com]. The microformats support is [http://sindice.com/developers/microformat detailed in the Sindice.com documentation].

=== Multimedia ===

Following the [http://en.wikipedia.org/wiki/Don't_repeat_yourself DRY principle], a pointer to tools in the realm of multimedia (origin: [http://www.w3.org/2005/Incubator/mmsem MMSEM-XG]):

* [http://www.w3.org/2005/Incubator/mmsem/wiki/Tools_and_Resources Multimedia Semantics Tools]

=== OAI-PMH ===

* [http://simile.mit.edu/repository/RDFizers/oai2rdf/ oai2rdf] harvests an [http://www.openarchives.org/OAI/openarchivesprotocol.html OAI-PMH] repository and transforms the captured metadata in an RDF representation thru pluggable XSLT stylesheets.

=== Outlook ===

Microsoft Outlook contains contact and event data, and so on in a proprietary format.

* [http://www.w3.org/2000/10/swap/pim/lookout.py Lookout.py] convers the Microsoft Outlook calendar and address format into RDF. (SWAP)
* [http://aperture.sourceforge.net aperture.sf.net] includes Java crawler for MS Outlook

=== Open Financial Exchange (OFX) ===

[http://www.ofx.net/ OFX] is the format for downloaded bank statements and other financial information.
There are various levels of OFX, the early ones being HTTP headers followed by SGML, the later ones being HTTP-like headers followed by XML.

* [http://www.w3.org/2000/10/swap/pim/financial/OFX-to-n3.py OFX-to-n3.y] converts OFX format to RDF/N3. The conversion is only syntactic. The OFX modeling is pretty well thought out, so taking it as defining an RDF ontology seems to make sense. Rules can then be used to define mapping into your favorite ontology.

=== Open [[CourseWare]] ===

* [http://simile.mit.edu/repository/RDFizers/ocw2rdf/ ocw2rdf] harvests metadata from the MIT [http://ocw.mit.edu/ OpenCourseWare] web site and transforms it into an RDF representation of [http://meta.wikimedia.org/wiki/IEEE_LOM IEEE LOM].

=== Palm OS ===

* [http://dev.w3.org/cvsweb/2001/palmagent Palmagent] converts the calendar format of PalmOS into RDF. (SWAP)

=== plist ===

The Apple OS-X property list (.plist) filetype is an XML fromat for arbitrary structured data.
Numeric keys are used as local IDs. OS X applications store many kinds uf data in these files, including configuration data, iPhoto almum and photo data, iTunes metadata, and so on.

To convert plists well, added information is necessary, such as a namespace for the properties.

[http://dev.w3.org/cvsweb/2000/10/swap/util/plist2rdf.xsl plist2rdf.xsl] is an XSLT script to convert a plist file into RDF/XML. It does not add namespaces to the exported data.

=== Quicken Interchange Format (QIF) ===

* [http://www.w3.org/2000/10/swap/pim/qif2n3.py qif2n3.py] Takes Quicken interchange format and converts to to RDF/N3. (SWAP)

=== Quick and Dirty CSV to RDF Converter (QUIDICRC) ===

* [http://www.mindswap.org/~anant/quidicrc/ quidicrc] A perl script for rapidly transferring csv to RDF with some translation in the middle. (not actively being maintained, available open source -- SWAP)

=== Random ===

Seriously.

* [http://simile.mit.edu/repository/RDFizers/random2rdf/ random2rdf] generates synthetic random graphs encoded in RDF/N3.

=== Spreadsheet ===

* An [http://lab.linkeddata.deri.ie/2010/grefine-rdf-extension/ RDF Extension] is available for [http://code.google.com/p/google-refine/ Google Refine]. It can export Excel, CSV, and other tabular data to RDF. The schema mapping can be defined in a graphical UI.
* [http://www.mindswap.org/%7Erreck/excel2rdf.shtml Esxcel2rdf] is a Microsoft Windows program (exe) that converts Excel files into valid RDF. It has been tested on Windows 98, and Windows 2000 Professional. ([[MindSwap]]) Export can be done via comma- or tab- separated values. See Flat Files above.
* [http://aperture.sourceforge.net aperture.sf.net] includes Java crawler for Excel and open document. Does only extract plaintext and basic metadata, though.
* [http://rdf123.umbc.edu/ RDF123] has Windows and Linux applications to download, a Java application and servlet.
* Cambridge Semantics' [http://www.cambridgesemantics.com/products/anzo_for_excel Anzo for Excel] extracts RDF data from Excel spreadsheets while keeping the spreadsheet in-sync with the underlying data as things change
* XLWrap, [http://xlwrap.sourceforge.net] wraps spreadsheets (including cross tables) to arbitrary RDF graphs; supports Excel/OpenDocument/CSV streamed processing, local/HTTP loading, expressions similar to Excel/OpenOffice Calc, custom functions, usage via API or SPARQL endpoint
* [http://purl.org/twc/id/software/csv2rdf4lod csv2rdf4lod] uses declarative RDF enhancement parameters to specify how to transform tabular data into well-structured, well-connected RDF. The tool uses identifiers for ''source'' organization, ''dataset'', and ''version'' to establish default namespaces for all URIs created and provides VoID and provenance metadata as part of the conversion output.

=== SQL ===

SQL databases are rich stores of relational data ideal for export as RDF.
Conference tracks and many papers cover this subject from different angles. See also: [[RdfAndSql]]

* [http://d2rq.org/ D2RQ] provides a mapping from a SQL server (tested with several brands), producing both linked virtual RDF data files and a SPARQL service. Uses a configuration file in Turtle. (DERI and FU Berlin)
* [http://www.w3.org/2000/10/swap/dbork/dbview.py dbview.py] provides a mapping from a SQL server (tested with mySQL), producing linked virtual RDF data files. Uses a configuration file in N3. (SWAP)
* [[VirtuosoUniversalServer|OpenLink Virtuoso]]'s [http://virtuoso.openlinksw.com/wiki/main/Main/VOSSQLRDF declarative N3/Turtle based Metaschema Language] enables the creation of RDF Instance Data for associated RDF Ontologies via RDF VIEWs of ODBC, JDBC, ADO.NET, and OLE-DB accessible SQL Data. It is important to note that these VIEWs also apply to Native Virtuoso Data and/or Heterogeneous Data from other Web Services, HTTP/WebDAV, NNTP, and other Data Sources known to Virtuoso. This is an enhancement of the traditional SQL VIEW concept than enables multiple use of the same base SQL Data from a variety of data access points.
* [http://triplify.org Triplify] is a small plugin for Web applications, which reveals the semantic structures encoded in relational databases by making database content available as RDF, JSON or Linked Data.
* [http://www.tao-project.eu/researchanddevelopment/demosanddownloads/RDBToOnto.html RDBToOnto] is a full-fledged conversion tool that can produce accurate RDF/OWL models from various types of relational databases and Excel spreadsheets. The conversion is fully automated while various parameters can be set through the user interface to refine the resulting models (e.g., derivation of rich class hierarchies, proper naming of instances, database optimization before conversion, etc).

Many RDF Triple stores are implemented using SQL databases, but that is not covered here.

=== Subversion ===

[http://subversion.tigris.org/ Subversion] is a code-management system.

* [http://simile.mit.edu/repository/RDFizers/svn2rdf/ svn2rdf] A pair of scripts; one can be used in a post-commit subversion hook to generate RDF/N3 with each commit, the other on a working copy. (Simile)

=== Tab Separated Text ===

See flat files.

=== Talis SW Format Converter ===

* [http://convert.test.talis.com/ Talis' converter], convert from various format to various formats (including RDF->RDF with various serializations, RDF->HTML, etc)

=== UML ===

* [http://www.topbraidcomposer.com TopBraid Composer] can convert UML Class Diagrams (XMI format) into RDF/OWL models.
* [http://eulergui.sourceforge.net/ EulerGUI] is a lightweight IDE that translates on the fly UML and eCore XMI into N3. Moreover there are N3 rules to convert UML to OWL.

=== VCARD, Addressbook, … ===

VACRD is a standard for interchange of contact data, such as business cards and address books.

[http://www.w3.org/TR/vcard-rdf "Representing vCard Objects in RDF/XML"] is a W3C note defining an [http://www.w3.org/2006/vcard/ns ontology] for VCARD. FOAF is widely used ontology covering some of the domain.

* [http://www.holygoat.co.uk/applications/address-book-foaf/projects/ab/ab.py code to convert your Apple Addressbook into FOAF file] (Richard Newman)
* [http://people.no-distance.net/ol/software/ab-foaf/ ab-foaf] does the same.
* [http://search.cpan.org/dist/XML-FOAFKnows-FromvCard/ XML::FOAFKnows::FromvCard], Perl extension to create FOAF dumps from vCards. Does not attempt to create a full model, just foaf:knows. It also has some privacy features. In addition to the module, which conforms with the Formatter API specification, comes with a command-line tool.

=== Weather ===

* [http://simile.mit.edu/repository/RDFizers/weather2rdf/ weather2rdf] Given a US city or ZIP code, retrieves weather report data from weather.com and returns it in RDF. (Simile)

=== XML ===

* '''GRDDL:''' Any XML files can be marked up with pointers to XSLT files which convert them to RDF. The standard for this is [http://www.w3.org/TR/grddl/ GRDDL]. A GRDDL pointer can even be put in an XML schema, so that automatically all XML documents written to that schema will have a defined RDF mapping which any GRDDL-aware processor will benefit from. Several XSLT conversion transformations can be found linked from [[MicroModels]]
* [http://kwarc.info/projects/krextor/ Krextor] is a framework for extracting RDF in various notations from various XML languages and can easily be extended for additional input languages. Support for RDFa and some mathematical markup languages is built in. The implementation is done in XSLT, with a command-line frontend and a Java wrapper.
* [http://www.topbraidcomposer.com TopBraid Composer] can convert XML Schema (and their XML instance files) into RDF/OWL models.
* Rhizomik [http://rhizomik.net/redefer/ ReDeFer] includes XSD2OWL and XML2RDF plus MPEG-7 to RDF (all XSLT-based)
* '''XHTML:''' Convert ''existing'' pages to RDF. For example, see [[HtmlToRdf]].
* [http://www.dblab.ntua.gr/~bikakis/SPARQL2XQuery.html SPARQL2XQuery] The SPARQL2XQuery Framework provides mechanisms for: (a) Query translation (SPARQL to XQuery) (b) Mapping specification & generation (Ontology to XML Schema) (c) Schema transformation (XML Schema to OWL) and (d) Data Transformation (XML to RDF and vice versa)

=== XMP ===

[http://www.adobe.com/products/xmp/ XMP] is an Adobe-sponsored specification for putting RDF metadata in virtually any form of file, including binary formats. XMP metadata is RDF data in fact, but it has to be extracted from the file.

* [http://www.inf.unideb.hu/~jeszy/xmp/ xmpextractor] extracts XMP data. ([http://www.inf.unideb.hu/~jeszy/ Jeszenszky Péter])
* [http://dev.w3.org/cvsweb/~checkout~/2004/PythonLib-IH/xmp.py A python script to extract XMP]. There is also a service to do that on-line, see [http://www.ivan-herman.net/WebLog/WorkRelated/SemanticWeb/xmpextract.html separate page]

== Frameworks ==

The following are general tools which provide conversion from many formats.

=== AnnoCultor ===

[http://annocultor.eu/ AnnoCultor] was built during several years of practical work on porting various datasets to RDF. It allows converting data from the following data sources:
* databases via SQL and JDBC;
* XML files, also in batch;
* RDF files,
* Solr servers,
* custom formats, via format-specific parsers written in Java.

AnnoCultor is specifically suited for the situations where XSLT is not sufficient.

It comes with built-in converters for Geonames and Getty vocabularies (AAT, ULAN, TGN), that are ready to use.
Several additional specific converters illustrate advanced use: converters for collections of Louvre and Joconde,
Institute Collection Netherlands, Dutch Museum of Asian Ceramics, Tropenmuseum Amsterdam.

As part of conversion, AnnoCultor can semantically tag (enrich) data with links to various vocabularies, with advanced customised disambiguation and term processing possibilities.
These vocabularies should be represented in RDF or SKOS to be imported via SPARQL queries.
AnnoCultor comes with built-in tagging with Geonames and a custom time ontology.

AnnoCultor is written in Java, but conversion rules are written in XML. They are extendible with either small Java snippets, or custom rules implementions in Java.
AnnoCultor has been practically used with datasets ranging from a few records to more than ten millions, containing up to dozens fields each.

=== Apache Any23 ===

[[http://www/incubator.apache.org/any23/ Apache Any23] is a Java library web service and command line tool for parsing multiple document formats and extracting structured data in RDF format from a variety of Web documents. Currently it supports the following input formats:

* RDF/XML, Turtle, Notation 3
* RDFa
* Microformats: Adr, Geo, hCalendar, hCard, hListing, hResume, hReview, License, XFN and Species

Apache Any23 is used in major Web of Data applications such as [http://sindice.com/ sindice.com] and [http://sig.ma/ sig.ma].

=== Aperture ===

* [http://aperture.sourceforge.net/ Aperture] is a project written in Java gathering RDF extractors for many formats, mentioned in the list above.
Aperture supports crawling, making it not a converter but a framework to crawl updates of data (like rsync).

=== [[PiggyBank]] ===

* [http://simile.mit.edu/piggy-bank/ Piggy-bank] is a [http://simile.mit.edu/ Simile] project which allows the Firefox-based clent to automatically load "[http://simile.mit.edu/RDFizers/ RDFizers]", javascript-based converters to RDF.
Piggy-bank associates given scarping scripts with given web sites. (How?)

=== Triplr ===

[http://triplr.org/ Triplr] is a general “Stuff in, triples out” system by Dave Beckett. Triplr handles GRDDL, RSS, Atom, and other formats.

=== Virtuoso Sponger ===

[[OpenLinkSoftware|OpenLink Software]] via the "[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger Sponger]" component of [[VirtuosoUniversalServer|Virtuoso]]'s SPARQL Processor and Proxy Web Service (used by default by [[OpenLinkDataExplorer| OpenLink Data Explorer]]) provides RDFization for:
* RDFa
* GRDDL
* Amazon Web Services
* eBay Web Services
* Freebase Web Services
* Facebook Web Services
* Yahoo! Finance
* XBRL Instance documents
* DOI (includes a custom resolver for HTTP)
* OAI
* RSS/Atom Feeds
* Digital Music Files (various formats via ID3 Tags)
* Image Files
* vCard
* iCalendar
* Microformats - hCard, hCalendar
* HR-XML Resumes
* Flickr
* Del.icio.us
* Bugzilla
* ODBC or JDBC accessible SQL Data
* [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSources Many others]

= Notes =

Historically, this list was made from a lists of [http://simile.mit.edu/RDFizers/ RDFizers] and
[http://www.w3.org/2002/Talks/09-lcs-sweb-tbl/slide31-1.html SWAP converters].
It has grown significantly from community input since then.

This should be in a data format like Semantic Media Wiki or in N3 -- TimBL

> Would there an advantage to have this kind of list in an RDF file specifically to make queries on it. Maybe if we add a format on how to declare it here, we could create a converter to RDF. -- [[KarlDubost]]

> The task force [http://esw.w3.org/topic/SweoIG/TaskForces/InfoGathering InfoGathering] from SWEO works on such a vocabulary, if you want to rewrite this list using this vocab, look here: [http://esw.w3.org/topic/SweoIG/TaskForces/InfoGathering/DataVocabulary DataVocabulary] or contact me -- [[LeoSauermann]] on 22.1.2007
----

----
[[Category:SwTools]] [[Category:SwTools]]

TaskForces/CommunityProjects/LinkingOpenData/DataSets/CKANmetainformation

Rcygania2 — Thu, 01 Dec 2011 13:05:52 GMT

Rcygania2:

= Guidelines for Collecting Metadata on Linked Datasets in CKAN =

For keeping the LOD cloud diagram up to date, the Linking Open Data community effort has started to collect meta-information about Linked datasets on [http://www.ckan.net CKAN], a registry of open data and content packages provided by the Open Knowledge Foundation.

This page explains how dataset publishers or other people that want a dataset to be added to the LOD cloud, describe datasets on CKAN.

The list of datasets about which we have already collected information is be found here:

http://www4.wiwiss.fu-berlin.de/lodcloud/ckan/validator/

== Which datasets are included into the LOD cloud diagram? ==

All datasets are included that fullfil the following requirements:

# Data items are accessible via dereferencable URIs
# The dataset sets at least 50 RDF links pointing at other datasets or at least one other dataset is setting 50 RDF links pointing at your dataset.

== How do I add a data set to CKAN or edit an existing data set? ==

# Please register with [http://www.ckan.net CKAN] before editing or adding any packages.
# Please confirm that your data set does not already exist on CKAN before adding a new data set.
# Add or edit your data set and describe it with the following minimum required information:
#* CKAN name (a unique id)
#* title
#* URL
#* number of triples
#* links to other data sets.
# Please tag newly added data sets with <code>''lod''</code>.
# If you are not aware of any in- or outlinks, tag it with <code>''lodcloud.nolinks''</code>.
# Please provide as much additional information as possible (e.g. SPARQL endpoint, voiD description, license, and the topic of the data set) as described below. This information helps the community to know more about the development state of the Web of Linked Data and is made available via the CKAN API.

=== Minimum Information ===

Please provide the following minimum information about your data set.

==== Standard CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/Examples'''
|-
|Name
|Unique ID for your data set on CKAN
|[a-z0-9-]+ "my-dataset"
|-
|Title
|Full name of your data set
|"My Dataset"
|-
|URL
|Link to data set homepage
|http://example.com/my-ds
|}

==== Custom CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/Examples'''
|-
|triples
|Approximate size of your data set in RDF triples
|100000, 62345123
|-
|links:xxx
|Number of RDF links pointing at data set with Data Hub ID xxx (http://thedatahub.org/dataset/xxx). Please provide separate links xxx statements for each data set your are linking to
|20000
|}

==== CKAN tags ====

Please use the following tags to provide meta-information about your data set.

We will use the topic information to color the LOD cloud later.

Please also list the vocabularies used by your data set so that the community can get an overview of which vocabularies are commonly used on the Web of Linked Data.

Linked Data published on the Web should be as [http://www.w3.org/2001/tag/doc/selfDescribingDocuments.html self-describing] as possible in order to make it easier for clients to understand and use the data. Important aspects of self-descriptiveness are making vocabulary terms dereferenceable according to the best practices described in [http://www.w3.org/TR/swbp-vocab-pub/ Publishing RDF Vocabularies], using terms from common vocabularies and providing vocabulary mappings for proprietary vocabulary terms. In order to allow the community to get an overview which data sets implement these best practices, please tag your data set accordingly.

{| border="1px" cellpadding="2" cellspacing="0"
|'''Tag'''
|'''Purpose'''
|-
|<code>''<topic>''</code>
|One of:
* <code>media</code>
* <code>geographic</code>
* <code>lifesciences</code>
* <code>publications</code> (including library and museum data)
* <code>government</code>
* <code>ecommerce</code>
* <code>socialweb</code> (people and their activities)
* <code>usergeneratedcontent</code> (blog posts, discussions, pictures, ...)
* <code>schemata</code> (structural resources, including vocabularies, ontologies, classifications, thesauri)
* <code>crossdomain</code>
|}

=== Enhanced Information ===

Please provide the following additional information about your data set. This information helps the community to know more about the development state of the Web of Linked Data and is made available via the CKAN API.

==== Standard CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/Examples'''
|-
|Version
|Last modification date or version of your data set
|"2010-04 (3.5)", "2006", "beta"
|-
|Notes
|Description of your data set
|some free text
|-
|Author
|Name of publishing org and/or person
|"Talis (Leigh Dodds)"
|-
|Author email
|Contact email
|leigh@ldodds.com
|-
|License
|Standard license drop-down
|OSI approved::MIT license
|}

==== Custom CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/examples'''
|-
|shortname
|Short name for LOD bubble
|"NY Times"
|-
|license_link
|Custom license link
|http://example.com/so-sue-me
|-
|sparql_graph_name
|Named graph in SPARQL store (if used by your SPARQL endpoint)
|http://species.geospecies.org
|-
|namespace
|Instance namespace
|http://dbpedia.org/resource/
|}

==== CKAN resource links ====

Links (other than dereferenceable URIs) that enable alternative access to the data set (e.g., via downloads or SPARQL endpoints) should be specified in the Resources section of the CKAN entry form. Please also provide links to the [http://vocab.deri.ie/void/guide voiD description] or [http://sw.deri.org/2007/07/sitemapextension/ Semantic Web Sitemap] describing your data set.

{| border="1px" cellpadding="2" cellspacing="0"
|'''Purpose'''
|'''Format'''
|'''Description'''
|-
|Download page
| —
|Download
|-
|XML Sitemap
|<code>meta/sitemap</code>
|XML Sitemap
|-
|SPARQL endpoint
|<code>api/sparql</code>
|SPARQL endpoint
|-
|voiD file
|<code>meta/void</code>
|voiD description
|-
|RDF/XML download
|<code>application/rdf+xml</code>
|Download
|-
|Turtle download
|<code>text/turtle</code>
|Download
|-
|N-Triples download
|<code>application/x-ntriples</code>
|Download
|-
|N-Quads download
|<code>application/x-nquads</code>
|Download
|-
|RDF Schema
|<code>meta/rdf-schema</code>
|Download link to RDF/OWL Schema used by your data set (in addition to having dereferenceable vocabulary URIs)
|-
|RDF/XML example link
|<code>example/rdf+xml</code>
|Link to an example data item within your data set (RDF/XML)
|-
|Turtle example link
|<code>example/turtle</code>
|Link to an example data item within your data set (Turtle)
|-
|N-Triples example link
|<code>example/ntriples</code>
|Link to an example data item within your data set (N-Triples)
|-
|HTML+RDFa example link
|<code>example/rdfa</code>
|Link to an example data item within your data set (RDFa)
|-
|Vocabulary Mappings, e.g., OWL, RDFS, RIF, R2R
|<code>mapping/''<format>''</code>
|If your data set uses proprietary vocabulary terms and you know these terms also exists in other vocabularies, you should set <code>owl:equivalentClass</code>, <code>owl:equivalentProperty</code>, <code>rdfs:subClassOf</code>, and/or <code>rdfs:subPropertyOf</code> links pointing at these terms or provide mapping expressed as RIF rules or using the R2R Mapping Language. If your mappings can be downloaded as a single file, please provide the link to the download.
|}

==== CKAN tags ====

Please use the following tags to provide meta-information about your data set.

We will use the topic information to color the LOD cloud later.

Please also list the vocabularies used by your data set so that the community can get an overview of which vocabularies are commonly used on the Web of Linked Data.

Linked Data published on the Web should be as [http://www.w3.org/2001/tag/doc/selfDescribingDocuments.html self-describing] as possible in order to make it easier for clients to understand and use the data. Important aspects of self-descriptiveness are making vocabulary terms dereferenceable according to the best practices described in [http://www.w3.org/TR/swbp-vocab-pub/ Publishing RDF Vocabularies], using terms from common vocabularies and providing vocabulary mappings for proprietary vocabulary terms. In order to allow the community to get an overview which data sets implement these best practices, please tag your data set accordingly.

{| border="1px" cellpadding="2" cellspacing="0"
|'''Tag'''
|'''Purpose'''
|-
|<code>format-''<prefix>''</code>
|A [http://esw.w3.org/TaskForces/CommunityProjects/LinkingOpenData/CommonVocabularies vocabulary] used by the data set, e.g., <code>format-skos</code>, <code>format-dc</code>, <code>format-foaf</code>. Use http://prefix.cc/ to find a prefix for a vocabulary. If a vocabulary is not in prefix.cc, then add it there or ignore that vocabulary.
|-
|<code>no-proprietary-vocab</code>
|Indicates that your data set does not use a proprietary vocabulary (defined within your top-level domain).
|-
|<code>deref-vocab</code>
<code>no-deref-vocab</code>
|Indicates whether the proprietary vocabulary terms used by your data set (the ones that are defined within your top-level domain) are dereferenceable according to the best practices for [http://www.w3.org/TR/swbp-vocab-pub/ Publishing RDF Vocabularies]
|-
|<code>vocab-mappings</code>
<code>no-vocab-mappings</code>
|Indicates whether you provide mappings for proprietary vocabulary terms (by setting <code>owl:equivalentClass</code>, <code>owl:equivalentProperty</code>, <code>rdfs:subClassOf</code>, and/or <code>rdfs:subPropertyOf</code> links, or publish mapping expressed as RIF rules or using the R2R Mapping Language).
|-
|<code>provenance-metadata</code>
<code>no-provenance-metadata</code>
|Indicates whether the data set provides provenance meta-information (creator of the data set, creation date, maybe creation method) as document meta-information or via a voiD description. For instance, using the <code>dc:creator</code> or <code>dc:date</code> properties.
|-
|<code>license-metadata</code>
<code>no-license-metadata</code>
|Indicates whether the data set provides licensing meta-information as document meta-information or via a voiD description. For instance, using the <code>dc:rights</code> property.
|-
|<code>published-by-producer</code>
<code>published-by-third-party</code>
|Indicates whether the data set is published by the original data producer or a third party.
|-
|<code>limited-sparql-endpoint</code>
|Indicates whether the SPARQL endpoint is not serving the whole data set.
|-
|<code>lodcloud.nolinks</code>
|Dataset has no external RDF links to other datasets.
|-
|<code>lodcloud.unconnected</code>
|Dataset has no external RDF links to or from other datasets.
|-
|<code>lodcloud.needsinfo</code>
|The data provider or dataset homepage do not provide mininum information (and information can't be determined from SPARQL endpoint or downloads).
|-
|<code>lodcloud.needsfixing</code>
|The dataset is currently broken. Provide details in the Notes.
|}

TaskForces/CommunityProjects/LinkingOpenData/DataSets/CKANmetainformation

Rcygania2 — Thu, 01 Dec 2011 13:02:07 GMT

Rcygania2:

= Guidelines for Collecting Metadata on Linked Datasets in CKAN =

For keeping the LOD cloud diagram up to date, the Linking Open Data community effort has started to collect meta-information about Linked datasets on [http://www.ckan.net CKAN], a registry of open data and content packages provided by the Open Knowledge Foundation.

This page explains how dataset publishers or other people that want a dataset to be added to the LOD cloud, describe datasets on CKAN.

The list of datasets about which we have already collected information is be found here:

http://www4.wiwiss.fu-berlin.de/lodcloud/ckan/validator/

== Which datasets are included into the LOD cloud diagram? ==

All datasets are included that fullfil the following requirements:

# Data items are accessible via dereferencable URIs
# The dataset sets at least 50 RDF links pointing at other datasets or at least one other dataset is setting 50 RDF links pointing at your dataset.

== How do I add a data set to CKAN or edit an existing data set? ==

# Please register with [http://www.ckan.net CKAN] before editing or adding any packages.
# Please confirm that your data set does not already exist on CKAN before adding a new data set.
# Add or edit your data set and describe it with the following minimum required information:
#* CKAN name (a unique id)
#* title
#* URL
#* number of triples
#* links to other data sets.
# Please tag newly added data sets with <code>''lod''</code>.
# If you are not aware of any in- or outlinks, tag it with <code>''lodcloud.nolinks''</code>.
# Please provide as much additional information as possible (e.g. SPARQL endpoint, voiD description, license, and the topic of the data set) as described below. This information helps the community to know more about the development state of the Web of Linked Data and is made available via the CKAN API.

=== Minimum Information ===

Please provide the following minimum information about your data set.

==== Standard CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/Examples'''
|-
|Name
|Unique ID for your data set on CKAN
|[a-z0-9-]+ "my-dataset"
|-
|Title
|Full name of your data set
|"My Dataset"
|-
|URL
|Link to data set homepage
|http://example.com/my-ds
|}

==== Custom CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/Examples'''
|-
|triples
|Approximate size of your data set in RDF triples
|100000, 62345123
|-
|links:xxx
|Number of RDF links pointing at data set with Data Hub ID xxx (http://thedatahub.org/dataset/xxx). Please provide separate links xxx statements for each data set your are linking to
|20000
|}

==== CKAN tags ====

Please use the following tags to provide meta-information about your data set.

We will use the topic information to color the LOD cloud later.

Please also list the vocabularies used by your data set so that the community can get an overview of which vocabularies are commonly used on the Web of Linked Data.

Linked Data published on the Web should be as [http://www.w3.org/2001/tag/doc/selfDescribingDocuments.html self-describing] as possible in order to make it easier for clients to understand and use the data. Important aspects of self-descriptiveness are making vocabulary terms dereferenceable according to the best practices described in [http://www.w3.org/TR/swbp-vocab-pub/ Publishing RDF Vocabularies], using terms from common vocabularies and providing vocabulary mappings for proprietary vocabulary terms. In order to allow the community to get an overview which data sets implement these best practices, please tag your data set accordingly.

{| border="1px" cellpadding="2" cellspacing="0"
|'''Tag'''
|'''Purpose'''
|-
|<code>''<topic>''</code>
|One of:
* <code>media</code>
* <code>geographic</code>
* <code>lifesciences</code>
* <code>publications</code> (including library and museum data)
* <code>government</code>
* <code>ecommerce</code>
* <code>socialweb</code> (people and their activities)
* <code>usergeneratedcontent</code> (blog posts, discussions, pictures, ...)
* <code>schemata</code> (structural resources, including vocabularies, ontologies, classifications, thesauri)
* <code>crossdomain</code>
|}

=== Enhanced Information ===

Please provide the following additional information about your data set. This information helps the community to know more about the development state of the Web of Linked Data and is made available via the CKAN API.

==== Standard CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/Examples'''
|-
|Version
|Last modification date or version of your data set
|"2010-04 (3.5)", "2006", "beta"
|-
|Notes
|Description of your data set
|some free text
|-
|Author
|Name of publishing org and/or person
|"Talis (Leigh Dodds)"
|-
|Author email
|Contact email
|leigh@ldodds.com
|-
|License
|Standard license drop-down
|OSI approved::MIT license
|}

==== Custom CKAN fields ====

{| border="1px" cellpadding="2" cellspacing="0"
|'''Field name'''
|'''Description'''
|'''Format/examples'''
|-
|shortname
|Short name for LOD bubble
|"NY Times"
|-
|license_link
|Custom license link
|http://example.com/so-sue-me
|-
|sparql_graph_name
|Named graph in SPARQL store (if used by your SPARQL endpoint)
|http://species.geospecies.org
|-
|namespace
|Instance namespace
|http://dbpedia.org/resource/
|}

==== CKAN resource links ====

Links (other than dereferenceable URIs) that enable alternative access to the data set (e.g., via downloads or SPARQL endpoints) should be specified in the Resources section of the CKAN entry form. Please also provide links to the [http://vocab.deri.ie/void/guide voiD description] or [http://sw.deri.org/2007/07/sitemapextension/ Semantic Web Sitemap] describing your data set.

{| border="1px" cellpadding="2" cellspacing="0"
|'''Purpose'''
|'''Format'''
|'''Description'''
|-
|Download page
| —
|Download
|-
|XML Sitemap
|<code>meta/sitemap</code>
|XML Sitemap
|-
|SPARQL endpoint
|<code>api/sparql</code>
|SPARQL endpoint
|-
|voiD file
|<code>meta/void</code>
|voiD description
|-
|RDF/XML download
|<code>application/rdf+xml</code>
|Download
|-
|Turtle download
|<code>text/turtle</code>
|Download
|-
|N-Triples download
|<code>application/x-ntriples</code>
|Download
|-
|N-Quads download
|<code>application/x-nquads</code>
|Download
|-
|RDF Schema
|<code>meta/rdf-schema</code>
|Download link to RDF/OWL Schema used by your data set (in addition to having dereferenceable vocabulary URIs)
|-
|RDF/XML example link
|<code>example/rdf+xml</code>
|Link to an example data item within your data set (RDF/XML)
|-
|Turtle example link
|<code>example/turtle</code>
|Link to an example data item within your data set (Turtle)
|-
|N-Triples example link
|<code>example/ntriples</code>
|Link to an example data item within your data set (N-Triples)
|-
|HTML+RDFa example link
|<code>example/rdfa</code>
|Link to an example data item within your data set (RDFa)
|-
|Vocabulary Mappings, e.g., OWL, RDFS, RIF, R2R
|<code>mapping/''<format>''</code>
|If your data set uses proprietary vocabulary terms and you know these terms also exists in other vocabularies, you should set <code>owl:equivalentClass</code>, <code>owl:equivalentProperty</code>, <code>rdfs:subClassOf</code>, and/or <code>rdfs:subPropertyOf</code> links pointing at these terms or provide mapping expressed as RIF rules or using the R2R Mapping Language. If your mappings can be downloaded as a single file, please provide the link to the download.
|}

==== CKAN tags ====

Please use the following tags to provide meta-information about your data set.

We will use the topic information to color the LOD cloud later.

Please also list the vocabularies used by your data set so that the community can get an overview of which vocabularies are commonly used on the Web of Linked Data.

Linked Data published on the Web should be as [http://www.w3.org/2001/tag/doc/selfDescribingDocuments.html self-describing] as possible in order to make it easier for clients to understand and use the data. Important aspects of self-descriptiveness are making vocabulary terms dereferenceable according to the best practices described in [http://www.w3.org/TR/swbp-vocab-pub/ Publishing RDF Vocabularies], using terms from common vocabularies and providing vocabulary mappings for proprietary vocabulary terms. In order to allow the community to get an overview which data sets implement these best practices, please tag your data set accordingly.

{| border="1px" cellpadding="2" cellspacing="0"
|'''Tag'''
|'''Purpose'''
|-
|<code>format-''<prefix>''</code>
|A [http://esw.w3.org/TaskForces/CommunityProjects/LinkingOpenData/CommonVocabularies vocabulary] used by the data set, e.g., <code>format-skos</code>, <code>format-dc</code>, <code>format-foaf</code>
|-
|<code>no-proprietary-vocab</code>
|Indicates that your data set does not use a proprietary vocabulary (defined within your top-level domain).
|-
|<code>deref-vocab</code>
<code>no-deref-vocab</code>
|Indicates whether the proprietary vocabulary terms used by your data set (the ones that are defined within your top-level domain) are dereferenceable according to the best practices for [http://www.w3.org/TR/swbp-vocab-pub/ Publishing RDF Vocabularies]
|-
|<code>vocab-mappings</code>
<code>no-vocab-mappings</code>
|Indicates whether you provide mappings for proprietary vocabulary terms (by setting <code>owl:equivalentClass</code>, <code>owl:equivalentProperty</code>, <code>rdfs:subClassOf</code>, and/or <code>rdfs:subPropertyOf</code> links, or publish mapping expressed as RIF rules or using the R2R Mapping Language).
|-
|<code>provenance-metadata</code>
<code>no-provenance-metadata</code>
|Indicates whether the data set provides provenance meta-information (creator of the data set, creation date, maybe creation method) as document meta-information or via a voiD description. For instance, using the <code>dc:creator</code> or <code>dc:date</code> properties.
|-
|<code>license-metadata</code>
<code>no-license-metadata</code>
|Indicates whether the data set provides licensing meta-information as document meta-information or via a voiD description. For instance, using the <code>dc:rights</code> property.
|-
|<code>published-by-producer</code>
<code>published-by-third-party</code>
|Indicates whether the data set is published by the original data producer or a third party.
|-
|<code>limited-sparql-endpoint</code>
|Indicates whether the SPARQL endpoint is not serving the whole data set.
|-
|<code>lodcloud.nolinks</code>
|Dataset has no external RDF links to other datasets.
|-
|<code>lodcloud.unconnected</code>
|Dataset has no external RDF links to or from other datasets.
|-
|<code>lodcloud.needsinfo</code>
|The data provider or dataset homepage do not provide mininum information (and information can't be determined from SPARQL endpoint or downloads).
|-
|<code>lodcloud.needsfixing</code>
|The dataset is currently broken. Provide details in the Notes.
|}

User:Rcygania2

Rcygania2 — Thu, 23 Jun 2011 21:01:33 GMT

Rcygania2: /* Nearby */

__NOTOC__


== Richard Cyganiak ==

Email: richard@cyganiak.de

Blog: http://dowhatimean.net/

Homepage (in German): http://richard.cyganiak.de/

== Nearby ==

* My [[/RulesOfThumb]]
* My [[/FavouritePapers]]
* Notes: [[/HashVsSlash]]

== Semwebby interests ==

* [[JenaFramework]]
* [[SPARQL]]
* RDF and relational databases -- [http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/ D2RQ], [http://www.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/ D2R Server], [http://jena.sourceforge.net/sparql2sql/ sparql2sql]
* [[NamedGraphs]] -- [[NamedGraphsApiForJena]]
* User interfaces for authoring and browsing RDF -- Snorql
* [[SemanticWiki]]
* FOAF
* [http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData Linking Open Data]
* [[DBpedia]]

----
[[Category:Homepage]]

RichardCyganiak/RulesOfThumb