http and https

I’m at W3C’s Technical Plenary event, and Pierre-Antoine Champin has been asking me whether the Schema.org community have advice about the different machine-readable representations for identifying schema.org terms. It is a little messy but I thought a simple list of facts might help clarify, at the risk of being extremely boring for most normal humans.

1. At launch, schema.org was on an http: -based website, rather than https:. This was common at the time for non-ecommerce or privacy-sensitive sites. It also closely shadowed its schema data structures to the website pages that documented each term (“/Person” etc.). In its RDF representations, terms had identifiers like “http://schema.org/Person”.
2. In a post-Snowden, more security conscious environment the industry moved quickly towards using https everywhere. This included Google software and services, which meant that Schema.org quickly looked archaic for hosting without https:. Schema.org also made its human-facing site available by default at https://schema.org/
3. RDF is a very identifier-centric approach to data, and changes in identifiers can be costly and complex. Within the project, and in its data dumps, Schema.org continued to use http: in data, even as it had https: pages for its documentation. This weakened the simple mapping between human-visible and machine-visible term identifiers, causing frequently asked questions about whether to use ‘http” or ‘https’.

At that point, the project FAQ stated:

There is a general trend towards using 'https' more widely, and you can already write 'https://schema.org' in your structured data. We have migrated the schema.org site itself towards using https: as the default version of the site and our preferred form in examples. However ‘http://schema.org’ -based URLs in structured data markup will remain widely understood for the foreseeable future and there should be no urgency about migrating existing data. This is a lengthy way of saying that both 'https://schema.org' and 'http://schema.org' are fine.

This made sense for publishers, SEOs etc., and left it to the consumers to deal with the resulting heterogeneity. As a search-centric project, this was reasonable – search engines deal with heterogeneity all the time.

For RDF users of Schema.org there are some additional considerations. When Schema.org embraced JSON-LD, it encouraged the use of idioms that referenced a remote “JSON-LD context” whose URI was the URI for the project homepage. This was originally http: and then later it became more common for the JSON markup to reference a context of http://schema.org. Initially Schema.org made the context available via HTTP Content Negotiation (i.e. it sent HTML to browsers, but JSON-LD context data in response to web fetches that specifically asked for it. However, when Schema.org switched to a 100% static file hosting, the option to do this went away. Fortunately the JSON-LD technology added the abiility to point to a JSON-LD context via HTTP headers:

danbri@Dans-MacBook-Air ~ % curl -sSI ‘https://schema.org/’ | grep -i ‘^link:’

link: </docs/jsonldcontext.jsonld>; rel=”alternate”; type=”application/ld+json”

This “context” file serves to make JSON-LD syntax slightly prettier for humans by hiding some details and mappings in a remote web document. Amongst other things, it establishes the base URL used when converting JSON-LD markup into RDF triples. To this end the context begins with a declaration which has to establish the URL used. In the case of Schema.org this was http: and has remained http:. If the project were to change this, then data systems all around the planet would suddenly have subtly different RDF data emitted from parsers, even if the parser software and its input files were entirely unchanged.

The consequence of this is that we have several things going on that mean heterogeneity is inevitable. There is a general trend towards https:, a usability need for schema.org machine formats and URLs to follow this, and a technical constraint ensuring that JSON-LD parsers will emit http:// based triples, even if the file being parsed references the context via a https: reference.

Aside from all of this, the Web now has a mix of microdata, rdfa, json-ld and other representations that use schema.org data. Some using http:, some using https:. Anyone exploring the Common Crawl extractions published by the Web Data Commons project will soon come to appreciate that this is the last of their worries, when trying to make sensible use of this huge corpus of stuff.

My advice is to canonicalise (most naturally towards https://) when aggregating and consuming schema.org data. There are other cases in schema.org and elsewhere where different kinds of normalization makes sense, e.g. shortcut use of strings in place of entities with name strings, or the various constructions for representing opening hours. Other widely used vocabularies often share these issues, eg. Dublin Core in RDF has taken various forms in the years since 1997. Developers with canonicalization / normalization code might get positive feedback from sharing their work with other members of the community group.

Dan