The Design of MapML
Posted on:MapML – What’s the big idea?
Geographic mapping applications are central to the numerous aspects of the digital world. They’re in our cars, on our phones, on the web. They’re used by industry, governments, and individuals trying to go about their day. A huge global industry exists, dedicated to the technology for acquiring, processing, and transforming raw spatial information into formats for use in applications in which maps form part or all of the content.
On the Web, the information from which Web maps are made is often, but not always, served by dedicated spatial server software, in the form of raw Web application data, typically encoded as text formats like JSON or XML, and images of various formats.
The Web Platform — that is, the set of standards implemented by agreement across browser engines — doesn’t currently include any features that specifically support mapping applications, such as elements with map-related semantics. Adding to the challenge, Websites that do rely on dedicated spatial server software are missing a way to encapsulate map content in a single map data document format. If an encapsulated map document existed, it could be made available at any arbitrary, unspecified or abstract URL; a URL would allow any HTML page or other mapping application to include, or link to, that map content.
The situation for maps in the Web Platform is similar to that of vector graphics on the Web prior to the introduction of SVG, or to that of video content prior to the definition of the <video> element in HTML5. There are other examples, but it’s important to note that the Web Platform has adapted and evolved to include these types. To help precipitate a similar evolution for maps, we’ve formed the Maps for HTML Community Group, to bring together the experts and interests necessary to push maps along the same trajectory these other media types have taken.
So, the big idea is this: MapML resolves the Web Platform gap by combining map and map data semantics into a hypermedia format that is syntactically and architecturally compatible with and derived from HTML. MapML provides a standardized way for declarative HTML content to communicate with custom spatial server software (which currently use HTTP APIs based on multiple queries and responses). MapML allows map and map data semantics to be either included in HTML directly, or referred to at arbitrary URLs that describe stand-alone layers of map content, including hyper-linked annotations to further content. This allows HTML and map authors to easily create markup inline or refer to Web map content from their documents without needing procedural programming (JavaScript) to fetch and integrate data. Finally, MapML defines a DOM structure for the map content that can integrate with Cascading Style Sheets and JavaScript, in exactly the same way HTML content does.
What is the scope of the “MapML proposal”
At the time of writing, the Maps for HTML proposal is scoped over two documents: one describes MapML content, the other describes the supporting extensions to HTML. The specifications include four entirely new elements in the HTML namespace: <layer> (a new sibling of the existing <area> element), with <layer> child elements <extent>,<tile> and <feature>. The new elements will need supporting extensions to existing elements, including <map> (or, an entirely new element with <map>-like semantics), <link> and <input>. In addition, the proposal specifies that the new and modified existing HTML elements may be found in a new doctype (“mapml”) which can be found by URL reference from a <layer src=”…”> attribute. A new document type is necessary for the use case where map content is included purely by reference. The new document type should be readily parseable with a (minimally) enhanced HTML parser, and should be distinguished at the HTTP header level by the “text/mapml” MIME type.
MapML layers will also need an image element, for which we hope to rely on the existing <img> element (although it is currently specified as the <image> element). In line with our design principles outlined below, we should try to not duplicate element semantics with minimally incompatible new forms. Instead, we should rely on existing element processing semantics wherever possible. Further, we believe it should be possible to directly use the semantics of many existing HTML elements, as well as SVG elements, but not limited to those. The reason is that it should be possible to use HTML elements to draw hypermedia annotations on a map layer, and there should be no need to re-invent those elements in order to do so. CSS should allow styling of map features in the same way that it allows styles of most of HTML and SVG elements.
MapML design principles
HTML Design Principles
Notwithstanding the specifications being currently spread over two documents, MapML is intended to be an extension of HTML. As such, features designed into MapML should follow the HTML Design Principles.
Among Web standards developers, one of the most potentially controversial elements of the design of the MapML proposal, is the proposed extension of the <map> element. At first, this might seem like the Map for HTML community is trying to shoehorn new functionality into an HTML element where it doesn’t belong. It’s worth taking a closer look at this proposal, instead of taking what might seem to be the less controversial path of adding an entirely new map-related embed.
Our Contract With the Web
One of the core design principles of HTML is “Support Existing Content”. In this context, this means that for many years, authors have created <map> elements with associated <area> elements, and they expect a certain processing behaviour to come of parsing and displaying their Web page that use these “client side image maps”. This is the contract of the Web, and it’s mostly unbreakable. This is often referred to as element semantics.
The key semantic of <img>, <map> and <area>, are that they form a visual stack of layers which share the coordinate system of the <img> layer — the areas in the map create a layer of interactive objects on top of the image. No other element group has this layering semantic. Because this is so, it would seem a reasonable extension point for a more dynamic layer stack of layers sharing a coordinate system, such as we’re proposing, and the key challenge will be to ensure that the contract between the browser and the Web is not broken.
Graceful Degradation
We believe this proposal aligns well with the criteria established for evaluating proposed changes to legacy features, in particular, because we hope and intend that it will be possible to make this extension without breaking the Web. Further, if we are able to make these changes without breaking the Web, we think this could create an excellent situation for proposing fallback behaviour, thanks to the existing layering semantics of <img>, <map> and <area>. When a new element is introduced, some consideration must be given to how the new markup will behave in existing user agents, ones that haven’t been updated to support the element. This is called “graceful degradation”. In the case of <map> with <layer> children, we hope that authors will be able to provide <img> and <area> elements that will serve the base (original, current) purpose, in older user agents, per existing semantics, without the extra interactive map features. This is possible because the core semantics of <img>, <map> and <area> should not be affected by the extension. If the author uses the proposed form, they could also provide the fallback markup (<img>, <map> +/- <area> elements). At least, that’s the theory of this proposal. We (the Maps for HTML CG) need and welcome further discussion of the topic, especially with individuals with deep and long experience developing Web browsers and Web standards.
Do Not Reinvent the Wheel
In suggesting that HTML designers not reinvent the wheel, the HTML Design Principles are talking about non-standard markup that authors have used on the Web (thanks to browser support of the feature, despite not being part of a formal standard), and how it is better to accept this fact and encode it as a standard, rather than trying to change authors existing behaviour and knowledge. There are a couple of aspects that relate to this, but in fact there is no Web map content out there (apart from that discussed above as <img>, <map> and <area>).
Mostly, Web maps are composed by JavaScript libraries, using server-side content APIs. The Maps for HTML community has undertaken an extensive review of existing JavaScript mapping library approaches, with a view to creating a thorough definition of the use cases and hence requirements to support standardization of Web maps in the platform.
The intent of standardization would be to create standardized DOM APIs to reflect the standard patterns that exist today only “virtually”, as a common thread across the JavaScript mapping libraries. What is currently done virtually, that is the creation and rendering of tiles, images and features, would be done by the browser rendering engine, through the parsing and rendering of markup. In other words, let’s take the wheel that clearly exists, and standardize it.
Pave the Cowpaths
This is related to the admonition to not reinvent the wheel, except that it refers to existing practice which is clearly not standard, nor implemented by browsers, but is widely practiced by authors. JavaScript Web mapping libraries are full of such practices. Unfortunately, it is only the open source libraries, such as Leaflet and OpenLayers, which allow us to draw insight into what such practices in fact are. Fortunately, Leaflet and OpenLayers are well documented and represented by members of the community group. We do have the ability to draw not only on the source code, but on implementer experience, to inform the Web platform of these cowpaths that may be pavable.
Evolution not Revolution
This relates strongly to how the original Web was designed, and this was described by Tim Berners-Lee as the Test of Independent Invention, or TOII. In the TOII, one performs a kind of thought experiment. Realizing that existing content has an insurmountable advantage when it comes to building new paradigms, the new system design must be able to consume or use existing content without requiring that content to change. There isn’t really an existing body of HTML content that represents maps, apart from the image maps discussed above. On the other hand, there is a huge number of geospatial Web services, containing > 1.5M thematic layers, representing an impossibly huge amount of spatial information from around the globe. This the existing content that a new standard must recognize and adapt to.
Solve Real Problems
Unstandardized Web maps are an under- or unrecognized major problem for the Web platform. In part, that’s because there is one major Web mapping commercial service that has effectively centralized mapping technology. As a result, Web platform advocates have not recognized or looked past the obvious need and opportunity for standardization. One of the major problems that results from centralization is loss of privacy. Although the solution to pollution is not dilution, decentralization may help solve the privacy crisis on the Web, by making it easy for authors to refer to private or public spatial data infrastructures based on need and means. There are free, public geospatial information infrastructures that are underutilized resources, because they can’t be integrated into websites and apps as easily as the big-company commercial map viewers can. If the public map data was compatible with a global standard for maps in the Web platform, it could help solve the Web’s privacy crisis, especially in regard to user geolocation.
Priority of Constituencies
The HTML priority of constituencies tells us to prioritize the needs of the billions of website users, over the millions of website authors, over the thousands of web browser implementers, over the hundreds of web specification writers.
Update 2020-04-28: Users will benefit from maps in HTML because they will be able to search for Web resources by location: any page that has a map in it is implicitly geocoded to the location(s) in the map(s) in the page, so search engines will be able to use that facet to index pages by location. Users will benefit from standardized HTML maps because their privacy will be less at risk because more Web map services will be realised based on standardized Web platform maps. Users will benefit because maps will now be available to Web authors of every skill level, supported on all the devices that support the Web platform itself. Users will benefit because the browser will not only understand their location, through the geolocation API, but will also understand the location of real-world features, because they are encoded in HTML. The browser will be able to provide events and accessibility information not only to readable text, as it can do today, but also to textual representations of the real-world objects. Such objects could be invisible, such as speed zones, but could be tangible, such as sidewalks, fire hydrants, crossing light switches, etc.
Authors will benefit because the means for map generation can now be built into HTML IDE autocomplete functionality. Authors will benefit because the APIs that can be used to customize maps can now be built into the DOM, and standardized.
Implementers will benefit because there is a huge volume of geospatial Web content that will be readily available to assure interoperability, and to allow implementers to immediately and tangibly obtain return on the investment in a standardized platform. Implementers will benefit from the large and vibrant standards and developer community associated to open source geospatial tools and data, and the synergy they will bring to the Web platform community through their participation in the platform. The geospatial community will become valuable and contributing stakeholders in the Web platform, making the platform stronger by their contributions.
Web platform standards developers will benefit, or at least not lose, because the Maps for HTML community is ready and willing to help in developing specs, polyfills and platform tests.
Secure by design
We are not certain what the implications of this design principle are for Web maps. It’s clear that spatial information delivers its value by being used in conjunction with other spatial data, as mashups, so we need to work with the Web platform security experts to ensure that browser and user security is not put at risk, while at the same time allowing the Web to unlock the potentially enormous value of layered mashups. We believe a useful solution is possible, given the existence of the <iframe>, CSP, CORS and other platform standards. Let’s work together to find a solution that fits the use cases and requirements of maps.
Separation of concerns
This principle relates to the desire to separate presentation from structure to the extent practical. Apart from the <map> and <layer> proposals, there are three main (completely new) elements proposed: <extent>, <tile> and <feature>. <extent> is a form-like affordance, and as such it is not practical to omit this element. It may be that we can improve the definition of the element and its concept through discussion and experimentation. Let’s have those discussions, and build those experiments. <tile> and <feature> reflect the things that make up map content, and we believe they are also essential. The markup of <feature> in particular may be worth discussing. In geospatial information models, the Simple Features model has come to be widely accepted. I’ve been told that the model derives from the object-property model of RDF. Anyway, it is a widely implemented model, from shapefiles to KML, GML, probably the protocol buffers that underlie Google Maps, and finally the Web developer friendly GeoJSON. The coordinate and markup model for <feature> is based on what Web developers actually do with GeoJSON in their scripts: they transform the GeoJSON to HTML and SVG (or canvas commands) for rendering. The markup of <feature> paves this cowpath: create an element that allows properties to be rendered as HTML, with the geometry model conforming to the standard simple features geometry model. The central idea behind having a content-based coordinate model, as opposed to an attribute based one (like SVG), is because the coordinates of features should themselves be available to markup and hence styling or making links out of them. In this view of geography, things (“features”) on earth, or in non-georeferenced technical diagrams for that matter, become links on and to the surface of the Earth, as human thought becomes hypertext in HTML.
It must be admitted at this point that the idea of linking and styling features according to the above model remains largely a vision for how this proposal should work, but given the existing HTML parser, and map viewers, it doesn’t seem too far from being a cowpath to pave. Some examples of styling using CSS have been created, but more work needs to be done, establishing the vision for styling feature geometries and their parts, as well as linking features and their parts.Why shouldn’t the geometry part simply be transformed to SVG? According to this HTML design principle: “Profound and detailed semantic encoding is not necessary if the end can be reached otherwise”, so it would seem to be logical to transform geometries to SVG for rendering. The reason a <feature> element is explicitly needed is because rendering the geometry model of simple features as SVG objects is not enough, even if it was fully compatible with the simple features geometry model. In simple features, geographic objects are reified (as features), whereas in SVG, geometric objects have strictly presentation / rendering semantics. All this is not to say that SVG objects aren’t useful in cartography, far from it. But as discussed above, geographic features are a well established standard that is widely implemented (this is a cow path), and that standard should also be supported by browsers in order to mediate real-world objects for the user, just as the browser mediates textual documents for the user i.e. for accessibility reasons, it’s not enough for an HTML element to be purely visual.
DOM Consistency
HTML is defined as a DOM (document object model) with two possible markup serializations — HTML and XHTML — that have different parsers but should produce consistent results for a valid document.The MapML proposal began as an attempt to capture Web map semantics in a separate but HTML parser-compatible serialization format, and so MapML, or text/mapml was specified as a “MicroXML” encoding of <extent>, <tile> and <feature>. MicroXML forbids namespaces, and attributes such as xmlns, so it might seem that this precludes parsing text/mapml documents as being content within the “http://www.w3.org/1999/xhtml” namespace. It is possible as a temporary measure perhaps, to impute this namespace value for documents parsed as text/mapml, although longer term it might be wise to specify an XML serialization of text/mapml in the same way that this is defined for HTML. In other words, it might have been a mistake to banish namespaces altogether. On the other hand, we have forked and hacked the Mozilla HTML parser to allow it to parse MapML documents in the MicroXML or HTML form with void elements etc. This form is signified by the <!doctype mapml> declaration. There are undoubtedly issues that will arise in further developing the parsing of MapML content, but we think the path forward looks good to this point. Evaluating how well the coordinate model can support internal markup such as <a> and <span> is important research that needs to be done. We welcome the participation of validation, parsing and CSS experts from the Web standards community to help validate and refine or alter these proposals. In particular we would like to request that those experts who self-identify in these categories join our community group to help research and to discuss the results of others’ research.
Well-defined behaviour
The initial draft MapML and map element specifications focus on high-level functionality; when it comes to details, the demo implementation relies on the existing behaviour of the Leaflet.js library. We know that this doesn’t yet meet the expectations of a modern web standard.We would like to use the detailed Use Cases and Requirements to define behaviour in a way that can be standardized in the platform. We welcome further contributions to the Community Group, in particular, as requested above. The behaviour should be testable, and we would like to develop Web Platform Tests that assure the interoperability of implementations. Similar to the requests immediately above, the Web standards community can help us develop our proposal to this degree, and we welcome that in the community group.
Avoid needless complexity
MapML is intended to be the simplest possible proposed extension to HTML that can support the core use cases of Web maps, kind of a minimum viable product. All such approaches are subjective, however, and we welcome contributions intended to make the proposal clearer, simpler etc.
Handle Errors
This seems to be part of the contract that Web browsers have with users and authors, in that they are not exposed to exceptions or confusing error messages that they can do nothing about. We have almost certainly not yet identified all the possible error situations that need to be handled gracefully, and we can probably define such error handling in our specifications in future iterations.
Accessibility
Limited knowledge of what even constitutes accessibility for maps has somewhat limited our efforts in this domain. We are actively seeking contributions from the accessibility community to help us define a vision for how Web platform maps could encode accessibility features for all users. One thing seems clear: the browser as the user’s agent, is in a unique position to be able to broker access to real-world phenomena described by the <feature> element. In this way, maps and geospatial information can be much more than a mere visual tool for sighted users.
Conclusion
We hope that the Web standards community will consider this proposal in the light described above, first, and make suggestions and contributions of improvements as required. There are on-going debates within the Web mapping community about how best to respond to the developing requirements for maps on the Web. These debates are a healthy product of collaborators thinking about the problem space, and they are welcome in support of the goal of progress towards browser support of Web maps. Such debates importantly include what constitutes the “cow paths” in Web mapping, as well as what abstractions are core to Web map functionality vs what are the actual absolutely required “primitives” that are necessary for browser support of Web maps, and from which other non-primitive abstractions can be reasonably built.