Richer UI access to HTML data

Bugreport 5756

Problem statement / use cases

• authors want to make use of HTML semantic facilities, often receive no perceptual feedback
• users want access to semantics encoded by authors even if that information is meant for fallback or machine purposes
• when authoring content fallback, authors are not always able to verify the content except through examination of the HTML source
• While HTML conformance expressly prohibits certain element hierarchies and avoids some attribute collisions, HTML currently provides no guidance to UAs on handling non-conforming content (e.g., an anchor link child of an anchor link)

Problem statement overview

HTML5 should provide UA guidance on user-interfaces to access specific HTML data and provide specific guidance on hierarchical attributes and elements. Each of the following sources of document data within an HTML document may require UI enhancements beyond the straightforward rendering of element contents:

• title attribute
• href attribute and the resource referenced by the attribute
• cite attribute and the resource referenced by the attribute
• src attribute and the resource referenced by the attribute
• longdesc and described-by attribute and the resource referenced by the attribute
• alt attribute
• the class attribute
• head metadata such as
  • contents of the title element
  • name / content pairs for meta elements
  • rel / href pairs for link elements and the resource referenced by the href attribute
• The contents of a replaced element
• Tables
  • Summary for tables
  • Abbreviations for header cells
  • Axis membership for table cells

Different UAs and different sources of document data have led to various methods of accessing the data. For example the use of the href attribute on an A (anchor) element leads to the activation of the link and typically the replacement of the document in the current viewport. Hover-views have been used by various UAs to display the value of both the alt attribute and the title attribute. The src attribute on an IMG element leads to the embedding of the referenced resource. These various methods provide users with access to the data provided by this document data. However, often times UAs provide insufficient UI for users to access all of this data without painstaking reading the source of the document. Currently this data can be conveyed to users through various means:

• replacement (embedding): where the element referencing other resources is replaced by some representation of the resource (e.g., an rendering of an image resource replaces the IMG element referencing that image)
• activation: where the element becomes a link to another referenced resource and that resource is loaded in the same viewport or a new viewport upon receiving a click event
• hover view: where a new frame is drawn above the main document view displaying other content
• display in the chrome: otherwise non-displayed content of an attribute or element is displayed in a portion of the window not dedicated to rendering the document itself
• display in or activation from a contextual menu
• swapping between alternate document data in-place (such as alt attribute values, or the fallback contents of a replaced element)
• display in the window’s title bar
• display through the UAs menu bar
• display in an info/inspector pane

UAs also traditionally provide a source view as a means of last resort for users to access document semantics, however an inspector pane would be a much more user-friendly place to gather such information.

Proposed solutions

The HTML 5 draft should add specific guidance that browser UAs must provide an inspector (whether pane, window, sidebar, etc.) to view additional document semantics not already rendered by the stylesheet including (recommended visual means in parentheses):

• title attribute (hoverview and inspector)
• href attribute (chrome)
• resource referenced by the href attribute (activation)
• cite attribute (inspector)
• resource referenced by the cite attribute (secondary activation or embedded in citation list)
• src attribute (inspector)
• resource referenced by the src (or data) attribute (embedded)
• longdesc / described-by attributes (inspector)
• resource referenced by the longdesc / described-by attributes (secondary/tertiary activation)
• alt attribute (inspector or embedded in the case of a failed resource or by indication of user defaults)
• the class attribute (inspector; while class attributes are not required to be human-readable they often are and may convey additional semantic information)
• head metadata such as
  • contents of the title element (window title bar)
  • name / content pairs for meta elements (inspector)
  • rel / href pairs for link elements (inspector)
  • resource referenced by the href attribute for for link elements (activation from the menu bar, the chrome, or activation from inspector)
  • stylesheet titles (selection of alternates by title from chrome or inspector)
• The contents of a replaced element (inspector or through a contextual menu selection to display alternates in place)
• Tables
  • Summary for tables
  • Abbreviations for header cells
  • Axis membership for table cells

Content from the above list displayed by other means may be omitted from the inspector pane, or UAs may provide all of this data in the inspector pane even when it is made available elsewhere at the UAs own discretion.

Other data made part of the document may be omitted from direct view or may be included in the inspector. UAs including other information in the inspector should maintain this information in a separate advanced inspector pane to avoid overwhelming novice users. Such advanced semantic data includes:

• script data
• stylesheet data
• all other content attributes
• DOM attributes

List of important document data likely requiring inspector

• title attribute (especially if CSS later provides mechanism for authors to control hover views)
• class attribute
• lang (or xml:lang) attribute (may provide a localized string for the language instead of the attribute value)
• alt attribute (inspector or embedded in the case of a failed resource or by indication of user defaults or with a contextual menu to display alternate in place)
• KeyBindingMarkup: element’s keybinding key combination and keybinding event
• URIs
  • cite attribute
  • src attribute
  • longdesc / described-by attributes
• Resources
  • resource referenced by the cite attribute
  • resource referenced by the longdesc / described-by attributes (alternately with a contextual menu to display long description fragment in place)
• Document relational data
  • associated definitions and abbreviation expansions
  • associated citations, quotations, attributions, and bibliographic references
  • associated subtext or asides
• Head metadata such as
  • name / content pairs for meta elements
  • rel / href pairs for link elements
• Document list data
  • table of contents
  • indexes
  • reference list
  • glossary
  • presentation legend
• Tables
  • Summary for tables
  • Abbreviations for header cells
  • Axis membership for table cells

Recommendations for secondary and tertiary link activation

Link activation occurs from a click event on an hyper-reference element. In cases where UAs want to give users access other information through activation on an element UAs should provide secondary and tertiary means of activating links. For example an element or element hierarchy could potentially have hyper-references to an anchor, a citation, and a described-by for an image:

<a href='ananchoruri'><img aria-describedby='adescriptionuri'></a>

A straight click on this document fragments corresponding frame should activate the anchor link. UAs must provide another means to activate the other links such as: a contextual menu; an inspector interface; or enabling/disabling buttons in the chrome.

UA handling of errant and non-errant hierarchical data

replacement

When replacing elements or embedding content, the most senior ancestor referencing a valid and available resource for embedding should override the processing of any other elements contained within. In this way the contents of the replaceable element serve as the fallback content for that element.

activation

When activating or displaying the URI for a link, UAs should use the URI value of the element corresponding to the current box even if that element is contained within another element with a different URI value. In other words, UAs should respect the hierarchy of the element as those elements are rendered in a hierarchy of box frames, changing the activated URI as the user changes the target of the click event.

hover view

When displaying the document data in a hover view, the data should be displayed for the current context of the user’s focus. In other words, UAs should display the data for the element corresponding to the current frame under the pointing device. Also when displaying the document data in a hover view, UAs should provide alternate access to this same data since authors may use their own hover views overriding the UAs user-interface.

display in the chrome

When displaying the document data in the chrome, the data should be displayed for the current context of the user’s focus. In other words, UAs should display the data for the element corresponding to the current frame under the pointing device.

display in or activation from a contextual menu

Like hover views, UAs must adjust data accessed from contextual menus to the user’s current context. This implies that for hierarchically arranged elements with relevant document data, the menu should focus on the current element in the user’s context. However, a contextual menu might also provide additional information ancestor elements. For example an errant A element contained in side another A element, each with their own href attribute, might be displayed from a contextual menu allowing activation of either link along with separate activation of multiple secondary and tertiary links in errant documents.

display in an info/inspector pane

An inspector is perhaps the most flexible way to display document data to users. Even when the document contains a hierarchy of data, UAs may provide inspectors that allow the user to traverse that element hierarchy: drilling down to access document data in descendant elements or for example activating a pop-up menu to access ancestors of the currently selected element.

Discussion and evaluation

Email

WG members should post feedback and other discussion to the WG’s list serve (the URI for the links below provides date information). Search on this email subject.

See also

• ABetterAlt: A Better ALT
• Bugreport 5756