User Input of Mathematics

From Educational Exercises and Activities Community Group

Introduction

Mathematics user input is an important topic. Mathematics user input facilitates scholarly and scientific communication scenarios as well as educational scenarios, providing new varieties of science, technology, engineering and mathematics exercises and activities.

To provide users with accessible mathematics user input, Web browser software applications could invoke operating system platform or third-party components and services. This resembles the way that touchscreen keyboards and handwriting recognition are provided on tablet computers.

Three varieties of components and services for mathematics user input are presently considered. Firstly, graphical user interface widgets with panels of buttons for entering mathematical notations. Secondly, mathematics handwriting recognition components and services. Thirdly, mathematics speech recognition components and services.

Input Elements for Mathematics

Extending the HTML5 form elements, here are some sketches of user input elements for mathematics. 

<input type="math" />

<matharea />

Mathematics Recognition Hints: A Metadata-based Approach

Contextual recognition hints can enhance the accuracy of both handwriting and speech recognition.

A metadata-based approach to mathematics recognition hints involves a new <metadata> element which can contain <meta> and <link> elements. Document elements can reference and connect to <metadata> elements via a new metadata attribute.

Noteworthy is an extension to the <meta> element such that it can also contain XML content. 

<metadata id="hints1">
  <meta name="..." content="..." />
  <meta name="..."><xml>...</xml></meta>
  <link rel="..." href="..." />
</metadata>

<input type="math" metadata="hints1" />

<matharea metadata="hints1" />

Specifying a Mathematics Subject

Hints for mathematics recognition can include specifying the subject of mathematics to be inputted, for instance arithmetic, algebra, geometry, trigonometry or calculus. The Mathematics Subject Classification can be of use as a controlled vocabulary of mathematical subjects. 

<metadata id="hints1">
  <meta name="subject" content="msc2010:12D05" />
</metadata>

Specifying Simple and Composite Symbols

Hints for mathematics recognition can include specifying variables or symbols expected to occur in user input. 

<metadata id="hints1">
  <meta name="symbol" content="x" />
</metadata>

<metadata id="hints1">
  <meta name="symbol">
    <math>
      <mover>
        <mi>a</mi>
        <mo>→</mo>
      </mover>
    </math>
  </meta>
</metadata>

<metadata id="hints1">
  <meta name="symbol">
    <math>
      <msub>
        <mi>λ</mi>
        <mi>i</mi>
      </msub>
    </math>
  </meta>
</metadata>

Specifying Possible User Inputs

Hints for mathematics recognition can include specifying a set of mathematical expressions which are each a possible user input. Recognition components or services may utilize such hints while processing user input. 

<metadata id="hints1">
  <meta name="hint">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>-</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
</metadata>

<metadata id="hints1">
  <meta name="hint">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>+</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
  <meta name="hint">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>-</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
</metadata>

Specifying Recognition Templates

Hints for mathematics recognition can include specifying result templates. A result template is a mathematics expression with one or more empty boxes or blanks.

See also: https://www.w3.org/Math/Documents/Notes/blanks/fill_in_the_blank.html 

<metadata id="hints1">
  <meta name="template">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>-</mo>
      <msup>
        <mi>?</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
</metadata>

Types of Recognition Templates

A result template may be displayed to users with empty boxes or blanks as editable regions. There may be two types of recognition templates: one for specifying a template such that the template is not intended to be visible to the user during input and another for specifying a template such that the template is intended to be visible to the user during input.

It may also be possible to utilize a template syntax in hints.

Template Blanks and Recognition Hinting

A topic for discussion is the capability to describe valid content for empty boxes or blanks in templates, possibly utilizing metadata hints recursively.

Also possible is use of MathML elements, <mn>, <mi>, <mo>, and <mrow>, with question marks indicating blanks for numbers, symbols, operators and expressions.

Document Markup Before and After Mathematics Recognition

It should be possible to provide MathML content for display in an input element before user input. After user input, MathML content should be provided for display. 

<input type="math" metadata="hints1">
  <math>
    <msup>
      <mi>x</mi>
      <mn>2</mn>
    </msup>
    <mo>-</mo>
    <msup>
      <mi>y</mi>
      <mn>2</mn>
    </msup>
  </math>
</input>

<matharea metadata="hints1">
  <math>
    <msup>
      <mi>x</mi>
      <mn>2</mn>
    </msup>
    <mo>-</mo>
    <msup>
      <mi>y</mi>
      <mn>2</mn>
    </msup>
  </math>
</matharea>

Interoperability with MathJax

It is desirable for MathJax, a popular JavaScript library, to interoperate with MathML markup intended for display before and after mathematics recognition.

Forms Processing

Forms submission data from mathematics user input elements is envisioned as MathML-based, e.g. encoded strings of MathML. 

<form method="post" action="math-enabled.php">
  <input type="math" name="math-enabled-input" metadata="hints1" />
  <input type="submit" value="Submit Form" />
</form>

Input Events

Level 2

Input Events Level 2 is scalable and extensible for mathematics input events. Input Events Level 2 provides a DataTransfer interface on the InputEvent interface for a number of scenarios involving text, rich text, hypertext and, perhaps, mathematics.

Other Scenarios for Document Element Metadata

There are other uses for the expressiveness possible with a new <metadata> element and with a metadata attribute on a broader set of document elements.

Accessibility

Document element metadata can provide new expressiveness for accessibility scenarios.

Hyperlinks

In addition to cryptographic hashes indicated on hyperlinks, we can (see also: .NET assembly linking and strongly named assemblies) consider the titles, versions, cultures/languages and the public keys or public key tokens, if linked-to resources are digitally signed. We could attach to hyperlinks the expected metadata of linked-to resources. 

<metadata id="resource1">
  <meta name="title" content="Example Resource" />
  <meta name="version" content="1.0.0.0" />
  <meta name="language" content="en-US" />
  ...
</metadata>

<a href="http://example.com/resource.html" metadata="resource1" />

Articles and Sections

Document element metadata can provide metadata for <article> and <section> elements. 

<metadata id="article1">
  ...
</metadata>

<article metadata="article1">
  ...
</article>

<metadata id="section1">
  ...
</metadata>

<section metadata="section1">
  ...
</section>

Hypertext

<metadata id="txt1">
  ...
</metadata>

<span metadata="txt1">This is a sentence of hypertext.</span>

Tables

Document element metadata can provide metadata for tables. 

<metadata id="table1">
  ...
</metadata>

<table metadata="table1">
  ...
</table>

Multimedia

As multimedia resources contain embedded metadata, <metadata> elements attached to multimedia document elements could, in addition to being specified, be populated when content is loaded. 

<metadata id="img1">
  ...
</metadata>

<img src="image.jpg" metadata="img1" />

Mathematics

<metadata id="equation1">
  ...
</metadata>

<math metadata="equation1">
  <mi>i</mi>
  <mi>ℏ</mi>
  <mfrac>
    <mo>∂</mo>
    <mrow>
      <mo>∂</mo>
      <mi>t</mi>
    </mrow>
  </mfrac>
  <mn>Ψ</mn>
  <mo>=</mo>
  <mfrac>
    <mrow>
      <mo>-</mo>
      <msup>
        <mi>h</mi>
        <mn>2</mn>
      </msup>
    </mrow>
    <mrow>
      <mn>2</mn>
      <mi>m</mi>
    </mrow>
  </mfrac>
  <msup>
    <mo>∇</mo>
    <mn>2</mn>
  </msup>
  <mn>Ψ</mn>
  <mo>+</mo>
  <mi>V</mi>
  <mn>Ψ</mn>
</math>

Speech Recognition

<metadata id="hints2">
  <link rel="lexicon" type="application/pls+xml" href="lexicon-file.pls" />
</metadata>

<input type="text" metadata="hints2" />

<metadata id="hints2">
  <link rel="grammar" type="application/srgs+xml" href="grammar-file.srgs" />
</metadata>

<input type="text" metadata="hints2" />

Multiple Metadata Attributes

Should the metadata attribute become a global attribute or an attribute for a broader set of document elements, we could utilize two attributes, metadata and input-metadata, to distinguish metadata describing the input document element from metadata describing the input. 

<input type="math" metadata="..." input-metadata="hints1" />

Examples

<metadata id="hints1">
  <meta name="subject" content="msc2010:12D05" />
  <meta name="symbol" content="x" />
  <meta name="symbol" content="y" />
  <meta name="hint">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>+</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
  <meta name="hint">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>-</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
</metadata>

<input type="math" metadata="hints1" />

<matharea metadata="hints1" />

<metadata id="hints1">
  <meta name="subject" content="chemistry" />
  <meta name="symbol" content="C" />
  <meta name="symbol" content="H" />
  <meta name="symbol" content="N" />
  <meta name="symbol" content="O" />
  <meta name="hint">
    <math>
      <msub>
        <mi>C</mi>
        <mn>8</mn>
      </msub>
      <msub>
        <mi>H</mi>
        <mn>10</mn>
      </msub>
      <msub>
        <mi>N</mi>
        <mn>4</mn>
      </msub>
      <msub>
        <mi>O</mi>
        <mn>2</mn>
      </msub>
    </math>
  </meta>
</metadata>

<input type="math" metadata="hints1" />

<matharea metadata="hints1" />
Retrieved from "https://www.w3.org/community/exercises-and-activities/wiki/index.php?title=User_Input_of_Mathematics&oldid=531"