Standard Generalized Markup Language (SGML)

• an international standard (ISO 8879:1986)
• non-proprietary
• explicit* markup
• descriptive markup
• document grammars (~ BNF)
• flexible character-set declaration

Extensible Markup Language (XML)

• a de facto standard (W3C Recommendation 1998)
• subset of SGML
• Web-friendly
• easier to parse than SGML
• few optional features
• Unicode

The World Wide Web Consortium is a member-supported organization which creates Web standards.

Our mission:

A transcription (re)produces a character sequence.

But virtually all transcriptions of texts do more.

Informally, we sometimes say they ‘add’ information to the text using markup — debatable.

Consider (virtually) any machine-readable transcription of any text.

                                      1875
                                   PEER GYNT
                                by Henrik Ibsen
  THE CHARACTERS
  ASE, a peasant's widow.
  PEER GYNT, her son.
  TWO OLD WOMEN with corn-sacks. ASLAK, a smith. WEDDING-GUESTS. A
    MASTER-COOK, A FIDDLER, etc.
  A MAN AND WIFE, newcomers to the district.
  SOLVEIG and LITTLE HELGA, their daughters.
  THE FARMER AT HEGSTAD.
  INGRID, his daughter.
  THE BRIDEGROOM and His PARENTS.
  THREE SAETER-GIRLS. A GREEN-CLAD WOMAN.
  THE OLD MAN OF THE DOVRE.
  A TROLL-COURTIER. SEVERAL OTHERS. TROLL-MAIDENS and TROLL-URCHINS. A
    COUPLE OF WITCHES. BROWNIES, NIXIES, GNOMES, etc.
  AN UGLY BRAT. A VOICE IN THE DARKNESS. BIRD-CRIES.
  KARI, a cottar's wife.
  Master COTTON, Monsieur BALLON, Herren VON EBERKOPF and
    TRUMPETERSTRALE, gentlemen on their travels. A THIEF and A RECEIVER.
  ANITRA, daughter of a Bedouin chief.
  ARABS, FEMALE SLAVES, DANCING-GIRLS, etc.
  THE MEMNON-STATUE (singing). THE SPHINX AT GIZEH (muta persona).
  PROFESSOR BEGRIFFENFELDT, Dr. Phil., director of the madhouse at
    Cairo.
  HUHU, a language-reformer from the coast of Malabar. HUSSEIN, an
    eastern Minister. A FELLAH, with a royal mummy.
  SEVERAL MADMEN, with their KEEPERS.
  A NORWEGIAN SKIPPER and HIS CREW. A STRANGE PASSENGER.
  A PASTOR. A FUNERAL-PARTY. A PARISH-OFFICER. A BUTTON-MOULDER. A
    LEAN PERSON.
    The action, which opens in the beginning of the nineteenth
  century, and ends around the 1860's, takes place partly in
  Gudbrandsdalen, and on the mountains around it, partly on the coast
  of Morocco, in the desert of Sahara, in a madhouse at Cairo, at sea,
  etc.
                                   ACT FIRST
  SCENE FIRST
  [A wooded hillside near ASE's farm. A river rushes down the slope.
  On the further side of it an old mill shed. It is a hot day in
  summer.]
  [PEER GYNT, a strongly-built youth of twenty, comes down the
  pathway. His mother, ASE, a small, slightly built woman, follows
  him, scolding angrily.]
  ASE
       Peer, you're lying!
  PEER [without stopping].
       No, I am not!
  ASE
       Well then, swear that it is true!
  PEER
       Swear? Why should I?
  ASE
       See, you dare not!
       It's a lie from first to last.

Some transcriptions have more information:

|s001
|l001 ich sâz ûf eime steine
|l002 und dâhte bein mit beine
|l003 dar ûf sazt ich den ellenbogen

or

|b001
|l001a Hw*a/et, we GAR-DENA
|l001b in geardagum t*rym gefrunnon
|l002a Hu t*a *a/ed*elingas 
|l002b ellen fremedon

Does this transcription include extra-textual information?

S0CF6003 v
[S [N TROUBLED_JJ [ morning_NNT1 television_NN1 ] 
station_NN1 GMTV_NP1 N] finally_RR [V had_VHD 
[N something_PN1 [Ti to_TO smile_VVI [P about_II
P]Ti]N][Nr last_MD night_NNT1 [Fr when_RRQ [N 
it_PPH1 N][V was_VBDZ revealed_VVN [Fn[N it_PPH1 
N][V gained_VVD [N an_AT1 extra_JJ million_NNO 
viewers_NN2 N][P over_II [N the_AT last_MD two_MC
weeks_NNT2 N]P]V]Fn]V]Fr]Nr]V] ._YSTP S]

More formally, we can oppose content and markup. A markup language:

• defines a vocabulary to use in markup.
• specifies how markup constructs can occur (containment, sequence, ...), providing a contract between data sources and data sinks.
• tells how to distinguish markup from content.

Some simple XML:

<!DOCTYPE greetings [
<!ELEMENT greetings (hello+) >
<!ELEMENT hello (#PCDATA) >
<!ATTLIST hello

          lang CDATA #IMPLIED >
<!ENTITY szlig   "&#223;" >
<!ENTITY uuml    "&#252;" >
]>

<greetings>

<hello lang="en">Hello, world!</hello>

<hello lang="fr">Bon jour, tout le monde!</hello>

<hello lang="no">Goddag!</hello>

<hello lang="de">Guten Tag!</hello>

<hello lang="de-franken">Gr&uuml;&szlig; Gott!</hello>

</greetings>

XML is a meta-language for defining markup languages.

Applications of XML are markup languages:

• XHTML
• TEI, CES, ...
• SVG (Scalable Vector Graphics)
• XSLT (XSL Transformations)
• MathML
• ...

The original plan:

• XML (subset of SGML)
• XLink (subset of HyTime)
• XSL (Extensible Stylesheet Language) (subset of DSSSL)

The plan, as modified by experience:

• XML
• Namespaces in XML
• XML Information Set
• XLink
• XPointer
• XSLT (XSL Transformations)
• XSL Formatting Objects
• XPath
• XML Schema
• XML Query
• Document Object Model
• SOAP (Simple Object Access Protocol)
• WSDL (Web Services Description Lanuage)
• ...

Origin pragmatic, not theoretical.

Later aligned (partially) with language theory.

Formal specification of validity rules → automated validation. (Cf. Algol vs. Fortran.)

Distinction between document type definition (DTD) and “the set of effective formal declarations”.

Document grammars may have several uses:

• in the struggle against dirty data
• as documentation of a contract between data provider and data consumer
• as documentation of the content of data flows
• as specification of client/server protocols

SGML/XML DTDs resemble Backus-Naur Form grammars, but:

• They describe bracketed languages* ...
• ... so ‘non-terminals’ are visible*.
• SGML allows ‘inclusion’ and ‘exclusion’ exceptions.
• They are not purely grammatical (notations, entities).
• Determinism rule.

Limericks and canzone:

poem     ::= limerick | canzone

limerick ::= trimeter trimeter dimeter 
             dimeter trimeter
trimeter ::= CHAR+
dimeter  ::= CHAR+

canzone   ::= aufgesang abgesang
aufgesang ::= stollen stollen
stollen   ::= line+
abgesang  ::= line+

Limericks and canzone:

<!ELEMENT poem (limerick | canzone) >

<!ELEMENT limerick (trimeter, trimeter, 
                    dimeter, dimeter, 
                    trimeter)>
<!ELEMENT trimeter (#PCDATA)>
<!ELEMENT dimeter  (#PCDATA)>

<!ELEMENT canzone   (aufgesang, abgesang) >
<!ELEMENT aufgesang (stollen, stollen) >
<!ELEMENT stollen   (l+) >
<!ELEMENT abgesang  (l+) >
<!ELEMENT l         (#PCDATA) >

In version 1 of this schema, we imitate the DTD slavishly.

At the outer level is a schema element:

<xsd:schema>
 <!--* element declarations go here *-->
</xsd:schema>

N.B. the schema does not identify a document-root element / start symbol.

 <xsd:element name="l">
  <xsd:complexType mixed="true">
   <xsd:sequence>
   </xsd:sequence>
  </xsd:complexType>
 </xsd:element>

or

 <xsd:element name="l" type="xsd:string"/>

Core functions quite distinct:

• XSLT: formatting.
• XPath: addressing.
• XML Query: tree manipulation.

Serious formatting requires both

• styling text blocks
  • font family (Times Roman, Helvetica, ...)
  • font treatment, weight (italic, bold, demi-bold, ...)
  • measure, color, ...
• tree transformation.
  • table of contents, list of figures, ...
  • indices
  • footnotes, endnotes
  • running headers and footers

A generic tree-transformation tool

• output-driven
• input-driven
• functional* language
• Turing complete
• XML syntax
• untyped

An industrial-strength query language for XML data

• explicit data model
• formal semantics
• keyword syntax
• Turing complete
• statically typed

Many applications need to ‘address’ parts of XML documents:

• formatting (e.g. XSLT)
• hyperlinking
• document construction
• query / search and retrieval
• schema / language specification
• ...

XPath captures the common functionality.

A document is an ordered tree with

• a root node
• element nodes
• text nodes
• attribute nodes
• namespace nodes
• processing instructions
• comment nodes

No structure sharing. No entity boundaries. Namespace prefixes resolved.

An expression is a sequence of steps:

Each step

• starts from some current node(s),
• moves to some result node(s).

Cf. HyTime, TEI location ladders.

A step is where E.g. descendant::figure[@rend="svg"]

• child (→ e, t, c, p)
• parent (→ e)
• attribute (→ a)
• following, following-sibling (→ e, t, c, p)
• preceding, preceding-sibling (→ e, t, c, p)
• self
• namespace (→ n)
• ancestor, ancestor-or-self (→ e)
• descendant, descendant-or-self (→ e, t, c, p)

XPath and XSLT already widely used for queries.

• select elements, attributes, strings, numbers
• select by position in tree, generic identifier, attribute name, value, content
• co-occurrence constraints

Some drawbacks:

• no data types
• no type safety
• variables are external

Sample search ...