| SGML

Document Management for Web Specs

Aaargh! Maintaining specs is a Royal Pain! We need to automate this!

See also:

• spec-prod@w3.org Mail Archives
• W3C XML Specification DTD (“XMLspec”)
• Jim Davis's HTML parser, with RFC generator
• HTML design notebook, with list of implementations
• HTML parser implementations (needs beefing up)

Requirements

• Single Source Format (for each spec, if not a common format for all specs)
• PDF Output
• EPSF Figures (reduced to SVG/PNG for online)
• Tables
• HTML output
• Plain Text output according to IETF formatting guidelines
• Automatic TOC generation, section numbering

Goals

• Open standard source format
• HTTP-based document management (PUT, version control, ...)
• Direct Manipulation Rich-Text Editing, ala FrameMaker, MS Word
• Direct Manipulation HyperLink Editing, ala Nexus (navipress is close)
• Other Automated navigation structures: index, cross references, glossary, references
• FrameMaker interoperability
• MS Word Interoperability
• Emacs/vi interoperability (for low-bandwidth situations)
• LaTeX interoperability (nice typesetting)
• Version control with change logs
• meaningfull diffs

Wishes

• Annotation support (for writer's comments, group comments, public comments)
• "structured sed" -- API for document manipulation (e.g. for TOC generation, glossary, etc.)
• BibTeX/refer-like database. Here's how it works:
  • The schema for the database is modeled after bibtex/refer: class (thesis, techreport, etc.), title, author, date, abstract ...
  • Each record also can be marked "surrogate," meaning the authoratative source is somewhere else: the IETF abstracts file, a W3C tech report, etc.
  • To refer an entry in a document, we use some stylized markup. For example, in the head: <link rel=bibliography href="/Bibliography?">. Then, at the point of the reference, some sort of transclusion link markup: <a rel="embed" href="/Bibliography?id=draft-ietf-http-v10-4;fields=title,author,date,status,abstract"> ... </a> (we might need to use RANGE or AS/AE to avoid nested A elements)
  • The server checks documents at PUT time. When it sees such a reference, it consults the bibliography database (which might involve updating the bib DB from the ietf drafts index) and fills in the appropriate fields.
  • Viola! We never need to manually cite documents again. Not only are the citations reusable (in specs, overviews, etc.) but the database itself can be a valuable browsing/searching resource in and of itself.
• Sharable elements/entities (boilerplate, cross-references)
• Author-chunks distinct from reader-chunks
• Equasion support
• PowerPoint interoperability (goes beyond the scope of specs into presentations)

Possible Solutions

Multiformat tools

This was used for the PNG spec

FrameMaker, WebMaker, ??? print-to-text tool

This is what Roy Fielding (and a lot of other folks) use.

• + Direct-Manipulation editing
• + WYSYWIG postscript output
• + Automatic TOC, cross references, index, section numbering
• + Automatic HTML output with TOC, chunking, navigation (prev/next/up)
• - generating plain-text is a bear
• - generating HTML requires a baroque toolset that we don't have a license to.
• - need a Frame license
• - no way to edit over a telnet connection
• - no way to edit over an HTTP connection (must have local access to files)

HTML+, dsr's tools

Dave Raggett edits the HTML with a text editor (mostly BBEdit on a Mac). He's got some little tools written in C to produce plain text.

• + Automatic TOC, headers/footers in text
• + renders HTML math in plain text output
• + HTML is easy to edit over a telnet connection
• + can work with Navipress to edit HTML via HTTP
• - no postscript output tools
• - text generation tool often requires manual post-processing
• - author chunks must be the same as reader chunks
• - manual maintenance of prev/next links in HTML
• - manual maintenance of cross-references, index, etc.
• - little structure in the source format (e.g. no explicit "Abstract" structure)

Snafu DTD, gf tools, Texi2HTML, COST, Joe English

This is what I ended up using for HTML 2.0

• + structured source format
• + open standard source format
• + Automated postscript output in IETF format with TOC, headers, footers, cross-references, References, section numbers, glossary
• + Automated plaintext output with same features
• + Automated HTML output with glossary, TOC, navigation links
• + LaTeX output
• + TeXinfo output
• + RTF output
• + meaningfull diffs (cuz I edited the source with a text editor)
• + low-bandwidth access (emacs over telnet works fine)
• - Tools require outside support (Joe English)
• - Tools are baroque
• - no WYSYWIG tools (perhaps SoftQuad Author/Editor?)

LinuxDoc

• - print-to-text tool isn't IETF happy
• - no direct-manipulation editing tools

LaTeX, latex2html, IETF print-to-text tools

MS Word, rtf2html, ??? print-to-text tool

Ideal Solution

Source format: HTML dialect

use a strict HTML dialect with: tables, class=abstract, possibly math.

Document Manipulation API: java interface

formerly:

There are lots of web libraries for python. We could eventually specify the interfaces in ILU and use them from lots of languages (C, C++, java, scheme, CommonLisp, Modula-3), but we'd prototype and develop using python.

I've already written little tools to do things like relativize links and such. Rather than doing TOC generation, section nubmbering, etc. during translation, we'd do it in-place in the source, but automatically

changed my mind, since java has at least the potential to address the installation bugs. Plus, it looks like we can write for the java VM in scheme (see kawa)

Chunking support: python scripts

This would handle chunking many HTML documents into one for printing, and many-to-many chunking for author/reader convenience.

PostScript Output: python implementation of Mosaic print tool

This code is already written. Guido translated the postscript printing code from Mosaic into python. We could adapt things like headers/footers for our needs. This eliminates the need for a TeX installation.

Postscript Output: libwww TeX module?

use HTTeXGen module in libwww to generate TeX. It doesn't currently support all the features we need, but it could work. It would rely on a many-to-one html-to-html filter

Postscript Output: html2lout?

lout is kinda like TeX, but it was written since the dawn of postscript, so there's less redundancy between lout and PS than between TeX and PS. The syntax of lout is also cleaner. Lout has table, equasion, etc. packages. A clean html2lout filter should be much more reliable and hands-free than anything based on TeX.

Plain-Text output: custom python app?

there is already python code to do simple html to text formatting, but handling multiple documents, tables etc. needs to be added, as well as IETF style

Plain-Text output: libwww module?

same feature enhancements would be needed.

Wish list

Direct manipulation grammar editor

for SGML DTDs, RFC822 grammars in HTTP specs, etc.

References

SDC

structured document conversion. in use at vuw.ac.nz. Gotta check it out...

Debiandoc-SGML markup manual

4 February 1997 Ian Jackson ijackson@gnu.ai.mit.edu.

source in debian archive under text

Dr. Thomas F. Gordon

GMD FIT - German National Research Center for Information Technology
Research Division Artificial Intelligence
53754 Sankt Augustin, Germany
email: thomas.gordon@gmd.de; phone: (+49 2241) 14-2665

The Collection of Computer Science Bibliographies

Copyright © 1995-1996 Alf-Christian Achilles

Great for the reference section!

spam

example of using spam to munge HTML

SGML tools

as used in The Linux Documentation Project

Postscript

Python

SGML

LinuxDocSGML

lout

Re: Lout to HTML Jin S. Choi (jsc@atype.com) Wed, 13 Nov 1996 19:34:47 -0500 . Nifty thread about LOUT, SGML, DSSSL, HTML, etc. I agree!

Joe English

Eric Raymond

Linux, computational linguistics, www-html <199512221800.NAA09004@locke.ccil.org>

IETF draft guidelines

@@ I know from first-hand experience that producing multi-purpose technical specifications (e.g. IETF plain text, online hypertext, and postscript) is tricky and tedious. I try to keep track of tools that might provide solutions to this problem.

SP

a new C++ based SGML parser by James Clark, the author of SGMLS

DTD Fragments

Another SGMLS/Perl formatter, DTD Fragments. It's not DTD specific and does output to HTML, ASCII and TROFF, it does require a DTD to generic element mapping in Perl for any specific DTD and comes with DocBook and Linuxdoc mappings. The next version will have RTF output, Snafu DTD mapping and better support for applying different styles to the output.

Ken MacLeod

SGMLSpm

Another perl5/ngmls toolet. Includes some support for DocBook->LaTeX, HTML conversion, though that part of the code looks like a one-time shot, not a complete implementation.

DTD2HTML

An SGML DTD documentation/navigation tool by Earl Hood
This tool translates an SGML DTD into HTML, providing hypertext navigation of the document structure. Handy for learning SGML.

PSGML

A GNU Emacs mode for SGML files

Getting HotMetaL by FTP

SoftQuad Inc. Panorama Press Release

Linux Doc/SGML

These guys have taken a very practical approach to SGML for technical documentation. They started with SGMLs from James Clark and the QWERTZ DTD, which mirrors LaTeX structure. Then they added down-translators for groff, HTML, and others. Looks promising.

Hmmm... on closer examination, this is something of a hack. They hacked the DTD, hacked the down-translators, etc. I like the idea of using a LaTeX-like DTD, but I think I'll wait till this matures a little more. also: distribution archive.

GF: General SGML Formatter

another SGMLs based SGML to HTML converter supporting a few sophisticated DTDs

Setting up PSGML and sgmls for HTML

Remote file ftp.jclark.com/pub/sp

CoST

Copenhagen SGMLs Tool -- SGMLs meets Tcl
maintained by Joe English