Uighur Gap Analysis

W3C Group Draft Note

More details about this document
This version:
https://www.w3.org/TR/2022/DNOTE-arab-ug-gap-20220712/
Latest published version:
https://www.w3.org/TR/arab-ug-gap/
Latest editor's draft:
https://w3c.github.io/alreq/gap-analysis/arab-ug-gap.html
History:
https://www.w3.org/standards/history/arab-ug-gap
Commit history
Editor:
(W3C)
Feedback:
GitHub w3c/alreq (pull requests, new issue, open issues)

Abstract

This document describes and prioritises gaps for the support of Uighur on the Web and in eBooks. In particular, it is concerned with text layout. It checks that needed features are supported in W3C specifications, in particular HTML and CSS and those relating to digital publications. It also checks whether the features have been implemented in browsers and ereaders. This is a preliminary analysis.

Status of This Document

This section describes the status of this document at the time of its publication. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

This document describes and prioritises gaps for the support of Hungarian on the Web and in eBooks. In particular, it is concerned with text layout. It checks that needed features are supported in W3C specifications, in particular HTML and CSS and those relating to digital publications. It also checks whether the features have been implemented in browsers and ereaders. It is linked to from the language matrix that tracks Web support for many languages.

This document is an individual contribution, and is not currently a work item in any group, however, you can contact the Internationalization Working Group for more information. We welcome contributions to this and/or other documents.

This document was published by the Internationalization Working Group as a Group Draft Note using the Note track.

Group Draft Notes are not endorsed by W3C nor its Members.

This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

The W3C Patent Policy does not carry any licensing requirements or commitments on this document.

This document is governed by the 2 November 2021 W3C Process Document.

1. Introduction

The W3C needs to make sure that the needs of scripts and languages around the world are built in to technologies such as HTML, CSS, SVG, etc. so that Web pages and eBooks can look and behave as people expect around the world.

This page documents difficulties people encounter when trying to use the Uighur language with the Arabic script on the Web.

Having identified an issue, it investigates the current status with regards to web specifications and implementations by user agents (browsers, e-readers, etc.), and attempts to prioritise the severity of the issue for web users.

A summary of this report and others can be found as part of the language matrix.

1.1 Work flow

This version of the document is a preliminary analysis

Gap analysis work usually starts with a preliminary analysis, conducted quickly by one or a small group of experts. Then a more detailed analysis is carried out, involving a wider range of experts. The detailed analysis may involve the development of tests, in order to illustrate issues and track results for browsers. The next phase is ongoing maintenance. It is expected that the resulting document will not be frozen: as gaps are fixed, this should be noted in the document. It is also possible that new gaps are noticed or arise, and they can be added to this document when that happens.

1.2 Prioritization

This document not only describes gaps, it also attempts to prioritise them in terms of the impact on the local user. The prioritisation is indicated by colour.

Key:

It is important to note that these colours do not indicate to what extent a particular features is broken. They indicate the impact of a broken or missing feature on the content author or end user.

Basic styling is the level that would be generally accepted as sufficient for most Web pages. Advanced level support would include additional features one might expect to include in ebooks or other advanced typographic formats. There may be features of a script or language that are not supported on the Web, but that are not generally regarded as necessary (usually archaic or obscure features). In this case, the feature can be described here, but the status should be marked as OK.

The decision as to what priority level is assigned to a described gap is down to the experts doing the gap analysis. It may not always be straightforward to decide. If a given section in this document refers to more than one feature that is broken, each with different impacts on Web users, the priority for the section should be the lowest denominator.

A cell can be scored as OK if the feature in question is specified in an appropriate specification, and is supported by user agents. A specification that is in CR or later and has two implementations in 'major' browsers will count. This means that the feature may not be supported in all browsers yet. (At some point in the future we may try to distinguish, visually, whether support is available in a specification but still pending in major browsers or applications.)

2. Text direction

See also General page layout & progression for features such as column layout, page turning direction, etc. that are affected by text direction.

2.1 Vertical text

Are the script requirements for vertically oriented text met? What about if you mix vertical text with scripts that are normally only horizontal? Do you need a switch to use different characters in vertical vs. horizontal text? Does the browser support short runs of horizontal text in vertical lines (tate-chu-yoko in Japanese) as expected? Is the orientation of characters and the directional ordering of characters supported as needed? See available information or check for currently needed data.

#242 CSS Writing-modes sideways values not implemented

This issue is applicable to most languages.

Vertical text may occur for special effects (the spine of a book, table column headings, etc). Typographically, it is simply horizontal text that is rotated. There is no way to do this effectively until browsers support the new CSS properties.

For more details, see this GitHub issue, which is being used to track this gap.

2.2 Bidirectional text

If this script runs right-to-left, are there any issues when handling that? Is bidirectional text adequately supported? What about numbers and expressions? Do the Unicode bidi controls and HTML markup provide the support needed? Is isolation of directional runs problematic? See available information or check for currently needed data.

#256 :dir lacks wide support

This issue is applicable to all languages with RTL orthographies.

Style sheets need to add special rules for RTL styles if they are not supported by logical properties or values.

One approach is to create a second style sheet which, when pulled into an HTML page, overrides styles in the main style sheet with settings for RTL text. This approach is not ideal because it requires maintaining the styles in two separate locations, which can therefore get out of synch, and it requires explicit addition of a call to the second style sheet in every page that will support RTL text.

The :dir() pseudo-class avoids these issues by allowing the content author to include the RTL variations in the same style sheet as the others. However, it is not yet supported by all major browser engines.

A workaround that precedes selectors with :root[dir=rtl] selector_here can help, but :dir() is a better way forward. There also appear to be Shadow DOM implications if this is not supported.

Specs: selectors-4 Describes the :dir() feature in CSS. html5 Describes the :dir() feature in HTML.

Tests & results: Can I Use indicates that Gecko supports :dir(), but Blink, and Webkit do not.

The functionality appears to be available in Blink, but behind a flag.

Browser bug reports: BlinkWebkit

Priority: This pseudo-class significantly improves the process of designing style sheets that can cater for both LTR and RTL pages. It is a valuable enabler for better internationalisation of the Web for the billion or so potential users of RTL scripts.

#244 Support for isolating formatting characters lacking in some browsers

This issue is applicable to all languages using a right-to-left script.

For support of bidirectional plain text, the Unicode Standard provides a number of formatting characters, which include RLI, LRI, PDI and FSI. See an explanation of how these work.

Although markup should be used most of the time in HTML pages, there are parts of an HTML document that don't support markup, such as the title element and title, alt, and other attributes. These characters can be necessary for managing inline runs of such text.

For more details, see this GitHub issue, which is being used to track this gap.

#217 Logical CSS keyword support needed

This issue is common to all RTL scripts.

Adoption of logical keywords such as -start and -end, rather than -left and -right needs to be completed.

For margins, padding, block size, border colour, width & style, logical keywords such as margin-inline-start or margin-block-end are widely supported by major browsers in their simplest forms (such as those just mentioned). However, logical properties are not well supported in shorthands such as margin-block or margin-inline or the margin property. The lack of support for shorthands is significant, since they are expected to have high use.

Support for the margin shorthand is currently held up by a discussion about the appropriate syntax. See this CSS issue.

Other logical properties that are not widely supported include border radius, caption side, and float. Firefox does support border-radius and floats, but otherwise these are not supported.

See a set of test results.

These new values also need to be widely supported in editing applications.

3. Characters and phrases

3.1 Characters & encoding

Are there any character repertoire issues preventing use of this script on the Web? Do variation selectors need attention? Are there any other encoding-related issues? See available information or check for currently needed data.

#251 Characters & encoding ok

No known problems in this area.

3.2 Fonts

Do the standard fallback fonts used in browsers (eg. serif, sans-serif, cursive, etc.) match expectations? Are special font or OpenType features needed for this script that are not available? See available information or check for currently needed data.

#252 Fonts ok

No known problems in this area.

3.3 Font styles, weight, etc

This covers ways of modifying the glyphs, such as for italicisation, bolding, oblique, etc. Do italic fonts lean in the right direction? Is synthesised italicisation problematic? Are there other problems relating to bolding or italicisation - perhaps relating to generalised assumptions of applicability? See available information or check for currently needed data.

3.4 Glyph shaping and positioning

Does the script in question require additional user control features to support alterations to the position or shape of glyphs, for example adjusting the distance between the base text and diacritics, or changing the glyphs used in a systematic way? Do you need to be able to compose/decompose conjuncts, or show characters that are otherwise hidden, etc? See available information or check for currently needed data.

3.5 Cursive text

If this script is cursive (eg. Arabic, N’Ko, Syriac, etc), are there problems or needed features related to the handling of cursive text? Do cursive links break if parts of a word are marked up or styled? Do Unicode joiner and non-joiner characters behave as expected? See available information or check for currently needed data.

#222 Inline elements/styling break cursive shaping

This issue is applicable to text in all cursive scripts.

When elements surround part of a cursive run of text, and apply styling, the results often break the cursive joins. (See the results of trying to colour individual letters in the illustration below – as expected above, unsuccessful below.)

Screenshot 2021-01-22 at 15 19 49

shaping-2

Specs: After some discussion, the CSS spec requires the following (see CSS Text: Shaping Across Element Boundaries):

  1. Markup alone around part of a joined up sequence must not disturb the joining behaviour.
  2. Styling that doesn't affect the characters, such as text-decoration, must not break the joins.
  3. Styling that does affect the shape of the characters should not break the joins, however the result is not well defined for complex glyph arrangements such as ligatures where the markup occurs between characters that make up the ligature.
  4. Non-zero margins, padding, and borders, will break the join, as will isolation boundaries.

Tests & results: Interactive test, A span with a colour change for one letter in an Arabic word doesn't break the joining behaviour

  • Gecko: ✅ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0*
  • Blink: ✅ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36*
  • Webkit: ❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15*
I18n test suite, Cursive joining
  • Gecko: ✅❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0*
  • Blink: ✅ ❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36*
  • Webkit: ❌ *Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15*
Webkit breaks cursive joining as soon as markup appears around a character, and so obviously fails for any type of styling application, too.

Gecko and Blink keep joins for styling that doesn't affect the shape of the characters (eg. text-decoration), and keeps it for colour changes, however Firefox fails for changes in font-weight, font-style, and font-size, as well as for markup such as em and b tags.

(Gecko and Blink also only pass some of the tests for non-zero margin/padding/border and bdi isolation. Which expect the cursive joins to be broken.)

Browser bug reports: GeckoBlinkWebkit

Priority: It would be useful to decide on the potential impact of the failures described here, so as to prioritise the issue. Is the inability to surround/style parts of a word a significant issue? It may be problematic when defining a term (using dfn) if the term is only part of the run of letters between spaces, eg. after the definite article.

#221 Text opacity shows glyph overlap

This issue is likely to apply to all cursive script text.

When opacity is applied to text, Gecko and Blink produce dark patches where the cursive glyphs overlap, but Webkit and Legacy Edge don't. See the test.

text-opacity

For more details, see this GitHub issue, which is being used to track this gap.

#220 Text stroke cuts joining glyphs apart

This issue is likely to apply to all cursive script text, but also to other scripts that have joins between letters.

When text stroke effects are applied to cursive text, they should not interrupt the cursive flow. Unfortunately, that's not the case for current implementations. Overlaps where the glyphs join produce double breaks in the cursive flow.

This example shows the typical result for Arabic.

text-stroke

The set of modern languages using cursive scripts includes those that use the following scripts: Arabic, Syriac, Mongolian, Mandaic, N'Ko, Adlam, Hanifi Rohingya. It also affects text in archaic scripts, including Phags Pa, Manichaean, Psalter Pahlavi, Sogdian, Old Uyghur, Chorasmian.

However, this also affects scripts such as Devanagari (Hindi, Marathi, etc), Gurmukhi (Punjabi), Bengali, Gondi, Modi, Newa, etc. This example is Hindi:

Screenshot 2022-02-08 at 17 12 09

Specs: Issue, Standardize text-stroke Open.

The property text-stroke is not yet in CSS, but has been implemented in major browsers under the name -webkit-text-stroke.

-webkit-text-stroke is currently specified in WhatWG's Compatibility spec.

Tests & results: Interactive test, Text stroke applied to Arabic text doesn't cut joining glyphs apart
i18n test suite, Text stroke

In Gecko, Blink, and Webkit the stroke around the text interrupts the cursive flow.

Priority: Marking as advanced because this is a decorative feature.

3.6 Baselines, line-height, etc

Does the browser support requirements for baseline alignment between mixed scripts and in general? See available information or check for currently needed data.

3.7 Transforming characters

Does your script need special text transforms that are not supported? Does your script convert letters to uppercase, capitalised and lowercase alternatives according to your typographic needs? Do you need to to convert between half-width and full-width presentation forms? See available information or check for currently needed data.

#253 Transforming characters ok

No known problems in this area.

3.8 Grapheme/word segmentation & selection

This is about how text is divided into graphemes, words, sentences, etc., and behaviour associated with that. Do Unicode grapheme clusters appropriately segment character units for your script? When you double- or triple-click on the text, is the expected range of characters highlighted? When you move through the text with the cursor, or backspace, etc. do you see the expected behaviour? (Some of the answers to these questions may be picker up in other sections, such as line-breaking, or initial-letter styling.) See available information or check for currently needed data.

#254 Grapheme/word segmentation & selection ok

No known problems in this area.

3.9 Punctuation & other inline features

Are there specific problems related to punctuation or the interaction of the text with punctuation (for example separation of punctuation from previous text, but allowing no line break between)? Are there issues related to handling of abbreviation, ellipsis, or iteration? Are there problems related to bracketing information or demarcating things such as proper nouns, etc? See available information or check for currently needed data.

3.10 Text decoration

This is about ways of marking text (see also specific sections dedicated to quotations and inline notes/annotations). Is it possible to express emphasis or highlight content as expected? Bold, italic and under-/over-lines are not always appropriate, and some scripts have their own unique ways of doing things, that are not in the Western tradition at all. Text delimiters mark certain items or sections off from the main text, such as book names in Chinese, quotations, head markers in Tibetan, etc, and often involve the use of punctuation. Is there any behaviour that isn't well supported, such as overlines for numeric digits in Syriac? Are there issues about the positioning or use of underlines? Some aspects related to the drawing of lines alongside or through text involve local typographic considerations. Do underlines need to be broken in special ways for this script? Do you need support for additional line shapes or widths? Does the distance or position of the lines relative to the text need to vary in ways that are not achievable? Are lines correctly drawn relative to vertical text? See available information or check for currently needed data.

3.11 Quotations

Are there any issues when dealing with quotations marks, especially when nested? Should block quotes be indented or handled specially? See available information or check for currently needed data.

3.12 Text spacing

Some scripts create emphasis or other effects by spacing out the words, letters or syllables in a word. Are there requirements for this script/language that are unsupported? (For justification related spacing, see below.) See available information or check for currently needed data.

3.13 Inline notes & annotations

The ruby spec currently specifies an initial subset of requirements for fine-tuning the typography of phonetic and semantic annotations of East Asian text, including furigana, pinyin and zhuyin fuhao systems. Is is adequate for what it sets out to do? What other controls will be needed in the future? What about other types of inline annotation, such as warichu? (For referent-type notes such as footnotes, see below.) See available information or check for currently needed data.

#255 Inline notes & annotations

No known problems in this area.

3.14 Data formats & numbers

If the script has its own set of number digits, are there any issues in how they are used? Does the script or language use special format patterns that are problematic (eg. 12,34,000 in India)? What about date/time formats and selection - and are non-Gregorian calendars needed? Do percent signs and other symbols associated with number work correctly, and do numbers need special decorations, (like in Ethiopic or Syriac)? How about the management of personal names, addresses, etc. in web pages: are there issues? See available information or check for currently needed data.

4. Lines and Paragraphs

4.1 Line breaking

Does the browser capture the rules about the way text in your script wraps when it hits the end of a line? Does line-breaking wrap whole 'words' at a time, or characters, or something else (such as syllables in Tibetan and Javanese)? What characters should not appear at the end or start of a line, and what should be done to prevent that? See available information or check for currently needed data.

See also hyphenation below.

4.2 Hyphenation

Is hyphenation used for your script, or something else? If hyphenation is used, does it work as expected? (Note, this is about line-end hyphenation when text is wrapped, rather than use of the hyphen and related characters as punctuation marks.) See available information or check for currently needed data.

#250 Uighur hyphenation should be supported

This issue is applicable to Uighur.

Unlike Arabic, which is never hyphenated, words in Uighur text in the Arabic script can be broken at line ends. A short horizontal stroke is added at the end of the line, separated from the previous text by a small space, and joining forms are retained for left-joining letters at line end and line start.

hyphenation

This hyphenation method needs to be supported in browsers.

Specs: css-level-3 provides controls for hyphenation, and alludes to the requirement to create joining letter forms at line end and start for Arabic-script text where hyphenation is allowed, but leaves it to the browser implementation to produce the specific type of hyphenation that is appropriate to a given language.

Tests & results: The following tests use the second half of the text in the image shown above.

interactive test, hyphens:auto makes the browser hyphenate Uighur text and uses a low stroke at the line end, and joining forms at line end and start.

Results:

  • Gecko: ❌ No hyphenation occurs *Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:98.0) Gecko/20100101 Firefox/98.0*
  • Blink: ❌ No hyphenation occurs *Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36*
  • Webkit: ❌ No hyphenation occurs *Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15*

interactive test, After setting hyphens:manual, the browser hyphenates Uighur text where soft hyphens occur. Hyphenation is shown by a low stroke at the line end, slightly separated from the foregoing text, and joining forms at line end and start.

Results:

  • Gecko: ✅❌ The lines break and the line-end and line-start letters have joining forms, but the marker used is an ordinary hyphen and not on the baseline. *Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:98.0) Gecko/20100101 Firefox/98.0*
  • Blink: ✅❌ The lines break and the line-end and line-start letters have joining forms, but the marker used is an ordinary hyphen and not on the baseline. *Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36*
  • Webkit: ✅❌ The lines break but the line-end and line-start letters don't have joining forms, and the marker used is an ordinary hyphen and not on the baseline. *Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15*

Browser bug reports: GeckoBlinkWebkit

Priority: Uighur hyphenation is common in printed material, so it should also work on the Web.

4.3 Text alignment & justification

When text in a paragraph needs to have flush lines down both sides, does it follow the rules for your script? Does the script need assistance to conform to a grid pattern? Does your script allow punctuation to hang outside the text box at the start or end of a line? Where adjustments are need to make a line flush, how is that done? Do you shrink/stretch space between words and/or letters? Are word baselines stretched, as in Arabic? What about paragraph indents, or the need for logical alignment keywords, such as start/end, rather than left/right? See available information or check for currently needed data.

4.4 Lists, counters, etc.

The CSS Counter Styles specification describes a limited set of simple and complex styles for counters to be used in list numbering, chapter heading numbering, etc.The rules plus more counter styles (totalling around 120 for over 30 scripts) are listed in the document Ready-made Counter Styles. Do these cover your needs? Are the details correct? Are there other aspects related to counters and lists that need to be addressed? See available information or check for currently needed data.

#227 Customisable counter-styles are not supported

Customisable counters are needed for many languages.

There is no way for users to create local counter styles that are not built in to the browser, and users also often want to tweak the counter style for particular contexts (esp. the prefix/suffix).

Arabic script text uses local counter styles. While some of these are supported by some browsers, the set of symbols used and their order varies by language.

For more details, see this issue.

4.5 Styling initials

Does the browser or ereader correctly handle special styling of the initial letter of a line or paragraph, such as for drop caps or similar? How about the size relationship between the large letter and the lines alongide? where does the large letter anchor relative to the lines alongside? is it normal to include initial quote marks in the large letter? is the large letter really a syllable? etc. Are all of these things working as expected? See available information or check for currently needed data.

5. Page & book layout

5.1 General page layout & progression

How are the main text area and ancilliary areas positioned and defined? Are there any special requirements here, such as dimensions in characters for the Japanese kihon hanmen? The book cover for scripts that are read right-to-left scripts is on the right of the spine, rather than the left. Is that provided for? When content can flow vertically and to the left or right, how do you specify the location of objects, text, etc. relative to the flow? For example, keywords 'left' and 'right' are likely to need to be reversed for pages written in English and page written in Arabic. Do tables and grid layouts work as expected? How do columns work in vertical text? Can you mix block of vertical and horizontal text correctly? Does text scroll in the expected direction? Other topics that belong here include any local requirements for things such as printer marks, tables of contents and indexes. See available information or check for currently needed data.

5.2 Footnotes, endnotes, etc.

Does your script have special requirements for footnotes, endnotes or other necessary annotations of this kind in the way needed for your culture? (There is a section above for purely inline annotations, such as ruby or warichu. This section is more about annotation systems that separate the reference marks and the content of the notes.) See available information or check for currently needed data.

5.3 Page headers, footers, etc.

Are there special conventions for page numbering, or the way that running headers and the like are handled? See available information or check for currently needed data.

5.4 Forms & user interaction

Are vertical form controls well supported? In right-to-left scripts, is it possible to set the base direction for a form field? Is the scroll bar on the correct side? etc. See available information or check for currently needed data.

6. Other

6.1 Culture-specific features

Sometimes a script or language does things that are not common outside of its sphere of influence. This is a loose bag of additional items that weren't previously mentioned. This section may also be relevant for observations related to locale formats (such as number, date, currency, format support).

6.2 What else?

There are many other CSS modules which may need review for script-specific requirements, not to mention the SVG, HTML, Speech, MathML and other specifications. What else is likely to cause problems for worldwide deployment of the Web, and what requirements need to be addressed to make the Web function well locally?

Show summary