This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 25136 - You have dropped support for encoding cp862!!
Summary: You have dropped support for encoding cp862!!
Status: RESOLVED WONTFIX
Alias: None
Product: WHATWG
Classification: Unclassified
Component: Encoding (show other bugs)
Version: unspecified
Hardware: All All
: P2 major
Target Milestone: Unsorted
Assignee: Anne
QA Contact: sideshowbarker+encodingspec
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-03-24 14:57 UTC by queency3
Modified: 2014-03-27 18:10 UTC (History)
7 users (show)

See Also:


Attachments

Description queency3 2014-03-24 14:57:09 UTC
firefox 28 had dropped support for ibm cp862 encoding.

This is very important feature for my busyness.

A lot of my clients need this kind of support and i expand this 
support to others as well.

many legacy systems create text files with encoding cp862
i have turned many system to work in firefox thus can read
those text files properly regardless the operation system running.

based on this feature i have transfer clients from windows to linux
as well.

Dropping this support will hurt firefox as a leading 
open source browser for me and make me think again on what browser
should i based my busyness on from now on.

I hope you consider again about that feature which i assure you 
make just good for firefox ,gnu/linux and the whole open source 
comunity.
Comment 1 Anne 2014-03-24 15:18:20 UTC
Why don't you convert the legacy output?

What web pages are breaking?
Comment 2 queency3 2014-03-24 15:49:33 UTC
The legacy systems i mentioned  ,is not mine.
 it belongs to various companies out there.

every file that i need to transfer from cp862 to utf-8
cost me 60 lines of python code to make  (which is  inconceivable 
amount to maintain)

firefox programmers are professionals that know how to do it
in generic fashion and very low amount of code.

for now i didn't see any web pages breaking because
i downgraded all the firefox systems to version 27 (which supports cp862)

if people will transfer legacy system to cloud (in the future)
I'm sure that this text files (downloading from the cloud) will brake as well 
I don't know if you can consider it as "web pages" 
but its a possibility you must take in consideration .
Comment 3 Anne 2014-03-24 15:56:34 UTC
I don't understand. You should only have to write the conversion code once. And then the legacy output you convert will be more interoperable across many systems.
Comment 4 queency3 2014-03-24 16:58:10 UTC
I write code in python not in c.
now 
i need to install python across all legacy computer who makes those 
reports.
i need to install my python code in the legacy computers
i need to change batch files across the legacy computers.

firefox in the other hand already support that encoding 
you do not need to change or do anything just keep that conversion 
in the next releases.

why is it so important for you to drop this support?
Comment 5 Anne 2014-03-24 17:04:14 UTC
We are aligning all browsers on the encodings they support.
Comment 6 queency3 2014-03-25 09:29:57 UTC
its seems odd.
this step is very worrying
is politic involved ?
Comment 8 queency3 2014-03-25 18:14:11 UTC
i can see here : http://wiki.whatwg.org/wiki/Web_Encodings#Encodings
that firefox ought to support ibm862 encoding !! so why did you drop it ?
Comment 9 queency3 2014-03-25 18:30:28 UTC
and may i add that iceweasel identify the ibm862 automatically 
and open my text files as i click on them .

this is how i see "the leading browser" operates .
firefox is a wide trademark its a shame you choose to drop its potential.
Comment 10 Leif Halvard Silli 2014-03-25 19:39:11 UTC
(In reply to Anne from comment #7)
> No, it's the result of research. E.g.
> http://lists.w3.org/Archives/Public/www-archive/2012Apr/att-0058/spectable.

The above posting describes the encodings that all browsers had in common. Didn’t you ever include an encoding in the Encodings spec unless it was supported by all the browsers?

> html and http://wiki.whatwg.org/wiki/Web_Encodings

I ask because this page says that both IE and Firefox, at time of writing, supported cp862.
Comment 11 Leif Halvard Silli 2014-03-26 04:11:43 UTC
Here is some data which I think warrants that the editor looks at the issue once more:

(In reply to Anne from comment #7)
> No, it's the result of research. E.g.

Anne, the research you pointed to was a message to www-archive@ and one wiki page:

> http://lists.w3.org/Archives/Public/www-archive/2012Apr/att-0058/spectable.
> html and http://wiki.whatwg.org/wiki/Web_Encodings

Based on that, the research seems to have some shortcomings:

First, Anne’s message to www-archive@ makes no evaluation of cp862.

Second, the Wiki page has no data on Safari on Chrome, except that they support an unknown subset of ICU (and, for Safari, TEC). No tests have been performed, it seems. (Safari can *detect* (via the charset label) many more encodings than those that are available in its Encoding menu.)

Third, there is no data about today’s situation.

Here is the data I have found:

* Support in IE and Safari is logical order
* Support in Firefox 28 was visual order
* Does was by default visual order: http://en.wikipedia.org/wiki/CP862

* Safari 7.0.2, OSX 10.9.2: cp862 is supported via the label
                       "dos-862", and may be other labels as well.

* IE 11 (Windows 8.1): Same as Safari.

  NOTE: In IE, cp862 may be selected by detection or manually,
        whereas in  Safari, encoding can be detected but cannot
        be selected manually.

* Firefox 28: Support has been dropped completely.
* Firefox 27: does not seem to support any labels at all, but it 
              is possible to manually select the encoding.
* Firefox older: Did previously detect labels, according to Anne’s
                 wikipage.

* Blink (Chrome/Opera): does not support it at all nowadays.
* Chromium (Chrome): probably support = worked liked Webkit, says
                     Anne’s wikipage
* Legacy Opera: not relevant anymore.


Summary: 

The way I understand how the Encoding spec has chosen its encodings, then CP862 was apparently excluded for the wrong reasons: Support was actually quite broad, cross browser. 

1) Safari and IE continue to support CP862. Firefox has just recently dropped support completely. ”New” kid on the block, Blink, does not support it. 
2) Support used to be more or less cross browser, but the encoding spec has, (probably) due to lack of testing/data about Safari and Chrome, deleted CP862 from itself based on the incorrect assumption that it was already not cross browser supported.
3) Due to what was in the Encoding spec, Firefox and Blink, in turn, do not support for Cp862 anymore. (For Firefox, this seems like a given, but I don’t know when Chrome actually stopped supporting the encoding.)

I wonder what will happen to e.g. old email messages in Thunderbird if support is dropped, just like that.

Btw, the Safari behavior seems quite advantageous: It allows documents to be supported, but, the lack of a sub menu item for CP862 in the Encoding menu, in praxis discourages its use and makes users forget about the encoding, which seems good.
Comment 12 Leif Halvard Silli 2014-03-26 04:29:23 UTC
(In reply to queency3 from comment #9)
> and may i add that iceweasel identify the ibm862 automatically 
> and open my text files as i click on them .
> 
> this is how i see "the leading browser" operates .
> firefox is a wide trademark its a shame you choose to drop its potential.

One reason to continue to *not* support CP862 is that Firefox (and therefore probably as well iceweasel/icecat) supports it as a *visual* order hewbrew encoding. 

Whereas IE and Safari/Webkit support it as a *logical* order - aka bidirectional - encoding.

Thus, unless Firefox is aligned with Safari/IE - or the other way around, we will not have compatible implementations. Thus, the spec must decide who is correct, IE+Safari or Firefox. My guess is that Firefox would have to be aligned with IE+Safari.

Since you speak about Firefox, I suppose that the files you are talking about, requires visual order. Am I right? Or would logical order work for you?

The spec cannot keep CP862 if the only reason is that you need the exact implementation that Firefox has. The encoding spec must isntead design something inter-operable. 

But from another angle: Since Firefox supported it as visual order, it could perhaps be said to be correct of Firefox to drop support regardless. That move does, in principle not prevent that it can be re-implemented in Firefox, as logical order.
Comment 13 Anne 2014-03-26 10:55:39 UTC
Actually, we decided to be conservative. So Chrome and Opera not supporting it was sufficient, unless there was strong evidence to the contrary (e.g. Chrome and Opera have open bugs). It further not being interoperable is only more justification.
Comment 14 queency3 2014-03-26 22:32:16 UTC
(In reply to Leif Halvard Silli from comment #12)

> Since you speak about Firefox, I suppose that the files you are talking
> about, requires visual order. Am I right? Or would logical order work for
> you?

I want to be exact here (because i don't familiar with the terms
 (visual or logical) )so i will describe my file and my view.

file hex code: abcd efg. .... ....
view in html :abcd efg. .... ....
the view is from the left to the right only.

hope it answers your question.
Comment 15 Jungshik Shin 2014-03-27 00:50:33 UTC
Chrome has 
never supported cp862.
Comment 16 Leif Halvard Silli 2014-03-27 02:07:30 UTC
(In reply to queency3 from comment #14)

> file hex code: abcd efg. .... ....
> view in html :abcd efg. .... ....
> the view is from the left to the right only.

OK, that means that you require visual order. However, for CP862, visual order does not work cross browser.

To get it in visual order in a cross browser compatible way you should use "ISO-8869-8" rather than "DOS-862". Here is what you could do:

1) Let your application convert each Hebrew letter to its numerical character reference equivalent. For example, convert Hebrew aleph into א

2) Apply the charset label "ISO-8869-8" to the web page. 
  (For example <meta charset="ISO-8869-8"/>)

3) Make sure the dir="" attribute is set to "ltr": <html dir="ltr">.


RESULTS: The letters will display in *visual* order not only in Firefox but also in Safari and Chrome. In IE it might not give you visual order since visual order only works in quirks mode.

(Of course, you should rather use UTF-8. But then I guess you must first convert the text from visual to logical order, which sounds more complicated, to me.)
Comment 17 Leif Halvard Silli 2014-03-27 02:10:52 UTC
(In reply to Anne from comment #13)
> Actually, we decided to be conservative. So Chrome and Opera not supporting
> it was sufficient, unless there was strong evidence to the contrary (e.g.
> Chrome and Opera have open bugs). It further not being interoperable is only
> more justification.

With regard to Internet Explorer, then, visual order for ISO-8859-8, seems to require Quirks-Mode in order to kick in. Thus, the expected visual order for ISO-8859-8 only works fully cross-browser when the page is in quirks mode.
Comment 18 Leif Halvard Silli 2014-03-27 02:26:53 UTC
(In reply to Leif Halvard Silli from comment #16)
> (In reply to queency3 from comment #14)

> 3) Make sure the dir="" attribute is set to "ltr": <html dir="ltr">.

Actally, the above detail might be irrelevant.
Comment 19 Martin Dürst 2014-03-27 02:43:23 UTC
(In reply to Leif Halvard Silli from comment #16)

> OK, that means that you require visual order. However, for CP862, visual
> order does not work cross browser.
> 
> To get it in visual order in a cross browser compatible way you should use
> "ISO-8869-8" rather than "DOS-862". Here is what you could do:

That should be iso-8859-8, not 8869. Same below.

> 1) Let your application convert each Hebrew letter to its numerical
> character reference equivalent. For example, convert Hebrew aleph into
> &#1488;

You can also convert it to the actual byte value, at least for those characters in DOS-862 that are also in ISO-8859-8

> 2) Apply the charset label "ISO-8869-8" to the web page. 
>   (For example <meta charset="ISO-8869-8"/>)
> 
> 3) Make sure the dir="" attribute is set to "ltr": <html dir="ltr">.

Instead of this, I'd use CSS to apply the same styling as for <bdo dir='ltr'>:
direction: ltr;
unicode-bidi: bidi-override;


> RESULTS: The letters will display in *visual* order not only in Firefox but
> also in Safari and Chrome. In IE it might not give you visual order since
> visual order only works in quirks mode.

Given this data and the web pages are very old, my guess would be that they result in quirks mode anyway. But I might be wrong.

> (Of course, you should rather use UTF-8. But then I guess you must first
> convert the text from visual to logical order, which sounds more
> complicated, to me.)

Yes.Converting from visual to logical order is impossible without intervention from a human being understanding the text.

But it is possible to convert to UTF-8 and have the text display in visual order. Put the text into <pre> elements or otherwise make sure that line breaks are preserved for display, and apply the style rules given above.
Comment 20 queency3 2014-03-27 06:46:17 UTC
(In reply to Leif Halvard Silli from comment #16)
> (In reply to queency3 from comment #14)
> 
> > file hex code: abcd efg. .... ....
> > view in html :abcd efg. .... ....
> > the view is from the left to the right only.
> 
> OK, that means that you require visual order. However, for CP862, visual
> order does not work cross browser.
> 
> To get it in visual order in a cross browser compatible way you should use
> "ISO-8869-8" rather than "DOS-862". Here is what you could do:
> 
> 1) Let your application convert each Hebrew letter to its numerical
> character reference equivalent. For example, convert Hebrew aleph into
> &#1488;
> 
> 2) Apply the charset label "ISO-8869-8" to the web page. 
>   (For example <meta charset="ISO-8869-8"/>)
> 
> 3) Make sure the dir="" attribute is set to "ltr": <html dir="ltr">.
> 
> 
> RESULTS: The letters will display in *visual* order not only in Firefox but
> also in Safari and Chrome. In IE it might not give you visual order since
> visual order only works in quirks mode.

I need to open TEXT files not html file.
Comment 21 Leif Halvard Silli 2014-03-27 09:44:51 UTC
(In reply to queency3 from comment #20)

> I need to open TEXT files not html file.

OK. 

Then you follow the advice that Martin Dürst gave in comment #19: instead of converting to numerical character references, you must convert directly to ISO-8859-8. Then, in Firefox, the user just need to select "Hebrew (Visual)" from the Character Encoding menu.

However, with a little hacking, you could *also* do what I said, namely convert the output to numeric character references: Just remember to replace the the file suffix .txt with the file suffix .html. If you do that, then Firefox will interpret the content as HTML, even if there is no HTML inside the file. Which means, once again, that the user just need to choose "Hebrew (Visual)" from the Character Encoding submenu.

Of course, if the text file contains lots of information, interpreting it as HTML will make all the whitespace be collapsed. To workaround that problem, the user could apply a User CSS (user style sheet) that do html{white-space:pre}. You may also read what Martin said about CSS in comment #19.

If you want to promote Firefox, then there it has a user style sheet manager:  https://addons.mozilla.org/en-US/firefox/addon/user-style-manager/

In fact, if you convert the output to numerical character references and apply the correct CSS via user style sheet, then users do not need to think about the Character Encoding menu. All they need to think about would be the user style sheet.
Comment 22 Leif Halvard Silli 2014-03-27 10:23:47 UTC
(In reply to Leif Halvard Silli from comment #21)
> (In reply to queency3 from comment #20)
> 
> > I need to open TEXT files not html file.
> 

> If you want to promote Firefox, then there it has a user style sheet
> manager:  https://addons.mozilla.org/en-US/firefox/addon/user-style-manager/

Example that I made using that add on:

html {            
   direction: ltr;
unicode-bidi: bidi-override;
 white-space: pre;
}

Btw, the add-on also allows you to create specific stylesheets for specific domains or URLs.
Comment 23 Martin Dürst 2014-03-27 10:51:41 UTC
(In reply to Leif Halvard Silli from comment #21)

> If you want to promote Firefox, then there it has a user style sheet
> manager:  https://addons.mozilla.org/en-US/firefox/addon/user-style-manager/

A user stylesheet isn't needed. If you have control over your server, you can make it add a HTTP link header. Anne can give the details.

Also, I don't think it's necessary for the file name to end in .html as long as it's served as Content-Type: text/html.
Comment 24 Leif Halvard Silli 2014-03-27 15:27:18 UTC
(In reply to Martin Dürst from comment #23)
> (In reply to Leif Halvard Silli from comment #21)
> 
> > If you want to promote Firefox, then there it has a user style sheet
> > manager:  https://addons.mozilla.org/en-US/firefox/addon/user-style-manager/
> 
> A user stylesheet isn't needed. If you have control over your server, you
> can make it add a HTTP link header. Anne can give the details.

Cool. But my suspicion is that Queency have in mind parsing of files via the file URL protocol.

> Also, I don't think it's necessary for the file name to end in .html as long
> as it's served as Content-Type: text/html.

There is also a "Force Content-Type" add-on for Firefox, which works fine. (Not sure it works for file: URLs, though.)

<https://addons.mozilla.org/en-US/firefox/addon/force-content-type/>

However, as it turns out, in Firefox, CSS applies to text/plain as well. So it might not be necessary to fiddle with the Content-Type - one can just apply the CSS - via User-Style-Manager or via HTTP link header.
Comment 25 Leif Halvard Silli 2014-03-27 18:10:40 UTC
(In reply to Leif Halvard Silli from comment #24)

> However, as it turns out, in Firefox, CSS applies to text/plain as well. So
> it might not be necessary to fiddle with the Content-Type - one can just
> apply the CSS - via User-Style-Manager or via HTTP link header.

Yup, this works fine. But one must use !important  on the unicode-bide rule:

*{unicode-bidi:bidi-override!important;}