EPUB 3 Working Group Telco — Minutes

Date: 2022-02-11

See also the Agenda and the IRC Log

Attendees

Present: Masakazu Kitahara, Ivan Herman, Wendy Reid, Ben Schroeter, Dave Cramer, Aimee Ubbink, Laura Brady, Brady Duga, Dan Lazin, Charles LaPierre, Matthew Chan, Matt Garrish, Zheng Xu (徐征), Avneesh Singh, Gregorio Pellegrino

Regrets: George Kerscher, Bill Kasdorf

Guests:

Chair: Dave Cramer

Scribe(s): Matthew Chan

Content:

Dave Cramer: we have a new/old member today! LauraB__ changed affiliations. Congrats!.

Laura Brady: i’m head of a11y at Legible Media.

1. Review CR Exit Criteria.

Dave Cramer: See initial proposal document.

Ivan Herman: I put this together a few weeks ago because i know from experience what we have to tell W3C about our plans.
… i’m sure most of you have heard this, but the CR period and status is there to test the recommendations.
… to test whether the recommendations to be is consistent and can be implemented (whatever implementation means in the specific case).
… we have to formalize exit criteria - i.e. how do we prove for each normative statement that this is so?.
… i looked at the 3 rec track documents (core, rs, a11y) to see whether what we’ve done so far and what we plan to do is okay.
… for RS we are in a good place, primarily thanks to dlazin.
… the RS must statements are usually some sort of behavioural requirement.
… e.g. RS must display this, or react on this event.
… for all of them we want to have at least one if not several tests.
… we want them to be granular tests, each one consisting of a small HTML document/epub.
… goal is that by end of CR we have at least two passing implementation for each test.
… that shows the normative statement is good.
… we currently have between 60-100 tests, quite a number.
… we have a reporting method for tests.
… in core the situation is more complicated.

Ivan Herman: See categories in the core document.

Ivan Herman: we do have some statements in core which can be treated the same way as in core - e.g. statements that are earmarked in both core and RS.
… e.g. list of Core Media Types.
… in core, we have also a 3rd category of normative vocab.
… each has an associated MUST behaviour.
… testing for these would mean we show that those terms are of value to the community, they are being deployed by the community.
… there are no requirements in spec about how those metadata values must be used, so we cannot test in same way as category 1.
… so we have to list these vocab in a doc, and then we must have at least 2 real life publishers that use those terms.
… those data can be found.
… i presume that RS like Kobo, Google, Apple are in position to collect info about deployment of those vocab.
… we have not yet done it, but we have about a year to complete it.
… there is another category of things like structural constraints on the format of the epub file (e.g. structure of container, structure of package, structure of smil, etc.).
… we do not define what RS should do if one of those structure is not adhered to be epub file.
… RS may not even see those types of files because they would be caught by conformance checkers.
… we have to define how we will test this type of requirement.
… there is also a similar set of requirements which are things like restrictions on HTML.
… i.e. valid HTML that we don’t allow in epub.
… i.e. formatting for the nav document.
… i.e. HTML conformance beyond what is generally required.
… my personal opinion is that the only thing we can really do is to pub epubcheck into the center of our testing.
… can epubcheck test any of these types of requirements.
… so we have to produce a test for each of those that violates conformance, and epubcheck has to say something about it.
… doesn’t make sense to have a requirement that epubcheck can’t test.

Ivan Herman: See categories in the a11y document.

Ivan Herman: for a11y, i am less familiar with it, i am hopeful that those who are more familiar will comment.
… it is kind of similar, except I couldn’t find behavioural normative requirements like we have with category 1 in RS.
… more structural and vocab.
… for both of these, we have to find an approach on what it means to “test” the a11y spec.
… have discussed with mgarrish and avneeshsingh.
… for vocab we can use the same definition of testing as above.
… mgarrish has put together a usage report.
… we can use the same for both vocab in content document and a11y spec.
… what the a11y document does is to define what has to be put on top of WCAG requirements for epub.
… but content document should still be some level of WCAG compliant.
… and wg had its own opinion on how to do exit criteria testing - i.e. by emulating the WCAG method of conformance levels.
… there has to be real published epub documents that abide to the conformance level.
… the a11y document has a concept of conformance level.
… strings identifying conformance level.
… we could say there must be at least 2 published epubs that abide to one of the conformance levels defined by the a11y spec.

Charles LaPierre: as far as the exit criteria re. making sure content documents are conforming to WCAG, this is already something that CGA does.
… though we are checking against original WCAG 1.
… once a11y becomes spec, we can start certifying against WCAG 2.

Ivan Herman: indeed, a little bit of chicken and egg situation.
… the exit criteria should be formulated such that publishers show they are producing experimental documents aimed at epub 33 core compliance.
… and how we would check this is also something that must be defined.
… epubcheck already has a beta release for epub 33 checking.

Matt Garrish: this isn’t that dissimilar from DPUB ARIA. If something isn’t a standard, how do you prove that you are doing it?.
… we just need to be able to show that it is possible to produce at the new level, and that publishers are committed to producing at that new level.

Ivan Herman: it’s slightly different in the sense that DPUB aria is more in line with the category of vocab tests.
… i’m not familiar enough with a11y spec to say whether there are other normative requirements that are not vocab.

Matt Garrish: it’s all content-based, so if we can get publishers to produce one book compliant with the new requirements, that would prove possibility. Not necessary to show that publishers will certify under the new spec.

Ben Schroeter: the epub a11y 1.1 doc is a collection of requirements that include best practices, sometime beyond WCAG standards.
… so we wouldn’t be able to incorporate tests for this into epubcheck.
… these sorts of requirements are contextual, not testable via automation.

Matt Garrish: DAISY has ACE for the parts that are testable via automation, and then SMART for everything that isn’t.
… i think we’re in fairly good shape there.

Ivan Herman: i didn’t realize there was a test suite for epubcheck already, so i’m trying to be pragmatic here to see what next steps should be.
… my exit criteria doc isn’t just for the wg, it’s something we need to refer to when asking for CR exit from director.

Avneesh Singh: Ivan your text looks fine for a11y.

Ivan Herman: mgarrish if you can put some text into the document to reflect your comments just now.
… for the content document testing, 1) in an ideal world there should be 2 implementations.
… however, there is only one epub checker in this world.
… no independent implementations.
… so 1) we will have to convince the director that epubcheck should be allowed to be the final arbitrator.
… 2) we will have to have a report, so if there is a comprehensive test suite, then we need to make sure that all our MUST statements are reflected in the test suite, that it is okay for W3C to refer to the test suite.
… 3) we will have to have some sort of report for content, the same way that we have for RS testing.
… not sure if epubcheck would consider that confidential info.

Matt Garrish: there are a lot of epub checkers that aren’t epubcheck, but they are all built on epubcheck….

Ivan Herman: no, that doesn’t work.
… is not considered independent.

Matt Garrish: okay, i don’t know anything else than epubcheck then.

Ivan Herman: okay, so we just need to explain that in the exit criteria then.

Dan Lazin: i suspect that amazon will have different epub conformance checker because their content is not just vanilla epub.

Wendy Reid: no, i can’t think of another epub conformance checker either.
… there is flightdeck, but that is based on epubcheck + a layer of platform specific tests they made themselves.

Avneesh Singh: epubcheck is having a lot of tests which are of features added to epub 33.
… but we also relaxed some tests, e.g. for ordering of toc.
… epubcheck beta will be out later this month, and there may be feedback from community.
… so epubcheck may not be 100% aligned with spec because it takes time to catch up.
… re. 5 publications from different publishers that conform to epub 11, WCAG level AA, what you have in your exit criteria document now is fine.

Ivan Herman: are those tests systematically available somewhere so that we can refer to them?

Avneesh Singh: i can table this during the next epubcheck management call, but i don’t see an issue.

Matt Garrish: i think we’re in good shape re the content related requirements.

Ivan Herman: after we get a round of reviews on the exit criteria document, can we submit to director?

Avneesh Singh: would you like to check with Romain?

Ivan Herman: good idea.
… somebody has to do those report patterns, like what mgarrish did for the a11y requirements.

Matt Garrish: i can take that on.

Wendy Reid: we’re not going to talk about Legal Entity is today, contrary to agenda.
… i would recommend reading minutes from AC Office Hours this week for background on what Legal Entity is.
… so we will return to that on 25th.

Avneesh Singh: CharlesL1 and gregorio, please help us by using your contacts to help us get those implementations re. a11y!

2. Reading Systems and Media Overlays… (issue epub-specs#1991)

See github issue epub-specs#1991.

Wendy Reid: we were doing our RS and MO testing, and realized that current language says that RS may support MO.
… given how MO are used these days, i think it makes sense to push that up to a should.
… this still gives room to those RS who can’t support audio, but there are definitely more than 2 independent implementations of MO these days.

Ivan Herman: i was writing tests for MO and realized this point.
… is this a leftover from when an earlier version of the spec, or was this deliberate?.

Matt Garrish: i think it came over from DAISY side in 3.0, since we weren’t sure at the time what the uptake from mainstream RS would be.
… don’t remember any specific discussion about the level of conformance at the time.
… it was just a brand new feature, so we didn’t want to start by positioning it as critical feature.
… but that’s just my opinion.

Dan Lazin: i would propose as a general guideline that support for features should never be below a SHOULD.
… MAY should be reserved for behavioural things.
… if I am RS, and a I see a mishmash of requirement levels, i’m not sure how to interpret that.

Ivan Herman: +1 to dan.

Matt Garrish: if you put it in context of audio support being required for RS, we weren’t requiring that either.
… maybe if you support audio in your RS, then you should also support MO playback.
… less broad than just saying SHOULD support MO across the board.

Dan Lazin: can whoever is making the PR review other features, and elevate in the same way?.

Proposed resolution: Elevate requirement for media overlays to a SHOULD, close issue 1991. (Wendy Reid)

Wendy Reid: i worry that that will turn up something where raising the conformance level will be controversial….

Ivan Herman: +1.

Dan Lazin: +1.

Zheng Xu (徐征): +1.

Matt Garrish: +1.

Laura Brady: +1.

Charles LaPierre: +1.

Avneesh Singh: +1.

Ben Schroeter: +1.

Wendy Reid: +1.

Masakazu Kitahara: +1.

Dan Lazin: then if in the course of reviewing you locate other similar feature support issues, can you open issues for those?.

Ivan Herman: yes, okay.

Resolution #1: Elevate requirement for media overlays to a SHOULD, close issue 1991.

3. AOB.

Ben Schroeter: circling back to first topic, i’m still confused about next step for validating that we have implementations for new epub spec and a11y spec.
… is the assumption that we have 5 publishers who have already produced epubs that conform to those new specs?.
… how do we find epubs that include the delta between the old spec and new spec, when the new spec isn’t… a spec yet.

Charles LaPierre: i think this goes back to comments that mgarrish said, about publisher only needing to show that it is possible.
… the spec doesn’t break backwards compatibility, it just opens the way for publishers to go further.

Matt Garrish: we are going to have to look at WCAG 2.1 and try to locate things that might trip up production.

Ben Schroeter: that might be quite an exercise.

Avneesh Singh: looking WCAG 2.1 exit criteria, it says that there should be 5 websites, it doesn’t say that each of those 5 websites should demonstrate all exit criteria.

Matt Garrish: we need representative epub content that can pass WCAG 2.1.
… some of the exit criteria isn’t even applicable to epub (e.g. hovering events).

Ben Schroeter: does an HTML validator count as an additional validator for epub?.

Avneesh Singh: epubcheck is synchronizing with HTML validator, but it is not based on HTML validator… so this is a good point.

Ivan Herman: our exit criteria tests pertain to features that we define for epub, so in this sense the HTML validators won’t help.

Matt Garrish: more to the point, epubcheck is universal, the whole ecosystem uses it.

Wendy Reid: we’re out of time, thank you everyone!.

4. Resolutions