14:37:30 <RRSAgent> RRSAgent has joined #epub
14:37:30 <RRSAgent> logging to https://www.w3.org/2022/02/11-epub-irc
14:37:32 <Zakim> RRSAgent, make logs Public
14:37:33 <Zakim> please title this meeting ("meeting: ..."), ivan
14:37:48 <ivan> ivan has changed the topic to: Meeting Agenda 2022-02-11: https://lists.w3.org/Archives/Public/public-epub-wg/2022Feb/0005.html
14:37:49 <ivan> Chair: dauwhe
14:37:49 <ivan> Date: 2022-02-11
14:37:49 <ivan> Agenda: https://lists.w3.org/Archives/Public/public-epub-wg/2022Feb/0005.html
14:37:49 <ivan> Meeting: EPUB 3 Working Group Telco
14:37:49 <ivan> Regrets+ George, BillK
14:50:48 <dauwhe> dauwhe has joined #epub
14:50:59 <mgarrish> mgarrish has joined #epub
14:58:22 <BenSchroeter> BenSchroeter has joined #epub
14:59:20 <MasakazuKitahara> MasakazuKitahara has joined #epub
14:59:30 <MasakazuKitahara> present+
14:59:55 <wendyreid> wendyreid has joined #epub
15:00:05 <ivan> present+
15:00:17 <wendyreid> present+
15:00:36 <ivan> present+ BenSchroeter
15:00:37 <dauwhe> present+
15:00:56 <LauraB__> LauraB__ has joined #epub
15:01:07 <ivan> present+ aubbink
15:01:09 <Aimee> Aimee has joined #epub
15:01:13 <ivan> present+ laurab
15:01:21 <Aimee> present+
15:01:28 <ivan> present+ brady
15:01:30 <dlazin> dlazin has joined #epub
15:01:33 <dlazin> present+
15:01:35 <MattChan> MattChan has joined #epub
15:01:41 <duga> duga has joined #epub
15:01:45 <duga> present+
15:01:56 <CharlesL1> CharlesL1 has joined #epub
15:01:59 <duga> Regrets, I will need to leave early today
15:02:02 <CharlesL1> present+
15:02:10 <MattChan> present+
15:02:24 <ivan> present+ wendy
15:02:30 <MattChan> scribe+
15:02:46 <ivan> present+ mgarrish
15:03:32 <MattChan> dauwhe: we have a new/old member today! LauraB__ changed affiliations. Congrats!
15:03:34 <zheng_xu> zheng_xu has joined #epub
15:03:43 <MattChan> LauraB__: i'm head of a11y at Legible Media
15:04:11 <zheng_xu> present+
15:04:16 <MattChan> TOPIC: Review CR Exit Criteria Prepared by Ivan
15:04:20 <dauwhe> https://w3c.github.io/epub-specs/epub33/reports/exit_criteria.html
15:04:55 <MattChan> ivan: i put this together a few weeks ago because i know from experience what we have to tell W3C about our plans
15:05:20 <avneeshsingh> avneeshsingh has joined #epub
15:05:23 <MattChan> ... i'm sure most of you have heard this, but the CR period and status is there to test the recommendations
15:05:27 <avneeshsingh> present+
15:05:58 <MattChan> ... to test whether the recommendations to be is consistent and can be implemented (whatever implementation means in the specific case)
15:06:22 <MattChan> ... we have to formalize exit criteria - i.e. how do we prove for each normative statement that this is so?
15:06:53 <MattChan> ... i looked at the 3 rec track documents (core, rs, a11y) to see whether what we've done so far and what we plan to do is okay
15:07:04 <MattChan> ... for RS we are in a good place, primarily thanks to dlazin
15:07:21 <MattChan> ... the RS must statements are usually some sort of behavioural requirement
15:07:27 <MattChan> ... e.g. RS must display this, or react on this event
15:07:43 <MattChan> ... for all of them we want to have at least one if not several tests
15:08:05 <MattChan> ... we want them to be granular tests, each one consisting of a small HTML document/epub
15:08:19 <MattChan> ... goal is that by end of CR we have at least two passing implementation for each test
15:08:29 <MattChan> ... that shows the normative statement is good
15:08:45 <MattChan> ... we currently have between 60-100 tests, quite a number
15:08:55 <MattChan> ... we have a reporting method for tests
15:09:00 <MattChan> ... in core the situation is more complicataed
15:09:08 <MattChan> s/complicataed/complicated
15:09:11 <ivan> -> categories in the core document https://w3c.github.io/epub-specs/epub33/reports/exit_criteria.html#epub-3.3-core
15:09:48 <MattChan> ivan: we do have some statements in core which can be treated the same way as in core - e.g. statements that are earmarked in both core and RS
15:09:55 <MattChan> ... e.g. list of Core Media Types
15:10:17 <MattChan> ... in core, we have also a 3rd category of normative vocab
15:10:27 <MattChan> ... each has an associated MUST behaviour
15:10:56 <MattChan> ... testing for these would mean we show that those terms are of value to the community, they are being deployed by the community
15:11:17 <MattChan> ... there are no requirements in spec about how those metadata values must be used, so we cannot test in same way as category 1
15:11:37 <MattChan> ... so we have to list these vocab in a doc, and then we must have at least 2 real life publishers that use those terms
15:11:47 <MattChan> ... those data can be found
15:12:02 <MattChan> ... i presume that RS like Kobo, Google, Apple are in position to collect info about deployment of those vocab
15:12:10 <MattChan> ... we have not yet done it, but we have about a year to complete it
15:12:56 <MattChan> ... there is another category of things like structural constraints on the format of the epub file (e.g. structure of container, structure of package, structure of smil, etc.)
15:13:19 <MattChan> ... we do not define what RS should do if one of those structure is not adhered to be epub file
15:13:39 <MattChan> ... RS may not even see those types of files because they would be caught by conformance checkers
15:13:46 <MattChan> ... we have to define how we will test this type of requirement
15:14:02 <MattChan> ... there is also a similar set of requirements which are things like restrictions on HTML
15:14:16 <MattChan> ... i.e. valid HTML that we don't allow in epub
15:14:23 <MattChan> ... i.e. formatting for the nav document
15:14:34 <MattChan> ... i.e. HTML conformance beyond what is generally required
15:15:01 <MattChan> ... my personal opinion is that the only thing we can really do is to pub epubcheck into the center of our testing
15:15:13 <MattChan> ... can epubcheck test any of these types of requirements
15:15:32 <MattChan> ... so we have to produce a test for each of those that violates conformance, and epubcheck has to say something about it
15:15:45 <MattChan> ... doesn't make sense to have a requirement that epubcheck can't test
15:16:09 <ivan> -> categorie in the a11y document https://w3c.github.io/epub-specs/epub33/reports/exit_criteria.html#epub-3.3-core
15:16:27 <MattChan> ... for a11y, i am less familiar with it, i am hopeful that those who are more familiar will comment
15:16:57 <MattChan> ... it is kind of similar, except I couldn't find behavioural normative requirements like we have with category 1 in RS
15:17:17 <MattChan> ... more structural and vocab
15:17:31 <MattChan> ... for both of these, we have to find an approach on what it means to "test" the a11y spec
15:17:46 <MattChan> ... have disucssed with mgarrish and avneeshsingh
15:18:00 <MattChan> ... for vocab we can use the same definition of testing as above
15:18:11 <MattChan> ... mgarrish has put together a usage report
15:18:25 <MattChan> ... we can use the same for both vocab in content document and a11y spec
15:18:39 <MattChan> ... what the a11y document does is to define what has to be put on top of WCAG requirements for epub
15:18:59 <MattChan> ... but content document should still be some level of WCAG compliant
15:19:28 <MattChan> ... and wg had its own opinion on how to do exit criteria testing - i.e. by emulating the WCAG method of conformance levels
15:19:47 <MattChan> ... there has to be real published epub documents that abide to the conformance level
15:20:01 <MattChan> ... the a11y document has a concept of conformance level
15:20:15 <MattChan> ... strings identifying conformance level
15:20:41 <MattChan> ... we could say there must be at least 2 published epubs that abide to one of the conformance levels defined by the a11y spec
15:20:51 <CharlesL1> q+
15:21:23 <MattChan> BenSchroeter: did you mean to put a link to the a11y doc?
15:22:04 <ivan> -> categories in the a11y document https://w3c.github.io/epub-specs/epub33/reports/exit_criteria.html#epub-accessibility-1.1
15:22:09 <wendyreid> ack CharlesL1
15:22:10 <dauwhe> ack ch
15:22:13 <MattChan> ivan: oops, here is the right one
15:22:42 <MattChan> CharlesL1: as far as the exit criteria re. making sure content documents are conforming to WCAG, this is already something that CGA does
15:22:59 <MattChan> ... though we are checking against original WCAG 1
15:23:21 <ivan> q+
15:23:25 <MattChan> ... once a11y becomes spec, we can start certifying against WCAG 2
15:23:35 <dauwhe> ack iv
15:23:45 <MattChan> ivan: indeed, a little bit of chicken and egg situation
15:25:00 <MattChan> ... the exit criteria should be formulated such that publishers show they are producing experimental documents aimed at epub 33 core compliance
15:25:13 <MattChan> ... and how we would check this is also something that must be defined
15:25:27 <MattChan> ... epubcheck already has a release for epub 33 checking
15:25:43 <MattChan> s/release/beta release
15:25:45 <BenSchroeter> q+
15:26:07 <MattChan> mgarrish: this isn't that dissimilar from DPUB ARIA. If something isn't a standard, how do you prove that you are doing it?
15:26:27 <MattChan> ... we just need to be able to show that it is possible to produce at the new level, and that publishers are committed to producing at that new level
15:27:05 <MattChan> ivan: it's slightly different in the sense that DPUB aria is more in line with the category of vocab tests
15:27:24 <MattChan> ... i'm not familiar enough with a11y spec to say whether there are other normative requirements that are not vocab
15:28:26 <MattChan> mgarrish: it's all content-based, so if we can get publishers to produce one book compliant with the new requirements, that would prove possibility. Not necessary to show that publishers will certify under the new spec
15:28:27 <wendyreid> ack BenSchroeter
15:28:56 <MattChan> BenSchroeter: the epub a11y 1.1 doc is a collection of requirements that include best practices, sometime beyond WCAG standards
15:29:10 <MattChan> ... so we wouldn't be able to incorporate tests for this into epubcheck
15:29:24 <MattChan> ... these sorts of requirements are contextual, not testable via automation
15:29:24 <ivan> q+
15:29:41 <MattChan> mgarrish: DAISY has ACE for the parts that are testable via automation, and then SMART for everything that isn't
15:30:05 <MattChan> ... i think we're in fairly good shape there
15:30:28 <wendyreid> ack ivan
15:30:54 <MattChan> ivan: i didn't realize there was a test suite for epubcheck already, so i'm trying to be pragmatic here to see what next steps should be
15:31:22 <MattChan> ... my exit criteria doc isn't just for the wg, it's something we need to refer to when asking for CR exit from director
15:31:34 <avneeshsingh> Ivan your text looks fine for a11y
15:31:36 <MattChan> ... mgarrish if you can put some text into the document to reflect your comments just now
15:32:03 <MattChan> ... for the content document testing, 1) in an ideal world there should be 2 implementations
15:32:22 <MattChan> ... however, there is only 1 epubchecker in this world
15:32:30 <MattChan> ... no independent implementations
15:32:49 <MattChan> ... so we will have to convince the director that epubcheck should be allowed to be the final arbitrator
15:32:52 <mgarrish> q+
15:33:22 <MattChan> ... 2) we will have to have a report, so if there is a comprehensive test suite, then we need to make sure that all our MUST statements are reflected in the test suite, that it is okay for W3C to refer to the test suite
15:33:45 <MattChan> ... 3) we will have to have some sort of report for content, the same way that we have for RS testing
15:33:55 <wendyreid> ack mgarrish
15:34:03 <MattChan> ... not sure if epubcheck would consider that confidential info
15:34:17 <MattChan> mgarrish: there are a lot of epub checkers that aren't epubcheck, but they are all built on epubcheck...
15:34:23 <MattChan> ivan: no, that doesn't work
15:34:31 <MattChan> ... is not considered independent
15:34:40 <avneeshsingh> what about chrome and edge?
15:34:53 <MattChan> mgarrish: okay, i don't know anything else than epubcheck then
15:34:55 <dlazin> q+
15:35:01 <wendyreid> ack dlazin
15:35:04 <MattChan> ivan: okay, so we just need to explain that in the exit criteria then
15:35:23 <MattChan> dlazin: i suspect that amazon will have different epub conformance checker because their content is not just vanilla epub
15:35:46 <avneeshsingh> q+
15:35:46 <MattChan> wendyreid: no, i can't think of another epub conformance checker either
15:36:13 <ivan> present+ gpellegrino
15:36:14 <wendyreid> ack avneeshsingh
15:36:15 <MattChan> ... there is flightdeck, but that is based on epubcheck + a layer of platform specific tests they made themselves
15:36:36 <MattChan> avneeshsingh: epubcheck is having a lot of tests which are of features added to epub 33
15:36:45 <MattChan> ... but we also relaxed some tests, e.g. for ordering of toc
15:37:08 <MattChan> ... epubcheck beta will be out later this month, and there may be feedback from community
15:37:22 <MattChan> ... so epubcheck may not be 100% aligned with spec because it takes time to catch up
15:37:57 <ivan> q+
15:38:00 <wendyreid> ack ivan
15:38:11 <MattChan> ... re. 5 publications from different publishers that conform to epub 11, WCAG level AA, what you have in your exit criteria document now is fine
15:38:36 <zheng_xu> Thanks CharlesL1
15:38:38 <MattChan> ivan: are those tests systematically available somewhere so that we can refer to them?
15:39:15 <MattChan> avneeshsingh: i can table this during the next epubcheck management call, but i don't see an issue
15:39:25 <zheng_xu> s/Thanks CharlesL1//
15:39:45 <MattChan> mgarrish: i think we're in good shape re the content related requirements
15:40:14 <MattChan> ivan: after we get a round of reviews on the exit criteria document, can we submit to director?
15:40:33 <MattChan> avneeshsingh: would you like to check with roman?
15:40:42 <MattChan> ivan: good idea
15:40:47 <MattChan> s/roman/romain
15:41:42 <MattChan> ivan: somebody has to do those report patterns, like what mgarrish did for the a11y requirements
15:42:25 <MattChan> mgarrish: i can take that on
15:42:59 <MattChan> wendyreid: we're not going to talk about Legal Entity is today, contrary to agenda
15:43:16 <MattChan> ... i would recommend reading minutes from AC question period this week for background on what Legal Entity is
15:43:20 <avneeshsingh> q+
15:43:25 <wendyreid> ack avneeshsingh
15:43:28 <MattChan> ... so we will return to that on 25th
15:45:07 <duga> duga has joined #epub
15:45:28 <wendyreid> s/AC question period/AC Office Hours/
15:46:00 <wendyreid> https://github.com/w3c/epub-specs/issues/1991
15:46:01 <MattChan> avneeshsingh: CharlesL1 and gregorio, please help us by using your contacts to help us get those implementations re. a11y!
15:46:27 <ivan> Topic: https://github.com/w3c/epub-specs/issues/1991
15:46:35 <MattChan> wendyreid: we were doing our RS and MO testing, and realized that current language says that RS may support MO
15:46:37 <ivan> github-bot, bye
15:46:37 <github-bot> github-bot has left #epub
15:46:53 <MattChan> ... given how MO are used these days, i think it makes sense to push that up to a should
15:47:07 <ivan> q+
15:47:11 <wendyreid> ack ivan
15:47:15 <MattChan> .. this still gives room to those RS who can't support audio, but there are definitely more than 2 independent implementations of MO these days
15:47:28 <MattChan> ivan: i was writing tests for MO and realized this point
15:47:46 <MattChan> ... is this a leftover from when an earlier version of the spec, or was this deliberate?
15:48:28 <MattChan> mgarrish: i think it came over from DAISY side in 3.0, since we weren't sure at the time what the uptake from mainstream RS would be
15:48:44 <MattChan> ... don't remember any specific discussion about the level of conformance at the time
15:49:10 <MattChan> ... it was just a brand new feature, so we didn't want to start by positioning it as critical feature
15:49:11 <dlazin> q+
15:49:14 <MattChan> ... but that's just my opinion
15:49:16 <wendyreid> ack dlazin
15:49:34 <MattChan> dlazin: i would propose as a general guideline that support for features should never be below a SHOULD
15:49:43 <MattChan> ... MAY should be reserved for behavioural things
15:50:05 <MattChan> ... if I am RS, and a I see a mishmash of requirement levels, i'm not sure how to interpret that
15:50:10 <ivan> +1 to dan
15:50:43 <MattChan> mgarrish: if you put it in context of audio support being required for RS, we weren't requiring that either
15:51:00 <MattChan> ... maybe if you support audio in your RS, then you should also support MO playback
15:51:12 <MattChan> ... less broad than just saying SHOULD support MO across the board
15:51:37 <MattChan> dlazin: can whoever is making the PR review other features, and elevate in the same way?
15:51:43 <ivan> q+
15:52:05 <wendyreid> Proposed: Elevate requirement for media overlays to a SHOULD, close issue 1991
15:52:06 <MattChan> wendyreid: i worry that that will turn up something where raising the conformance level will be controversial...
15:52:07 <ivan> q-
15:52:12 <ivan> +1
15:52:14 <dlazin> +1
15:52:15 <zheng_xu> +1
15:52:16 <mgarrish> +1
15:52:16 <LauraB__> +1
15:52:17 <CharlesL1> +1
15:52:17 <avneeshsingh> +1
15:52:18 <BenSchroeter> +1
15:52:18 <wendyreid> +1
15:52:19 <MasakazuKitahara> +1
15:52:25 <MattChan> dlazin: then if in the course of reviewing you locate other similar feature support issues, can you open issues for those?
15:52:29 <MattChan> ivan: yes, okay
15:52:34 <wendyreid> RESOLVED: Elevate requirement for media overlays to a SHOULD, close issue 1991
15:52:48 <BenSchroeter> q+
15:52:52 <wendyreid> ack BenSchroeter
15:53:06 <ivan> q+
15:53:06 <ivan> Topic: aob
15:53:28 <MattChan> BenSchroeter: circling back to first topic, i'm still confused about next step for validating that we have implementations for new epub spec and a11y spec
15:53:33 <ivan> q-
15:53:45 <MattChan> ... is the assumption that we have 5 publishers who have already produced epubs that conform to those new specs?
15:54:06 <MattChan> ... how do we find epubs that include the delta between the old spec and new spec, when the new spec isn't... a spec yet
15:54:34 <MattChan> CharlesL1: i think this goes back to comments that mgarrish said, about publisher only needing to show that it is possible
15:55:29 <MattChan> ... the spec doesn't break backwards compat, it just opens the way for publishers to go further
15:56:02 <avneeshsingh> q+
15:56:40 <tzviya> tzviya has joined #epub
15:56:41 <MattChan> mgarrish: we are going to have to look at WCAG 2.1 and try to locate things that might trip up production
15:56:55 <wendyreid> ack avneeshsingh
15:56:57 <MattChan> BenSchroeter: that might be quite an exercise
15:57:27 <MattChan> avneeshsingh: looking WCAG 2.1 exit criteria, it says that there should be 5 websites, it doesn't say that each of those 5 websites should demonstrate all exit criteria
15:57:46 <MattChan> mgarrish: we need representative epub content that can pass WCAG 2.1
15:58:06 <MattChan> ... some of the exit criteria isn't even applicable to epub (e.g. hovering events)
15:58:36 <ivan> q+
15:58:48 <wendyreid> ack ivan
15:58:52 <MattChan> BenSchroeter: does an HTML validator count as an additional validator for epub?
15:59:32 <MattChan> avneeshsingh: epubcheck is synchronizing with HTML validator, but it is not based on HTML validator... so this is a good point
16:00:53 <MattChan> ivan: our exit criteria tests pertain to features that we define for epub, so in this sense the HTML validators won't help
16:01:11 <MattChan> mgarrish: more to the point, epubcheck is universal, the whole ecosystem uses it
16:01:22 <MattChan> wendyreid: we're out of time, thank you everyone!
16:01:31 <ivan> rrsagent, bye
16:01:31 <RRSAgent> I see no action items