12:25:33 RRSAgent has joined #epub 12:25:33 logging to https://www.w3.org/2020/10/23-epub-irc 12:25:35 RRSAgent, make logs Public 12:25:36 please title this meeting ("meeting: ..."), dauwhe 12:25:55 meeting: EPUB 3 Working Group Telecon, Virtual TPAC Day 2 13:03:03 zakim, start meeting 13:03:03 RRSAgent, make logs Public 13:03:04 please title this meeting ("meeting: ..."), ivan 13:03:23 Meeting: EPUB 3 Working Group vF2F — 2nd Day 13:03:27 romain has joined #epub 13:03:44 ivan has changed the topic to: Meeting Agenda 2020-10-23: https://lists.w3.org/Archives/Public/public-epub-wg/2020Oct/0034.html 13:03:45 Chair: wendy 13:03:45 Date: 2020-10-23 13:03:45 Agenda: https://lists.w3.org/Archives/Public/public-epub-wg/2020Oct/0034.html 13:03:52 regrets+ mattg 13:27:39 wendyreid has joined #epub 13:39:44 duga has joined #epub 13:51:18 present+ 13:52:55 present+ 13:53:02 present+ brady 13:53:04 present+ 13:53:10 present+ avneesh 13:53:17 present+ tzviya 13:53:21 present+ wendy 13:53:38 guest+ dauwhe 13:53:49 present+ mattchan 13:54:02 MattChan has joined #epub 13:55:39 Avneesh has joined #epub 13:55:42 dkaplan3 has joined #epub 13:56:13 present+ 13:57:06 Yanni has joined #epub 13:57:21 toshiakikoike has joined #epub 13:57:32 present+ 13:57:47 present+ 13:57:56 present+ 13:57:56 MasakazuKitahara has joined #epub 13:58:40 George has joined #epub 13:58:48 present+ 14:00:09 present+ 14:00:09 plh has joined #epub 14:00:19 As of this point the attendees have been tzviya, ivan, brady, duga, avneesh, wendy, mattchan, toshiakikoike, dkaplan, Yanni, George, romain 14:00:24 Garth has joined #epub 14:00:31 present+ 14:00:33 present+ 14:00:34 guests+ dkaplan3 14:00:37 present+ 14:00:38 present+ 14:00:59 guests+ courtneyb 14:01:17 scribe+ MattChan 14:01:42 gpellegrino has joined #epub 14:01:53 LauraB__ has joined #epub 14:02:01 guests+ plh 14:02:04 jmcshane_ has joined #epub 14:02:06 present+ 14:02:07 present+ 14:02:17 present+ 14:02:26 JuanCorona has joined #epub 14:02:29 Courtney_B has joined #epub 14:02:38 guests+ judy 14:02:38 guests+ karen 14:02:54 guests+ Julie_Johannessen 14:03:01 CharlesL has joined #epub 14:03:27 present+ toshiaki 14:03:31 circularken has joined #epub 14:03:36 fchasen has joined #epub 14:03:39 present+ 14:04:04 marisa has joined #epub 14:04:05 dauwhe: welcome everyone to day 2 to virtual TPAC for epub 3 wg 14:04:22 ... we had a good meeting last night, sorted through a couple highly commented issues 14:04:22 Garth has joined #epub 14:04:35 present+ 14:04:42 ... nav vs spine order, external entity declarations, started convos about testing and fxl accessibility 14:04:55 ... now continuing with those last two topics 14:05:17 guests+ fchasen 14:05:23 ... do we have any new people who want to introduce themselves? 14:05:46 guests+ Ken_Jones 14:06:07 present+ Garth 14:06:19 Judy has joined #epub 14:06:29 present+ 14:06:30 Judy introduces herself 14:06:43 present+ Rachel_Yager 14:06:54 present+ 14:07:03 Guests+ Daniella_Levy-Pinto 14:07:03 fchasen: i'm also new, from scribd. Just looking on as an observer for now and perhaps will join the process 14:07:27 PLH, W3C 14:07:34 guests+ Danny_Faris 14:07:35 plh: hi, project manager for the w3, responsible for all the wg, thank you for your work 14:07:41 zhengxu has joined #epub 14:07:46 JulieJ has joined #epub 14:07:52 Guests+ Rachel_Yager 14:08:08 dkaplan3: hi, i'm a project coordinator for NNELS, happy to be here 14:08:11 present+ 14:08:16 q+ 14:08:28 Danny, also from NNELS, an accessibility developer, introduces himself 14:08:30 ack tz 14:08:53 q? 14:08:56 q+ 14:08:59 tzviya: explains the irc queueing mechanism to the new folks 14:09:07 s/dkaplan3/Daniella/ 14:09:28 Topic: testing 14:10:05 dauwhe: epub 3.2 was done by the community group as a report, but didn't go through the full w3c wg process 14:10:22 ... we want to take advantage of w3c experience in developing spec 14:10:44 ... we need to prove that everything in our spec is implementable, and we do that by showing 2 independent implementations 14:11:18 ... yesterday we talked about history of previous attempts at doing this, e.g. epubtest 14:11:40 ... it tried to be something like caniuse for epub, providing authors info about which features work on which rs 14:12:11 ... epubtest faced significant problems, there are lots and lots of rs, and those rs behave differently even depending on which os they are on 14:12:23 ... work kind of overwhelmed the volunteers 14:12:32 present+ Hadrien 14:12:40 ... it was also focused on testing large scale features vs normative statements in the spec itself 14:12:52 guests+ Karen 14:12:54 ... the newer version of epubtest is working very well, is a great resource 14:13:09 ... we going to need people to work on testing, including a test lead 14:13:09 old epubtest: https://daisy.github.io/old-epub3-support-grid/features/ 14:13:13 Bill_Kasdorf_ has joined #epub 14:13:18 ... if you can, please volunteer! 14:13:21 new epubtest: http://epubtest.org 14:13:30 present+ 14:13:38 ... we're going to need a lot of planning re. our test approach, define what the scope of testing will be 14:14:01 ... we can't do every web platform test for css, but wrapped up in an epub 14:14:05 present+ MasakazuKitahara 14:14:29 ... ivan, should this be a separate github repo, just for testing? would be difficult to include it in the spec repo 14:14:36 q? 14:14:45 repo for the old epubtest test contents: https://github.com/IDPF/epub-testsuite 14:14:53 ivan: the spec repo is already pretty big, and will be subject to a clean up for order 14:15:06 dauwhe: maybe we can talk offline about how to do this 14:15:09 ack Geor 14:15:15 q+ to ask about testing approach 14:15:20 ivan: if you assign this to me, i'll do it 14:16:36 dauwhe: other thing that came up yesterday, browsers can be tested using test harnesses that can automate tests 14:16:48 ... we don't have that, so a lot of our testing is going to be manual, more laborious 14:17:18 present+ romain 14:17:24 q- 14:17:46 ... the other thing that came up last night, we need to do some experimentation, starting with a small corner of the spec, to determine how will we document, how we will collect normative statements from the spec/identify testable assertions, create testable content, how we will maintain all our epub test files... 14:18:22 ... let's try to work at a small scale to figure out what will work, and what won't, before we start to try to tackle everything 14:18:43 ... we'll have to have a taskforce within the wg to do a lot of this, meeting and talking separately from the regular wg calls 14:18:55 zakim, who is here? 14:18:55 Present: tzviya, ivan, brady, duga, avneesh, wendy, mattchan, toshiakikoike, dkaplan, Yanni, George, romain, MasakazuKitahara, wendyreid, plh, LauraB__, gpellegrino, jmcshane_, 14:18:58 ... CharlesL, marisa, Garth, JuanCorona, Rachel_Yager, Karen, zhengxu, Hadrien, Bill_Kasdorf_ 14:18:58 On IRC I see Bill_Kasdorf_, JulieJ, zhengxu, Judy, Garth, marisa, fchasen, circularken, CharlesL, Courtney_B, JuanCorona, jmcshane_, LauraB__, gpellegrino, plh, George, 14:18:58 ... MasakazuKitahara, toshiakikoike, Yanni, dkaplan3, Avneesh, MattChan, duga, wendyreid, romain, RRSAgent, Zakim, ivan, tzviya, Karen, dauwhe 14:19:00 q+ 14:19:03 q+ 14:19:21 q+ 14:19:30 ack plh 14:19:53 https://docs.google.com/spreadsheets/d/1wOe-6MqFk296dCoRUmrd3ghmVXH3yM-i97hFOuGlX4k/edit#gid=708869567 14:20:06 plh: you have different categories of tests, e.g. unique tests 14:20:07 q- 14:20:22 ... we have test harness tech 14:20:45 ... then you have tests that test sections, we don't have any of those 14:20:53 ... then you have marketing tests 14:21:18 q+ 14:21:35 ... these tests were not so much aimed at testing implementation, but more at motivating conformance 14:21:55 ... e.g. the happy face that showed up if you passed this type of test 14:22:14 q- 14:22:23 http://acid2.acidtests.org/#top 14:22:56 http://acid3.acidtests.org/ 14:24:00 wendyreid: for content authors, what testing information are you looking for, or would like to see as a result of the testing? 14:24:28 q+ 14:24:32 ... this is aiming at the same thing as the acid tests (i.e. the final category), so authors can see how many rs have a smiley face for a feature they want to use 14:24:34 ack wen 14:25:18 tzviya: responding to wendy, good point. A big thing that discourages authors from using a feature is that they are unsure about interoperability 14:25:23 ... i really like the idea of the acid tests 14:25:23 q? 14:25:38 ack tz 14:25:41 q+ 14:25:52 ... a lot of devs do unit testing for their own rs, could we reuse some of these? 14:25:57 q+ 14:25:59 q+ 14:26:37 ack dauwhe 14:26:40 dauwhe: the tests that some of these rs have might be embedded into their development framework, and not reusable 14:27:06 ... or they might be dealing with handling of invalid content, i.e. handling things that would never get through epubcheck 14:27:10 q? 14:27:11 ... these tests wouldn't help us 14:27:12 ack gp 14:27:26 qq+ 14:27:28 gpellegrino: is css support outside scope of these tests? 14:27:35 ... i.e. using new css rules, etc. 14:27:37 dkaplan31 has joined #epub 14:27:49 ... also, it would be useful to determine if metadata is used by rs 14:27:57 q+ 14:28:07 ... also if rs use toc, landmark, etc. 14:28:33 dauwhe: i've started on a script that turns browser css tests into tests for epubs 14:28:41 ... but if we start testing all css features, we're never going to finish 14:29:00 ... maybe css support is more an area for acid testing, as opposed to comprehensive unit testing 14:29:02 q+ 14:29:06 https://docs.google.com/spreadsheets/d/1wOe-6MqFk296dCoRUmrd3ghmVXH3yM-i97hFOuGlX4k/edit#gid=1730700660 14:29:12 ack dauwhe 14:29:12 dauwhe, you wanted to react to gpellegrino 14:29:42 q+ 14:29:43 ... re. the metadata test, that is definitely going to get tested. Some of these things correspond to assertions in the spec 14:29:59 q? 14:30:02 ack ivan 14:30:20 ivan: how do we relate to the existing testing environment for css, svg, mathml? 14:30:29 q+ 14:30:35 ... because epub's content document are made of those, so we shouldn't repeat work 14:30:38 ack mari 14:31:07 marisa: we should mine the old epubtest for content 14:31:29 https://github.com/IDPF/epub-testsuite 14:31:38 ... the process for building those test books should be streamlined, but once we determine what we want to test, we could look through the epubtest repo and see what we can reuse 14:31:44 dauwhe: agreed 14:31:45 ack dug 14:31:52 q? 14:32:24 duga: it'll be harder for us to go through our own testing framework and pull tests for reuse vs just going through the spec and making new tests 14:32:45 ... when it comes to the acid tests, as someone who tried to pass, I didn't find them terribly useful 14:33:12 ack plh 14:33:22 ... you essentially had to implement everything before you could get a pass, no indication as to how close you are to passing, it was pass/fail 14:34:05 plh: the acid test was a marketing test, purely meant as motivation to implementors, that's why it didn't give an indication as to how close you were to being feature complete (according to the test) 14:34:09 q? 14:34:16 q+ 14:34:27 q+ 14:35:12 ack rom 14:35:23 ... you won't be able to test everything, so prioritize which areas of the spec are most crucial for testing 14:36:04 romain: one of the primary sources of interop problems between rs is that spec is not clear on how rs should process content that is invalid 14:36:16 ... should spec provide some guidance on error handling? 14:36:24 Hadrien has joined #epub 14:36:25 +1 to romain 14:36:40 +1 to romain 14:37:24 dauwhe: one interesting source of tests is bugs, if we want to decide how rs should respond to erroneous content, its difficult to do this in the abstract 14:37:31 present+ 14:38:16 q+ 14:38:22 ... about the scope of testing, the scope should depend on the epub community and the wider ecosystem, but it also depends on demands of w3c management 14:38:27 ack we 14:38:28 ... may have to consult the director 14:38:44 s/the director/"The Director"/ 14:39:58 wendyreid: any rs which fail our test shouldn't see this as a mark against them, what we are testing is the quality of _the spec_ 14:40:15 q+ 14:40:19 ack dauwhe 14:40:22 +1 wendyreid 14:40:22 ack plh 14:40:24 ... it could be either that the rs hasn't implemented properly, OR that the spec is really hard to implement/needs to be rewordd 14:40:30 s/rewordd/reworded 14:40:32 ack George 14:40:34 q+ 14:41:25 George: at epubtest we will be continuing our accessibility testing, but in that process we end up testing a whole lot of things that the epub wg will end up testing 14:41:37 q? 14:41:50 ... maybe when we report these relevant results, we can report to the epub wg too 14:41:54 q+ 14:42:05 ack plh 14:42:35 plh: about talking to the w3, look at it like a negotiation entering CR phase 14:42:41 ... w3 will be reasonable 14:42:45 although we need to consider that epubtest.org tests are all manual tests 14:42:46 q? 14:42:55 ... our goal is to make the web work, not to nitpick on the spec 14:43:02 q+ 14:43:11 ack garth 14:44:10 q+ 14:44:23 Garth: re. George's last comment, tests that determine things not covered by an in-spec assertion aren't as relevant to our testing here 14:44:25 q+ 14:44:44 ... those tests may be relevant to the individual rs, but might not be as relevant to us 14:45:33 ... re. wendy's comment, we should start with the must assertions, and just try to get the 2 independent implementations 14:45:47 ack judy 14:45:47 re. marissa's comment, we don't need to test a lot of those higher level features 14:46:27 s/marissa/marisa 14:46:41 Judy: re. manual testing vs testing automation, large scale testing that w3 has done have been automated 14:47:21 ... but testing in accessibility has been manual, so it would be good to point this out, and make a specific plan for these more laborious tests 14:47:35 ... this also applies to the entire class of tests that have to be done manually 14:48:02 0% fully automatable I'm afraid :/ 14:48:05 ... what percent of your tests are: fully automatable/semi-auto/fully-manual req. human experts? 14:48:32 dauwhe: i think fully automatable tests is 0% since we aren't working in browsers, and cannot assume scripting support 14:48:37 q? 14:49:14 Judy: i suggest you look for appointments with Phillipe or myself, to specifically look at testing challenges on rec-track, and how to scale that in light of the manual nature of a lot of your tests 14:49:20 q+ 14:49:22 ... i was hoping more of your tests were already automated 14:49:35 https://docs.google.com/spreadsheets/d/1wOe-6MqFk296dCoRUmrd3ghmVXH3yM-i97hFOuGlX4k/edit#gid=1730700660 14:49:51 dauwhe: this doc is trying to start identifying normative statements in our spec 14:50:09 ... categorizing by topic and subtopic, listing the assertion itself, whether it is must/should 14:50:12 s/or myself/and if needed also myself/ 14:50:17 ... this is an example of some of the work we need to start doing 14:50:35 q+ 14:50:42 ack dauw 14:50:43 ... i think someone mentioned tooling that could automatically find some of the normative statements 14:50:45 ack zh 14:51:14 zhengxu: we need to start with the must items, no need to spend time on css errors in the beginning 14:51:44 ... re. css support, we can let the publishers work directly with the css cg 14:52:38 q? 14:52:38 ... we should start with a smaller scope of testing, and gradually expand scope as we develop our testing methods 14:52:39 q? 14:53:13 George: though js support is not required, some do. Can we automate testing on the rs that do support js? 14:53:31 ack Judy 14:53:34 dauwhe: not sure... that might present issues in terms of how we report results... 14:53:34 ack geor 14:54:09 Judy: i understand someone from publishing area has been liaising with accessibility testing taskforce 14:54:11 Romain from DAISY is in ACT 14:54:19 q? 14:54:23 q+ 14:54:29 I am (was really) in that group 14:54:46 ... has this resulted in any advances in getting testing automation? 14:55:41 romain: i have an idea of what is going on in that taskforce, but that is more concerned with how those tests are worded, vs. how the testing is actually done 14:55:56 q+q+ 14:56:06 q- q+ 14:56:08 q+ 14:56:10 Judy: will follow up with the taskforce, and see if we need to focus more on enabling testing automation 14:57:05 ack zh 14:57:13 q+ 14:57:14 zhengxu: if we don't try automation testing, we'll never know how well automation testing works 14:57:22 During accessibility testing we observed 60% of the tested RS had Java Script support 14:57:51 ... the rs who can't support these automatic tests might be able to make some changes to support these tests 14:57:53 ack plh 14:58:20 plh: there are around 270 musts in the spec as is, and you have to test both the positive case, and the negative case 14:58:43 ... plus, there are some assertions which aren't phrased as must, but effectively are must statements 14:59:19 ... then, where the spec includes algorithms, these effectively imply must statements as well 14:59:44 ack mar 14:59:52 ... remember that your time isn't unlimited, timebound your testing 14:59:56 https://daisy.github.io/old-epub3-support-grid/features/ 15:00:18 https://github.com/IDPF/epub-testsuite/tree/master/content/30/epub30-test-0102/EPUB/xhtml 15:00:26 marisa: re. epubtest, there is a small section about scripting, beyond this section, we weren't able to automate anything 15:00:40 ... for that number of musts, are those content musts, or rs conformance musts? 15:00:45 circularken has left #epub 15:00:51 plh: that's both 15:00:55 ken has joined #epub 15:01:44 q? 15:01:46 q+ 15:02:05 marisa: i'm noticing some musts that will be hard to test 15:02:13 q+ 15:02:21 ack roma 15:03:18 romain: part of this work will involve reworking conformance classes 15:04:34 ... e.g. your content must have nav document - this is about content, this is easy to test, only a matter of creating a test content that has these features 15:05:19 dauwhe: re. marissa's last, i've also found some musts that are hard to test. This process will help us reword the spec to make it more clear 15:05:48 s/marissa/marisa/ 15:05:53 ... next step is to collect the people in the group who are interested in working on this 15:06:48 wendyreid: is there anyone interested in volunteering today? 15:07:05 George: i chaired the rs accessibility working group so... 15:07:13 dauwhe: if possible I will contribute 15:07:18 certainly interested in contributing too! 15:07:19 marisa: me too 15:07:22 zhengxu: me too 15:07:36 duga: i would like to participate 15:07:55 dkaplan31: in theory, include me! 15:08:36 ivan: we need someone really willing to lead 15:08:46 wendyreid: we'll organize that later 15:09:03 Daniella will also participate! 15:09:25 15:09:46 I can try to scribe 15:10:38 Thanks MattChan 15:21:59 as long as having fallback it should be fine? 15:26:23 dkaplan31 has left #epub 15:26:29 dkaplan31 has joined #epub 15:27:31 Scribe: Karen 15:27:59 Topic: FXL A11y 15:28:09 https://comica11y.humaan.com/ 15:28:09 Wendy: this part of the session we will continue conversation we started last night on EPUB accessibility 15:28:14 ...this link is fun, cool 15:28:17 ...It's a comic 15:28:32 ...get to look at comics as part of work...who doesn't love that! 15:28:45 ...I did this introduction yesterday but will do it again today because we have a different audiences 15:28:47 love the responsive design! 15:28:49 ...to make sure everyone gets the context 15:28:58 ...one of the deliverables that we have in this WG 15:29:19 ...is to update a number of documents, including the EPUB Accessibility Guidelines; 1.0 or 1.1 thanks to ISO 15:29:31 ...one of biggest questions is fixed layout content, fixed layout accessibility 15:29:52 ...a lot of publishers use for comics, cookbooks, magazines, text books; many use cases and needs 15:30:04 ...especially as more publishers move to born accessible workflows 15:30:11 ...we want things to be consistently accessibilty 15:30:14 s/accessible 15:30:30 ...core spec has all the tools we need to make fixed layout accessible; HTML; SVG; CSS 15:30:38 ...but we need guidelines to make a consistent experience 15:30:44 ...put out a call on twitter, talked with people 15:30:51 ...about content accessibility experiences 15:30:57 ...Studio called Human in Australia 15:31:05 ...spoke to developer Paul to walk me through the process 15:31:16 ...it is a completely unsustainable way to produce content at scale 15:31:27 ...what might be an outcome is creation of a tool 15:31:36 ...for an automated tool for writing captions 15:31:51 ...talking to other people, SVG was recommended a lot for description text and overlays 15:31:56 ...other things were core best practices 15:32:05 ...put in agenda an item written by Ken Jones 15:32:20 ...we have outlined some in accessibility, but these core principles are very important 15:32:26 q? 15:32:34 q- 15:32:36 ...see the rec and say reading order is important, but how do I apply it; how do I watch out for this problem 15:32:42 ...other problem is tooling, if we can apply that 15:32:54 q+ 15:32:54 ...not saying lack of tools is causing problem, but there is that 15:33:00 ...some output from tools I have seen is crazy 15:33:02 ack Garth 15:33:05 ack Garth 15:33:07 Garth: A lot of time allocated to this topic 15:33:13 ...wonder what we think the output will be 15:33:17 ...you started out correctly 15:33:29 ...you can write accessible content using fixed layout in text 15:33:31 ...can use SVG 15:33:41 ...and stroke every curve of every letter and it can be horrible 15:33:52 ...are these...do we think what comes out of this is a set of best practices 15:33:56 ...is that in the scope of this WG/ 15:34:08 ...or do we envision changes to FXL spec 15:34:16 ...I'm curious how we envision "winning"? 15:34:25 Wendy: what I would like to see come out of this conversation 15:34:32 q+ 15:34:37 ...I have some ideas about directions, but as a group would like to discuss what is the best output 15:34:46 ...I don't think it's changes to FXL stuff, tools are there 15:34:53 ...I definitely see additions to the accessibility docs 15:34:58 ...but I am opening up the floor 15:35:03 ...I would like us to come out of this meeting 15:35:24 ...with either a single direction to research, or have a few people go off to experiment and research different options 15:35:41 ...we need a solution or series of recommendations that are implementable for content authors and reading systems 15:35:58 ...one things from last night's conversations, we might have to recommend different things for different content types 15:36:07 q+ 15:36:07 ...for example, manga and text books may differ 15:36:07 ack Avneesh 15:36:18