EPUB 3 Working Group F2F, 1st Day — Minutes
Date: 2022-09-12
See also the Agenda and the IRC Log
Attendees
Present: Dave Cramer, Ivan Herman, Wendy Reid, Brady Duga, Tzviya Siegman, Shinya Takami (高見真也), Gregorio Pellegrino, Dan Lazin, Masakazu Kitahara, Romain Deltour, Avneesh Singh, Rick Johnson, Charles LaPierre, Daihei Shiohama (塩濱大平), GeorgeK, Matt Garrish, John Roque, Bill Kasdorf, YuYu Lin, Mateus Teixeira, Judy Brewer, David Hall
Regrets: Toshiaki Koike
Guests: David Hall, David Singer, Judy Brewer
Chair: Dave Cramer, Wendy Reid, Shinya Takami (高見真也)
Scribe(s): Brady Duga, Gregorio Pellegrino
Content:
Charles LaPierre: RIP Ben, We will deeply miss you.
Wendy Reid: Different format today.
Tzviya Siegman: CEPC https://www.w3.org/Consortium/cepc.
Wendy Reid: will be a presentation from Ivan.
… Then we will focus on tests, including running them.
1. where are we with our work?.
Ivan Herman: [Sharing slides on screen].
… brief overview of where we are.
Ivan Herman: odd group, since we are not starting from a blank slate.
… WG is starting with a long history of work and requirements.
… 2017 is the starting point of W3C work on epub, but was already on version 3.1.
… First job was to “fix” 3.1 with 3.2.
… Now we are on 3.3 on Rec track.
… Had to go through horizontal reviews, need tests, plus some editorial things.
… we do require all 3.2 documents to be valid 3.3 documents (in our charter).
… And epubcheck is coming with 3.3 support.
… beta release of epubcheck is already available.
Wendy Reid: https://github.com/w3c/epubcheck/releases.
Ivan Herman: limited new features (webp and opus media types, and dir attribute in package).
… epub:type should not be used for a11y anymore, use aria instead.
… Editorial changes - major reorg.
… It wasn’t obvious where to find things in the old organization.
… there wasn’t clear delineation of author requirements and reading system requirements.
… Very hard for authors to read through it all and figure out what they should do.
… New organization breaks out core (for creators), reading systems, and a11y.
… very thorough review which led to improvements in the spec.
… some clarification of what we meant (eg URI handling) plus some entirely new sections.
… Privacy and security sections added.
… during these reviews we pulled some things out as they weren’t really used and were an attack vector.
… interoperability wasn’t very clear, mainly just vague comments.
… IDPF had some old tests, but they were for a different purpose.
… created a new test suite for epub.
… We are not duplicating existing suites (eg html, svg, css).
… No need to test them, since reading systems are typically built on browser engines which are already tested.
… Focus only on epub specific features.
… each test focuses on a single normative statement.
… Currently around 150 tests today.
… If we can’t get two implementations, then we have a problem (more to come).
… Tests have a pretty table with details of each test.
Wendy Reid: https://github.com/w3c/epub-tests.
Ivan Herman: spec and tests are mutually linked (tests point at spec, spec points at test).
Wendy Reid: https://w3c.github.io/epub-tests/.
Ivan Herman: [showing mathml test as example].
… another example, language, which has no impact on the text of the content, just the metadata.
… tests uses the quote element.
… just two examples of what tests look like.
… we will have a final report.
… just sample data now, but we need to fill it out with real data.
… so what happens to statements in the core spec? How do we test those?.
… epubcheck is the answer.
… Will take all the epubcheck tests and make a report from those.
… What about a11y and metadata?.
… Will depend on usage.
… need enough organizations using them.
… What about unimplemented features?.
… That will be a topic for this f2f.
… Some features were added because they sounded good at the time and hoped someone would implement it.
… sometimes that worked, sometimes it didn’t.
… currently called “under-implemented”.
… Risky to use, epubcheck accepts them, but they do not pass the CR tests.
… Usually in W3C it is more straightforward - just remove the feature.
… but our charter binds us to backwards compat.
… So no 3.2 document can be rejected in 3.3.
… Unusual, but basically our only option.
… Deprecated features are essentially removed, there are no tests for these.
… There are also legacy features for epub 2 support.
… same as deprecated, no tests.
Charles LaPierre: We didn’t add any new features, correct?.
Ivan Herman: Only added new features when they were absolutely needed (eg dir attribute).
… and some in security and privacy.
… examples of under-implemented features.
… rendition:flow turns off pagination.
… RS do implement this.
… but RS do it at user request, not author request.
… If all RS ignore the author request then we may need to mark as under implements.
… fallbacks is a trickier case.
… Given a chain of media types, RS should pick what they can support.
… May not have many implementations.
… some features are already removed due to no usage and no implementations.
… 3.3 is here, technical changes are generally done.
… start using it for content now.
… final standard will be published winter next year.
… please run the tests and return a report.
… That’s all!.
Wendy Reid: Questions? Comments?.
Dave Cramer: That was awesome, thank you!.
Wendy Reid: Intros. Will not be scribing this bit..
2. Testing hackaton.
Wendy Reid: https://docs.google.com/spreadsheets/d/13wgptApSVbQbYzi2L5VXe3D_1QLzJOTGySUAtOxn4sA/edit#gid=0.
Wendy Reid: today we’ll continue the meeting focusing on testing.
… there are still few tests that needs to been written.
Wendy Reid: https://w3c.github.io/epub-tests/.
Wendy Reid: then we have to run the tests!.
… the goal is to finish the creation of the tests and start testing the apps.
Ivan Herman: while writing tests I’ve run them locally on my mac.
Wendy Reid: the other part is to be sure that the tests are clear.
… and understandable.
… tomorrow we want to discuss the results of today.
… to understand if some tests have to be revisited.
… check if there are features that are under-implemented.
Wendy Reid: gpellegrino: Do we need to compile the test results into a JSON file?.
Wendy Reid: dlazin: Yes, that would be the easiest way.
Wendy Reid: gpellegrino: Second question, most reading systems do tweaks on the files before giving them to end users, if we sideload we might get different results?.
Wendy Reid: ivan_: We are testing the correctness of the recommendation, if during the publication process, user agent A does something to the file, compared to the author’s version, we have no way of controlling that.
Wendy Reid: https://w3c.github.io/epub-tests/contributing.
Brady Duga: we may fail with sideloading.
… but we may get results if we pass the file trough the publisher pipeline.
Ivan Herman: what happens when a file is sideloaded?.
Brady Duga: the file goes to a completely different pipeline.
Rick Johnson: this is true for us (VitalSource Bookshelf) as well. Sideloading vs. publisher loading are different processes, and have some feature issues associated with them.
Ivan Herman: from my point of view those are two different implementations.
… in theory I could run all the tests on my mac on Apple Books.
… so I think ideally that the test results from an engine should come from the engine itself.
Matt Garrish: since there are a lot of tests, do we have a short-list of tests that may be “problematic”, so we can start with those?.
Brady Duga: is there a repository for the files?.
Wendy Reid: it’s in the github repo.
… it would be easier to have a sub set of tests, but at the same time I think one goal is to have a feedback on how and where EPUB features work.
… so having a table with the results I think would be useful.
Rick Johnson: in the early days of epubtest.org we had problems with maintaining it updated.
Ivan Herman: for sure it is complicated to maintain it updated.
… normally in W3C when a WG ends we create a new maintenance WG for updating the results.
Rick Johnson: the JSON file should contain the version number of the app and the date of the testing.
Ivan Herman: we have a granularity to track implementations on different systems.
Tzviya Siegman: I share Rick’s concerns, but we can’t continue the specs without the tests. Eventually those will become historical document of the implementations in this year.
Dan Lazin: https://w3c.github.io/epub-tests/contributing#implementation-report-files.
Wendy Reid: can we use the JSON metadata so tell if the files where sideloaded or pass through the publishers’ pipeline.
GeorgeK: maybe we have the possibility to test if the reading system is accessible, for example testing MathML.
Judy Brewer: I think there are WGs that do accessibility testing on they’re specs.
Ivan Herman: I think is up to us to check implementations that are accessible.
… at this moment our first concern is to test the specs as they are.
Wendy Reid: Maybe we may have a separate feedback for accessibility, also accessibility testing is not always a yes/no test.
… I think we should focus first on general testing.
Dan Lazin: To prepare an implementation report (ideally for “your” reading system), copy the xx-template.json file here: https://github.com/w3c/epub-tests/tree/main/reports.
Ivan Herman: we may add to some tests some descriptions for defining what we aspect from screenreader consumption of the content.
Dan Lazin: Once committed, your RS will appear as a column at https://w3c.github.io/epub-tests/results#implementations-results.
Tzviya Siegman: I think for the vast majority of tests we don’t need it since we’re based on HTML specs.
… so we don’t have to test it for accessibility.
Wendy Reid: gpellegrino: My proposal would be to add an additional field to say whether it’s been tested with a screen reader.
Wendy Reid: ivan_: Add an additional field in results.
Matt Garrish: I agree with tzviya: we should not re-testing the web technologies for accessibility.
… I would focus on EPUB specific features.
Dan Lazin: And in general, instructions here: https://w3c.github.io/epub-tests/contributing.
Wendy Reid: ok, I think we can start testing.
Dan Lazin: This is mentioned in that documentation, but the needed (unwritten) tests are https://tinyurl.com/epub-tests.
Wendy Reid: dlazin_ has put all the info in the IRC.
David Hall: how do we submit the results? Using PRs?.
Wendy Reid: yes, using PRs.
Wendy Reid: https://github.com/w3c/epub-tests.
David Hall: For iOS/Apple Books, you can airdrop an .EPUB and it will ask if you want to open in the Books application..
Dan Lazin: PR for empty implementation reports: https://github.com/w3c/epub-tests/pull/202.
Dan Lazin: Committed now, so if you fork https://github.com/w3c/epub-tests/tree/main/reports, you can start filling out the major platforms’ implementation reports. Have fun!.
Dan Lazin: The current proposal is that, to get us to 2 implementations ASAP, we federate the work of testing Apple, Google, Kobo, and Thorium. (Kobo not committed yet; Wendy is working on it.).
Dan Lazin: So please call out which tests you’re going to run (say “I will do all the cnt-* tests for Apple Books on macOS”) and we can all merge our work together..
Dan Lazin: And, in the spirit of that … I will do all the cnt-* tests for Apple Books on macOS :).
Dan Lazin: To map between test titles and IDs, see https://w3c.github.io/epub-tests/.
David Hall: Hearing conversation around cnt-svg-css - a missing .epub?.
David Hall: Oh never mind, I’m seeing it now..
David Hall: I can attempt the cnt-* tests for Apple Books on iOS.
Dan Lazin: Thanks, dhall!.
Dan Lazin: Here’s the PR for my first batch, as an example: https://github.com/w3c/epub-tests/pull/204.
Dan Lazin: And yes, alphabetization in Finder is funny, so if you’re on a Mac (and maybe other platforms) you may find the .epubs are alphabetized in a funny way relative to the folders..
Dan Lazin: I will now do all the css-* tests for Apple Books on macOS :).
Romain Deltour: I can start with all the pub-cmt-* for Thorium on macOS.
David Hall: Suggestion: Add test name (“cnt-css-fonts”) to test documents themselves..
Dave Cramer: it turns out epubcheck doesn’t like it when the epub is encrypted :).
Dave Cramer: wrote a test of zip encryption of the container. Having trouble with my GitHub tokens… I’ll do a PR when I get that sorted out..
David Hall: iOS cnt-* tests all pass. Now to get my GitHub account setup properly to push and create PR….
David Hall: *Apple Books tests that is..
Romain Deltour: @dlazin_ it seems I don’t have push permissions on the gh repo. shall I use a fork instead or can you easily add all wg members to it?.