14:01:31 <RRSAgent> RRSAgent has joined #epub
14:01:31 <RRSAgent> logging to https://www.w3.org/2020/11/20-epub-irc
14:01:34 <Zakim> RRSAgent, make logs Public
14:01:35 <Zakim> please title this meeting ("meeting: ..."), ivan
14:01:37 <ivan> ivan has changed the topic to: Meeting Agenda 2020-11-20: https://lists.w3.org/Archives/Public/public-epub-wg/2020Nov/0010.html
14:01:38 <ivan> Chair: wendy
14:01:38 <ivan> Date: 2020-11-20
14:01:38 <ivan> Agenda: https://lists.w3.org/Archives/Public/public-epub-wg/2020Nov/0010.html
14:01:38 <ivan> Meeting: EPUB 3 Working Group Telco
14:01:38 <ivan> Regrets+ daihei
14:32:09 <Karen> Karen has joined #epub
14:32:11 <zhengxu> zhengxu has joined #epub
14:52:31 <dauwhe> Guest+ dauwhe
14:55:22 <MattChan> MattChan has joined #epub
14:57:18 <tzviya> tzviya has joined #epub
14:57:45 <ivan> present+
14:57:57 <MattChan> present+
14:57:59 <ivan> present+ avneesh
14:58:48 <wendyreid> wendyreid has joined #epub
14:59:08 <MasakazuKitahara> MasakazuKitahara has joined #epub
14:59:31 <toshiakikoike> toshiakikoike has joined #epub
14:59:43 <toshiakikoike> present+
14:59:56 <BenSchroeter> BenSchroeter has joined #epub
15:00:01 <MasakazuKitahara> present+
15:01:01 <Avneesh> Avneesh has joined #epub
15:01:07 <tzviya> present+
15:01:16 <duga> duga has joined #epub
15:01:25 <BenSchroeter> present+
15:01:32 <wendyreid> present+
15:01:56 <MattChan> scribe+
15:02:01 <ivan> present+ brady
15:02:49 <duga> present+
15:03:11 <duga> Hopefully I didn't miss anything - audio trouble when I first joined
15:03:31 <ivan> present+ BillK
15:03:33 <Avneesh> present+
15:03:37 <CharlesL> CharlesL has joined #epub
15:03:43 <CharlesL> present+
15:05:04 <ivan> present+ george
15:06:00 <MattChan> wendyreid: welcome. We're mainly going to be talking about testing this week
15:06:13 <mgarrish> mgarrish has joined #epub
15:06:52 <George> George has joined #epub
15:06:58 <ivan> topic: testing
15:07:04 <George> present+
15:07:12 <Bill_Kasdorf_> Bill_Kasdorf_ has joined #epub
15:07:19 <Bill_Kasdorf_> present+
15:07:20 <MattChan> dauwhe: there's been progress since the f2f, ivan created a repo for tests
15:07:23 <dauwhe> https://github.com/w3c/epub-tests/
15:07:32 <ivan> present+ hadrien
15:07:37 <MattChan> ... let's go over some of the big questions about testing
15:07:42 <dauwhe> https://github.com/w3c/epub-tests/wiki/People-interested-in-testing
15:07:43 <MattChan> ... e.g. who is doing what work?
15:07:57 <ivan> present+ mattg
15:07:58 <MattChan> ... there's a wiki on the repo about this topic
15:08:01 <zhengxu> present+
15:08:17 <MattChan> ... check the repo if you volunteered to make sure you're on the list
15:08:29 <MattChan> ... we still need an official test lead to manage and drive the testing effort
15:08:30 <mgarrish> present+
15:08:31 <zhengxu> (join only from irc for a few mins)
15:08:42 <MattChan> ... if you are that person, please let us know!
15:08:53 <MattChan> ... Q2 what do we test?
15:09:23 <MattChan> ... i.e. What is testable in the spec, What is the scope of testing? What doesn't need to be tested? What will satisfy W3 management?
15:09:45 <MattChan> ... How do we write and organize test?
15:10:12 <MattChan> ... I've started a PR in the test repo, a folder structure based on how the spec is organized
15:10:25 <MattChan> ... folder for core spec, folder for RS conformance
15:11:00 <MattChan> ... there are also a couple tests already in there that I've written in the past
15:11:26 <MattChan> ... e.g. if epub version = 0
15:11:56 <MattChan> ... yes, go ahead and merge the PR
15:12:25 <MattChan> s/merge the PR/merge the PR Ivan
15:12:39 <MattChan> ... Q. How do we deal with all the overhead of our tests?
15:13:22 <MattChan> ... Most of these tests will be an HTML file that shows an issue, or a package file that shows an issue. But for it to be an epub it will still have to have all the other elements that make a complete epub
15:13:41 <tzviya> q+
15:13:41 <MattChan> ... could each test be that bare minimum html or package file + a script that builds it into a complete epub?
15:13:51 <MattChan> ... i've experimented with that approach a little
15:14:05 <ivan> q+
15:14:07 <MattChan> ... or should we just have a generic epub that we modify to have whatever specific assertion that we are testing?
15:14:24 <MattChan> ... not sure which way is better
15:14:50 <George> q+
15:14:55 <MattChan> ... maybe we should just pick a small piece of the spec and write a handful of tests for that, and discover best-practice based on that
15:14:56 <wendyreid> ack tzviya
15:15:07 <MattChan> tzviya: agree with dave about experimenting to see what will work
15:15:43 <MattChan> ... did Richard say that he had already started experimenting with something like this, or a parallel effort?
15:15:52 <mgarrish> q+
15:16:06 <wendyreid> ack ivan
15:16:08 <MattChan> ... we should figure out what he has that we can build on
15:17:02 <duga> q+
15:17:03 <MattChan> ivan: If we had a script that could make an epub from a test html, it should be possible to have github automatically turn it into a epub
15:17:10 <MattChan> wendyreid: that's github actions
15:17:16 <gpellegrino> gpellegrino has joined #epub
15:17:17 <wendyreid> ack George
15:17:18 <MattChan> ... does the w3c have access?
15:17:24 <MattChan> ivan: i wouldn't worry about that
15:17:42 <gpellegrino> present+
15:17:48 <MattChan> George: at epubtest.org, each test book has a unique id for each test, along with the success criteria for that individual test
15:18:13 <MattChan> ... when we go to the website and query a RS that we are testing against, we then mark that test as fail/pass, and the grid gets updated
15:18:28 <MattChan> ... the books are built though github scripts that we maintain
15:18:42 <MattChan> ... the tests are manually carried out
15:19:01 <MattChan> ... would something like this be helpful to this group?
15:19:20 <MattChan> ... also, would the existing books that we have for epubtest be used for our testing in this group?
15:19:31 <MattChan> ... should I share my screen and do a demo?
15:19:41 <wendyreid> ack mgarrish
15:19:42 <MattChan> wendyreid: let's go through the queue first
15:20:39 <MattChan> mgarrish: i think this could get difficult, esp. with package files that contain special settings
15:20:44 <wendyreid> ack duga
15:20:46 <Bill_Kasdorf_> q+
15:20:51 <MattChan> ... may wish to consult roman
15:21:18 <MattChan> duga: i would like as little in the github as possible, to keep it clear what is being tested in each case
15:21:41 <MattChan> ... prefer not to have dummy files that clutter up the repo
15:22:01 <MattChan> ... but i also want something easy to test, i.e. a packaged up epub that is ready to be tested
15:22:28 <wendyreid> ack Bill_Kasdorf_
15:22:30 <dauwhe> q+ to respond to duga
15:22:39 <MattChan> ... some sort of scripting mechanism will probably be necessary, but trying to do it cleanly is going to be interesting
15:23:01 <MattChan> Bill_Kasdorf_: when you start from a master and make variants from it, we have to be careful not to have variants and variants
15:23:02 <wendyreid> ack dauwhe
15:23:02 <Zakim> dauwhe, you wanted to respond to duga
15:23:11 <MattChan> ... quickly gets out of hand with duplicates
15:23:36 <MattChan> dauwhe: maybe some of this can be handled with careful file naming conventions
15:24:58 <dauwhe> q+
15:25:20 <wendyreid> ack dauwhe
15:25:23 <MattChan> wendyreid: yes, we'll have to have a robust naming convention for different versions of files
15:25:53 <MattChan> dauwhe: one thing that happens in the css test is that the html files have a prescribed structure and use meta elements in the html to describe things about the test
15:26:10 <Bill_Kasdorf_> s/variants and variants/variants of variants/
15:26:23 <MattChan> ... e.g. the a meta element contains the test assertions, a meta element that contains a link to the section of the spec being tested
15:26:35 <duga> q+
15:26:54 <wendyreid> ack duga
15:27:39 <MattChan> duga: looking at the version = 0 test, I wouldn't know what this test was meant to show
15:27:57 <MattChan> duga: that's why that test comes with an html file that describes the test
15:28:37 <MattChan> duga: if you had a script that could generate a package based on a number of named parameters, we could include a desc of the test in those parameters
15:29:03 <MattChan> dauwhe: web developers would probably do this using a json data structure that goes into the script
15:29:35 <MattChan> duga: well, most of this is just boilerplate, so the json structure could just say version = 0, that would work
15:30:35 <MattChan> wendyreid: another thing to consider is that the tests should have very clear pass/fail states
15:31:22 <MattChan> ... e.g. in some of the tests I've run in the past, there were partial-passes, which made things difficult for testers, complicates efforts to use automation
15:31:52 <MattChan> ... at Kobo we've had automated rendering tests that just use a compare, if you run it enough its very easy to tell if anything has changed
15:32:04 <MattChan> ... took testing time down from ~day to ~15 minutes
15:32:23 <Bill_Kasdorf_> +1 to Wendy on the tests should be pass/fail and automatable; automation also reduces the variations you get from various people doing the testing
15:34:11 <MattChan> dauwhe: this is a difficult subject to hash out with out being in person, with a whiteboard...
15:34:32 <MattChan> wendyreid: w3c has etherpad... but there are also other similar tools
15:34:59 <MattChan> dauwhe: or what about more information collaboration? IRC outside of official meeting hours
15:35:08 <duga> {package: {version: 0} result: {text: "This test checks if a reading system will open an EPUB with package version attribute less than 3"}}
15:35:14 <MattChan> ... I'm going to keep filling the testing repo, and welcome people to comment
15:36:46 <MattChan> dauwhe: what about when tests start involving interactions between files? Could the json be modified to describe these interactions?
15:37:07 <MattChan> duga: the json could be modified to include a reference to a file name, yeah
15:38:10 <George> q+
15:38:13 <MattChan> dauwhe: okay, we have lots of ideas, so let's go and experiment
15:38:29 <MattChan> ... tell us what you've done and what the results are in the repo using issues
15:39:41 <MattChan> wendyreid: we also need to pull out a new list of assertions
15:40:11 <MattChan> dauwhe: how should we time that with the FPWD?
15:40:18 <wendyreid> ack George
15:40:30 <MattChan> ... maybe wait until FPWD?
15:41:18 <MattChan> George: I like the idea of developing a unique ID for each assertion, or each test of each assertion, maybe pointing to the specific section of the spec being testing
15:41:25 <MattChan> Topic: FPWD
15:41:31 <ivan> Topic: FPWD
15:41:42 <MattChan> wendyreid: mgarrish has made a number of changes to a bunch of the documents
15:42:00 <dauwhe> q+
15:42:06 <MattChan> ... it feels like now is a good time to publish the FPWD
15:42:16 <MattChan> ... the group just has to agree to publish
15:42:30 <MattChan> ... there are a number of open PRs right now, ~7
15:42:41 <MattChan> mgarrish: yes, some are for the a11y spec though
15:42:59 <dauwhe> q-
15:43:00 <MattChan> ... i'm going away for a while, so you guys can review and merge during that interim period
15:43:08 <ivan> q+
15:43:18 <wendyreid> ack ivan
15:43:42 <MattChan> ivan: mgarrish, how long will you be gone for?
15:44:03 <MattChan> mgarrish: probably from next wed thru to ~Dec 20 I'll be on the road
15:44:16 <MattChan> ... i'll still be contactable though
15:44:31 <MattChan> ivan: we won't be able to publish by next Wed
15:44:48 <Avneesh> q+
15:44:49 <MattChan> ... i would prefer to wait until you are back
15:45:36 <MattChan> ... even if we agree today to go to FPWD, I think we should agree not to pub until January. But we can get the process of approvals for FPWD started now
15:45:55 <wendyreid> ack Avneesh
15:46:10 <MattChan> Avneesh: I think Jan is fine. Our deadline for FPWD isn't until Feb
15:46:38 <MattChan> ... regarding a11y, we've just started. I'd like to have some substance in this section of the spec before we publish
15:47:16 <MattChan> ... I'd prefer if we stick to the timeline
15:47:54 <MattChan> ivan: maybe we can make a formal decision about publishing FPWD in december, and I can start the publication process then?
15:48:03 <MattChan> Avneesh: fine with me
15:48:40 <Bill_Kasdorf_> I'm not sure 2020 will concede.
15:49:14 <wendyreid> Proposal: Publish the EPUB Core and Reading System documents as FPWD in January 2021
15:49:28 <mgarrish> +1
15:49:29 <gpellegrino> +1
15:49:29 <tzviya> +1
15:49:31 <BenSchroeter> +1
15:49:32 <MattChan> +1
15:49:32 <ivan> +1
15:49:32 <duga> +1
15:49:33 <wendyreid> +1
15:49:34 <Avneesh> +1
15:49:36 <CharlesL> +1
15:49:38 <Bill_Kasdorf_> +1
15:49:50 <George> +1
15:50:05 <MasakazuKitahara> +1
15:50:12 <zhengxu> +1
15:50:19 <wendyreid> Resolved: Publish the EPUB Core and Reading System documents as FPWD in January 2021
15:50:30 <Hadrien> Hadrien has joined #epub
15:50:38 <George> q+
15:50:45 <Hadrien> present+
15:50:47 <wendyreid> Proposal: Use the ECHIDNA publishing system
15:50:55 <tzviya> +1
15:50:57 <mgarrish> +1
15:51:09 <Bill_Kasdorf_> +1
15:51:15 <MattChan> +1
15:51:28 <Avneesh> +1
15:51:32 <ivan> +1
15:51:40 <CharlesL> +1
15:52:11 <duga> +1
15:52:13 <wendyreid> +1
15:52:27 <BenSchroeter> +1
15:52:34 <wendyreid> Resolved: Use the ECHIDNA publishing system
15:52:37 <George> +1
15:53:00 <wendyreid> ack George
15:53:26 <MattChan> wendyreid: is there AOB?
15:53:37 <MattChan> dauwhe: What about the patent policy?
15:54:03 <MattChan> wendyreid: I submitted the survey. Ivan, will this be processed in the beginning of Dec?
15:54:04 <Avneesh> q+
15:54:05 <MattChan> ivan: yes
15:54:12 <wendyreid> ack Avneesh
15:54:29 <MattChan> wendyreid: so everyone, look out for an email about the policy/re-joining
15:54:48 <MattChan> Avneesh: Did this group have a resolution about patent policy?
15:55:24 <MattChan> ivan: Yes, some weeks ago. It happened during a japanese time meeting, but we should not separate the groups. A resolution for one applies to the other as well
15:55:31 <MattChan> Avneesh: is that true?
15:56:08 <MattChan> wendyreid: Yes... but when a resolution is raised, we won't act on it until a few days later, to give the rest of the members time to comment via email mailing list, to contact the chairs, etc.
15:56:17 <MattChan> ... but we won't re-do the resolution, no
15:56:40 <MattChan> ... Note: we are not going to be meeting next week
15:57:16 <MattChan> ... happy Thankgiving to those celebrating next week!
15:57:25 <MattChan> s/Thankgiving/Thanksgiving
15:57:56 <ivan> zakim, end meeting
15:57:56 <Zakim> As of this point the attendees have been ivan, MattChan, avneesh, toshiakikoike, MasakazuKitahara, tzviya, BenSchroeter, wendyreid, brady, duga, BillK, CharlesL, george,
15:57:59 <Zakim> ... Bill_Kasdorf_, hadrien, mattg, zhengxu, mgarrish, gpellegrino
15:57:59 <Zakim> RRSAgent, please draft minutes
15:57:59 <RRSAgent> I have made the request to generate https://www.w3.org/2020/11/20-epub-minutes.html Zakim
15:58:01 <Zakim> I am happy to have been of service, ivan; please remember to excuse RRSAgent.  Goodbye
15:58:05 <Zakim> Zakim has left #epub
15:58:15 <CharlesL> CharlesL has left #epub
15:58:50 <ivan> rrsagent, bye
15:58:50 <RRSAgent> I see no action items