16:32:35 <RRSAgent> RRSAgent has joined #pwg
16:32:35 <RRSAgent> logging to https://www.w3.org/2019/11/18-pwg-irc
16:32:36 <ivan_> rrsagent, set log public
16:32:36 <ivan_> Meeting: Publishing Working Group Telco
16:32:36 <ivan_> Chair: wendy
16:32:36 <ivan_> Date: 2019-11-18
16:32:36 <ivan_> Agenda: https://lists.w3.org/Archives/Public/public-publ-wg/2019Nov/0037.html
16:32:36 <ivan_> ivan_ has changed the topic to: Meeting Agenda 2019-11-18: https://lists.w3.org/Archives/Public/public-publ-wg/2019Nov/0037.html
16:32:37 <ivan_> Regrets+ tzviya
16:49:43 <ivan> regrets+ luc
16:50:54 <Karen> Karen has joined #pwg
16:51:43 <ivan> regrets+ Laurent
16:56:06 <ivan> present+
16:56:46 <NickRuffilo> NickRuffilo has joined #pwg
16:58:00 <dkaplan3> dkaplan3 has joined #pwg
16:59:20 <mattg> mattg has joined #pwg
16:59:52 <dauwhe> present+
17:00:15 <dkaplan3> present+
17:01:05 <mgarrish> mgarrish has joined #pwg
17:01:32 <Kenneth> Kenneth has joined #pwg
17:02:09 <timCole> timCole has joined #pwg
17:02:52 <wendyreid> present+
17:03:00 <timCole> present+
17:03:32 <geoffjukes> geoffjukes has joined #pwg
17:04:00 <NickRuffilo> ScribeNick: NickRuffilo
17:04:10 <wendyreid> https://www.w3.org/publishing/groups/publ-wg/Meetings/Minutes/2019/2019-11-11-pwg
17:04:18 <NickRuffilo> Wendy: Minutes from last week - approval?
17:04:21 <garth> garth has joined #pwg
17:04:29 <ivan> resolved: last week's minutes approved
17:04:32 <garth> present+ Garth
17:04:32 <NickRuffilo> ... Approved.
17:04:33 <Nellie> Nellie has joined #pwg
17:04:57 <ivan> present+ nellie
17:05:08 <ivan> present+ Kenneth
17:05:52 <franco> franco has joined #pwg
17:05:54 <ivan> Topic: email voting
17:05:58 <franco> present+
17:05:59 <wendyreid> Proposal: for publishing the Lightweight Packaging Format note, with the shortname of LPF, and full name Lightweight Packaging Format
17:06:03 <NickRuffilo> ... First agenda item - for the last week we held a vote on moving ahead with CR and publishing and naming LPF.  I tallied the votes and for LPF - the proposal was: "To publish the Lightweight Packaging Format note with the shortname LPF and long name Lightweight Packaging Format."
17:06:11 <mattgarrish> mattgarrish has joined #pwg
17:06:22 <NickRuffilo> ... The vote was 16 +1, 5 0s, and 1 -1, so that is approved.
17:06:24 <Mutter> Mutter has joined #pwg
17:06:31 <Mutter> present+
17:06:35 <wendyreid> RESOLVED: for publishing the Lightweight Packaging Format note, with the shortname of LPF, and full name Lightweight Packaging Format
17:06:36 <romain> romain has joined #pwg
17:06:37 <david_stroup> david_stroup has joined #pwg
17:06:39 <duga> duga has joined #pwg
17:06:40 <rkwright> rkwright has joined #pwg
17:06:45 <duga> present+
17:06:47 <rkwright> present+
17:06:58 <g_pellegrino> present+
17:07:00 <NickRuffilo> ... Second part of the vote is to move audiobooks.  I'm happy to say it was unanimous with 21 votes to go to CRs.
17:07:08 <david_stroup> present+
17:07:14 <romain> present+
17:07:18 <wendyreid> Proposal: Move Audiobooks and Publication Manifest to CR.
17:07:22 <geoffjukes> present+
17:07:26 <wendyreid> RESOLVED: Move Audiobooks and Publication Manifest to CR.
17:07:29 <ivan> q+
17:07:33 <wendyreid> ack ivan
17:07:50 <NickRuffilo> Ivan: From an administrative point of view, we need a resolution on when we plan to close (at which date)
17:08:24 <NickRuffilo> Wendy: That would be the next matter - the first matter would be publishing, correct?  First proposal...
17:08:31 <wendyreid> Proposal: Publish Workings Drafts of both Audiobooks and Publication Manifest
17:08:45 <dkaplan3> +1
17:08:46 <wendyreid> +1
17:08:47 <ivan> +1
17:08:49 <NickRuffilo> +1
17:08:50 <dauwhe> +1
17:08:54 <geoffjukes> +1
17:08:55 <timCole> +1
17:08:55 <g_pellegrino> +1
17:08:57 <romain> +1
17:08:59 <garth> +1
17:09:02 <Nellie> +1
17:09:12 <bigbluehat> present+
17:09:15 <bigbluehat> +1
17:09:26 <rkwright> +1
17:09:38 <wendyreid> RESOLVED: Publish Working Drafts of both Audiobooks and Publication Manifest
17:09:47 <NickRuffilo> Wendy: We will be publishing the working Giraffes, Draughts, no, wait, drafts....
17:09:59 <duga> +🦒
17:10:38 <NickRuffilo> ... the second proposal today is to set up the window of CR (deadline for our implementers to implement by) and then we go to a 4-6 week proposal period.  The chairs discussed this and we thought the end of march was a good window to aim for.
17:10:41 <ivan> q+
17:10:47 <NickRuffilo> ... we decided that end of march would be the CR period.
17:10:48 <wendyreid> ack ivan
17:11:22 <NickRuffilo> Ivan: just an explanation - what this means is that we will not (shall not) move to recommendation before that date.  It doesn't mean that we MUST move on that date, just gives a notice to implementers so that they can safely plan for that work.
17:12:15 <rkwright> Yes, 2020
17:12:23 <wendyreid> Proposal: The end of the CR period will be March 31
17:12:35 <ivan> +1
17:12:36 <NickRuffilo> Wendy: so the end of the CR period would be March 31st.
17:12:38 <dkaplan3> +1
17:12:38 <NickRuffilo> +1
17:12:39 <wendyreid> +1
17:12:41 <timCole> +1
17:12:43 <rkwright> +1
17:12:46 <geoffjukes> +1
17:12:48 <garth> +1
17:12:52 <Nellie> +1
17:12:56 <romain> +1
17:12:59 <bigbluehat> +1
17:13:11 <g_pellegrino> +1
17:13:20 <wendyreid> RESOLVED: The end of the CR period will be March 31.
17:13:22 <dauwhe> +1
17:13:34 <NickRuffilo> Wendy: Awesome - resolved.  The end of the CR Period is March 31st.
17:13:38 <NickRuffilo> ... 2020
17:14:25 <NickRuffilo> ... One of the things we have to do in the next few weeks is that we have to prepare the tests.  There is a framework in place if you go on to our github, you will notice there is a PR (thanks to Benjamin) setting up the manifest test.  The next step is to work on the tests.
17:14:50 <NickRuffilo> ... so the implementers can test their implementations against the spec.  I have never done this, and we do have help from Benjamin and Tim and Lars will be helping soon as well.
17:15:23 <NickRuffilo> ... Ben/Tim either of you want to talk about how annotations went?
17:16:00 <NickRuffilo> Tim: It worked well for annotations once we had the schemas in place.  I'm happy to help Benjamin but he's further in this than me.
17:16:02 <wendyreid> https://github.com/w3c/pub-manifest/pull/130
17:17:27 <NickRuffilo> Tim: The state of the tests now is really just the foundation.  There is not much more there.  Breaking them into groups helped, but as mentioned in the PR, it's not avoidable.  I can work with whomever is willing to break things down
17:17:28 <ivan> q+
17:18:10 <NickRuffilo> ...  We do need to have some documentation about how to run the tests, but they should just run - especially for the validation ones.  It should just run a series of tests and tell you if you pass or not.
17:18:23 <NickRuffilo> ... if it's broken into small steps, it gives better feedback.
17:18:27 <wendyreid> ack ivan
17:18:45 <NickRuffilo> (( the above is Benjamin not Tim)
17:18:51 <duga> duga has joined #pwg
17:19:22 <NickRuffilo> Ivan: I have something like 30+ tests, where tests are a very small JSON-LD file that is just a manifest.  The goal is that the implementation should either process it fine or generate errors/warnings.  I would be happy to put these tests into a framework for it to work.
17:19:51 <NickRuffilo> ... and have a methodology on how to report.  That's what I have to offer.  They aren't manifests that have anything to do with a JSON schema, but they test the API implementation.  What would be the next step for those?
17:20:28 <NickRuffilo> Wendy: We have some for the schema, some for the API - which tests different parts of our spec - we just need to implement them (document them) so it's clear what is being run/tested.
17:21:15 <NickRuffilo> Ivan: I have a stored description for each of them, but for the time being, I have very simple command line stuff that runs from my machine and it tells me ya-or-nay for a test.  Every implementor would be able to see what works and doesn't work.  Even for that, we need some sort of structure to move on with.
17:21:44 <mateus> mateus has joined #pwg
17:21:57 <NickRuffilo> Wendy: We would need a spreadsheet - pass, fail, or not implemented.  We don't want to say it fails because it doesn't use a recommended element...  We'll have to think about how to structure the test suite.
17:21:58 <timCole> q+
17:22:05 <wendyreid> ack timCole
17:23:02 <NickRuffilo> Tim: For annotation we did make a spreadsheet.  it ported how each submitted file did against the schemas.  It told us if they were violating the spec and how.  but it doesn't get at the API.  It's fairly easy to run a file through a manifest and say the data types are correct, it's harder to say if the manifest works with the API.
17:23:36 <NickRuffilo> Ivan: Matt also has an implementation, but what really matters are real audiobook readers that take the manifest and take them or not.
17:24:02 <NickRuffilo> Wendy: We need to start producing schemas for implementers to test... (correct: manifests)
17:24:13 <dauwhe> q+
17:24:14 <NickRuffilo> ... I have a handful for audiobooks as well.
17:24:17 <wendyreid> ack dauwhe
17:24:56 <NickRuffilo> Dave: I don't think we should be - we're hoping implementers can make their own manifests.  We're testing how easy it is to make these things.  I hope we don't drift into having some approved manifest created by the working group that are the only ones used during testing...
17:24:57 <NickRuffilo> Wendy
17:25:27 <duga_> duga_ has joined #pwg
17:25:40 <NickRuffilo> Wendy: Some implementers may produce their own file - some might just make things that display manifests - so we have to address all of those scenarios.  For example, if they're just implementing play and display.  That will be interesting to see what variety we get.
17:26:07 <romain> q+
17:26:11 <wendyreid> ack romain
17:26:13 <NickRuffilo> ... sounds like we have the frameworks for both kinds of tests.
17:26:42 <ivan> q+
17:27:15 <NickRuffilo> Romain: I'm just curious about the manifests - how they're going to be used by the implementers.  We're discussing the processing alogrithm, as things get converted to an internal data format - and we can't test that...  How do we want to propose that.  Maybe for some limited scenarios you have 2 manifests and the processing algorithm must treat them the both the same...
17:27:19 <NickRuffilo> ... so what's the plan?
17:27:21 <wendyreid> ack ivan
17:28:13 <NickRuffilo> Ivan: I don't have a direct answer, but one thing that can/should be done is that the algorithm itself does 2 things - it should find cases where there is a validation error, or a fatal error.  These are well defined, so part of the test I did, part of the expected value is no return.
17:28:53 <NickRuffilo> ... or, it must give me a warning about - such as a linked resource without a URL - all are testable.  How you test the result is more complicated.  Maybe we have good manifests and bad manifests (incomplete and complete)
17:29:08 <NickRuffilo> ... Do we want to test automatically or do we believe the implementer?
17:29:09 <NickRuffilo> q+
17:29:21 <wendyreid> ack NickRuffilo
17:29:24 <Karen> Karen has joined #pwg
17:30:18 <wendyreid> NickRuffilo: As much as we can automate, which is great, but going to UI/frontend validation from a few years ago, there's a few inputs and some expected outputs. We want to give a very clear what is/isn't a success. It's in their best interest for the manifest to work as expected.
17:30:34 <timCole> q+
17:30:56 <wendyreid> ... if the implementor has a splash screen, but the manifest says page x needs to appear first, the implementor is correct but would it pass? Keep tests
17:31:00 <wendyreid> ...small
17:31:02 <wendyreid> ack timCole
17:31:51 <NickRuffilo> Tim: One thing we have to be careful of is going beyond what's actually in the specification.  On the consumer side, it's going to be reasoanbly easy to establish if a manifest contributed by a producing spec meets the requirements.  It'll be harder that consumers of the manifest are able to demonstrate all the features we support...
17:32:05 <ivan> q+
17:32:32 <wendyreid> ack ivan
17:32:33 <NickRuffilo> ... or we'll have implementations that support features a little different than originally defined.  It will be hard to find a set of implementations that will take the manifests and do what we envision.  If no implementations are using a feature, then we have to mark it so we can review.
17:33:25 <NickRuffilo> Ivan: this is a challenge, but it depends if we can convince the director.  One thing that did come up is to show a mapping between the various terms we have with the equivalent terms that are used in epub today.  If the term is widely used in the epub world, then we can reasonably conclude that those terms are important for this version as well and that may be enough.
17:34:05 <NickRuffilo> ... i think that mapping - Matt has already started that...
17:34:50 <NickRuffilo> Wendy: Tim makes a good point that we need to look at what kinds of manifests are produces and consumed and see what features aren't used.  As much as auto-testing is nice, we might need to start with manual.
17:35:38 <ivan> q+
17:35:41 <NickRuffilo> ... we have to allow implementors to report the result to us.
17:35:41 <wendyreid> ack ivan
17:36:34 <timCole> q+
17:36:35 <NickRuffilo> Ivan: I would need a URL with some, more or less empty page - where the implementation report would go at the end of the period.
17:36:40 <wendyreid> ack timCole
17:36:42 <NickRuffilo> Wendy: Can't it just live in the publication manifest?
17:37:13 <NickRuffilo> Tim: What i've seen done is that the report was put somewhere on Github - in a separate repository, as it makes it much easier for implementors to deposit their segment of the report.
17:37:23 <NickRuffilo> Ivan: I would propose we put it on the Working Group repository.
17:38:06 <NickRuffilo> Wendy: Would anyone like to build a 1990s under construction page?
17:39:47 <NickRuffilo> Wendy: Next steps - push to CR, Matt and I will publish the next working draft, get those URLs in the CR report.  Ivan  - what else are we missing?
17:40:41 <NickRuffilo> Ivan: Have a feeling of when we will think we have the CR published.  Tomorrow or wednesday I can issue a request for CR.  Next week is Thanksgiving in the US (most people won't be doing much)...
17:41:12 <NickRuffilo> ... Directory will have a response the week of the 2nd, so either the 5th of the 10th of December seems a safe date.  This is just a date for now and it depends on what the director will say about it.
17:41:40 <NickRuffilo> Wendy: Lets try to get this by the 5th.
17:42:20 <NickRuffilo> ... Lets get our push to CR then I'll hit up key people about getting the tests ready.
17:43:17 <dkaplan3> dkaplan3 has left #pwg
17:43:18 <NickRuffilo> Wendy: Thank you everyone for your hard work and help
17:43:44 <ivan> rrsagent, draft minutes
17:43:44 <RRSAgent> I have made the request to generate https://www.w3.org/2019/11/18-pwg-minutes.html ivan
17:43:44 <ivan> zakim, bye
17:43:44 <ivan> rrsagent, bye
17:43:44 <RRSAgent> I see no action items
17:43:44 <Zakim> leaving.  As of this point the attendees have been ivan, dauwhe, dkaplan, wendyreid, timCole, Garth, nellie, Kenneth, franco, Mutter, duga, rkwright, g_pellegrino, david_stroup,
17:43:44 <Zakim> Zakim has left #pwg