Publishing Working Group Telco — Minutes

Date: 2019-11-18

See also the Agenda and the IRC Log

Attendees

Present: Ivan Herman, Dave Cramer, Deborah Kaplan, Wendy Reid, Tim Cole, Garth Conboy, Nellie McKesson, Kenneth Dougherty, Franco Alvarado, mutter, Brady Duga, Ric Wright, Gregorio Pellegrino, David Stroup, Romain Deltour, Geoff Jukes, Benjamin Young

Regrets: Tzviya Siegman, Luc Audrain, Laurent Le Meur, Joshua Pyle, Teenya Franklin, Avneesh Singh, Charles LaPierre

Guests:

Chair: Wendy Reid

Scribe(s): Nick Ruffilo, Wendy Reid

Content:

Wendy Reid: https://www.w3.org/publishing/groups/publ-wg/Meetings/Minutes/2019/2019-11-11-pwg

Wendy Reid: Minutes from last week - approval?

Resolution #1: last week’s minutes approved

Wendy Reid: Approved.

1. CR Transition

Wendy Reid: First agenda item - for the last week we held a vote on moving ahead with CR and publishing and naming LPF. I tallied the votes and for LPF - the proposal was: “To publish the Lightweight Packaging Format note with the shortname LPF and long name Lightweight Packaging Format.”
… The vote was 16 +1, 5 0s, and 1 -1, so that is approved.

Proposed resolution: for publishing the Lightweight Packaging Format note, with the shortname of LPF, and full name Lightweight Packaging Format (Wendy Reid)

Resolution #2: for publishing the Lightweight Packaging Format note, with the shortname of LPF, and full name Lightweight Packaging Format

Wendy Reid: Second part of the vote is to move audiobooks. I’m happy to say it was unanimous with 21 votes to go to CRs.

Proposed resolution: Move Audiobooks and Publication Manifest to CR. (Wendy Reid)

Resolution #3: Move Audiobooks and Publication Manifest to CR.

Ivan Herman: From an administrative point of view, we need a resolution on when we plan to close (at which date)

Wendy Reid: That would be the next matter - the first matter would be publishing, correct? First proposal…

Proposed resolution: Publish Workings Drafts of both Audiobooks and Publication Manifest before CR (Wendy Reid)

Deborah Kaplan: +1

Wendy Reid: +1

Ivan Herman: +1

Nick Ruffilo: +1

Dave Cramer: +1

Geoff Jukes: +1

Tim Cole: +1

Gregorio Pellegrino: +1

Romain Deltour: +1

Garth Conboy: +1

Nellie McKesson: +1

Benjamin Young: +1

Ric Wright: +1

Resolution #4: Publish Working Drafts of both Audiobooks and Publication Manifest before CR

Wendy Reid: We will be publishing the working Giraffes, Draughts, no, wait, drafts….

Brady Duga: +🦒

Wendy Reid: the second proposal today is to set up the window of CR (deadline for our implementers to implement by) and then we go to a 4-6 week proposal period. The chairs discussed this and we thought the end of march was a good window to aim for.
… we decided that end of march would be the CR period.

Ivan Herman: just an explanation - what this means is that we will not (shall not) move to recommendation before that date. It doesn’t mean that we MUST move on that date, just gives a notice to implementers so that they can safely plan for that work.

Ric Wright: Yes, 2020

Proposed resolution: The end of the CR period will be March 31 (Wendy Reid)

Ivan Herman: +1

Wendy Reid: so the end of the CR period would be March 31st.

Deborah Kaplan: +1

Nick Ruffilo: +1

Wendy Reid: +1

Tim Cole: +1

Ric Wright: +1

Geoff Jukes: +1

Garth Conboy: +1

Nellie McKesson: +1

Romain Deltour: +1

Benjamin Young: +1

Gregorio Pellegrino: +1

Resolution #5: The end of the CR period will be March 31.

Dave Cramer: +1

Wendy Reid: Awesome - resolved. The end of the CR Period is March 31st.
… 2020
… One of the things we have to do in the next few weeks is that we have to prepare the tests. There is a framework in place if you go on to our github, you will notice there is a PR (thanks to Benjamin) setting up the manifest test. The next step is to work on the tests.
… so the implementers can test their implementations against the spec. I have never done this, and we do have help from Benjamin and Tim and Lars will be helping soon as well.
… Ben/Tim either of you want to talk about how annotations went?

Tim Cole: It worked well for annotations once we had the schemas in place. I’m happy to help Benjamin but he’s further in this than me.

Benjamin Young: The state of the tests now is really just the foundation. There is not much more there. Breaking them into groups helped, but as mentioned in the PR, it’s not avoidable. I can work with whomever is willing to break things down
… We do need to have some documentation about how to run the tests, but they should just run - especially for the validation ones. It should just run a series of tests and tell you if you pass or not.
… if it’s broken into small steps, it gives better feedback.

Ivan Herman: I have something like 30+ tests, where tests are a very small JSON-LD file that is just a manifest. The goal is that the implementation should either process it fine or generate errors/warnings. I would be happy to put these tests into a framework for it to work.
… and have a methodology on how to report. That’s what I have to offer. They aren’t manifests that have anything to do with a JSON schema, but they test the API implementation. What would be the next step for those?

Wendy Reid: We have some for the schema, some for the API - which tests different parts of our spec - we just need to implement them (document them) so it’s clear what is being run/tested.

Ivan Herman: I have a stored description for each of them, but for the time being, I have very simple command line stuff that runs from my machine and it tells me ya-or-nay for a test. Every implementor would be able to see what works and doesn’t work. Even for that, we need some sort of structure to move on with.

Wendy Reid: We would need a spreadsheet - pass, fail, or not implemented. We don’t want to say it fails because it doesn’t use a recommended element… We’ll have to think about how to structure the test suite.

Tim Cole: For annotation we did make a spreadsheet. It ported how each submitted file did against the schemas. It told us if they were violating the spec and how. but it doesn’t get at the API. It’s fairly easy to run a file through a manifest and say the data types are correct, it’s harder to say if the manifest works with the API.

Ivan Herman: Matt also has an implementation, but what really matters are real audiobook readers that take the manifest and take them or not.

Wendy Reid: We need to start producing schemas for implementers to test… (correct: manifests)
… I have a handful for audiobooks as well.

Dave Cramer: I don’t think we should be - we’re hoping implementers can make their own manifests. We’re testing how easy it is to make these things. I hope we don’t drift into having some approved manifest created by the working group that are the only ones used during testing…

Nick Ruffilo: Wendy

Wendy Reid: Some implementers may produce their own file - some might just make things that display manifests - so we have to address all of those scenarios. For example, if they’re just implementing play and display. That will be interesting to see what variety we get.
… sounds like we have the frameworks for both kinds of tests.

Romain Deltour: I’m just curious about the manifests - how they’re going to be used by the implementers. We’re discussing the processing alogrithm, as things get converted to an internal data format - and we can’t test that… How do we want to propose that. Maybe for some limited scenarios you have 2 manifests and the processing algorithm must treat them the both the same…
… so what’s the plan?

Ivan Herman: I don’t have a direct answer, but one thing that can/should be done is that the algorithm itself does 2 things - it should find cases where there is a validation error, or a fatal error. These are well defined, so part of the test I did, part of the expected value is no return.
… or, it must give me a warning about - such as a linked resource without a URL - all are testable. How you test the result is more complicated. Maybe we have good manifests and bad manifests (incomplete and complete)
… Do we want to test automatically or do we believe the implementer?

Nick Ruffilo: As much as we can automate, which is great, but going to UI/frontend validation from a few years ago, there’s a few inputs and some expected outputs. We want to give a very clear what is/isn’t a success. It’s in their best interest for the manifest to work as expected.
… if the implementor has a splash screen, but the manifest says page x needs to appear first, the implementor is correct but would it pass? Keep tests small

Tim Cole: One thing we have to be careful of is going beyond what’s actually in the specification. On the consumer side, it’s going to be reasoanbly easy to establish if a manifest contributed by a producing spec meets the requirements. It’ll be harder that consumers of the manifest are able to demonstrate all the features we support…
… or we’ll have implementations that support features a little different than originally defined. It will be hard to find a set of implementations that will take the manifests and do what we envision. If no implementations are using a feature, then we have to mark it so we can review.

Ivan Herman: this is a challenge, but it depends if we can convince the director. One thing that did come up is to show a mapping between the various terms we have with the equivalent terms that are used in epub today. If the term is widely used in the epub world, then we can reasonably conclude that those terms are important for this version as well and that may be enough.
… i think that mapping - Matt has already started that…

Wendy Reid: Tim makes a good point that we need to look at what kinds of manifests are produces and consumed and see what features aren’t used. As much as auto-testing is nice, we might need to start with manual.
… we have to allow implementors to report the result to us.

Ivan Herman: I would need a URL with some, more or less empty page - where the implementation report would go at the end of the period.

Wendy Reid: Can’t it just live in the publication manifest?

Tim Cole: What i’ve seen done is that the report was put somewhere on Github - in a separate repository, as it makes it much easier for implementors to deposit their segment of the report.

Ivan Herman: I would propose we put it on the Working Group repository.

Wendy Reid: Would anyone like to build a 1990s under construction page?
… Next steps - push to CR, Matt and I will publish the next working draft, get those URLs in the CR report. Ivan - what else are we missing?

Ivan Herman: Have a feeling of when we will think we have the CR published. Tomorrow or Wednesday I can issue a request for CR. Next week is Thanksgiving in the US (most people won’t be doing much)…
… Directors will have a response the week of the 2nd, so either the 5th of the 10th of December seems a safe date. This is just a date for now and it depends on what the director will say about it.

Wendy Reid: Lets try to get this by the 5th.
… Lets get our push to CR then I’ll hit up key people about getting the tests ready.
… Thank you everyone for your hard work and help

2. Resolutions