16:42:10 <RRSAgent> RRSAgent has joined #pwg
16:42:10 <RRSAgent> logging to https://www.w3.org/2019/12/09-pwg-irc
16:42:11 <ivan> rrsagent, set log public
16:42:11 <ivan> Meeting: Publishing Working Group Telco
16:42:11 <ivan> Chair: wendy
16:42:11 <ivan> Date: 2019-12-09
16:42:11 <ivan> Agenda: https://lists.w3.org/Archives/Public/public-publ-wg/2019Dec/0010.html
16:42:11 <ivan> ivan has changed the topic to: Meeting Agenda 2019-12-09: https://lists.w3.org/Archives/Public/public-publ-wg/2019Dec/0010.html
16:42:12 <ivan> Regrets+ Luc, mateus
16:54:10 <NickRuffilo> NickRuffilo has joined #pwg
16:59:52 <Teenya> Teenya has joined #pwg
17:00:32 <geoffjukes> geoffjukes has joined #pwg
17:01:03 <wendyreid> present+
17:01:10 <Avneesh> Avneesh has joined #pwg
17:01:15 <dauwhe_> present+
17:01:24 <marisa> present+
17:01:30 <Teenya> present+
17:01:55 <NickRuffilo> present+
17:02:28 <romain> present+
17:02:29 <George> George has joined #pwg
17:02:39 <ivan> present+
17:02:39 <bigbluehat> present+
17:03:05 <George> present+
17:03:12 <laurent_> present+
17:04:07 <NickRuffilo> ScribeNick: NickRuffilo
17:04:09 <CharlesL> CharlesL has joined #pwg
17:04:18 <CharlesL> present+
17:04:19 <wendyreid> https://www.w3.org/publishing/groups/publ-wg/Meetings/Minutes/2019/2019-12-02-pwg
17:04:26 <Avneesh> present+
17:04:28 <NickRuffilo> Wendy: First task - minutes, approval?
17:04:38 <NickRuffilo> ... approved.
17:04:39 <ivan> +1
17:04:43 <ivan> resolved: last week's minutes approved
17:04:59 <wendyreid> https://github.com/w3c/publ-tests
17:05:26 <garth> garth has joined #pwg
17:05:29 <NickRuffilo> ... The topic of the day is about testing.  (link posted above).  As discussed last week, we've created a separate repository for the tests.  Because we've named it broadly, it can be the testing repo for all publishing activities.
17:05:34 <franco> franco has joined #pwg
17:05:42 <franco> present+
17:05:54 <mattg> mattg has joined #pwg
17:06:02 <Bill_Kasdorf> Bill_Kasdorf has joined #pwg
17:06:10 <NickRuffilo> ... there are separate folders for audiobooks and manifests, reports...  Everything will be located here.  Matt and Ivan did a ton of work over the last couple days to ensure this is ready for view.  Ivan - have I missed anything?
17:06:19 <mattg> present+
17:06:22 <Bill_Kasdorf> present+
17:06:53 <rkwright> rkwright has joined #pwg
17:06:57 <NickRuffilo> Ivan: I think we got everything.  If you go one step deeper, such as the publication manifest repository - it already has 2 subdirectories for various types of testing.  Manifest processing and TOC processing.
17:07:06 <duga> duga has joined #pwg
17:07:10 <duga> present+
17:07:23 <garth> present+ Garth
17:07:29 <NickRuffilo> ... it's something Matt has started.  It is about testing the algorithm to extract TOC from an HTML file.  There may be other testing for audiobooks, there is only Manifest processing for now.
17:07:57 <NickRuffilo> Wendy: I made a few but put them in the wrong area.  I have a few things I need to clean up and potentially make them submittable.  In the same format the other tests are in.
17:08:00 <BenSchroeter> BenSchroeter has joined #pwg
17:08:03 <NickRuffilo> Ivan: You did something that ....
17:08:05 <BenSchroeter> present+
17:08:06 <rkwright> present+
17:08:09 <NickRuffilo> Wendy: They are just in text format right now.
17:08:17 <ivan> https://github.com/w3c/publ-tests/tree/master/audiobooks/manifest_processing/tests
17:08:30 <tzviya> present+
17:08:39 <NickRuffilo> Ivan: If I go to (pasted) directory, that's meant only for tests related to the manifest.  If you want user agent behavior test, it should be a separate folder
17:08:50 <NickRuffilo> Wendy: That's correct, I just haven't had a chance to do that yet
17:09:25 <ivan> https://github.com/w3c/publ-tests/tree/master/test_reports
17:09:44 <NickRuffilo> Ivan: the manifest is relatevely rich.  It's fairly easy to submit a new test.  Finally there is a test report directory which we (pasted) have here.  The intention is that the various types of testing should ahve their own subdirectory
17:09:47 <ivan> https://github.com/w3c/publ-tests/blob/master/test_reports/manifest_processing/index.html
17:10:07 <ivan> https://w3c.github.io/publ-tests/test_reports/manifest_processing/
17:10:38 <NickRuffilo> ... Ideally every test section, whether TOC or the user agent behavior or whatever else, should have a report file like the one provided
17:10:47 <NickRuffilo> Wendy: Matt - anything to add?
17:11:19 <CharlesL> q+
17:11:31 <NickRuffilo> Matt: Nope.  Like Ivan said, I've started with the TOC tests.  I'm having some issues making the tests atomic - as there's alot that has to go into loading a TOC.  In general, it's in progress.  Just still work to do
17:11:37 <wendyreid> ack CharlesL
17:12:03 <NickRuffilo> Charles: Just a question - in the results directory, there is an index.html -> is that where the visual representation of the results will end up?  Should we have a readme that points to the github IO?
17:12:13 <NickRuffilo> Ivan: Which file - can you paste a link?
17:12:20 <Nellie_> Nellie_ has joined #pwg
17:12:21 <CharlesL> https://github.com/w3c/publ-tests/tree/master/test_reports/manifest_processing
17:12:35 <NickRuffilo> Charles: In that directory, there is an index.html
17:12:43 <NickRuffilo> Ivan: that's the result of the test on Manifest processing
17:13:02 <ivan> https://w3c.github.io/publ-tests/test_reports/manifest_processing/
17:13:04 <NickRuffilo> Charles: can we have a readme that points to the github io about how this workS?
17:13:24 <NickRuffilo> Ivan: Ultimately, the whole repository can be looked at from the IO.
17:13:33 <Nellie_> present+
17:13:41 <NickRuffilo> Charles: and if you have a main dashboard that pulls in the main index files - i'm not sure if we want to do that...
17:13:58 <NickRuffilo> Ivan: we will have to do that at some point.  We have to provide a report at the end.  For the time being, that's the only report.
17:14:32 <NickRuffilo> Wendy: I think we're going to have to bump up the readme file on the mainpage to explain where to look/contribute.  I think we've burried that in the subfolder for now.
17:14:41 <NickRuffilo> Ivan: there is a readme, it just needs to be improved
17:14:44 <NickRuffilo> Wendy: agreed
17:14:59 <wendyreid> q?
17:15:40 <NickRuffilo> Wendy: In terms of test coverage, aside from the TOC test, I believe we have pretty much the manifest processing tests are almost complete.  TOC are in progress.  The user agent behavior tests are written but need to be made more usable.
17:16:02 <NickRuffilo> ... I don't know if anyone has attempted to run the tests in any way.  I know Ivan, Matt and I have given a chance.  But if others could try and provide some comments on that...
17:16:11 <gpellegrino> gpellegrino has joined #pwg
17:16:26 <NickRuffilo> ... Lars is very busy with Colibrio right now, so we wont' get an answer from him for a little bit
17:16:42 <ivan> q+
17:16:42 <NickRuffilo> wendy: ((breathing noises))
17:16:45 <wendyreid> ack ivan
17:17:31 <NickRuffilo> Ivan: My question is - what kind of other tests do we need to put there for this round of CR?  I have heard about the TOC, the user behavior testing...  Are there any categories for this sort that we need additionally?
17:17:35 <timCole> timCole has joined #pwg
17:17:36 <gpellegrino> gpellegrino has joined #pwg
17:17:41 <gpellegrino> gpellegrino has joined #pwg
17:17:59 <George> q+
17:18:04 <wendyreid> ack George
17:18:38 <NickRuffilo> George: I don't know if this is relevant, but if audio is encoded in variable bit rate, is there a place to note a file is problematic.  Do we want to include tests for that?
17:19:11 <NickRuffilo> Wendy: I think it's out of scope, and it sounds like the user agent would want to run against their implementation.  You would want to test that bitrates seek in a specific format.
17:19:18 <NickRuffilo> George: for variable bitrate, it could change...
17:19:24 <CharlesL> q+
17:19:30 <wendyreid> ack CharlesL
17:19:31 <NickRuffilo> Wendy: I think that falls into the user agents - hopefully their standard testing would check that
17:19:37 <CharlesL> https://w3c.github.io/publ-tests/test_reports/manifest_processing/index.html
17:20:00 <NickRuffilo> Charles: I found the github IO for the index -> if you wanted to see what those tests look like, you can see that a whole bunch of tests are passing with a few fails.
17:20:17 <ivan> q+
17:20:21 <wendyreid> ack ivan
17:21:32 <NickRuffilo> Ivan: Do we want to have tests about LPF and the way it handles files.  Maybe we want, maybe we don't want.  There are some statements in the manifest processing about the duration values being consistent - such as the duration overall should be the sum of the parts.  But other specifications around media specific items.  Anything we require in the spec?
17:22:15 <NickRuffilo> ... there might be some features - like what George referred to, but more generally.  What we test is the specifics of our specifications - so what items require testing of certain features.
17:22:34 <NickRuffilo> ... we do refer to sync media - but again, a question.
17:23:10 <NickRuffilo> Wendy: the LPF - there's requirements within LPF, such as not zipping audio files.  Laurent - would you consider writing tests around LPF or do you think it needs testing?
17:23:45 <NickRuffilo> Laurent: I think it's a bit premature.  We should first finalize the testing the publication manifest, then we can think about the DF.  There would only be verification of names of the files and the deflates...  Would be very quick
17:23:51 <NickRuffilo> ... we do have time for that
17:23:57 <NickRuffilo> Wendy: those all sound like valid tests.
17:24:03 <tzviya> q+
17:24:06 <gpellegr_> gpellegr_ has joined #pwg
17:24:23 <wendyreid> ack tzviya
17:24:24 <NickRuffilo> ... I'm already getting LPF questions / packaging.  It's worth having a handful of tests to make sure that things are named properly.
17:24:49 <NickRuffilo> Tzviya: You pretty much said what I was going to say.  Especially since Laurent said it's straightforward, I think it's worth doing even though it's just a note.
17:25:13 <NickRuffilo> Wendy: would you consider writing some tests Laurent?
17:25:15 <NickRuffilo> Laurent: Sure.
17:25:25 <NickRuffilo> Wendy: just a few - such as naming and deflate mechanism.
17:25:59 <NickRuffilo> ... the other question is about sync media.  I haven't gotten many questions about it, and it might be too early to test and it might be complicated.  I think it's a bit early, but it could include something in the User Agent behavior.
17:26:12 <NickRuffilo> ... such as "if present, display, if not, don't"
17:26:22 <NickRuffilo> Marisa: I think it is premature, but that basic case is reasonable.
17:26:39 <NickRuffilo> Wendy: because it's a prepheral feature, we don't have to do it just yet, but worth reviewing
17:26:48 <ivan> q+
17:26:53 <wendyreid> ack ivan
17:27:34 <NickRuffilo> Ivan: there was another work started by Matt and something else which is a very different kind of testing.  Whether the terms we define are really useful and used by the community.  That's also important.  Reminds me of the DPUB ARIA testing we had to do.
17:27:49 <mattg> https://github.com/w3c/pub-manifest/wiki/Publication-Manifest-to-EPUB-Mapping
17:28:06 <NickRuffilo> ... I think there was work on mapping the epub terms to our terms, indirectly proving that those terms are used by the community.  This is important to be properly documented and would be in the testing directory
17:28:49 <NickRuffilo> Matt: I just pasted in a link to the document, which is pretty complete at this point.  I've gone through all of the packages spec for epub and cross-referenced to pub manifest.  If we want to move it over or find a new home in the testing repository, that's easy to do.
17:28:57 <NickRuffilo> Ivan: I think it would be better to have everything in one place.
17:29:00 <CharlesL> q+
17:29:04 <wendyreid> ack CharlesL
17:29:32 <NickRuffilo> Charles: I was just looking that over and specifically under accessibility - I'm wondering about the 2 we don't use often, the accessibility API and accessibility control.
17:29:50 <NickRuffilo> Matt: I'm not sure if we still recommend them.  They are the general rule - that you can express both because they are in schema.org...
17:30:13 <NickRuffilo> Charles: we only tell publishers to put that in if they've added in special controls such as special ARIA - so we could see it happen, but it's usually not there.
17:30:22 <ivan> q+
17:30:30 <NickRuffilo> Matt: It can go in there if it makes things more complete - it's not a recommendation, just a mapping.
17:30:47 <NickRuffilo> Ivan: I don't  believe they are listed in the manifest.
17:30:58 <wendyreid> ack ivan
17:31:05 <NickRuffilo> Matt: we didn't list them in the manifest.  You could list them because they are schema.org, but they aren't specifically named
17:31:43 <NickRuffilo> Ivan: to be selfish, our goal is to provide the testing for candidates.  If a feature like this is not mentioned in the CR, we are in no obligation to have information about it.  We can, but what we have is 5 accessibility terms that are in the document and those are important.
17:31:55 <NickRuffilo> Matt: I agree, just pointing out.
17:32:33 <NickRuffilo> Wendy: Back to Ivan's point, are there any tests we're missing?  Or categories?
17:33:20 <NickRuffilo> ... The goal is to have the tests completed by the end of the year.  The only thing we need to finish are the User Agent Behavior, the LPF, and the rest of the manfists.  We have one more meeting next week, and it's the last for the year.  I don't want to put the same pressure on everyone else
17:33:48 <rkwright> q+
17:34:01 <NickRuffilo> Wendy: So we'll round out the testing by next week.
17:34:01 <wendyreid> ack rkwright
17:34:56 <ivan> q+
17:35:04 <NickRuffilo> Rick: One question: I am looking at the tests that are posted, they look very impressive, but it reminds me of the epub test suite.  You test each aspect or feature.  What happens sometimes is that if you have several features that are all brought together, thats when you see problems.
17:35:19 <NickRuffilo> ... is the goal to just verify that each individual feature is there, or is this intended to help the reading systems to regression test?
17:35:21 <wendyreid> ack ivan
17:36:19 <NickRuffilo> Ivan: The goal - since I'm not a reading system implementor, I didn't have the complex situations, but implementors might run into these problems and they would be kind enough to submit to the test suite.  The test suite came out of Matt and I testing the algorithm.
17:36:46 <NickRuffilo> ... because we didn't have to do such global ones, and the same way we produce those tests, others can create them and submit back.
17:37:41 <NickRuffilo> Wendy: That's a great question, I'm having that exact issue with epub this week.  I feel like we have to think about this.  I have to think about how things integrate - so that might be the challenge for the week.
17:37:57 <George> q+
17:38:03 <wendyreid> ack George
17:38:06 <NickRuffilo> Wendy: I'll see if there is a way to test relational issues.
17:38:24 <NickRuffilo> George: Are we saying that we want to stress test a reading system?  Complex TOC and monster audio files?
17:38:27 <NickRuffilo> q+
17:39:20 <NickRuffilo> Wendy: this comes down to more like - i'll use my epub example - right now we're looking at rendition properties.  Some conflict with user properties.  So - have we created a situation where certian properties conflict or cause issue
17:39:20 <mattg> q+
17:39:21 <NickRuffilo> q-
17:39:23 <Avneesh> q+
17:39:40 <wendyreid> ack mattg
17:40:28 <NickRuffilo> Matt: Just going to say - hopefully - we don't have the same degree of problems when it comes to manifest and what epub has - but I do agree with Rick that it's hard to think up the problems that might cause issues.  With TOC there's likely odd issues that could pop up, but it often takes people creating content to find the issues.
17:40:43 <wendyreid> ack Avneesh
17:40:46 <NickRuffilo> ... as Ivan says, it's something that hopefully people can provide info back.
17:41:21 <NickRuffilo> Avneesh: Regarding George's notes.  The scope of this testing can impact improvement in the specification.  What George noted about stress testing - the scope is anything that leads to improving the specification.
17:41:23 <rkwright> q+
17:41:25 <ivan> +1 to Avneesh
17:41:31 <wendyreid> ack rkwright
17:42:11 <NickRuffilo> Rick: I agree with Avneesh.  These sets of tests, these atomic tests, are for testing the specification.  The complicated ones would more be on the user agents.  Should they be somewhere else to ensure people don't get confused?
17:42:12 <NickRuffilo> q+
17:42:18 <wendyreid> ack NickRuffilo
17:42:30 <tzviya> q+ to ask about sample content
17:42:39 <wendyreid> NickRuffilo: I agree with everything said, but I think there's a line where some are spec related and some are RS related
17:43:18 <wendyreid> ... if we have 2 conflicting properties, or cause things to blow up, but are mutually exclusive, we will see that in testing
17:43:31 <wendyreid> ... it's all going to come in the testing process
17:43:40 <wendyreid> ... we'll see as it comes
17:43:44 <wendyreid> ack tzviya
17:43:44 <Zakim> tzviya, you wanted to ask about sample content
17:43:50 <NickRuffilo> Tzviya: I'm wondering if someone wants to create some sample content for us to test?
17:43:59 <NickRuffilo> ... Someone may have already done that???
17:44:28 <mattg> q+
17:44:40 <NickRuffilo> Wendy: It's just like in epub, you can make a spec perfect epub, but publisher X creates on and breaks everything because we didn't think of something specific.  If people are interested and they have content they want to test - then run it through the tests and see what happens.
17:44:47 <Nellie_> q+
17:44:49 <wendyreid> ack mattg
17:44:52 <NickRuffilo> ... if someone wants to help create a bank of test content for implementors to use - that would be helpful
17:45:28 <NickRuffilo> Matt: One thing that worries me is - we've had this over and over in epub.  If we put metadata in, but don't put in priority, when it comes to things like author - if author 3 is the one that displays, is that something we should test for?  Should we note some of those issues?
17:45:30 <wendyreid> ack Nellie_
17:45:57 <NickRuffilo> Nellie: I could potentially create an output to create files for testing.  My only concern is that I have alot going on and I'm not sure if I can do it now or in 2 weeks.
17:46:11 <ivan> +1 to Nellie
17:46:15 <NickRuffilo> Wendy: I would say that owuld be super amazing, and it doesn't need to be done in the next 2 weeks.
17:46:38 <NickRuffilo> ... The tests I'd like done in 2 weeks, but sample content and other resources, the only deadline there is end of March.
17:46:41 <wendyreid> +1 to Nellie!
17:46:50 <NickRuffilo> Nellie: OK, in a couple of weeks I should be able to have something for people to use.
17:47:20 <NickRuffilo> Wendy: On that note... We need User Agen Behavior, Manifest, and LPF.  Last meeting is next week - everyone have a great week.  Talk next week!
17:47:25 <ivan> rrsagent, draft minutes
17:47:25 <RRSAgent> I have made the request to generate https://www.w3.org/2019/12/09-pwg-minutes.html ivan
17:47:25 <ivan> zakim, bye
17:47:25 <ivan> rrsagent, bye
17:47:25 <RRSAgent> I see no action items
17:47:25 <Zakim> leaving.  As of this point the attendees have been wendyreid, dauwhe_, geoffjukes, marisa, Teenya, NickRuffilo, romain, ivan, bigbluehat, George, laurent_, CharlesL, Avneesh,
17:47:25 <Zakim> Zakim has left #pwg
17:47:27 <CharlesL> CharlesL has left #pwg