16:03:30 RRSAgent has joined #html-media
16:03:30 logging to http://www.w3.org/2015/01/06-html-media-irc
16:03:32 RRSAgent, make logs public
16:03:32 Zakim has joined #html-media
16:03:34 Zakim, this will be 63342
16:03:34 ok, trackbot; I see HTML_WG()11:00AM scheduled to start 3 minutes ago
16:03:35 Meeting: HTML Media Task Force Teleconference
16:03:35 Date: 06 January 2015
16:03:40 scribe: joesteele
16:03:42 chair: paulc
16:03:44 agenda: http://lists.w3.org/Archives/Public/public-html-media/2015Jan/0007.html
16:04:05 Zakim, who is here?
16:04:05 HTML_WG()11:00AM has not yet started, joesteele
16:04:07 On IRC I see RRSAgent, jdsmith, BobLund, pal, cyril, davide, markw, joesteele, adrianba, trackbot
16:05:51 acolwell has joined #html-media
16:05:56 paulc has joined #html-media
16:06:07 zakim, who is on the call?
16:06:07 HTML_WG()11:00AM has not yet started, paulc
16:06:09 On IRC I see paulc, acolwell, Zakim, RRSAgent, jdsmith, BobLund, pal, cyril, davide, markw, joesteele, adrianba, trackbot
16:06:20 trackbot, start meeting
16:06:22 RRSAgent, make logs public
16:06:24 Zakim, this will be 63342
16:06:24 ok, trackbot; I see HTML_WG()11:00AM scheduled to start 6 minutes ago
16:06:25 Meeting: HTML Media Task Force Teleconference
16:06:25 Date: 06 January 2015
16:06:25 rrsagent, generate minutes
16:06:25 I have made the request to generate http://www.w3.org/2015/01/06-html-media-minutes.html joesteele
16:06:36 zakim, who is on the call?
16:06:36 HTML_WG()11:00AM has not yet started, paulc
16:06:37 On IRC I see paulc, acolwell, Zakim, RRSAgent, jdsmith, BobLund, pal, cyril, davide, markw, joesteele, adrianba, trackbot
16:06:50 Zakim, this is html
16:06:50 ok, adrianba; that matches HTML_WG()11:00AM
16:07:10 I joined but am in a noisy place
16:07:26 zakim, who is on the phone?
16:07:26 On the phone I see [Microsoft], +1.408.536.aabb, markw, Vincent, davide, jdsmith, BobLund, Aaron_Colwell
16:08:02 zakim, [Microsoft] is me
16:08:02 +paulc; got it
16:08:03 Zakim, aabb is me
16:08:04 +[Microsoft]
16:08:04 +joesteele; got it
16:08:14 zakim, [Microsoft] is me
16:08:14 +adrianba; got it
16:08:30 http://lists.w3.org/Archives/Public/public-html-media/2015Jan/0007.html
16:08:32 agenda: http://lists.w3.org/Archives/Public/public-html-media/2015Jan/0007.html
16:09:01 topic: role call
16:09:19 Aaron is here with a cold
16:09:26 Zakim, who is here?
16:09:26 On the phone I see paulc, joesteele, markw, Vincent, davide, jdsmith, BobLund, Aaron_Colwell, adrianba
16:09:28 On IRC I see paulc, acolwell, Zakim, RRSAgent, jdsmith, BobLund, pal, cyril, davide, markw, joesteele, adrianba, trackbot
16:09:59 + +1.650.458.aacc
16:10:08 paulc: meeting will be on MSE topics
16:10:30 topic: update on editors draft
16:10:39 paulc: bugs on are agenda for later
16:10:57 acolwell: since last set of changes have not been able to make more edits yet
16:11:01 ... still working on it
16:11:11 ... clear path for everything except bug filed today
16:11:16 ... fine with doing a heartbeat
16:11:33 ... not sure whether folks want me to resolve all existing bugs -- at least a couple may be non-trivial
16:11:42 zakim, aacc is me
16:11:42 +pal; got it
16:11:43 paulc: wait till we go through the bugs
16:11:52 acolwell: we can mark the ones we want to do
16:11:58 topic: MSE test suite status
16:12:25 paulc: F2F had ACTION-71 re: Cyril figuring out where we stood
16:12:26 ACTION-71?
16:12:26 ACTION-71 -- Paul Cotton to Investigate testing status to move things forward -- due 2014-11-07 -- CLOSED
16:12:26 http://www.w3.org/html/wg/media/track/actions/71
16:12:42 paulc: cyril sent a report
16:13:01 http://lists.w3.org/Archives/Public/public-html-media/2014Dec/0012.html
16:13:06 ... can you step through the reports and remark on any changes since then?
16:13:18 ... please use the queue for questions
16:13:34 ... want to figure out how we should make progress
16:13:52 cyril: the report is in three parts
16:14:14 .... part 1 is about prducing the reports of the runner -- took a while to setup correctly
16:14:28 ... needed to setup on my machine to work correctly, but now working correctly again
16:14:46 ... I have run the test suite we have in the runner on 3 diff browsers -- Firefox, Chrome, IE
16:15:05 ... reports are in JSON and are in the git repository called "test results"
16:15:19 ... that is the initial report but folks can update as needed
16:15:35 ... for IE had lots of crashes at the time, so reports might not be good
16:15:50 paulc: link in the report was for which one?
16:16:00 ... #3 is the raw HTML I think?
16:16:11 cyril: that is the report generated by the online runner
16:16:23 paulc: is there a way to look at that in human readable form?
16:16:48 acolwell: its basically the same way we do specs -- raw git URL
16:16:57 cyril: even that gives you plaintext
16:16:59 Raw data of results: https://github.com/w3c/test-results/commit/265eaeb832a4deb19412dfa863fd422889263ffc
16:17:16 s/prducing/producing/
16:17:33 paulc: I can't see it, can you give us a short summary of the trends
16:17:36 ... ?
16:17:45 cyril: let me find a way to view the results
16:17:52 http://rawgit.com/w3c/test-results/265eaeb832a4deb19412dfa863fd422889263ffc/media-source/all.html
16:18:21 boblund: this should give you a human readable version
16:18:41 paulc: lots of red
16:18:52 ... lots of IE timeouts
16:19:08 ... have you figured out how to get around that problem?
16:19:22 cyril: I was running on Win8 IE11 lots of problems at that time
16:19:37 paulc: are the green and yellow things that were not run in IE
16:19:54 ?1: there were not results in those columns
16:20:15 paulc: seems like a lot of fails in UC10
16:20:37 cyril: not sure what those are -- they were in the tests at that time
16:20:46 paulc: anyone know what the status is in safari
16:20:58 acolwell: they have been filing bugs so I know they are working
16:21:06 paulc: let's keep going
16:21:12 cyril: that was part 1
16:21:26 ... for part 2 started looking at the test files -- very long task
16:21:41 ... I looked at one file in particular and filed some bugs on that
16:21:55 ... Aaron commented but have not had time to respond yet
16:22:06 paulc: so you thought some tests needed changing?
16:22:35 cyril: this test was making calls to isTimeSupported -- which is static -- was assuming WebM or MR4? to be supported
16:22:55 ... my main suggestion was to split this into a core test and separate tests for each byte stream format
16:22:57 q+
16:23:08 s/or MR4?/format/
16:23:27 acolwell: I put some comments, in general I approve of his approach
16:23:41 ... not sure whether this is establishing a base or documenting all possible use cases
16:23:59 ... pretty sure we will not have interop in some cases
16:24:06 ... for example for some codecs
16:24:11 q+
16:24:21 paulc: cyril can you explain the tests you added and why?
16:24:27 https://github.com/cconcolato/web-platform-tests/commit/0499e3fc0103f99fb64f386e5db070af47e5a62a
16:24:45 ack ac
16:25:04 cyril: the MP4 file format is a container for many types of codecs
16:25:16 ... I looked at all existing codecs and proposed a test for those codecs
16:25:31 ... I agree with Aaron that most of these codecs will not be implemented interoperably
16:25:40 ... but some codecs and variations of codecs that would be worth testing
16:25:48 paulc: where did you pull the codecs from?
16:26:04 cyril: from the MP4 registration authority that maintains a list of standards and codecs
16:26:25 ... any vendors can add a codecs -- mp4.org?
16:26:37 ack adr
16:27:12 adrianba: I think the goal that we have is to test interop of MSE itself, and of the normative requirements. MSE does not mandate any particular format support.
16:27:22 ... the spec itself does not have specific codec requirements
16:27:53 ... using isTypeSupported as an example -- we want to make sure implementation do this correctly and do not enumerate a large number of codec to see which are supported
16:28:16 q+
16:28:22 paulc_ has joined #html-media
16:28:38 ... if someone has an implementation tey want to sumbit, which does not support WebM or other codecs, it would be reasonable for them to request a new codec to be added to indicate their support for that API
16:28:52 MSE registry: http://www.w3.org/2013/12/byte-stream-format-registry/
16:28:56 ... but having a long list of codecs is probably not needed and we should be cautious about the ones we add
16:29:17 s/mp4.org/mp4ra.org/
16:29:23 paulc: when I asked the question about where the list was from -- wondered if it was from this registry
16:29:31 q+
16:29:40 ... that would have been the place to start
16:29:53 cyril: the registry only lists the byte stream formats, not the codecs
16:29:54 ack cyril
16:30:05 ... you have to go to the other registry for the codecs
16:30:24 ... I agree with adrina that we want to test the behavior not the byte stream formats
16:30:42 ... what I thought was needed to test MSE specifically were things like --
16:30:51 q+
16:31:04 ... what happens when one files has multiple tracks and some tracks use codecs that are not supported
16:31:14 ack acol
16:31:46 acolwell: I agree with Adrian that we should keep the tests focuses on testing MSE not the byte streams, but I also agree that we cover behavior where there are unsupported codecs
16:31:54 q+
16:32:00 ack ad
16:32:19 ddorwin has joined #html-media
16:32:19 rrsagent, generate the minutes
16:32:19 I have made the request to generate http://www.w3.org/2015/01/06-html-media-minutes.html paulc_
16:32:35 adrianba: was not suggesting that we have tests that do not use the codecs, this could be similar to how we did testing in HTML5 -- HTML4, ogg format or WebM
16:32:56 ... did not matter what format the files where in as long as you could play video and the interaction worked correctly.
16:33:08 ... but we had to have media content to test the elements
16:33:21 +ddorwin
16:33:25 ... in this case we will have to have media that does supported what is being testing
16:33:56 ... thing that is different than going to the MP4 registry and enumerating all those formats and testing whether they are supported
16:34:07 ... don't need to be exhaustive
16:34:31 ... ultimately we are looking to see how many passes for each test, we will have to explain why failures are not important
16:34:33 ack joe
16:34:33 q+
16:35:09 joesteele: is there an unsupported codec?
16:35:18 cyril: yesy I did add that to the tests
16:35:18 ack cyr
16:35:46 ... I think I agree, I did trim the tests to keep the codecs tested to a minimum
16:35:53 ... would like those MP4 codecs to a separate test
16:36:10 paulc: does anyone disagree with that strategy?
16:36:23 acolwell: so you want to kee the tests in a spearate file?
16:36:53 cyril: some would be kept in the tests, some might be kep in a separate file (e.g. on my website)
16:37:04 adrianba: not sure what the split out tests would be for
16:37:17 ... definitely agree that breaking things down and not having all in one place is helpful
16:37:30 ... in the end want it to be easy to anaylse the results
16:37:55 ... e.g. isTypeSupported is implemented interoperably across multiple browwers
16:38:11 cyril: you can give me an action to do that and I will review
16:38:38 ... I will define first a codec that is supported and one that is not supported for each implementation, I I will test both cases for each
16:38:46 ... what matters is that it is a supported coded
16:39:02 paulc: let's pop back up a level -- describing the testing you were doing
16:39:10 cyril: in part 3
16:39:23 ... this work was suggested by Paul at the TPAC
16:39:33 ... looking for coverage of the test suite
16:40:08 ... I looked at the spec and tried to determine how many tests were needed per section of the spec, and tried to see what was there, what was missing and what I had added during my anaylze
16:40:17 ... this is preliminary work
16:40:18 https://docs.google.com/spreadsheets/d/1XKjIuGWjEaSvMkf31HiTaYYkLbyuNg4saP2Sf-iW-cs/edit?usp=sharing
16:40:46 cyril: there are two spreadsheets
16:41:20 markw has joined #html-media
16:41:27 ... this is test coverage -- listing tests that are needed per section, tests available, and link to existing tests for that section
16:41:48 paulc: given that this is organized by section, are you going to revisit each section?
16:42:15 cyril: there are sections I have not scanned yet -- e.g. the audio splice algorithm -- don't know how many tests are needed there yet.
16:42:22 ... from that perspective it is not final yet
16:42:31 ... also I have not linked all the tests to a given section
16:42:55 ... the first tab in that google doc is the test coverage, the second is the list of existing files in that test suite and some notes
16:43:04 s/browwers/browsers/
16:43:11 cyril: probably need to work on the notes
16:43:40 paulc: do you need help from other people to generate tests, or finish the "tested in" column first?
16:43:59 cyril: if folks think this is the right thing to do, need to get agreement on each test
16:44:16 paulc: has anyone reviewed these tests and have comments
16:44:48 jdsmith: I think this approach is solid. intent is to identify testable assertions in the spec. we are going to find out whether we have full coverage.
16:45:18 +1
16:45:23 q+
16:45:24 ... we have to prioritize where we have gaps. I think Cyril is doing a good job and I am impressed with what he had done. We can split things up and help him if he needs it
16:45:32 ack ac
16:45:38 cyril: I would be happy to have help but I will continue on this
16:45:56 https://github.com/w3c/web-platform-tests/pull/1238
16:46:10 acolwell: I am also happy with what Cyril has done. and have a pull request also that I would appreciate review on
16:46:31 cyril: my intent was to get this to a stable state and then review that pull request as well
16:46:46 acolwell: this might reduce the amount of tests that need to be reviewed
16:46:59 paulc: want to have some time to review existing bugs and discuss heartbeat
16:47:12 ... should we schedule another meeting in a month?
16:47:34 ... want to identify places you could use help from others
16:47:39 cyril: that makes sense
16:47:46 topic: MSE bugs
16:48:09 paulc: there are 5 bugs -- Aaron do you want to discuss those bug you mentioned
16:48:15 topic: bug 27649
16:48:34 acolwell: think we have agreements on this
16:48:39 topic: bug 27599
16:48:59 acolwell: think we also have agreement on this need to rwch out and get clarification
16:49:05 topic: bug 27399
16:49:34 acolwell: need to get clarification on how to approach that
16:49:45 s/27399/27239/
16:49:48 s/bug 27399/bug 27239/
16:49:58 topic: bug 27758
16:50:18 acolwell: Bob and I talked awhile ago on this -- can he refresh?
16:50:33 https://www.w3.org/Bugs/Public/show_bug.cgi?id=27758
16:51:06 BobLund: think we had agreement that we should have an information reference to the src? specification -- question was in the byte stream spec or the specification?
16:51:38 ... there was a statement tht the bytestream spec should include the track attrbiute reference
16:51:55 paulc: does your proposal fill that in for each existing item in the regoistry?
16:52:10 BobLund: yes it just says it should follow the existing media format
16:52:19 acolwell: I think that is fine
16:52:24 topic: bug 27242
16:52:24 +1
16:52:38 acolwell: have not started this yet -- might be ab edge case folks have not hit yet
16:52:48 ... buffer ranges can do wierd things with key frame content
16:53:06 ... depending on how folks interpet what is buffered in that content -- need the right test cases
16:53:29 s/key frame/B frame/
16:53:38 s/ab edge/an edge.
16:53:40 s/ab edge/an edge/
16:53:45 topic: MSE heartbet
16:53:47 I propose we include 27599 and 27649 in the heartbeat with 27758 being optional depending on the amt of work to resolve 27758
16:53:55 s/MSE heartbet/MSE heartbeat/
16:54:21 paulc: I am proposing you do the bugs you said were ready, and leave to the editors whether the other are included
16:54:26 acolwell: that sounds fine
16:54:31 I can help with heartbeat if you like
16:54:40 s/rwch/reach/
16:54:55 paulc: put the folks in the CC list and CC me as well
16:55:11 acolwell: sure -- will coorindate with Adrian as well
16:55:25 paulc: next week will continue with EME -- revisit this in about a month
16:55:37 s/this/MSE/
16:55:42 s/revisit this/revisit MSE/
16:55:47 paulc: think we are done for today
16:56:02 paulc: thanks Cyril for attending and all your work!
16:56:07