14:55:12 <RRSAgent> RRSAgent has joined #htmlt
14:55:12 <RRSAgent> logging to http://www.w3.org/2010/05/04-htmlt-irc
14:55:18 <gsnedders> zakim, this will be htmlt
14:55:18 <Zakim> ok, gsnedders; I see HTML_WG(HTMLT)11:00AM scheduled to start in 5 minutes
14:55:24 <gsnedders> zakim, what's the code?
14:55:24 <Zakim> the conference code is 48658 (tel:+1.617.761.6200 tel:+33.4.89.06.34.99 tel:+44.117.370.6152), gsnedders
14:59:06 <Zakim> HTML_WG(HTMLT)11:00AM has now started
14:59:14 <Zakim> +[Microsoft]
15:01:59 <krisk> krisk has joined #HTMLT
15:02:33 <krisk> hello
15:02:44 <krisk> zakim, what's the code?
15:02:44 <Zakim> the conference code is 48658 (tel:+1.617.761.6200 tel:+33.4.89.06.34.99 tel:+44.117.370.6152), krisk
15:03:12 <krisk> trackbot, status
15:03:12 <trackbot> This channel is not configured
15:03:40 <krisk> trackbot, prepare telcon
15:03:40 <trackbot> Sorry... I don't know anything about this channel
15:03:40 <trackbot> If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
15:04:00 <krisk> trackbot #htmlt
15:04:42 <krisk> trackbot, prepare telcon
15:04:42 <trackbot> Sorry... I don't know anything about this channel
15:04:42 <trackbot> If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
15:05:24 <plh> plh has joined #htmlt
15:05:36 <krisk> Zakim, this will be htmlt
15:05:36 <Zakim> ok, krisk, I see HTML_WG(HTMLT)11:00AM already started
15:05:48 <krisk> trackbot, prepare telcon
15:05:48 <trackbot> Sorry... I don't know anything about this channel
15:05:48 <trackbot> If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
15:05:59 <plh> trackbot-ng, start telcon
15:06:01 <trackbot> RRSAgent, make logs public
15:06:03 <trackbot> Zakim, this will be HTML
15:06:04 <trackbot> Meeting: HTML Weekly Teleconference
15:06:04 <trackbot> Date: 04 May 2010
15:06:04 <Zakim> ok, trackbot, I see HTML_WG(HTMLT)11:00AM already started
15:06:16 <Zakim> +Plh
15:07:30 <adrianba> adrianba has joined #htmlt
15:08:53 <krisk> zakim, Microsoft is adrianb, kkrueger
15:08:53 <Zakim> I don't understand 'Microsoft is adrianb, kkrueger', krisk
15:09:01 <Zakim> + +04613479aaaa
15:09:12 <gsnedders> zakim, aaa is me
15:09:14 <Zakim> sorry, gsnedders, I do not recognize a party named 'aaa'
15:09:19 <gsnedders> zakim, aaaa is me
15:09:19 <Zakim> +gsnedders; got it
15:09:19 <krisk> should we start the meeting?
15:09:29 <adrianba> zakim, Microsoft has adrianba, krisk
15:09:30 <Zakim> +adrianba, krisk; got it
15:09:30 <krisk> anyone want to scribe?
15:09:32 <plh> --> http://lists.w3.org/Archives/Public/public-html-testsuite/2010May/0000.html agenda
15:10:06 <adrianba> scribenick: adrianba
15:10:11 <adrianba> scribe: adrianba
15:10:15 <adrianba> chair: krisk
15:10:21 <adrianba> agenda: http://lists.w3.org/Archives/Public/public-html-testsuite/2010May/0000.html
15:10:38 <adrianba> TOPIC: Check for any bugs on approved tests (currently zero)
15:11:21 <adrianba> krisk: currently zero, but we can have some bugs once we make changes to the harness
15:11:37 <adrianba> ...to support james graham's async feedback
15:12:18 <adrianba> gsnedders: don't want to start with one harness and to keep hacking another on top and one on top of that
15:12:28 <adrianba> krisk: do you have an idea what that should look like?
15:12:35 <adrianba> gsnedders: not really
15:12:42 <gsnedders> jgraham: ping
15:12:49 <adrianba> krisk: in the windows tests i think i did something along those lines
15:13:08 <adrianba> plh: i wonder how independent we can be from the harness
15:14:01 <adrianba> ...so that the tests can be written in a way that makes it easy to switch to a new harness
15:14:13 <adrianba> ...if we get another harness that is better
15:14:29 <adrianba> ...we don't have to re-write everything
15:14:34 <jgraham> FWIW I think the async stuff is the stuff that makes it hard to switch harnesses too
15:14:45 <adrianba> krisk: is this about consistency so that we can depend upon certain things being in the test
15:15:07 <adrianba> gsnedders: the main thing is about having the assert functions that check for what the test is testing
15:15:12 <adrianba> ...assertEquals, etc.
15:15:43 <adrianba> krisk: if you are too specific about what has to be written then you lose flexibility
15:15:51 <adrianba> ...but it has to say pass/fail in a consistent manner
15:16:09 <jgraham> Given a set of assertFunctions and two harnesses that implement them, it should be quite easy to swicth synchronous tests between the two harnesses
15:16:18 <adrianba> ...there will be some tests where assertEquals isn't going to work for you
15:16:30 <adrianba> gsnedders: most harnesses have these kind of functions
15:16:41 <adrianba> ...the main thing that varies is how you say what tests should be run
15:17:32 <adrianba> krisk: nothing would prevent you from changing later - for example i changed from jsunit by replacing a lot of the include files
15:17:50 <jgraham> but with an async test, you are typically deep into the harness-specific behaviour to determine what you need to write, when it gets called, how you express failure, and so on
15:17:58 <adrianba> ...in that regard, if a browser vendor didn't like how the tests ran they could make updates to run in their own way
15:18:49 <adrianba> krisk: the basic contract between the test and the harness is all that has to be defined
15:19:01 <adrianba> ...even if you had an async test you could poll and know the test was starting to run
15:19:11 <adrianba> ...and through some setTimeouts continue to check if the work has been done
15:19:25 <adrianba> ...and at some point the harness can say the work didn't complete and move on
15:19:33 <jgraham> That sounds like a harness-specific design
15:19:38 <adrianba> ...but in the end it needs to look for a pass or test completed result in a consistent way
15:19:52 <jgraham> You can't just replace a test designed to work in that way with one designed to work in a different way
15:20:05 <adrianba> plh: right now, kris you provided one?
15:20:12 <adrianba> krisk: i wouldn't say a harness, more like a loader
15:20:21 <adrianba> ...could be extended to do more
15:20:29 <adrianba> plh: do we need to do more? i suggest we don't need to
15:20:40 <adrianba> ...once we have plenty of tests we can focus on that if necessary
15:20:50 <adrianba> gsnedders: once we have plenty of tests it will be harder to change
15:21:02 <adrianba> ...for example if someone gives lots of tests for a section of the spec
15:21:08 <jgraham> FWIW my point of view is that we should write a harness and provide hooks for people to get the results out for whatever purpose they have
15:21:28 <jgraham> e.g. customs regression-detection systems
15:21:34 <jgraham> *custom
15:21:46 <adrianba> plh: the reason i'm not in a hurry to develop another test harness is to avoid duplicating work
15:21:56 <adrianba> ...would rather delay discussion on test harness until june
15:22:00 <adrianba> ...see where we are then
15:22:16 <adrianba> krisk: regardless of the harness we should define the consisten way of seeing a pass
15:22:24 <adrianba> plh: yes, we should define assertEquals, etc
15:22:32 <adrianba> krisk: i mean even like tag id 
15:22:39 <adrianba> gnsedders: why do we need to define that?
15:22:53 <adrianba> krisk: a number of tests have different ways of saying the pass
15:22:57 <jgraham> one assert != one passing test
15:23:05 <adrianba> ...at the moment mostly it is reading what is on the screen
15:23:26 <adrianba> gsnedders: i sent e-mail an hour ago about how to programatically get this out
15:23:46 <gsnedders> s/how to/getting this/
15:24:08 <krisk> this post ?
15:24:10 <krisk> http://lists.w3.org/Archives/Public/public-html-testsuite/2010May/0005.html
15:24:21 <gsnedders> yeah
15:24:51 <adrianba> krisk: when you call assertEquals, the implementation of that will go call something else
15:24:57 <adrianba> ...so you take the logging out of the test
15:25:06 <adrianba> ...then you can change assertEquals to do whatever you want
15:25:13 <jgraham> So one approach is that you have a callback funtion that you call when the test has a result
15:25:16 <adrianba> gsnedders: this is what most of the JS libraries do
15:25:33 <adrianba> krisk: otherwise you have a consistent place - the harness knows to go look there
15:25:43 <adrianba> gsnedders: the problem is that you have to keep polling the DOM
15:25:54 <adrianba> ...but a function means you can send it back as soon as you get the call
15:26:05 <adrianba> ...makes it easy to send back to regression tracking systems
15:26:11 <jgraham> To be clear, a single assertEquals should not be considered a single result. One result may be the combination of several assertions
15:26:44 <adrianba> krisk: should the harness be in control? what happens if the test blows up?
15:26:58 <adrianba> gsnedders: the harness should catch all exceptions
15:27:27 <adrianba> ...for critical issues in a browser you won't be able to access the DOM either
15:27:32 <adrianba> ...i don't think there's a difference
15:28:01 <adrianba> krisk: it sounds like we would want to change the DOM tests JS implementation of assertEquals to fit into a harness at some point
15:28:09 <adrianba> gsnedders: yeah
15:28:18 <adrianba> krisk: sounds like we have agreement
15:29:03 <krisk> wiki -> http://www.w3.org/html/wg/wiki/Testing
15:29:03 <plh> http://www.w3.org/html/wg/wiki/Main_Page
15:30:28 <adrianba> TOPIC: Review Current Tests Posted To List For Approval
15:30:43 <adrianba> krisk: we've covered this too
15:30:52 <adrianba> TOPIC: Ask group for any upcoming tests to be approved/submitted
15:31:00 <adrianba> krisk: is anyone thinking about submitting more tests?
15:31:29 <adrianba> plh: one of the reasons I was asking this - do we have anything that says which areas we are looking for tests for
15:32:00 <adrianba> ...for example, progress events for video aren't implemented in some browsers - if we ask for video tests we might get tests for things that aren't stable
15:32:15 <adrianba> ...i remember we said that we wanted tests for window because that was more stable
15:32:19 <adrianba> krisk: video would be one
15:32:26 <adrianba> plh: yes, as long as you don't go too far
15:32:33 <adrianba> ...for example, testing the js play function is fine
15:33:24 <adrianba> plh: i might start working on video tests, probably not in may
15:33:25 <jgraham> Writing video tests is one way to ensure that we support async tets well :)
15:33:31 <jgraham> *tests
15:34:02 <adrianba> krisk: some people at microsoft have some tests that we'd like to submit soon too
15:34:14 <adrianba> plh: i don't want to duplicate effort
15:34:24 <adrianba> ...if there are people who will submit video tests soonish
15:34:37 <adrianba> gsnedders: i will when i have time
15:34:48 <adrianba> ...hopefully this month
15:35:17 <adrianba> plh: video will be important - <video> gets the most buzz
15:35:35 <adrianba> krisk: do we want to use <video> to test the async part of our requirements?
15:35:38 <adrianba> plh: we could
15:35:42 <adrianba> krisk: we should
15:35:49 <adrianba> gsnedders: okay
15:36:04 <adrianba> TOPIC: Any other business
15:36:21 <plh> http://lists.w3.org/Archives/Public/public-html-testsuite/2010Apr/0017.html
15:36:28 <adrianba> plh: sent a report last week about getting things moved to the W3C site
15:36:46 <adrianba> ...after discussing with system folks they would like to avoid putting thousands of tests into main site
15:36:53 <adrianba> ...running into issues with css2.1
15:37:05 <adrianba> ...will provide access to a web site by end of may
15:37:18 <adrianba> ...will have to different hosts, two different domains in fact
15:37:25 <adrianba> ...so that we can do cross-origin tests if needed
15:37:46 <adrianba> ...also looking at whether test harness runs on this server or if we need it on a third server
15:37:54 <plh> tests.w3.org and tests.www.org
15:37:57 <krisk> for example....
15:38:04 <krisk> tests.w3.org/html5/
15:38:07 <adrianba> krisk: i'd make one primary - i think one ending in w3.org is important
15:38:32 <adrianba> plh: these sites will also host the css2.1
15:38:47 <adrianba> ...would prefer not to run php or whatever on the same site
15:38:53 <adrianba> ...prefer to run harness somewhere else
15:39:06 <adrianba> ...but it's not possible to differentiate which is the harness at the moment
15:39:12 <adrianba> ...still considering how to approach the problem
15:39:23 <adrianba> ...but if harness only uses js then not an issue
15:39:35 <adrianba> ...only an issue if we want to run server-side php or python
15:39:53 <adrianba> gsnedders: we will want some tests eventually for, e.g. slow loading images
15:40:09 <adrianba> plh: at some point we will need something on the server-side but not in the immediate future
15:40:23 <adrianba> ...so we should run with this idea by end of may and this is the main priority right now
15:40:27 <adrianba> ...expect more information next week
15:40:49 <adrianba> ...not going to get deployment right away because of that but will have a better solution by the end of the month
15:40:50 <adrianba> krisk
15:40:56 <plh> htmt5tests.w3.org
15:41:01 <adrianba> krisk: i think this is a good starting point
15:41:33 <adrianba> krisk: keep it simple in the short term seems good to me
15:41:42 <adrianba> krisk: anything else?
15:42:13 <gsnedders> q+ video format
15:42:13 <krisk> When I push to HG now i get a permission denied error
15:42:57 <krisk> I hit the error on 2 differnet systems
15:43:02 <krisk> specific error
15:43:04 <krisk> permission denied: .hg/store/lock
15:46:18 <adrianba> ack next
15:46:49 <adrianba> ack next
15:47:19 <adrianba> gsnedders: for the video tests, we're going to have to use some video format
15:47:22 <adrianba> plh: we can use two
15:47:30 <adrianba> gsnedders: yes, we can use two
15:47:42 <plh> http://www.w3.org/2008/12/dfxp-testsuite/web-framework/START.html
15:47:44 <adrianba> plh: this is similar to what happened in the timed text working group
15:48:01 <adrianba> ...there is a video and it is available in multiple formats
15:48:07 <adrianba> ...it's not a big deal
15:48:39 <adrianba> ...we could use the same videos - they were done for the purpose of testing the timing
15:48:52 <adrianba> ...but it doesn't contain sound
15:48:58 <adrianba> gsnedders: we have test videos too
15:49:14 <adrianba> plh: yes, if people want to submit videos as long as they have the rights we can use them
15:49:29 <adrianba> ...and we can transform into whatever format we need to
15:49:41 <adrianba> krisk: sounds like we have agreement that we'll have to support different formats
15:49:56 <adrianba> ...and plh will help with converting
15:50:08 <adrianba> plh: we will run into some problems for example testing the src attribute
15:50:35 <adrianba> ...but we could have some JS hacks testing which the browser supports
15:50:54 <adrianba> krisk: any other business?
15:52:04 <adrianba> krisk: okay, meeting adjourned
15:52:10 <Zakim> -gsnedders
15:52:11 <Zakim> -Plh
15:52:12 <Zakim> -[Microsoft]
15:52:14 <Zakim> HTML_WG(HTMLT)11:00AM has ended
15:52:16 <Zakim> Attendees were Plh, +04613479aaaa, gsnedders, adrianba, krisk
15:52:20 <adrianba> rrsagent, make minutes
15:52:20 <RRSAgent> I have made the request to generate http://www.w3.org/2010/05/04-htmlt-minutes.html adrianba
15:52:28 <adrianba> rrsagent, make logs public
17:00:32 <plh> plh has left #htmlt
17:54:32 <Zakim> Zakim has left #htmlt
19:21:45 <krisk> krisk has joined #HTMLT