<trackbot> Date: 14 June 2012

<krit> nikos: no party?

<nikos> heh. It's a pretty exclusive party

<birtles> ed, are you still having trouble joining?

<scribe> scribeNick: ed

Status of Shepherd integration / Test The Web Forward

http://www.w3.org/mid/4FD260CC.5050404@w3.org

DS: heycam wanted to work on that, not sure if there's any progress on it
... the focus for svg is for css transforms, we can do it with the css testsuite at the moment
... and then transfer the tests later

ED: is there any info on how to contribute on the TTWF site?

DS: there will be a presentation on how to do that

ED: just making sure the materials will be available online as the event takes place, to enable people participating even though they're not physically there

DS: I will publish them right after doug's presention

TB: peter linss is looking at integrating some of the changes I made for converting html/css test to svg
... it's perhaps not generic enough, my code was for the submitted adobe tests

ED: so there was a question about whether a pass on a test (regardless of the format) should be a pass for that feature or not

TB: i think you have to have separate results, e.g for transforms in svg and for transforms in html/css
... I don't think any of the browsers support the new transforms things for svg

ED: anything else needed from us in time for the event?

TB: would be good to have a couple of approved tests in our repo

DS: don't think that's necessary
... we can use the same process as the csswg uses for review/approval
... can=need

TB: we used to require tests to have a reviewer, are we giving up on that?

DS: no, css requires that too
... you have a creator, a reviewer, and a third person to approve it
... we could say reviewer and approver could be the same person if we want

TB: how does the test become approved?

DS: the shepherd tool moves the approved tests to another directory

TB: right now we have nothing in our approved directory
... shouldn't we have a few in there?

DS: I think so yes
... only a few people have committed tests so far
... I'll look at reviewing and approving some of the tests

TB: i'll move some of my tests to the submitted folder

DS: right, only those will be picked up by shepherd

<scribe> ACTION: Dirk to review (and approve) Tav's submitted svg2-tests [recorded in http://www.w3.org/2012/06/14-svg-minutes.html#action01]

<trackbot> Created ACTION-3308 - Review (and approve) Tav's submitted svg2-tests [on Dirk Schulze - due 2012-06-21].

ED: https://dvcs.w3.org/hg/svg2-tests/raw-file/tip/contributors/tavmjong/submitted/template_001.svg is this the template to use?

TB: yes

<ChrisL> http://www.w3.org/Graphics/SVG/WG/wiki/Test_Suite_Overview#How_to_Create_Tests

DS: all tests must be BSD-licensed, right?

CL: yes

<ChrisL> http://www.w3.org/Consortium/Legal/2008/03-bsd-license.html

TB: how does it work with linking to spec sections, since the spec is still pretty much in flux?

<ChrisL> template should be the same as http://www.w3.org/Graphics/SVG/WG/wiki/SVG2/Testing_Requirements#Revised_structure_after_comment_by_Peter_Linss

DS: you should link to the toplevel section

CL: so the file ED linked to isn't the latest template, should be revised

TB: I thought we agreed to allow link elements

DS: I think we have a resolution for that already

CL: for link peter said it was easier for him to import if it was in the html namespace
... this is all documented in the wikipage I linked to
... this is based on discussions with peter last week

DS: so the wikipage represents what we want, ok

<scribe> ACTION: tav to update the svg2 test template https://dvcs.w3.org/hg/svg2-tests/file/tip/contributors/tavmjong/submitted/template_001.svg to be in sync with the agreed format in http://www.w3.org/Graphics/SVG/WG/wiki/SVG2/Testing_Requirements#Revised_structure_after_comment_by_Peter_Linss [recorded in http://www.w3.org/2012/06/14-svg-minutes.html#action02]

<trackbot> Created ACTION-3309 - Update the svg2 test template https://dvcs.w3.org/hg/svg2-tests/file/tip/contributors/tavmjong/submitted/template_001.svg to be in sync with the agreed format in http://www.w3.org/Graphics/SVG/WG/wiki/SVG2/Testing_Requirements#Revised_structure_after_comment_by_Peter_Linss [on Tavmjong Bah - due 2012-06-21].

BB: did we come to a conclusion on the format for the reference images?

CL: we did discuss that
... if possible we agreed that if it's easy to do solid green for pass for example then that's preferred, but there are cases where that can give false positives and cases where it's very difficult, like filters

BB: right... another suggestion is to use one standard text string for tests with text

TB: I have objections to having just green rects

BB: in gecko we use tens of thousands of tests, it's easy to quickly see pass if the pass images are always green

DS: if you see green it's passed, if you see red it's failed, basic principle

<Tav> http://tavmjong.free.fr/SVG/svg2-tests/contributors/tavmjong/incoming/

TB: here are the transforms tests i wrote a while ago
... red indicates failure
... and you can tell what's being tested

BB: how important is it to know what's being tested?

TB: in inkscape it was useful, to show someone

<krit> http://dev.w3.org/csswg/css3-transforms/#transform-functions

BB: if inkscape had a testsuite with 10k tests, then it's still not easy

DS: every test is specified to test one specific thing, it's testing a part of the spec

<krit> http://test.csswg.org/harness/test/CSS3-TRANSFORMS_DEV/single/svg-matrix-008/format/html5/

DS: for that test it's just the matrix value
... it has a red rect behind that will show if there's something wrong

TB: but you can't tell what it's testing just by looking at it

DS: the filename tells you, and the test metadata
... every test should describe what it's testing inside the metadata
... and the pass criteria

BB: one difference is that you should be able to look at a test and see what it's testing, but that's not so important in an automated system

<krit> http://test.csswg.org/harness/test/CSS3-TRANSFORMS_DEV/single/svg-matrix-008/format/html5/

DS: right, because then the automated engine doesn't care what it's testing, just compares the results
... it's up to the author to provide the information
... in the metadata, but it should be there
... I think it's quite clear what it's testing

TB: what do people think?

CL: valuable to run automated tests, but when you get the list of failed tests it's useful if a human can quickly tell whats wrong
... and then it's useful to know what those tests are testing exactly
... this means we should have welldocumented testcases

DS: that's why we do review on them

TB: maybe we should have them separate? it's nice to have to some visual tests (like in SVG 1.1)

DS: we have reftests that can do that, two images have to look the same, otherwise it's a fail
... I agree that visual tests are good, and that if you see red it's failed

<krit> http://test.csswg.org/harness/test/CSS3-TRANSFORMS_DEV/single/svg-scale-001/format/html5/

DS: here is a test, two rects...

<krit> http://test.csswg.org/harness/test/CSS3-TRANSFORMS_DEV/single/svg-skewx-020/format/html5/

DS: this next test fails in all browsers
... anyway, we can also discuss further on the mailinglist

<ChrisL> the 'show reference' link is broken btw

DS: and it means others can follow the discussion

CL: i'd like to say another thing about the template
... the template doesn't use a particular font
... which means every implentation may use a different font
... suggest we standardize on a particular WOFF font

DS: but we don't need that with reftests, because it ensures the font is the same on both reference and testcase
... AHEM is often used in css tests

CL: ahem is not always useful though

DS: I think it's wrong to require a particular font

CL: why is consistency bad?

DS: but it doesn't matter, because we have reftest

CL: what is the problem? we've had unreadable text, and people assuming a particular default font
... if you're testing svg it's not going to reflow text for example

DS: if you add more dependencies then that's an additional thing that can fail

CL: all browsers support this, explain why the pass criteria would make it fail?

DS: but you add unnecessary complexity

CL: so should we also take out the metadata?

DS: that's different

ED: I think it's quite nice to have consistent fonts used throughout the testsuite
... looking back at the SVG testsuite, yes svgfonts/webfonts is additional complexity, but it's also nice to get consistent rendering across platforms

DS: webfonts are a requirement or just something we support in svg?

TB: inkscape doesn't support webfonts

DS: the problem is that if webfonts isn't a requirement for svg

CL: we have resolved that webfonts, and woff, are a requirement for svg

ED: so, can we agree on having a consistent font used when possible?

DS: don't want to require webfonts for the tests

TB: dirks tests are simple, don't require labeling, but svg1.1 tests are more visual
... with labels and so on

nikos: it's a risk if the layout obscures the text
... you don't necessaryly know the output you're going to get

CL: right, you might get unexpected results

TB: but the risk is pretty small, but I prefer the svg11 tests though
... all it would take is to put in a style section in the template to use a webfont as the default font

<ChrisL> yes it would just take one @font-face rule plus a font family and size on the main group

TB: inkscape would ignore it

(discussion on inkscape and testing with references)

<ChrisL> so you would no longer need to make speccial inkscape test versions with all text elements removed

DS: i'm not strongly opposed to adding WOFF fonts, I just think that we should reduce the tests as much as possible

CL: that's why I pushed hard for pass criteria, because it doesn't matter if the WOFF is supported or not if the only thing that needs to be done is to render a rect for example
... unless the pass criteria says you have to look exactly like a given font

DS: for automated tests it's still additional complexity, requirements for passing the test

CL: but if we have flags, then we should just not add the flag "need webfont" if it's not necessary

DS: how do you style the elements?

CL: that's why it should be in the template

DS: but we should only have that if it's needed for passing the test
... don't think we can agree on having this right now

CL: I don't accept your arguments that unflagged tests would fail

TB: don't care so much either way

nikos: no strong opinion from me, but it should be in the template i think so that it's no risk to be missed

ED: i'd be fine with having two templates, one for tests that use text in the visual output, and one that doesn't (where the webfont isn't needed)

CL: that would be ok with me too
... would that be fine with you DS?

DS: yes

TB: ok, so two templates, one for automated tests (without the webfont), one for visual tests (that have the webfont)
... I'll make those templates, what font do we want to use?

CL: freesans would be good

DS: do we also want to have other fonts, serif, bold etc?

CL: probably, but not in the template maybe, but it's good to have a library of fonts that can be used

<ChrisL> <!-- your test suould turn this rect green--><rect x= y= width= height= />

strokebbox

DS: I'd like to add getting the stroked bbox to the spec, is that fine?

CL: yes, we agreed to do that, should be fine

DS: other kinds of bboxes too, like markers, filters etc?

CL: we tried to limit it to stroke I think
... anything that affects strokes should say how it affects the strokebbox

DS: should markers be included?

CL: possibly

DS: ok, i'll try to do this next week (don't need an action)

This is scribe.perl Revision: 1.136  of Date: 2011/05/12 12:01:43  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/my/doug's/
Succeeded: s/new svg things for transforms/new transforms things for svg/
Succeeded: s/we require/we support/
Succeeded: s/webfonts is/webfonts, and woff, are/
Succeeded: s/arguments/arguments that unflagged tests would fail/
Found ScribeNick: ed
Inferring Scribes: ed
Default Present: Cyril, birtles, +61.2.980.5.aaaa, nikos, +1.415.832.aabb, krit, +1.612.789.aacc, Tav, ed, ChrisL
Present: Cyril birtles +61.2.980.5.aaaa nikos +1.415.832.aabb krit +1.612.789.aacc Tav ed ChrisL
Agenda: http://lists.w3.org/Archives/Public/public-svg-wg/2012AprJun/0096.html

WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth

Found Date: 14 Jun 2012
Guessing minutes URL: http://www.w3.org/2012/06/14-svg-minutes.html
People with action items: dirk tav