<foolip> RRSAgent: make logs public

<gsnedders> ScribeNick: boaz

<astearns> “The name comes from earlier variants on the same theme”

<scribe> scribenick: boaz

presentation beginning

foolip: ill be presenting some updates on our work on wpt for the last year
... wpt is our collection of tests for most of the web platform
... in the last year we've added 5k tests (a lot from css)
... also since last year is more results on more browsers
... worked with bocoup to improve the documentation this last year. we added search. we deduped and made the content easier to navigate.

<foolip> slides are https://docs.google.com/presentation/d/1DqzI_GhOE86CKBjJj0SPhf2oUxNE5w94dYzuACGa2oc/edit?usp=sharing

<gsnedders> https://web-platform-tests.org/ for new documentation

foolip: slides are https://docs.google.com/presentation/d/1DqzI_GhOE86CKBjJj0SPhf2oUxNE5w94dYzuACGa2oc/edit?usp=sharing
... we improved fuzzy reftests

astearns: how many have fuziness turned on

jgraham: a bunch

foolip: added comparison tool for ref test results
... we also added filtering to the the search bar of wpt.fyi
... live demo of the filtering and reftest comparison
... priorities for 2019/2020 are to 1) improve interop, looking at driving down browser specific failures
... ... 2) making PR results more actionable and reliable by taking our pr bot which runs tests, and making the output tell you what to do
... 3-7) also test failure triage, search test by metadata, mobile testing, address flakiness, and wpt.fyi ui improvments (ease of use, ally)

astearns: there are a ton of open PRs on wpt. where does getting through that bag log fit in your priorities?

jgraham: we have a reviewer system that consists of yml files that name people as reviewers on a per directory basis

zcorpan: if there is a subject matter expert not in there, you should ask them to add themselves

jgraham: we have a fundamental problem with this, because we can't get reviewers for certain technologies

astearns: I think thats fair. I know a lot of the PRs are on CSS tests. so I think it is on my to push people towards adding themselves as reviewers.

<foolip> boaz ^

AmeliaBR: on the issue of pr backlogs, I think it is worth having a discussion about making better tooling to have things like reassigning issues with no reviews after a week, or on a semireg basis sending an email to the working group about their unreviewed tests

astearns: I could try that manually for CSS

<foolip> present lukebjerring

lukebjerring: there is a tool called github-health.appspot.com that uses the google philosophy.

<zcorpan> https://github-health.appspot.com

foolip: there is a tradeoff, sometimes you have to close out reviews that you can't get to, or sometimes you have to go to amillion. crbug is at amillion.

JohnJansen: there is not an easy mapping between wpt dir structure and wgs and specs.

jgraham: it is supposed to be good.

JohnJansen: asks a q to prove that it is not

jgraham: the PRs get labeled

JohnJansen: at my work, I want to go review some tests. I'm not in the wg or working on a spec, but I want to find the technology area that I am interested in

jgraham: im not sure this is possible
... labels should address this

zcorpan: this is similar to how to find a test for the feature I'm interested in
... you can use grep to do this

jgraham: good idea to email wgs, if you try that in css, please let us know how it goes.

astearns: experimenting manually will help me figure out what to do

<Zakim> AmeliaBR, you wanted to ask about managing not-ready-to-merge PRs

boaz: will you summarize?

astearns: yes

AmeliaBR: in addition to not getting reviews, and sometimes the review leads to a spec issue.

gsnedders: we have a label for this in the PR

AmeliaBR: can I get docs to that?

<foolip> boaz: let's talk later

scribnic none

foolip: yah, or link that in the 2020 priorities doc

scribenic: boa
... boaz

<JohnJansen> scribenick: boaz

foolip: yes I have an issue open

JohnJansen: it sounds like some members of css wg have filed tests expecting someone outside of the wg to review

astearns: no, its an internal complaint
... what about the meta data tagging system

lukebjerring: we are thinking of adding a manifest
... we have a pr for showing open PRs in wpt.fyi

<JohnJansen> +1 to lukebjerring design to let me see PRs open for current directory. that would be much better for my mental model than searching labels

JohnJansen: +1s to lukebjerring design to let me see PRs open for current directory. that would be much better for my mental model than searching labels

foolip: we have identified in the past that test automation is tricky. is that something people are blocked by?

AmeliaBR: im not sure what you mean by automation, but I love the wptests.center

lukebjerring: we're going to integrate that

AmeliaBR: to be able to use that for reftests that'd be great

<JohnJansen> temporary url for feedback on test authoring: https://wptest.center/#/new

foolip: I was refering more to manual tests. how often do you have to write a tests for manual tests

AmeliaBR: we have a legacy issue with svg, where our existing tests were from a manual testing era
... manual comes up with interaction tests. there could be a way to automate those.

foolip: we have a way to automate mouse/keyboard input, we support that in all the browsers so long as you are writing testharness.js tests
... do you need it for ref tests?

<BitBot> (14wpt) [PR] jgraham 03merged 13#19029 into 07master: [Gecko Bug 1574690] P3 longer timeout to ease intermittent COEP test - https://git.io/JemhQ

AmeliaBR: we do have declaritive animations that should work with js turned off

foolip: that is tricky
... how many manual svg tests to you have

?

gsnedders: 514 manual tests in svg

boaz: has anyone recently written a manual test?

AmeliaBR: we did one recently because doing it in an automated way was too hard

<BitBot> (14wpt) [PR] jgraham 03merged 13#19116 into 07master: Fixed lint error in #18886 - https://git.io/JeOgS

foolip: tell us when you run into this and we can advise you

zcorpan: also please file an issue with type:untestable

lukebjerring: if we fix the issues that make svg need 514 manual tests, whats the liklihood that they will be replaced?

AmeliaBR: that's a big to do

SimonSapin: in the case of SVG I think its not just conversion work. buttt its hard to make a reference for some svg features without using the svg features

AmeliaBR: this has definitly been an issue

zcorpan: my immediate thought about how to automate those tests is to have an image

<foolip> boaz: let's keep going with this topic for a bit

zcorpan: and then add fuzziness

<astearns> Replacing actual eyeballs with fuzzy matching

lukebjerring: is there tooling that could walk you through manual tests and generate reftests from that

gsnedders: problem with that is text

jgraham: using images as references isn't a great pattern

boaz: how about mdn survey data and wpt data

kadir: now that we know that interop is a big problem it could be nice to dive into what the specific issues are

foolip: work on a compat/interop follow up survey, then look at what we have in wpt and figure out what to follow up on
... we want to see these things line up but my guess is that it wont

AmeliaBR: one thing on this is that the wpt approach to interop is different than the web developer's. because wpt only focuses on latest browser, where as web devs have to support old versions.

rachelandrew_: I agree, that a lot of the issues will be older browsers

boaz: we should have a consistent vocab between wpt and mdn survey

lukebjerring: yes, i agree
... it could be good to name a feature in wpt, and ask a dev if they think the feature is interoperable

boaz: not sure if we can ask web devs about this in terms of wpt

AmeliaBR: I think we'll need to group specs together to ask people
... I also think we'll need to separate frustrations from legacy browsers vs current ones

<BitBot> (14wpt) [PR] Honry 04closed 13#18886: [sensor] Upstream sensor layout tests to wpt - https://git.io/fjjly

<BitBot> (14wpt) [PR] Honry reopened 13#18886: [sensor] Upstream sensor layout tests to wpt - https://git.io/fjjly

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/JohnJansen: ok im satisfied/ /
Succeeded: s/greate/great/
Succeeded: s/type untestable/type:untestable/
Succeeded: s/just conversion/not just conversion/
Present: boaz foolip AmeliaBR myles jsbell cbiesinger JohnRiv Rachel Andrew SimonSapin JohnJansen lukebjerring jgraham
Found ScribeNick: boaz
Found ScribeNick: boaz
Found ScribeNick: boaz
Inferring Scribes: boaz

WARNING: No "Topic:" lines found.


WARNING: No meeting title found!
You should specify the meeting title like this:
<dbooth> Meeting: Weekly Baking Club Meeting


WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth


WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: No "Topic: ..." lines found!  
Resulting HTML may have an empty (invalid) <ol>...</ol>.

Explanation: "Topic: ..." lines are used to indicate the start of 
new discussion topics or agenda items, such as:
<dbooth> Topic: Review of Amy's report


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)