Meeting minutes
<jgraham_> RRSAgent: make minutes
<jgraham_> RRSAgent: make logs public
jgraham_: I have a small number of slides about what interop is... "What is interop: Metric measuring pass rate of a curated selection of wpt... not very interesting
jgraham_: why is interop is a more interesting question... we have lots of historic test failures and each browser is failing unique ones in many cases -- it's hard to know which ones to pick up/prioritize
jgraham_: there wasn't a way to really include end users or have discussions among browsers around how to do this prioritization
jgraham_: the point of interop is that it gives us a means of identifying interop priorities, clear metrics, makes it easy to communicate with developers and helps ensure users have a good experience on the web
jgraham_: the next obvious question is "How do you decide priorities"... there's various criteria we can use, and we can talk about what that was like for 2023/2024... broadly though: If we know things are broken for end users, or iff we're getting reports about pain from developers because of lack of interop...
jgraham_: it could be that it has lots of stack overflow questions, or high use counters, etc
jgraham_: 2023 had lots of focus areas (20?) - you can see that we have scores for the major browser engine browsers _and_ we have this interop score which measures the intersection that is passing on all because, again, browsers can fail unique tests
jgraham_: we also have investigation areas, which can be for example, non-standards work that can allow us to achieve interop in the future - this year we looked into better how to test mobile and a11y
jgraham_: so we have some graphs, "up and to the right" - that looks like victory
jgraham_: so we're getting ready to announce interop 2024... If you're thinking "Is {thing I am interested in} a viable candidate"? here are some criteria
jgraham_: it has to have a mature spec, on a standards track... It has to have good automated test coverage (preferably in wpt), and have posititive signals of cross-vendor support... it's not a venue to force browsers to implement something
jgraham_: so if it is widely implemented but has poor interop, if it is an upcomming/partially implemented feature that has very high developer interest
(container queiries/has/aspect ratio are examples)
jgraham_: or features that you can make the case would markedly improve the web for some or all users (for example, a11y)
jgraham_: sept 14 we will send out a call for proposals (tomorrow_)
jgraham_: it will be on github (will ammend the slide to add the url)... The call will be open until october 4... questions? things you want to discuss?
<zcorpan> GitHub link: web-platform-tests/
vmpstr: It's a great yearly cadence effort -- what is your sense of projects that could be bigger than 1 year... could they fit?
jgraham_: at some level anything bigger than a year has to be broken down into a series of smaller project. It seems completely reasonable we could add things along the way that could be parts of interop for multiple years... Did you have a specific example ?
vmpstr: we're working on view transitions -- we're estimating it's probably more than 1 year, and it seems hard to break it down
<Romain> color and colorspaces is already a multi-year focus area
jgraham_: if the estimate is that it is going to take more than a year, I guess the best thing would be to get people to start and then we can add it in the fture
ntim: yeah, we are adding things from color spaces along the way
<ntim> main point is that color spaces can be broken down
rnyman: {? was looking for nick}
zcorpan: We can also sometimes start it and then they can carry over...
jgraham_: there is ongoing discussion about how that will work because we may not finish and what happens, we also have new stuff - how do we decide if we are finished.
jgraham_: Especially if you have things like necessary engine refactors as a prerew
patrickbrosset: If there are more proposals, what's the selection process
jgraham_: the fundamental process is concensus based. All of the participating orgs have to agree we will accept something in interop. In previous years we've just used that directly... This year we're going to try somtehing a lot more specific about how proposals come in and get filtered out more easily.. Asking for supporting evidence and things that filter out. You're not required to provide all of it. Then at that point individual
members can go try to individually organize them by whatever criteria
jgraham_: then we sort of merge those lists and find agreement and then try to decide a cutoff point where we agree we can reasonably make very good process. It's not to come up with a global ordered list, but at least a common list we can agree is high priority
<rbyers> [takes over scribing]
<rbyers> bkardell: We've had lots of conversations on exactly this. Difficult to explain - we have 100 tasks, are they all good tasks? Could say yes. But if I say "you have one year what can you get done" you can't pick 100. Why did you pick A, B,C. instead of D,E,F
<rbyers> It's complicated. Reasonable people can disagree. People trying to do our best. Not a global list, but an intersection of lists we can all agree are good and common and achievable.
<rbyers> I hope that when developers submit things, we're able to somehow communicate that.
<rbyers> Hopefully everyone in the room understands that and helps explain it
<rbyers> I know folks submit something and it doesn't get chosen ....
jgraham_: (shows proposal template ongithub)
<rbyers> [scribing back to Brian]
rbyers: One pattern I see is: obviously not everything can make it in... I worry about how we talk about interop. There are people on the chrome team who might say "we have an interop problem" but it doesn't make itinto interop 2023, so it doesn't matter... You have to be careful how you say that because it's not what we mean to say... Do you agree there are things that are important for interop that wont make itinto interop 2024?
zcorpan_: yes
jgraham_: yes
bkardell_: yes
everyone: yes
jgraham_: Yes we have a years long interop problem -- css zoom, it can't make it into interop just now, but it is hugely impoprtant... or editing APIs... they are super important but they aren't currently in a place we could accept them
jgraham_: there are definitely things that we can't put in here currently -- having the investigation areas are a little bit of a way to include _some_ of those things, but we just can't fit everything. Even just given resource constraints -- there are many reasons, we don't have good tests for it. Maybe we should be more intentoional when we talk about it, because it is certianly true
<lea> is there any way to identify these kinds of "future interop" opportunities a) so that they don't fall off the radar and b) to get consensus and prioritize?
rbyers: one thing I am seeing/fighting against is that people on the chrome team might be getting the idea if it doesn't make it in here it isn't as important. I worry it could cause us to _lessen_ our investment in interop somehow because it chnages our conception
jgraham_: I think at the moment there isn't much communication - the standards bodies focus on the idea of what they are interested in but there isnt a more whole web platform narrative and thinking on priorities
jgraham_: it could be interesting to see how we take some of htose things like css zoom and could articulate something helpful
rnyman: The first year it was much smaller number of vendors, and since we have grown it in terms of collecting and giving thought to the priortization -- we now have more information and an easy grab bag we can use to go beyond this
jgraham_: to lea's question... like css zoom you shouldn't submit that because it doesn't meet the basis of the criteria. Last year we didn't articulate that well enough perhaps and we had too many proposals and we spent too much of people's time on things we shouldn't have been considering. Last year there were > 80 proposals
jgraham_: and you have to take them to your engineering teams and that takes a lot of time that then they're not doing the actual engineering work... So we are trying to focus on making this more efficient, so it's very interesating that something like css zoom would be immediately filtered out, but it is interesting somehow...
lea: I guess the question is it doesn't meet the baseline criteria today, but will it tomorrow? What if there is a way to post something not for today but for future consideration... I'm not really submitting it for this year's interop
jgraham_: that could be itneresting, a kind of holding area... the obvious worry is it becomes just another bug tracker that hasn't been triaged/prioritized... but between what you and rbyers said there is I think something there... It would be good to have something
lea: you mentioned these areas are already known - is there an FAQ somewhere or something?
jgraham_: no, it mostly exists in people's heads - it's a big list of bugs people know exist - places where we've heard about pains, etc.
andreubotella: how does the list of investigation focus areas differ?
jgraham_: standards work needs to happen in standards bodies - the interop project isn't that. CSS Zoom has work that needs to happen in the CSS working group, because it's not just implementation, it's spec issues. It would be maybe problematic to demand the CSS WG work at a certain pace
andreubotella: but the interop project has some importance or influence, could it help press the css working group
<gsnedders_web> Also some concern about the work happening around the Interop project instead of in the WG where there's an IPR policy.
ntim: I dont think we want to fall into the trap of making interop a standards body.. We had a bunch of investigation areas about SVG, for example that belong to an SVG working group
ntim: we also have things to improve a11y testing or mobile testing
jgraham_: yes, I think the thing to do with investigations is that it's for things that aren't immediately actionable, but aren't standards work... but it is similar people involved and so I assume there is influece through simple osmosis
jgraham_: maybe there is a way of documenting these things better... I dont think theinvestigation areas are that
bkardell_: agree
jgraham_: The simple fact that we have this project does sort of raise some awareness of these
zcorpan_: I think there's maybe a misperception with the interop project. The features that are in aren't ncessarily the most important features... They are the issues that aren't interoperable and needs a push.. Features become interoperable all the time just because we do a good job. Features in JS do a fair job of just being interoperable, for example.
zcorpan_: So this project really came about because of places where we failed to do that. If your feature isn't in theinterop project that could be a success
<fscholz> What happens after a feature was an interop focus area? I assume we send a strong signal to web devs that the thing is now usuable/stable? Do we know if features see larger adoption thanks to interop? Is there less frustration with certain features?
jgraham_: there are also some kind of different critera. Like container queries went in as soon as we had agreement becauase we could see that it is univerally desired/important - and we want to make sure that it launches closer and well coordinated/highly interoperable from day one and not a ragged, frustrating several years and then a real interop problem for some time
jgraham_: subgrid is a bit like that... subgrid was in firefox for a long time, but now it's coming along in other browsers and we can see firefox has some interoperability bugs
jgraham_: to fscholz that's a good question. So far there haven't been a lot of things ... in 2022 I think there were 15 focus areas and like 5 we didn't carry over... it wasn't big stuff, sticky positioning I think. WE haven't tried to do the analysis "did this have an impact"... it's hard to have a really good control for that kind of experiment - you'd want a feature which wasn't in interop that year... I'm not sure how you would
design that experiment. We know there were bad cases in history where we don't want to repeat
jgraham_: there will be big features that maybe could provide stronger evidence for pre/post interop
<Romain> Getting this feedback could be part of the state of CSS survey? Likely this data is already gathered.
<jgraham_> RRSAgent: make minutes