IRC log of beyond-interop on 2023-09-13

Timestamps are in UTC.

06:44:18 [RRSAgent]
RRSAgent has joined #beyond-interop
06:44:22 [RRSAgent]
logging to https://www.w3.org/2023/09/13-beyond-interop-irc
06:44:22 [Ian]
RRSAgent, do not leave
06:44:24 [Ian]
RRSAgent, make logs public
06:44:26 [Ian]
Meeting: Beyond Interop: Baseline and more
06:44:28 [Ian]
Chair: Kadir Topal
06:44:30 [Ian]
Agenda: https://github.com/w3c/tpac2023-breakouts/issues/60
06:44:32 [Ian]
clear agenda
06:44:34 [Ian]
agenda+ Pick a scribe
06:44:36 [Ian]
agenda+ Reminders: code of conduct, health policies, recorded session policy
06:44:38 [Ian]
agenda+ Goal of this session
06:44:40 [Ian]
agenda+ Discussion
06:44:42 [Ian]
agenda+ Next steps / where discussion continues
06:44:46 [Ian]
Ian has left #beyond-interop
07:13:03 [Ian]
Ian has joined #beyond-interop
07:13:10 [Ian]
Ian has left #beyond-interop
08:05:35 [kadirtopal]
kadirtopal has joined #beyond-interop
08:33:13 [fscholz]
fscholz has joined #beyond-interop
08:52:31 [dom__]
dom__ has joined #beyond-interop
08:57:38 [cathiechen]
cathiechen has joined #beyond-interop
09:01:00 [romain]
romain has joined #beyond-interop
09:01:00 [dom__]
Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Sep/att-0011/Beyond_Interop_Baseline_and_more.pdf
09:01:17 [gsnedders_web]
gsnedders_web has joined #beyond-interop
09:01:18 [dom__]
Present+
09:01:31 [tidoust]
tidoust has joined #beyond-interop
09:02:35 [tidoust]
present+
09:02:35 [tidoust]
scribe+ tidoust
09:06:09 [bkardell_]
bkardell_ has joined #beyond-interop
09:06:16 [jgraham_]
jgraham_ has joined #beyond-interop
09:06:59 [jgraham_]
Meeting: Beyond Interop
09:06:59 [jgraham_]
Chair Kadir Topal
09:06:59 [jgraham_]
Chair: Kadir Topal
09:06:59 [jgraham_]
RRSAgent: make logs public
09:07:30 [jgraham_]
RRSAgent: make minutes
09:07:31 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/09/13-beyond-interop-minutes.html jgraham_
09:07:38 [fscholz]
fscholz has joined #beyond-interop
09:09:49 [bkardell_]
present+
09:09:49 [jgraham_]
present+
09:09:49 [dom__]
[slide 2]
09:09:49 [tidoust]
kadirtopal: First question is "Why baseline"?
09:09:49 [dom__]
[slide 3]
09:09:49 [tidoust]
... We recently shared one of our internal developers survey.
09:09:49 [tidoust]
... We asked devs about top pain points
09:09:49 [tidoust]
... We've been runnning these for years now and top pain point is keeping up with changes
09:09:49 [tidoust]
... new APIs, practices, etc.
09:09:58 [dom__]
-> https://github.com/web-platform-dx/developer-research/blob/9c39c2b1927c9bb17b3cac932f1ea78ec3e41d14/vendor-research/Web%20Dev%20Sat%20report%20to%20be%20shared%20with%20WebDX%20CG.pdf Research results from which this slide is quoted
09:09:58 [tidoust]
... Even if you think about subgrid, landed in Firefox in 2016 I think, then you have a gap for 5 years to get another implementation, then another year for a third.
09:09:58 [tidoust]
... As a dev, how you can keep up with that?
09:09:58 [tidoust]
... New browser versions are coming up every few weeks, changing what the web platform is
09:09:58 [dom__]
[slide 4]
09:11:28 [tidoust]
kadirtopal: At some point, after a number of years developing, you need to be able to tell developers that the feature is here!
09:11:28 [dom__]
[slide 5]
09:11:28 [tidoust]
... But there's no feedback loop.
09:11:28 [tidoust]
kadirtopal: That's where Interop kicked in
09:11:39 [tidoust]
... Interop is a huge step forward.
09:11:46 [jgraham_]
RRSAgent: make minutes
09:11:47 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/09/13-beyond-interop-minutes.html jgraham_
09:11:53 [tidoust]
... But there's a step before, and a step after.
09:12:07 [tidoust]
... It still makes sense to look at interop, but also before and after.
09:12:17 [dom__]
[slide 7]
09:13:00 [dom__]
[slide 8]
09:13:06 [tidoust]
kadirtopal: We got together with like-minded people and the WebDX CG was created about a year ago to look at the research, which will be discussed this afternoon in another breakout, and on what comes after Interop, which is the topic of this session.
09:13:31 [tidoust]
kadirtopal: We want to make it easier for devs to track the list of features that are widely available.
09:13:37 [romain_]
romain_ has joined #beyond-interop
09:13:37 [dom__]
[slide 9]
09:13:46 [dom__]
[slide 10]
09:13:49 [tidoust]
... One aspect is naming feature and grouping features.
09:13:59 [tidoust]
kadirtopal: But before we get to that, there's baseline.
09:14:13 [tidoust]
... When a feature is widely available.
09:14:27 [dom__]
[slide 11]
09:14:40 [tidoust]
... Initial definition was "in 2 last major versions of Chrome, Firefox, Safari, Edge".
09:14:50 [dom__]
[slide 12]
09:15:05 [tidoust]
kadirtopal: And then we can set some baseline badge, displayed on top of MDN pages.
09:15:09 [dom__]
[slide 13]
09:15:23 [tidoust]
... We've been iterating on the definition of baseline in the past few months.
09:15:43 [tidoust]
kadirtopal: There is a point at which the feature is supported across browsers, and then there is uptake.
09:15:59 [tidoust]
... There is a power curve: adoption, then diminution of increase.
09:16:06 [tidoust]
... You never reach 100% of support for any feature.
09:16:23 [past]
past has joined #beyond-interop
09:16:59 [tidoust]
... As long as IE was to be supported, we were stuck. Since then, all browsers are evergreen. However, there are still issues with that path.
09:17:16 [tidoust]
... The vast majority of people don't even have to worry about that, they run the latest version.
09:17:44 [tidoust]
... But others may run on specific platforms that no longer are updated. Or running on a platform that is updated every 6 months, or that is managed.
09:18:02 [tidoust]
... Lots of reasons that explain why the curve is not an immediate massive adoption.
09:18:11 [tidoust]
... Most adoption is within the first 6 months.
09:18:35 [dom__]
[have we confirmed the rough shape of the uptake curve, out of curiosity?]
09:18:44 [tidoust]
... But still some significant, say ~20% of users, that will need more time to get the features.
09:19:30 [tidoust]
... Some of that is on smartphones. But also on TVs where the uptake cycle is vastly different and can take 10 years.
09:19:37 [tidoust]
... That explains why the curve looks like this.
09:20:11 [tidoust]
... What this brings us to is: there is one point, and then there is an interval. When can you say that a feature is widely available?
09:20:43 [tidoust]
... Very diminishing returns some time between 6 months and 2 years. 3 years seems a max.
09:20:54 [tidoust]
... That's what we're discussing.
09:21:32 [tidoust]
kadirtopal: All of this requires being able to talk about features and feature groups.
09:21:42 [tidoust]
Topic: Web features
09:22:05 [miriam]
miriam has joined #beyond-interop
09:22:09 [miriam]
present+
09:22:16 [tidoust]
daniel: When we talk about features, what are we talking about?
09:22:40 [zcorpan]
zcorpan has joined #beyond-interop
09:22:47 [zcorpan]
present+
09:22:50 [past]
present+
09:23:04 [tidoust]
... Most of discussions on features from an implementation perspective are specification specific, for internal purpose. Devs have a different perspective.
09:23:11 [whimboo_]
whimboo_ has joined #beyond-interop
09:23:12 [dom__]
q+ to ask how confident we are on the shape of the uptake curve (illustrated on slide 13)
09:23:32 [tidoust]
... The web-features project is an NPM package that tries to identify features.
09:23:44 [tidoust]
... We try to capture the things that we hope make sense for developers.
09:23:52 [dom__]
-> https://github.com/web-platform-dx/web-features web-features repo
09:24:06 [tidoust]
... We give features a short identifier, link to specifications, compat data, Can I Use entries
09:24:11 [dom__]
-> https://www.w3.org/2023/09/TPAC/demos/web-features.html Daniel's recorded presentation on web-features
09:24:26 [whimboo_]
Hi, I just joined and wonder if the room is muted?
09:24:41 [tidoust]
-> https://www.w3.org/2023/09/TPAC/demos/web-features.html Video presentation on web-features
09:25:31 [tidoust]
daniel: Flags about implementation, baseline, deprecation status, etc.
09:26:05 [tidoust]
... This may be helpful as a tool for more accurate documentation of features and compat data, e.g., on MDN.
09:26:18 [tidoust]
... This is going to be an important underlying thing to the overall project.
09:26:19 [rbyers]
rbyers has joined #beyond-interop
09:26:29 [tidoust]
q?
09:26:29 [rbyers]
rbyers has left #beyond-interop
09:26:40 [rbyers]
rbyers has joined #beyond-interop
09:26:49 [rbyers]
present+
09:26:54 [ddbeck]
ddbeck has joined #beyond-interop
09:26:56 [tidoust]
Topic: Open discussion
09:27:05 [zcorpan]
q?
09:27:08 [jgraham_]
ack dom__
09:27:08 [Zakim]
dom__, you wanted to ask how confident we are on the shape of the uptake curve (illustrated on slide 13)
09:27:31 [tidoust]
dom: How confident we are that the adoption curve has that shape?
09:27:32 [jgraham_]
q+
09:27:42 [tidoust]
kadirtopal: We spent quite a bit of time studying that, yes.
09:28:09 [tidoust]
... One problem is tracking usage metrics. Quite tricky to get access to valid and representative public usage data.
09:28:23 [tidoust]
... Can I Use agglomerates usage data from StatCounter.
09:28:51 [tidoust]
... But all versions and regions are not represented in that usage data.
09:29:09 [tidoust]
... Anyway, we've looked at all data available, and it always looks like a power curve.
09:29:48 [tidoust]
... Depending on when the interop moment happens, you may start at 80% already or lower than that. That depends on which browser is the last one to ship the feature.
09:30:07 [tidoust]
daniel: One thing that is interesting is that this curve has gone steeper over time.
09:30:32 [bkardell_]
q+
09:30:41 [tidoust]
... A 2015 feature took longer to be widely available than a 2020 feature.
09:31:00 [jgraham_]
ack jgraham_
09:32:02 [tidoust]
jgraham_: About the shape of the curve, there's a bit of approximation. At the start, it goes up over the initial weeks. But then you might see bugs in the released feature.
09:32:39 [patrickbrosset]
patrickbrosset has joined #beyond-interop
09:32:44 [tidoust]
... Or at some point the browser drops support for a particular old platform, and people will stop using some older version of a browser. Or people using Firefox ESR.
09:32:52 [tidoust]
... That will create bumps in the curve.
09:33:11 [tidoust]
... If you look at stats, a few percents still on Windows 7 for instance.
09:33:43 [tidoust]
... Browser vendors drop support for an OS correlated with how much users have upgraded.
09:33:57 [ddbeck]
Dan Fabulich used caniuse features to analyze the curve, with the summary table here: https://github.com/dfabulich/baseline-calculator#cohort-analysis. Individual features have their own individual curves, of course.
09:34:25 [tidoust]
... In an ideal world, we would just mention versions and the marketshare. But the usage data is indeed not super reliable.
09:34:47 [bkardell_]
ack bkardell_
09:34:58 [tidoust]
... No way to track the different steps which in the end create the curve.
09:35:13 [tidoust]
bkardell_: The notion of having a power curve seems very natural.
09:35:41 [tidoust]
... given what we discussed. You expect to get a big chunk of users, then additional updates over time.
09:36:08 [tidoust]
... Does that happen incrementally? I don't really know. This may change over time.
09:37:13 [tidoust]
... We've talked about not dropping X% of users. I agree with the goal.
09:37:29 [tidoust]
... But we're not counting for instance Samsung Internet, which should account for some users.
09:37:45 [tidoust]
jgraham_: I don't think that it's true when we look at usage data
09:38:04 [jgraham_]
q+
09:38:09 [tidoust]
bkardell_: Right, but the definition does not include Samsung Internet. Significant browser in the UK for example.
09:38:47 [tidoust]
... Anything that we do is ultimately going to have simple rules. It's a delicate balance to be found.
09:38:48 [dom__]
[part of my question on the shape of the curve is to understand if we can measure our impact in making it more vertical]
09:39:33 [tidoust]
... I don't have an answer but need to carefully consider all perspectives.
09:39:38 [romain_]
q+ Can we embrace the fact that it's going to be wrong and focus on finding a process to iterate?
09:40:02 [jgraham_]
ack jgraham_
09:40:06 [tidoust]
kadirtopal: Back to the top pain point raised by developers. No definition is going to be perfect indeed.
09:40:17 [romain]
romain has joined #beyond-interop
09:41:00 [tidoust]
jgraham_: About ignoring Samsung Internet, that's true the way baseline has been defined so far. Any definition that mentions marketshare would not ignore Samsung Internet.
09:41:16 [tidoust]
... If we're using a proxy for marketshare, Samsung Internet is still being indirectly included.
09:41:33 [bkardell_]
q+ to clarify something
09:41:57 [tidoust]
... If we have other criteria: all versions of browsers that still receive critical updates for instance, then we should be able to make sure that we include needed browsers and users.
09:42:34 [bkardell_]
ack bkardell_
09:42:34 [Zakim]
bkardell_, you wanted to clarify something
09:42:34 [tidoust]
... I think you can work with data even if the vendor is not in the room.
09:43:09 [tidoust]
bkardell_: I didn't mean to suggest that we're intentionnally excluding anyone. We're relying on StatCounter, and there may be much better data elsewhere.
09:43:15 [tidoust]
... We're doing with the data that we have.
09:43:29 [dom__]
q+ to ask about the duration of ~FPWD to baseline
09:43:43 [tidoust]
... Developers do not have this level of conversation. They may look at boxes, check they're green and be done.
09:44:25 [tidoust]
kadirtopal: Goal is to offer baseline as a tool to developers and gather feedback on whether it's useful. Then iterate.
09:44:39 [tidoust]
... I think that we can adjust over time, act on the feedback.
09:44:40 [romain]
q+ how would an iterative process account for biases?
09:44:57 [dom__]
q+ romain to ask how would an iterative process account for biases?
09:45:06 [miriam]
q+
09:45:08 [tidoust]
... The other part of it, raised by Patrick, is to provide feedback on features that are not baseline.
09:45:31 [tidoust]
... A set of features that is not yet interoperable, is there opportunity to provide feedback to developers?
09:45:57 [tidoust]
... Also, does the feature actually solve the problem it was developed for? Is there opportunity to gather that feedback?
09:46:42 [tidoust]
patrickbrosset: The baseline status comes with a little bit of a risk of hiding what's not baseline out there. I would hate for a world where everyone just considers baseline features and ignore the rest, leaving it for us to play with these features.
09:47:35 [bkardell_]
q+
09:47:39 [tidoust]
... An opportunity for us to track complaints to these things.
09:48:34 [jgraham_]
ack dom__
09:48:34 [Zakim]
dom__, you wanted to ask about the duration of ~FPWD to baseline
09:48:34 [tidoust]
... Maybe not something that the WebDX CG should work on, could be done by Can I Use, MDN, etc.
09:48:48 [tidoust]
dom: One metric that would be interesting to look at is maybe the time from certain phases of standardization to baseline status.
09:49:11 [jgraham_]
q+ to talk about time to baseline
09:49:13 [tidoust]
... Also could be used to evaluate the impact of the discussions.
09:50:23 [tidoust]
... Time to interop and time to baseline. Giving that kind of feedback to the working groups so that they get a better sense of durations.
09:50:35 [tidoust]
kadirtopal: I don't think that's been raised before, fantastic idea
09:50:38 [kadirtopal]
Zakim, ack romain
09:50:38 [Zakim]
romain, you wanted to ask how would an iterative process account for biases?
09:50:38 [jgraham_]
ack romain
09:50:40 [Zakim]
I see miriam, bkardell_, jgraham_ on the speaker queue
09:51:29 [tidoust]
romain: One thing that worries me is introducing bias with feedback. Those who have less access to Internet will be likely to give feedback.
09:51:54 [ddbeck]
q+
09:52:00 [tidoust]
s/likely/less likely
09:52:50 [tidoust]
... Iterating is interesting. First definition was wrong, we're somehow over-compensating the thing. Surveying developers about the definition without introducing bias is hard.
09:52:58 [jgraham_]
ack miriam
09:53:54 [tidoust]
miriam: When talking to developers about new features, I spend time convincing them that all green is not baseline. Big difference between container queries and cascade layers as well.
09:54:10 [tidoust]
... One can be added on top of it, the other would break your content.
09:54:17 [tidoust]
... Not conveyed in the definition of baseline.
09:54:35 [tidoust]
kadirtopal: Very important point.
09:54:54 [tidoust]
... At some point, we had mainline on top of baseline.
09:55:09 [tidoust]
... The more you simplify, the more you ignore things that do have an impact.
09:55:21 [tidoust]
... Progressive enhancement is maybe still worth folding into this.
09:56:07 [dom__]
RRSAgent, draft minutes
09:56:09 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/09/13-beyond-interop-minutes.html dom__
09:57:21 [tidoust]
rachel: [missed]
09:57:29 [dom__]
q?
09:57:50 [jgraham_]
ack bkardell_
09:57:53 [tidoust]
ack bkardell_
09:58:26 [kadirtopal]
rachel: defining a line in the sand for "widely supported" means that people don't have to worry about progressive enhancements for things before that line
09:58:43 [tidoust]
bkardell_: The current definition that we're working on, is 6 months to 3 years after interop. Most developers won't be caught by surprise on something that shipped that could be qualified as new.
09:58:56 [kadirtopal]
rachel: and can focus on making decisions about progressive enhancements for things after that
09:58:58 [jgraham_]
Zakim, close the queue
09:58:58 [Zakim]
ok, jgraham_, the speaker queue is closed
09:59:38 [tidoust]
... Devs could use "why isn't that in baseline?" to report on lack on interoperability. But we may be getting feedback on things that got fixes some time ago, just not with baseline status yet.
10:00:29 [jgraham_]
ack jgraham_
10:00:29 [Zakim]
jgraham_, you wanted to talk about time to baseline
10:01:13 [tidoust]
kadirtopal: If we do this with 3 stages, not widely supported, interop, baseline, the message could be specialized to the different stages.
10:01:55 [tidoust]
jgraham_: About the time baseline, there's always a danger. Browsers changing the frequency of releases. Also some versions that were shipped long long time ago that could be still be worth supported.
10:02:24 [tidoust]
... I'm always worried about making it a metric. How do you make sure that you're promoting the right thing.
10:03:22 [tidoust]
... If you can polyfill something with 0 performance impact, why would we standardize it? That will often be considered by users as broken if we tell them to use a polyfill.
10:03:33 [tidoust]
... There's a balance to be found.
10:03:36 [tidoust]
ack ddbeck
10:04:10 [tidoust]
daniel: General response. We will have to get better than we are now. We have real limitations that we cannot change.
10:04:52 [tidoust]
... No matter the definition, we'll have to teach devs that it is an answer to some questions, but not all of them.
10:05:01 [tidoust]
... We also need to communicate with them about what's missing.
10:05:11 [jgraham_]
s/Also some versions/Also some OS or hardware versions/
10:05:17 [dom__]
[re accessibility support, a breakout on the topic in the next session: https://github.com/w3c/tpac2023-breakouts/issues/53 ]
10:05:28 [tidoust]
kadirtopal: WebDX CG meets every other week, you're all most welcome to join!
10:05:32 [tidoust]
RRSAgent, draft minutes
10:05:33 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/09/13-beyond-interop-minutes.html tidoust
10:10:38 [bkardell_]
bkardell_ has left #beyond-interop
10:13:20 [jgraham_]
10:13:37 [dom__]
dom__ has left #beyond-interop
12:05:13 [tidoust]
tidoust has joined #beyond-interop