14:41:19 <RRSAgent> RRSAgent has joined #webmachinelearning
14:41:19 <RRSAgent> logging to https://www.w3.org/2021/12/16-webmachinelearning-irc
14:41:20 <Zakim> RRSAgent, make logs Public
14:41:22 <Zakim> please title this meeting ("meeting: ..."), anssik
14:41:27 <anssik> Meeting: WebML WG Teleconference – 16 Dec 2021
14:41:32 <anssik> Chair: Anssi
14:41:39 <anssik> Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2021-12-16-agenda.md
14:41:44 <anssik> Scribe: Anssi
14:41:49 <anssik> scribeNick: anssik
15:00:21 <anssik> Present+ Anssi_Kostiainen
15:00:34 <anssik> Present+ Dominique_Hazael-Massieux
15:00:50 <anssik> Present+ James_Fletcher
15:00:57 <anssik> Present+ Ganesan_Ramalingam
15:02:13 <anssik> Present+ Rafael_Cintron
15:02:32 <anssik> Present+ Geunhyung_Kim
15:02:34 <Rama> Rama has joined #webmachinelearning
15:03:58 <Geun-Hyung> Geun-Hyung has joined #webmachinelearning
15:04:46 <Geun-Hyung> present+
15:04:49 <anssik> RRSAgent, draft minutes
15:04:49 <RRSAgent> I have made the request to generate https://www.w3.org/2021/12/16-webmachinelearning-minutes.html anssik
15:04:55 <anssik> Topic: Announcements
15:05:01 <RafaelCintron> RafaelCintron has joined #webmachinelearning
15:05:06 <anssik> anssik: welcome everyone to the last WebML WG teleconference of 2021
15:05:10 <dom> scribe+
15:05:20 <anssik> ... we have a special guest today and a packed agenda as usual
15:05:20 <dom> anssik: we're welcoming James to talk to us about ethics in a minute
15:05:27 <anssik> anssik: we've created a "how to" resource to folks who want to help with scribing, linked from the agendas from now on
15:05:32 <anssik> -> https://github.com/webmachinelearning/meetings/blob/main/scribe-howto.md Scribe howto
15:05:45 <anssik> -> https://groups.google.com/a/chromium.org/g/blink-dev/c/w7Oq2_x2D8U/m/8BhYLGCIBwAJ Intent to Prototype in Chromium: Model Loader API
15:06:34 <anssik> -> https://doodle.com/poll/arstwzhv6ew6t36p WebML CG telcon for Model Loader API poll results
15:06:37 <dom> anssik: we're planning to run Sydney-friendly meetings on the model loader API
15:07:01 <dom> ... plan to kickoff on Jan 12 at 5am UTC
15:08:13 <dom> Geun-Hyung: will check my availabilities
15:09:36 <anssik> Present+ Jonathan_Bingham
15:09:43 <Jonathan> Jonathan has joined #webmachinelearning
15:10:47 <dom> Present+ Rachel_Yager
15:11:09 <anssik> Present+ Rachel_Yager
15:11:28 <anssik> Topic: TPAC meeting follow-up (cont'd)
15:11:55 <dom> Subtopic: Ethical issues in using Machine Learning on the Web
15:12:08 <dom> anssik: one of the topics we discussed briefly at TPAC was ethics
15:12:22 <dom> ... it got also highlighted in the intent to prototype discussion in Chromium
15:12:27 <dom> ... work has started on this
15:12:41 <dom> ... I'm happy to introduce James Fletcher today who is interested in helping us
15:12:47 <dom> ... Dom and I met a couple of times with him
15:13:08 <dom> ... and James is bringing his ethics experience in this space
15:13:20 <dom> Slideset: https://lists.w3.org/Archives/Public/www-archive/2021Dec/att-0004/1216_W3C_Ethical_WebML_Intro.pdf
15:13:30 <dom> [slide 1]
15:13:43 <dom> James: happy to be getting involved in this project
15:13:53 <dom> ... want to give a quick intro on myself and the BBC work in this space
15:14:02 <dom> ... and ideas on how to move this ethics work forard
15:14:07 <dom> s/ard/ward/
15:14:18 <dom> [slide 3]
15:14:40 <dom> ... the challenge I'm getting involved in is the development of the ethical considerations around ML and potential mitigations
15:14:40 <anssik> James: The challenge "What are the ethical considerations that should inform the development of the Web Neural Network API, and what mitigations should be taken into account?"
15:14:42 <dom> [slide 4]
15:15:01 <dom> James: I lead the work in the BBC in how we use ML as a user-facing organization
15:15:45 <dom> ... I'm not a technical person, but hopefully I can bring a different perspective
15:15:53 <dom> [slide 6]
15:16:13 <dom> James: BBC plays a role in standard as part of its R&D of work, as our charter recognizes
15:16:29 <dom> ... we work in ~40 standards groups, including W3C, esp in the audio/video/accessibility areas
15:16:42 <dom> ... It's exciting that we're getting involved in the ethical side of things
15:16:47 <dom> [slide 7]
15:17:27 <dom> James: wrt AI/ML, our approach is anchored to the response we gave the house of lords committee in 2017
15:17:48 <dom> ... my role is anchored in our 3rd commitment to responsible technical development
15:17:55 <dom> ... #2 is also relevant to getting involved here
15:18:09 <dom> [slide 10]
15:18:30 <dom> James: our work in this space is anchored in MLEP, Machine Learning Engineering Principles
15:18:36 <dom> s/Engineering/Engine/
15:18:46 <dom> [slide 11]
15:19:12 <dom> James: trust is a key value for BBC, which is reflected in our approach in this space
15:19:17 <dom> [slide 12]
15:20:35 <anssik> [slide 13]
15:20:40 <anssik> James: The Principles
15:20:57 <anssik> ... Reflecting the BBC’s Values
15:20:57 <anssik> ... Improving Audience Experience
15:20:57 <anssik> ... Clear Explanations
15:22:02 <anssik> ... Editorial Values & Broadening Horizons
15:22:02 <anssik> ... Taking Responsibility: Review, Security & Fairness
15:22:02 <anssik> ... Human in the Loop
15:22:52 <anssik> [slide 14]
15:23:01 <anssik> James: MLEP Checklist 2.0 Overview
15:23:08 <anssik> [A diagram depicting the checklist]
15:23:47 <anssik> [slide 15]
15:23:52 <anssik> James: Scoping your ML project
15:24:01 <anssik> ... Intended ML applications
15:24:01 <anssik> ... Impact
15:24:01 <anssik> ... Risks, opportunities and consequences
15:24:06 <anssik> [slide 16]
15:24:17 <anssik> James: Consequence Scanning Toolkit
15:24:32 <anssik> ... to think through ethical considerations
15:24:37 <anssik> [slide 17]
15:24:42 <anssik> James: AI & ML at the BBC
15:24:48 <anssik> ... two dimensions:
15:24:55 <anssik> ... internal-facing <-> audience-facing
15:24:55 <anssik> ... content data <-> audience data
15:25:55 <anssik> [slide 18]
15:26:00 <anssik> James: How might we apply an ethical approach to the web neural network API?
15:26:06 <anssik> [slide 19]
15:26:10 <anssik> James: Key considerations
15:26:14 <anssik> ... Build on existing W3C ethics work
15:26:18 <anssik> -> https://webmachinelearning.github.io/ethical-webmachinelearning/ Ethical Web Machine Learning
15:26:52 <anssik> ... Involve a wide range of stakeholders
15:26:52 <anssik> ... Use internal + external expertise
15:26:52 <anssik> ... Build towards broader web machine learning principles
15:27:17 <anssik> [slide 21]
15:27:30 <anssik> James: Discover > Define > Develop > Deliver
15:27:59 <anssik> [slide 22]
15:28:05 <anssik> James: How might we get there?
15:29:19 <anssik> ... Discover: Internal + external literature review, Consult experts
15:29:19 <anssik> ... Define: Guiding principles
15:29:19 <anssik> ... Develop: Workshops: Issues & Mitigations
15:29:19 <anssik> ... Deliver: Ethics Recommendations
15:29:48 <anssik> [slide 23]
15:29:52 <anssik> James: Questions from me ...
15:29:59 <anssik> ... Thoughts on the proposed process
15:29:59 <anssik> ... Timeline considerations
15:29:59 <anssik> ... Recommendations for key stakeholders, internal and external experts
15:30:27 <anssik> anssik: thanks for the great presentation!
15:32:14 <anssik> anssik: As for timeline
15:32:18 <anssik> ... we're aiming for a spec milestone "Candidate Recommendation" in 1H 2022, an ambitious goal but an important one
15:32:22 <anssik> ... would love to have a fleshed out document on Ethical Web Machine Learning to link to by that time.
15:32:57 <anssik> James: would the ethics recommendations include mitigations that are prototyped?
15:36:44 <anssik> Rachel: thanks for the great presentation and Anssi for connecting all the people
15:37:34 <anssik> ... about the framework, have thoughts of organizing an event where we can conduct policy research across different companies
15:38:52 <anssik> ... does policy research across companies make sense to you?
15:39:39 <anssik> ... companies outside of W3C also
15:40:01 <anssik> ... what is achievable from my end, within W3C community and outside at the corporation level
15:40:23 <anssik> ... including government, community and corporations
15:40:50 <anssik> ... policy research initiative to see how this is reflected at each corporate level
15:41:02 <anssik> James: AI ethics policies companies, govs have?
15:41:11 <anssik> ... that'd be part of literature review
15:41:31 <anssik> ... gov bodies and pan-gov entities have, this is meta research that looks into similarities
15:41:43 <anssik> ... similar themes are surfacing from many places
15:43:38 <anssik> Rachel: I'm proposing us to facilitate research across companies, within W3C community and outside, looking at existing policies
15:44:37 <anssik> James: literature review is meant to provide view into that
15:48:58 <anssik> Guen-Hyung: what is the approach in context of web technologies?
15:50:57 <anssik> ... the scope of ethics, includes bias
15:51:39 <anssik> Rachel: excellent question between policy and implementation
15:51:50 <anssik> ... policy driven implementation and the other way around
15:52:08 <anssik> ... we hear news people are surprised how companies use AI today
15:52:25 <anssik> ... everyone see ethics differently, but there are some key themes that surface
15:52:34 <anssik> ... there are gray areas
15:53:07 <anssik> James: two thoughts, in terms interpretations of ethics, absolutely!
15:53:20 <anssik> ... getting people together with diverse perspectives is the way to go
15:53:31 <anssik> ... we have UNESCO working on global level
15:53:55 <anssik> ... principles such as autonomy, allows the users to make their own decisions
15:54:19 <anssik> ... a lot of ethical principles are like that, privacy, security safeguarding individuals
15:55:46 <anssik> Topic: Path to Candidate Recommendation
15:56:01 <anssik> anssik: we are chartered to push toward Candidate Recommendation specification maturity in 2022.
15:56:27 <anssik> ... this is an important spec, essentially says "the spec is feature complete and can be implemented as is"
15:56:59 <anssik> ... initially, this was a spec publication that triggered implementation work, in practice today implementation work start before CR
15:57:29 <anssik> ... to increase transparency and allow us to move toward this CR milestone in a coordinated fashion, I created to meta issues that act as trackers.
15:57:34 <anssik> ... the first one, Candidate Recommendation readiness tracker, turn process requirements into checkboxes we want to tick:
15:57:39 <anssik> -> https://github.com/webmachinelearning/webnn/issues/240 Candidate Recommendation readiness tracker #240
15:57:58 <anssik> ... key requirements demystified:
15:58:08 <anssik> ... 1) "must show that the specification has met all Working Group requirements,"
15:58:18 <anssik> ... spec addresses the requirements defined at chartering time and satisfies its use cases
15:58:32 <anssik> ... the W3C process does not micromanage the requirements or use cases, up to the WG to be responsible
15:58:49 <anssik> ... 2) "must document how adequate implementation experience will be demonstrated"
15:59:09 <anssik> ... essentially tests and early implementations, with a caveat implementations do not need to exist to get to CR
15:59:15 <anssik> ... also no requirement to get explicit support signals from all implementers
15:59:31 <anssik> ... 3) "must show that the specification has received wide review"
15:59:42 <anssik> ... this is an important W3C process step we must complete, we're tracking this separately:
15:59:47 <anssik> -> https://github.com/webmachinelearning/webnn/issues/239 Wide review tracker #239
16:01:18 <rach> rach has joined #webmachinelearning
16:02:53 <rach_> rach_ has joined #webmachinelearning
16:03:05 <anssik> Jonathan: I these are two complementary approaches, good to get feedback from web developers for both
16:03:17 <anssik> ... WebNN is more mature at this point, fine to be further along and it makes sense
16:03:31 <anssik> ... we need to allow time for wider feedback and OTs for both of them
16:05:10 <anssik> Topic: Happy Holidays!
16:05:34 <anssik> anssik: We plan to resume WG telcons 13 Jan 2022, check the invite for details
16:05:44 <anssik> ... and kick off CG telcons on 12 January
16:05:50 <anssik> ... please enjoy the upcoming Holiday season!
16:06:02 <anssik> ... 2022 looks exciting for this WG with many important milestones.
16:06:29 <anssik> Rachel: thanks for your leadership Anssi and James for an amazing presentation!
16:06:58 <anssik> RRSAgent, draft minutes
16:06:58 <RRSAgent> I have made the request to generate https://www.w3.org/2021/12/16-webmachinelearning-minutes.html anssik
16:07:55 <anssik> s/important spec/important spec milestone
16:07:57 <anssik> RRSAgent, draft minutes
16:07:57 <RRSAgent> I have made the request to generate https://www.w3.org/2021/12/16-webmachinelearning-minutes.html anssik
16:08:42 <anssik> s/to meta issues/two meta issues
16:11:06 <anssik> s/I these are two/I see the Model Loader API incubation and WebNN API on the Rec Track as
16:11:10 <anssik> RRSAgent, draft minutes
16:11:10 <RRSAgent> I have made the request to generate https://www.w3.org/2021/12/16-webmachinelearning-minutes.html anssik
17:37:04 <Zakim> Zakim has left #webmachinelearning