Meeting minutes
<wseltzer> https://
<GarrettJohnson> Garrett+
<blassey> although regrets for the second half of the call
[Wendy asks people to "present+" if on irc channel]
Wendy: Let's start with a look through our agenda
… any other business?
Agenda-curation, introductions
Wendy: any introductions; anyone new to the call who would like to introduce themselves to the group?
… feel free to unmute
Claire Lukasiewicz: Hello, I am in product management at Indeed, based in Tokyo
Danny Doyle: I work in ad product team, based in London
Les: I'm with Audigent, invited to call today
Lon Pilot: new joiner, GroupM
Charles Wolrich: work at Hearst, based in NY
Luke Magerko: report to Charles, lead data scientist at Hearst
Wendy: Welcome to new folks and returning folks
… we have 127 people on today's call
Q&A continued for Publisher reporting use cases
<wseltzer> https://
Wendy: last week reporting publisher reporting use cases
… invite people to queue up if there are things you want to discuss
Wendy: Don, did you hear anything in the mean time, that you wanted to add to the discussion?
Don_Marti: No, I had a few questions that I answered offline
… happy to take further questions
… and I asked a reporting related question on the FLEDGE repository
… happy to participate on that as well
Wendy: we can certainly bring back questions at any point if people have additional questions
Roundup/updates on proposal status: TURTLEDOVE+
<wseltzer> https://
Wendy: the issues in that repository are open for asynchronous feedback
… thanks again for sharing that input
… Sounds as though we are interested to hear about FLEDGE
… Michael (Kleber), Google, has written an explainer
… and written that you plan to hold meetings in WICG to review details of the proposals
… Michael, what would you like to share about FLEDGE
Michael: Hello to those of you who are new
… you are coming into a conversation that has been going on for a long time
… some topics attendees have been discussing for the past year
… to take a running start
… a year ago I wrote a proposal for TURTLEDOVE
… where an ad network gets to build the network an ad will be targeted at
… and can later show an ad to people in that audience
… and participate in an auction
… but has dramatically different properties
… than today which is based on third-party cookies
… and real-time bidding auctions
… so there is a way in JS to say, hey, join my audience
… and browser is responsible for remembering which audiences it is part of
… and there is a mechanism for on-device auction
… appears in ad slot; on-device auction chooses winner
… using some more traditional means like data targeting
… We've been discussing that general topic for a year
… Turtledove was the second proposal along those lines
… Pigeon was first one we threw out
… there have been a large number of proposals and suggestions
… for ways to change TD
… and a bunch of counter proposals that were entirely new documents of their own
… most of which had bird names
… won hearts and minds, thank you for that
… So the FLEDGE document I put out on Friday and that we are discussing today
… is my attempt to bring together all of the proposals, discussions
… and integrating into a somewhat coherent picture
… and describe what Chrome is going to do
… to build infrastructure to make this whole
… and give everyone opportunity to try it out
… in 2021
… Timeline is intense interest
… we expect Chrome to remove third-party cookies sometime in 2022
… to experiment in this brave new world it's important
… to give sense of FLEDGE and properties, how they differ from before
… Some FLEDGE properties will sound familiar
… browser maintains IGs that person is in, for use in bidding and rendering
… those IGs in FLEDGE approach are allowed to have additional metadata to inform their bids
… and use at rending time on what appears on screen as long as there are K anonymity properties
<wseltzer> https://
Michael: these will be familiar from product-level and out-come based TD proposals that RTB House discussed
… at the time on-device bidding is happening
… it is possible for on-device bidding functions to get data about publisher page
… and data about IG and the user
… in addition to data downloaded in advance
… FLEDGE allows to load some amount of data in real-time
… to allow auctions to take time-sensitive actions into account
… to turn ad budgets on/off
… use of trusted server to help with the aucgtion but not itself responsible for maintaining info but is a stable server
… info introduced in Criteo's SPARROW
… and another similar proposal called DOVEKEY
… that built about that
… so it includes trusted server
… gymnastics of on-device auction in FLEDGE
… pulls apart rolls that were poorly squished together earlier proposal
… distinguishes roles of multiple buyers
… who are responsible for deciding for particular slot, how much to bid, etc.
… pulling apart the sell-side roll, whose JS gets to look at bids and further data and make decision about highest ranked one, so winner
… so two distinguished rolls
… isolated where bids get passed along to selection side of logic
… that will sound familiar to people paying attention to PARRROT proposal
… distinguishing rolls from person running auction
… and @ device...will be familiar to those who looked at TERN
Wendy: when would you like to take questions?
Michael: one last thing
… finally, reporting side of things is one of the relatively hard to work out details
… to work out over last year
… in the FLEDGE proposal
… aimed at getting us at an interim state
… no third-party data; not flowing through @ pipes
… reporting after the fact is much less encumbered than what we are talking about
… in the FLEDGE proposal
… ads are allowed to do after the fact, higher fidelity reporting, event-level
… what ad showed up in that slot
… stop there to take questions
Wendy: Thank you very much
… more detail is available online
… lots of folks in the queue
James: Thank you for publishing this
… you added a lot of interesting reading over the past five days
… yesterday there was an announcement of the 95% comparison with personalized marketing
… folks on this call are eager to understand what is going on
… and lots of work to do in short time
… C-suites wondering what's going on
… wondering is there is additional info to support the 95% claim
… it does affect the justification for the work we will all need to do on the back of this
<btsavage> I recommend reading the whitepaper: https://
Michael: is there someone from Google ads on the call?
<btsavage> It's very clear and detailed
Michael: the 95% number James refers to was something announced in a blog post
… put up by the Ads side of Google
<jrosewell> 21st Oct. 2020 data - https://
Michael: based on experiments they ran
… based on public information we made public
… they made their own FLOC style clustering algorithm
… to create FLOCS and create IG based target marketing
… and give some type of measurement
<wseltzer> Google Ads blog post
Michael: instead of using browsing histories
… there was no Chrome FLOC data
… we just announced we expect to do in M-89, goes stable in March
… if someone from Google wants to talk about what they actually did
Deepak: Basically, results were not for re-targeting, but were for IG ads, based on taxonomies
… for in-market and affinity verticals
… these details published in a white paper on FLOC
<blassey> https://
James: could you publish the link?
Wendy: yes
<joshua_koran> Was the 95% just seeing converters belonged to a FLOC-like OR usng FLOC-like clusters can drive 95% of converters?
Kanishk: shows a 1.5% increase over random audience
… how do they jump from that to 95% accuracy
Deepak: methodology is same in experiements
… A/B
… have system with FLOC; how these experiments set up
… Accuracy in these things
… some numbers being thrown around
… different metrics to evaluate
… publisher conversion rate or RY, or CPD
… a blend of the two
… if advertiser spent $100 in old system, and $95 in new system, they would get roughly same results
James: can you explain more; seems more complicated
<GarrettJohnson> Same results for less money does not make sense.
Deepak: experiment has people either in cohorts or in cookies
<bleparmentier> Why didn't you publish it?
Deepak: if cohort doesn't have sufficient number of people...
… so and A/B experiment test
James: any more details?
Deepak: white paper describes a methodology
… if you want, we can write a one-pager on how certain things are achieved
… but it''s not more interesting than what was already published
James: and it's based on the white paper from October?
Deepak: yes
<GarrettJohnson> Google should absolutely provide detail on how the experiment was constructed.
@: works for 99% of advertisers; a lot of distortion could be there
<kleber> Hi folks: I've lost audio
@: good to look at analysis and experiment
<kleber> I'm going to reconnect
<blassey> (call has dropped for me)
@: 95 is a great number, we'd like to understand it more
… type of advertiser, how big the advertiser is
… was not addressed in the blog post
James: a lot of businesses have to answer a lot of questions and need more information
… we are going to need to justify, so I would appreciate that information
Deepak: we are also thinking to write a paper for a scientific conference that will be independently vetted
… thank you for your feedback
<mikesmith_Hearst> Mike Smith from Hearst Magazines has a related question about publisher yield testing relating to this Google post from 2019. https://
<mikesmith_Hearst> decreased by 52%, with a median per-publisher decline of 64%."
<blassey> webex is down for me
Wendy: Michael Kleber says he lost audio
<bleparmentier> (can I jump in on the same topic?
@: I am also having audio trouble
Wendy: let me suggest people turn off their video to cut down on the load
… and keep going
<peligio_epc> Webex is very flaky atm
Lionel: Hi everyone
First thank Chrome team for FLEDGE and for incorporating the feedback
… two questions
… first one regarding features in FLEDGE; end-goal, better use cases
… we seek to cover high-level use cases
… has Chrome team reviewed ads-use-cases in light of FLEDGE
… and let us know which ones you covered
… second question regarding experimentation part
… says how to propose some
… first focus on measuring
… and research
… with ad display using @ and FLEDGE
… would you consider these two objectives to the FLEDGE experimental part?
<kleber> I'm sorry, I'm reading the transcript on IRC (thanks Karen!), but I'm not able to call back into the meeting
Wendy: we have a challenge
… Michael is unable to connect to audio
<angelina> recommend picking up the convo next week when he can reconnect
<kleber> Trying from three different computers but no luck so far :-(
Wendy: doesn't sound good
<kleber> Ah I'll try by phone, just a minute
Wendy: we are nearly 150 people hear and straining to make the tech work
… let's see if phone gets us back on track; otherwise we will have to take a pause
<AramZS> Wendy you are going in and out
<lbasdevant> Yes, I'm having trouble hearing you too
<wseltzer> [It seems we're having technical trouble]
<Brendan_eyeo_GmbH> A significant portion of attendees swapping to "Audio Only Mode" might assist WebEx in handling the ~150 attendees
Michael: can you hear me?
Wendy: yes, can you hear us?
<grzegorz> Wendy, I can hear you.
Wendy: I cannot see options in my meeting settings that would be helpful
… unless things clear up, we may need to take this up on issues
… and return next week, 2 February
… I'll inquire with our contacts at Webex whether is anything we should prepare to improve our odds for a successful meeting
… Technology has worked well in the past, but today it's not on our side
… Thank you for preparing the detailed explainer document
… Michael, and for the pointer to web platform incubator community group
<wseltzer> https://
Wendy: where we invite people to discuss calls on FLEDGE specifically
… and our call next week
… and then look forward to future opportunities to discuss this
… thank you Lionel, Jonasz, Brian, Aram, apascoe, Lon, Andrew, please take questions to repository
… thank you and thanks all for efforts
<ajknox> thank you
Wendy: we will send these notes out
… along with notes on what's next for suggestions for the technology
[for the next call]
<wseltzer> [adjourned]
<AramZS> How big are those calls?
<AramZS> Perhaps we should move to a conference line
<ErikAnderson> We've done 80+ people on Zoom for some Privacy CG meetings.
Wendy: yes, we are adjourning now and will pick it up next week
<kleber> We've had 120+ people on WebEx before, so I don't think it's just the number of participants
<kleber> It's true that those previous large calls were audio-only, and this time there were more people with video on
[Post-call note, there was a FIOS outage in the Northeast U.S. that likely affected many participants, including those in the Boston area.]