Improving Web Advertising BG

26 January 2021


ajknox, apascoe, AramZS, arnoldrw, blassey, bleparmentier, bmay, br-rtbhouse, Brendan_eyeo_GmbH, btsavage, cpn_, dinesh, dkwestbr, dmarti, ErikAnderson, eriktaubeneck, gendler, hong, ionel, jonasz, jrobert, jrosewell, Karen, kleber, kris_chapman, lbasdevant, marguin, Mike_Pisula, Mikjuo, mjv, mserrate, pbannist, pedro_alvarado, piwanczak, pl_mrcy, shigeki, Shivani, wbaker, wseltzer
Wendy Seltzer
Karen, Karen Myers

Meeting minutes

<wseltzer> https://github.com/dmarti/in-browser-auction-publisher-issues/blob/main/reporting.md https://lists.w3.org/Archives/Public/public-web-adv/2021Jan/0003.html

<GarrettJohnson> Garrett+

<blassey> although regrets for the second half of the call

[Wendy asks people to "present+" if on irc channel]

Wendy: Let's start with a look through our agenda
… any other business?

Agenda-curation, introductions

Wendy: any introductions; anyone new to the call who would like to introduce themselves to the group?
… feel free to unmute

Claire Lukasiewicz: Hello, I am in product management at Indeed, based in Tokyo

Danny Doyle: I work in ad product team, based in London

Les: I'm with Audigent, invited to call today

Lon Pilot: new joiner, GroupM

Charles Wolrich: work at Hearst, based in NY

Luke Magerko: report to Charles, lead data scientist at Hearst

Wendy: Welcome to new folks and returning folks
… we have 127 people on today's call

Q&A continued for Publisher reporting use cases

<wseltzer> https://github.com/dmarti/in-browser-auction-publisher-issues/blob/main/reporting.md

Wendy: last week reporting publisher reporting use cases
… invite people to queue up if there are things you want to discuss

Wendy: Don, did you hear anything in the mean time, that you wanted to add to the discussion?

Don_Marti: No, I had a few questions that I answered offline
… happy to take further questions
… and I asked a reporting related question on the FLEDGE repository
… happy to participate on that as well

Wendy: we can certainly bring back questions at any point if people have additional questions

Roundup/updates on proposal status: TURTLEDOVE+

<wseltzer> https://github.com/WICG/turtledove/blob/master/FLEDGE.md

Wendy: the issues in that repository are open for asynchronous feedback
… thanks again for sharing that input
… Sounds as though we are interested to hear about FLEDGE
… Michael (Kleber), Google, has written an explainer
… and written that you plan to hold meetings in WICG to review details of the proposals
… Michael, what would you like to share about FLEDGE

Michael: Hello to those of you who are new
… you are coming into a conversation that has been going on for a long time
… some topics attendees have been discussing for the past year
… to take a running start
… a year ago I wrote a proposal for TURTLEDOVE
… where an ad network gets to build the network an ad will be targeted at
… and can later show an ad to people in that audience
… and participate in an auction
… but has dramatically different properties
… than today which is based on third-party cookies
… and real-time bidding auctions
… so there is a way in JS to say, hey, join my audience
… and browser is responsible for remembering which audiences it is part of
… and there is a mechanism for on-device auction
… appears in ad slot; on-device auction chooses winner
… using some more traditional means like data targeting
… We've been discussing that general topic for a year
… Turtledove was the second proposal along those lines
… Pigeon was first one we threw out
… there have been a large number of proposals and suggestions
… for ways to change TD
… and a bunch of counter proposals that were entirely new documents of their own
… most of which had bird names
… won hearts and minds, thank you for that
… So the FLEDGE document I put out on Friday and that we are discussing today
… is my attempt to bring together all of the proposals, discussions
… and integrating into a somewhat coherent picture
… and describe what Chrome is going to do
… to build infrastructure to make this whole
… and give everyone opportunity to try it out
… in 2021
… Timeline is intense interest
… we expect Chrome to remove third-party cookies sometime in 2022
… to experiment in this brave new world it's important
… to give sense of FLEDGE and properties, how they differ from before
… Some FLEDGE properties will sound familiar
… browser maintains IGs that person is in, for use in bidding and rendering
… those IGs in FLEDGE approach are allowed to have additional metadata to inform their bids
… and use at rending time on what appears on screen as long as there are K anonymity properties

<wseltzer> https://github.com/WICG/turtledove/blob/master/FLEDGE.md#1-browsers-record-interest-groups

Michael: these will be familiar from product-level and out-come based TD proposals that RTB House discussed
… at the time on-device bidding is happening
… it is possible for on-device bidding functions to get data about publisher page
… and data about IG and the user
… in addition to data downloaded in advance
… FLEDGE allows to load some amount of data in real-time
… to allow auctions to take time-sensitive actions into account
… to turn ad budgets on/off
… use of trusted server to help with the aucgtion but not itself responsible for maintaining info but is a stable server
… info introduced in Criteo's SPARROW
… and another similar proposal called DOVEKEY
… that built about that
… so it includes trusted server
… gymnastics of on-device auction in FLEDGE
… pulls apart rolls that were poorly squished together earlier proposal
… distinguishes roles of multiple buyers
… who are responsible for deciding for particular slot, how much to bid, etc.
… pulling apart the sell-side roll, whose JS gets to look at bids and further data and make decision about highest ranked one, so winner
… so two distinguished rolls
… isolated where bids get passed along to selection side of logic
… that will sound familiar to people paying attention to PARRROT proposal
… distinguishing rolls from person running auction
… and @ device...will be familiar to those who looked at TERN

Wendy: when would you like to take questions?

Michael: one last thing
… finally, reporting side of things is one of the relatively hard to work out details
… to work out over last year
… in the FLEDGE proposal
… aimed at getting us at an interim state
… no third-party data; not flowing through @ pipes
… reporting after the fact is much less encumbered than what we are talking about
… in the FLEDGE proposal
… ads are allowed to do after the fact, higher fidelity reporting, event-level
… what ad showed up in that slot
… stop there to take questions

Wendy: Thank you very much
… more detail is available online
… lots of folks in the queue

James: Thank you for publishing this
… you added a lot of interesting reading over the past five days
… yesterday there was an announcement of the 95% comparison with personalized marketing
… folks on this call are eager to understand what is going on
… and lots of work to do in short time
… C-suites wondering what's going on
… wondering is there is additional info to support the 95% claim
… it does affect the justification for the work we will all need to do on the back of this

<btsavage> I recommend reading the whitepaper: https://github.com/google/ads-privacy/blob/master/proposals/FLoC/FLOC-Whitepaper-Google.pdf

Michael: is there someone from Google ads on the call?

<btsavage> It's very clear and detailed

Michael: the 95% number James refers to was something announced in a blog post
… put up by the Ads side of Google

<jrosewell> 21st Oct. 2020 data - https://github.com/google/ads-privacy/commit/c82c2c41cc6219d6f6e3de28d3ac94c68e41b941

Michael: based on experiments they ran
… based on public information we made public
… they made their own FLOC style clustering algorithm
… to create FLOCS and create IG based target marketing
… and give some type of measurement

<wseltzer> Google Ads blog post

Michael: instead of using browsing histories
… there was no Chrome FLOC data
… we just announced we expect to do in M-89, goes stable in March
… if someone from Google wants to talk about what they actually did

Deepak: Basically, results were not for re-targeting, but were for IG ads, based on taxonomies
… for in-market and affinity verticals
… these details published in a white paper on FLOC

<blassey> https://github.com/google/ads-privacy/blob/master/proposals/FLoC/FLOC-Whitepaper-Google.pdf

James: could you publish the link?

Wendy: yes

<joshua_koran> Was the 95% just seeing converters belonged to a FLOC-like OR usng FLOC-like clusters can drive 95% of converters?

Kanishk: shows a 1.5% increase over random audience
… how do they jump from that to 95% accuracy

Deepak: methodology is same in experiements
… A/B
… have system with FLOC; how these experiments set up
… Accuracy in these things
… some numbers being thrown around
… different metrics to evaluate
… publisher conversion rate or RY, or CPD
… a blend of the two
… if advertiser spent $100 in old system, and $95 in new system, they would get roughly same results

James: can you explain more; seems more complicated

<GarrettJohnson> Same results for less money does not make sense.

Deepak: experiment has people either in cohorts or in cookies

<bleparmentier> Why didn't you publish it?

Deepak: if cohort doesn't have sufficient number of people...
… so and A/B experiment test

James: any more details?

Deepak: white paper describes a methodology
… if you want, we can write a one-pager on how certain things are achieved
… but it''s not more interesting than what was already published

James: and it's based on the white paper from October?

Deepak: yes

<GarrettJohnson> Google should absolutely provide detail on how the experiment was constructed.

@: works for 99% of advertisers; a lot of distortion could be there

<kleber> Hi folks: I've lost audio

@: good to look at analysis and experiment

<kleber> I'm going to reconnect

<blassey> (call has dropped for me)

@: 95 is a great number, we'd like to understand it more
… type of advertiser, how big the advertiser is
… was not addressed in the blog post

James: a lot of businesses have to answer a lot of questions and need more information
… we are going to need to justify, so I would appreciate that information

Deepak: we are also thinking to write a paper for a scientific conference that will be independently vetted
… thank you for your feedback

<mikesmith_Hearst> Mike Smith from Hearst Magazines has a related question about publisher yield testing relating to this Google post from 2019. https://services.google.com/fh/files/misc/disabling_third-party_cookies_publisher_revenue.pdf Excerpt reads "We disabled access to cookies for a small fraction of randomly selected users (the treatment group). We observed that for the top 500 global publishers, average revenue in the treatment group

<mikesmith_Hearst> decreased by 52%, with a median per-publisher decline of 64%."

<blassey> webex is down for me

Wendy: Michael Kleber says he lost audio

<bleparmentier> (can I jump in on the same topic?

@: I am also having audio trouble

Wendy: let me suggest people turn off their video to cut down on the load
… and keep going

<peligio_epc> Webex is very flaky atm

Lionel: Hi everyone

First thank Chrome team for FLEDGE and for incorporating the feedback
… two questions
… first one regarding features in FLEDGE; end-goal, better use cases
… we seek to cover high-level use cases
… has Chrome team reviewed ads-use-cases in light of FLEDGE
… and let us know which ones you covered
… second question regarding experimentation part
… says how to propose some
… first focus on measuring
… and research
… with ad display using @ and FLEDGE
… would you consider these two objectives to the FLEDGE experimental part?

<kleber> I'm sorry, I'm reading the transcript on IRC (thanks Karen!), but I'm not able to call back into the meeting

Wendy: we have a challenge
… Michael is unable to connect to audio

<angelina> recommend picking up the convo next week when he can reconnect

<kleber> Trying from three different computers but no luck so far :-(

Wendy: doesn't sound good

<kleber> Ah I'll try by phone, just a minute

Wendy: we are nearly 150 people hear and straining to make the tech work
… let's see if phone gets us back on track; otherwise we will have to take a pause

<AramZS> Wendy you are going in and out

<lbasdevant> Yes, I'm having trouble hearing you too

<wseltzer> [It seems we're having technical trouble]

<Brendan_eyeo_GmbH> A significant portion of attendees swapping to "Audio Only Mode" might assist WebEx in handling the ~150 attendees

Michael: can you hear me?

Wendy: yes, can you hear us?

<grzegorz> Wendy, I can hear you.

Wendy: I cannot see options in my meeting settings that would be helpful
… unless things clear up, we may need to take this up on issues
… and return next week, 2 February
… I'll inquire with our contacts at Webex whether is anything we should prepare to improve our odds for a successful meeting
… Technology has worked well in the past, but today it's not on our side
… Thank you for preparing the detailed explainer document
… Michael, and for the pointer to web platform incubator community group

<wseltzer> https://github.com/WICG/turtledove/issues/88

Wendy: where we invite people to discuss calls on FLEDGE specifically
… and our call next week
… and then look forward to future opportunities to discuss this
… thank you Lionel, Jonasz, Brian, Aram, apascoe, Lon, Andrew, please take questions to repository
… thank you and thanks all for efforts

<ajknox> thank you

Wendy: we will send these notes out
… along with notes on what's next for suggestions for the technology

[for the next call]

<wseltzer> [adjourned]

<AramZS> How big are those calls?

<AramZS> Perhaps we should move to a conference line

<ErikAnderson> We've done 80+ people on Zoom for some Privacy CG meetings.

Wendy: yes, we are adjourning now and will pick it up next week

<kleber> We've had 120+ people on WebEx before, so I don't think it's just the number of participants

<kleber> It's true that those previous large calls were audio-only, and this time there were more people with video on

[Post-call note, there was a FIOS outage in the Northeast U.S. that likely affected many participants, including those in the Boston area.]

Minutes manually created (not a transcript), formatted by scribe.perl version 127 (Wed Dec 30 17:39:58 2020 UTC).