Overview slides with links

<Karen> Scribe: Karen

<Garrett_Johnson_> My slide deck is here: https://docs.google.com/presentation/d/1PKHVtO6hgwBJS1vafLyvG_lwupLhfCSelzczvltxjqc/edit?usp=sharing

Intros

<scribe> Chair: Wendy

Wendy: Welcome everyone to Improving Web Advertising Virtual F2F
... hope you have been able to join other TPAC events
... and are looking forward to a set of breakout sessions scheduled for next week as well
... Give you a brief welcome
... and reminder that this meeting, like all W3C meetings, is governed by our code of ethics and professional conduct
... we just had a great training session on that yesterday
... and opportunities to review that material with others in the W3C community
... Our agenda is posted and sent out; built from many conversations with the community
... Today we have Demystifying Advertising and Economics of Identity
... we have also built breaks into schedule
... after break we will hear about Privacy and Optimization
... and Browser and media modes, led by Wendell Baker
... We will conclude at 19:00 UTC today
... Tomorrow we will pick up on state of the art multi-party computation; measurement, Gatekeeper
... and time end of tomorrow for wrap up and next steps
... if there is something pressing you would like to make sure we cover, queue it up there
... if it doesn't fit into the other discussions
... We work in Github repository and take minutes in irc
... and present+ to indicate you are participating

[http: //irc.w8.org/?channels=#web-adv]
... please q+ if you would like to speak
... not everyone is in there [irc]
... I will also listen to verbal queue pluses and look for hands in webex
... a lot of screens, so if you can make irc work for you that is ideal
... with that, I think we can get to any logistical questions?

Wendy: thanks, Karen Myers for picking up scribe pen
... reminds me if there is anyone new to the group who would like to introduce yourself today?
... Jeff, please say hello

Jeff Jaffe: I'm CEO of W3C, thank you for participating

Philippe Le Hegaret: I am the project management lead responsible for the working groups

Wendy: Thanks for joining us

Don: I'm Don Marti, new with CafeMedia
... working on ecosystem innovation; new web standards to enable small publishers

Wendy: Welcome, Don
... thanks for joining us
... anyone else?
... Let's jump in
... We have some backgrounders prepared to help set the stage and bring everyone together
... first Demystifying advertising with Steve Hulkower from Magnite
... and then Economics of digital ad identity from Garrett Johnson from Boston University
... if you have questions or clarifications
... you can "q+ to ask"
... let's me know if it's queuing right away, or to ask a clarifying question
... Let's get to Steve's presentation

An attempt to demystify advertising

Wendy: Ready to go, Steve?

Steve: you will drive and I'll say next slide
... Good to meet everybody
... Steve Hulkower from Magnite
... I really appreciate what all of you are doing
... hope this is useful
... some may already know all this
... If this is really boring, tell me to go faster
... next slide
... Agenda
... talk about adtech

<wseltzer> Steve's Slides

Steve: picture of me 15 years ago as a product manager
... product management for 8 years, at Magnite
... started running campaigns for big brands
... learned about adtech
... I put this into rules
... do not read our marketing
... we are good at marketing; make ourselves sound amazing and doing cool, state of the art tech
... but we exaggerate

<weiler> steve++ for candor

Steve: do not learn from marketing, press releases; they are high level and exaggerate
... Second rule is this is simpler than it looks
... Some examples of examples of drawings of how adtech works
... it is hard
... some pieces don't exist, some do
... they don't all talk to each other at the same time
... they can go in different directions
... layers upon layers
... add complexity and make it hard to draw
... this is the turning money into people machine
... and idea of the players and data and that fun stuff
... three of these visuals
... How programmatic advertising works
... we have incorporated a little bit of privacy
... but still hard to tell what is going on
... IAB slide
... this is the worst one
... different buying types
... different deals, market buys
... this is confusing
... I am not good at art, but here is my rendition
... we have sellers who have ad space and buyers who want to buy ad space
... Sellers want good ads to show up; they want controls to support buyers
... and make money; current model for how we compensate publishers
... on the buy side
... they are trying to serve the right ads in the right places
... it's about targeting - make sure the right ads show up in the right places
... frequency control is showing the right number of ads
... then Attribution is about results
... showing user what I want them to do
... that goes to return on ad spend
... if I spend $100 on ad spend, I want to make $110 back on it
... Most of you are familiar with the Lumascape design
... there are a ton of these guys
... this is a simple version
... Cheifmarktec.com slide - the industry is more like this - there are a ton of adtech companies in the space
... confusing to show them all
... now you get my rendition
... I put everyone into a small number of boxes
... Everyone uses the same tech
... everyone uses and ad server
... buy side stores image
... sell side delivers the page
... if you need system to collect bids or both, everyone does this
... things that add to why we do something; creative vendors, audience modeling, brand protection
... all those companies fall into the data and services box
... two models of buying
... Direct buying, you have ad tags
... you don't need to bid anymore
... buyer stuffs everything into the ad tag and it's delivered
... so you just need ad servers
... the other model is programmatic
... that is more complicated
... Programmatic users RTB
... requires a bid sender and receiver to deliver those ads
... creative still in ad server
... seller delivers ad ultimately
... another layer of bidders taking to each other over RTB protocol
... Third rule

<jeff_> [RTB=Real Time bidding]

Steve: how an ad gets to a page
... I call it ad delivery system
... still same concenpt
... ad delivery system gets the code
... not always, but from there you end up in server land
... transmits everything on page from server to server
... the ID for the user is being picked up by that bidding system from the page
... User agents...targeting buyers
... page URLs target domains to make sure ads appear in right places
... location, we use IP address; awesome for countries, but not for zip codes
... all that info is picked up
... next bidding system maps to everyone elses ID and places calls
... bidders process info
... match to targeting and rules
... figure out which is best ad to serve
... processes publishes business rules and chooses a winner
... can be 1-4 layers
... a bunch of info gets processed and gets shrunk back
... may happen twice, three, four times
... but it's all the same thing
... all the bidders return info to delivery system and return code
... ID for the user
... since we are using serveres
... you rely on idea that first link in chain knew who user was
... first link has to pass ID and know how to map it
... very real times when bidder knows who user is
... but somewhere in chain there is no ID available
... and you lose that important info
... last bid is bidders process request
... what they care about is ID
... powers attribution
... as Garrret will show
... requests without user IDs don't generate a lot of user demand
... Last part
... Fourth rule: Identity matters
... being ridiculous to show all these companies again
... thanks to whoever built that
... The User ID is super important
... main role of ID is tapping an attribution, but also powers audience targeting
... sometimes scarey
... but it's the big draw
... Attribution is most important; knowing your money was spent well, you can adjut
... IDs, @ smash into each other
... comes from same ID
... if you can fix that, that's cool
... ID is super important because you have hundreds of silos
... everyone collects and stores own data
... some like how data is collected, others not
... we have to communicate with each other
... how we use the data is through that ID
... cookies...we are drowning
... and if you don't fix this, we will be drowning in third party IDs
... Look forward to a better version of how we do this

Wendy: thank you, Steve
... people with questions specific to Steve can please queue up

<Zakim> weiler, you wanted to ask re: fundamentals and to ask re: utility of 1p cookies

Wendy: and we will have a larger period of discussion

weiler: thank you for the presentation
... how does drowning in first party cookies work?

Steve: main switch is persistence
... third party cookies...stay there and can be transmitted and mapped
... you can build an ID graph
... of first party IDs and how they turn into one
... shifiting from one third party Id
... in browser
... versus ID for this user on every single publisher, trying to figure out how to tie together
... and have a zillion IDs

Weiler: you talked about how fundamental the ID is
... I have seen ecosystems shift before
... as we get different amounts of fraud, prices of ads change
... balances
... I wonder to what degree if whole ecosystem changes, we still have advertising that is not individually targeted?
... I still see billboards, TV
... will system rebalance?

Steve: If you all fail at what we are trying to do, this will rebalance
... what's changing is our ability to track better
... we are not using 75-yr-olds in Iowa who watched your TV show
... billboards have better measuring
... dollars have been proposition; ability to measure
... on high level
... if that all crashed and burned, you would still have digital advertising
... and you would figure out better ways to measure
... when you are self-isolated, you can do first party cookie and attribution
... for open web, all digital ad money flocks to where they can do this
... cool audiences that look exciting, but attribution is key
... This industry will survive
... but will be far better off if we can do privacy forward mode
... fine if we lose audience data

<wseltzer> vq?

Steve: some aren't ok with that but I am

Wendy: Closing queue after those
... and come back after

Jeff: Thanks for the presentation; quite interesting
... you siad in end if we lose third party cookies, we will end up with a mass of first party IDs
... that model of first party IDs being sent around
... is that being incubated somewhere?

Steve: it has already started
... Prebid has a user ID model
... don't know how many companies are in that module
... more and more prebid module systems to have consistent identifying to be broadcast out

<Jeff_Wieland> Prebid Userid modules: https://docs.prebid.org/dev-docs/modules/userId.html

Steve: we on sell side are being asked to target ids
... shifting to using others' IDs and lots of them

<weiler> "uBlock Origin has prevented the following page from loading:

<weiler> https://docs.prebid.org/dev-docs/modules/userId.html"

Steve: more and more being pushed around
... you can look at Prebids' module; will make you sad likely

Kris: I was going to comment on whether or not without programmatic if there will still be advertising
... I agree there will be ads online if programmatic doesn't exist
... but industry shifted to programmatic because it shows how consumers convert
... it will be a worse experience if we only do contextual and lose programmatic
... not argument of is there advertising
... but argument is whether people are interested in buying
... not look at it as something only interesting to buy and sell sides
... it matters to consumers as to why it should still be supported

Steve: thank you

bleparmentier: I want to
... say not exactly what Kris said
... there will be some of that, it's certain
... when you do an ad, this is lost
... at overall level
... without targeting, we will show irrelevant ads
... ad spend return goes down
... ads go elsewhere
... to walled garden that keeps info
... to TV
... this will mean reduced publisher revenue
... less spend because we cannot measure or optimize
... important to keep it
... there will still be advertising
... publishers will put up pay walls
... in my mind
... we need to find something that works
... Say something else that Steven said
... need to find something that finds something that works well enough
... if not usable, I am afraid people will use emails, which are PII
... and that will lead to less profitable web for publishers and worse privacy
... PIIs will be exchanged everywhere
... I think...explain why it is very important to find something that works well enough
... and replace
... and not find ways around the new restrictions

Wendy: Thank you

<Garrett_Johnson_> My slide deck is here: https://docs.google.com/presentation/d/1PKHVtO6hgwBJS1vafLyvG_lwupLhfCSelzczvltxjqc/edit?usp=sharing

Wendy: this has helped to tee up the big questions this group is trying to address
... How to build an ecosystem that works for everyone
... and works in the middle of these transactions, and how to serve them
... thanks for kicking us off with the importance there
... Shall I drive your slides?

Garrett: I will drive
... can you hear me?

Wendy: yes

Garrett: thank you

<wseltzer> Garrett's slides

<weiler> [seems to make a case for per-party email addrs.]

Economics of Digital Ad Identity

Garrett: many of the questions that came up will be addressed
... I am professor of marketing at Boston University
... an expert in ad identity
... I'm a Canadian who grew up on the prairies
... I have a 2020 publication that addresses this issue, my research examines issues in this space
... Look at GDPR
... and co-author of ghost ads methodology
... I think I am only academic on these calls
... bring an outside perspective; no stake, just hoping for good policy
... Three questions to discuss

[slide]

Garrett: Begin with the value of cookies
... focus on the status quo
... last presentation brought up the issues of why cookies are important
... programs the three key value generators
... sexy stuff is targeting
... but reach management is valued by all advertisers
... this identity has solved eyeballs problem
... do cross-site attribution and evaluate advertising
... these targeting and measurement tools allow optimization at scale
... Several studies have looked at the value of cookies
... without cookies, prices fall by up to 70%
... important to think about what these studies do or don't do
... Begin with paper by Goldfarb & Tucker (2011)
... they found measurements of purchase intent (ad effectiveness) fell by 65%
... these are chaired professors at MIT and Univ of Toronto
... EU in 2020 provides world post cookies
... we don't get revenue figures for study and data are 20 yrs old
... next is Beales & Eisenach (2014) study
... ads without cookies have 66% lower prices
... they provide revenue and price info
... industry study
... some lack of attention to detail
... a couple "i's" to be dotted
... My research
... we studied opt-out
... opt-out users have 52% lower ad exchange prices
... we find this drop is spread proportionally to these different agents in the marketplace
... this is a published and peer reviewed paper
... limitations is we looked at opt-out impressions versus cookieless, and looking at one ad exchange
... another study Marotta, Abhishek, & Acquisti
... they say ads without cookes have 11% lower publishing revenues; but another model says 4% lower publisher revenue
... this paper has not been peer reviewed
... raw difference in prices in 37%
... a big question is how you get from 37 to 4%?
... I think authors got a bit carried away
... long version can wait until Q&A
... Google study (2019)
... came up with same 52% lower revenue without cookies
... Used randomized experiment
... and looked at top 500 publishers on Google
... some of you may be skeptical
... some detail in study, but not a lot
... UK CMA Report (2020) looked at the same Google data
... removed users without cookiers
... accounted for missing unsold impressions
... i was impressed with this work
... Googles does not control the entire adtech stack
... other DSPs were getting cookies and that affects the result as well
... Additional evidence
... much lower prices from Safari/Firefox users
... drops in FB publisher revenue
... My broad takeaway
... Ad ID increases ad revenue by 2X-3X
... this is the conclusion that I come to you with
... Question of what advertising looks like without cookies
... Assumption that market will adjust and money will flow
... important to think about how will market be different
... think about different advertisers and how they will respond
... Brand advertisers tend to cling to premium publishers; but they really want cross-site frequency
... Direct advertisers are more about money in, money out
... value the consumer intent
... put a premium on ad measurement, attribution and ability to optimize
... expect ad pie to shrink and be divided more unequally
... there are some case studies
... NYTimes.com increased revenue
... might be flight to safety where not everyone is protecting privacy
... also in The Netherlands, NPO increased revenue
... From my industry, if you offer a lesser product
... there will be challenges
... we have seen reduced revenue today for Zoom-only educational model
... speaks to economics in advertising
... When we get rid of cookies
... Context, sign-ins, walled gardens, and W3C work, which I will refer to as "the birds"
... Context has work to do to recover the revenue
... general content performs worse than niche content
... we know this from the data
... Goldfarb and Google papers who this
... if you are showing finance users finance content, that is going to be more valuable than cook book content
... challenge algorithmically
... referred to as the "sperm whale" problem
... and algorithm is going to tell a publisher to stay away from a page that mentions "sperm" ten times or more
... price may go up when some advertisers hold back their spend
... and another potential danger is not scale to satisfy budgets
... of the advertisers, so pie shrinks
... if you are publisher with brand safety benefit, you will do better
... but context should recover the 2x-3x
... but context competes with sign-in and walled gardens
... a concern for small publishers is if you lose more than 2-3X
... move to sign-in
... provides and identity solution
... first party cookies reference
... this provides a cross-device identifier which is useful
... but you get lower scale
... makes it harder for small advertisers to participate
... winners and losers depends upon who has first party relationship with users
... Dollar Shave club has this info, but Campbells and Quaker Oats do not
... creates barriers for advertiser who do not have these email addresses
... and creates tension
... over coordinating identity solution
... challenge is there will be some benefits
... for reason we talked about on last slide - make slide bigger
... so again, we expect pie to shrink
... irony that cookies being replaced by emails that are personally identifiable
... do require consumer consent
... next come to walled gardens
... Rich get richer
... giants will be hurt by limiting 3rd party data
... FB has 40% of display ad spending, cross device, have scale
... get value from long tail advertisers
... Google represents 40% of digital ad spending
... cross-device ID among logged-in users
... have scale
... facility for advertisers to be a one-stop shop
... we expect walled gardens to do better in this new world
... i have been participating on these calls since January
... I am struck by the intelligence and hard work of the people on these calls
... but also understand we filter things depending upon our perspectives
... I can pretty much predict your perspectives whether you work for publisher, browser, ad tech
... of it you own ad stack or not
... Important to keep an open mind and learn from each other's perspectives
... Last question
... be brief and high level
... How can privacy and modern advertising work?
... See "The Birds" work at W3C
... not talk individually but more at the high level problem trying to solve
... Advertisers don't care about an audience of one
... Advertisers care about reaching audiences; that goal is consistent with privacy
... advertisers want to measure clicks and conversions
... and which are click-through and view-through
... and advertisers care about retargeting, recency
... which is valuable
... and the very basic, unsexy thing
... advertisers care about reach and frequency, especially across sites
... and advertisers want to run experiments
... Conclusion here
... is once we layer in all these use cases, the information becomes more personally identifiable
... doing away with identity is really hard
... first tension is between privacy, and measurement and targeting
... this is going to be more difficult
... Second tension is competition
... by nature, will be harder for smaller advertisers and publishers
... Third tension is between centralization and decentralization
... whether need a trusted gatekeeper
... will be session on gatekeepers later today or tomorrow
... Concluding thoughts
... this is really important
... danger that pie will shink
... tensions dictate how we will come out of this
... and not have the pie be distributed unequally

<bleparmentier> Just want to say, Amazing prez!

Wendy: Thank you, Garrett; lots of good info and references to follow up on
... I see a big queue

<kleber_> https://docs.google.com/presentation/d/1PKHVtO6hgwBJS1vafLyvG_lwupLhfCSelzczvltxjqc/edit#slide=id.p1 Garrett's slides

Valentino: this was incredible, good quality analysis; hope this deck will be available
... revenue response to cookies and noncookies
... see an exaggerated effect
... cookies on one side, not on other
... no one wants that
... take away cookies and becomes status quo
... and drop will be lower
... not sure if studies have confirmed this
... how do you see a good way to respond to this data
... that this is not the case

Garrett: you are correct; agree effect is exaggerated
... but not starting off with 2 or 5 %
... looking at 50-70%
... but doing better could still lose a lot on the table
... speaks to economics of ID
... they are extremely valuable
... why are we moving to a world with first party identifiers and emails
... tremendously valuable

<weiler> scribe: weiler

<inserted> scribenick: Karen

Garrett: means these contextual solutions for smaller publishers who have to compete with walled gardens and first party solutions will have a really hard time

Aram: I agree there is a difficulty to understand these things
... where advertisers go from data rich to data poor bid

<weiler> aram: f/u on valentino: difficulty understanding measurement in status quo, when advertsiers can go from data-rich to data-poor bid.

Aram: like 70% a total loss, make it worse

<weiler> ... maybe we make that a little worse.

Aram: entire marketplace changes
... do we really have enough data in these studies that this will cause money to leave the digital ad marketplace and go somewhere else

<weiler> ... if there are only data-poor bids, do we have enough data to show that $ will leave?

Aram: you talked about impact on small publishers
... is there any data of small publishers being representatives of niche markets
... that might be beneficial
... of course I have perspective of large publisher

Garrett: data I presented were from both large and small publishers
... if you cannot offer cross device and @ attribution, it is more difficult
... market is unknowable; haven't done this experiement
... not in Europe post GDPR
... have to rely on how valuable identity is and what are solutions that can take the change
... you brought up point from data rich to data poor
... post market adjustments; agree that will be reduced
... not so bad
... but in post market adjustment, we will still see data rich and data poor adjustments
... walled gardens and sign-in models will be more data rich
... will make it harder for smaller publishers to compete
... If you are a niche advertiser and flying blind
... some of these players will disappear from the marketplace
... concern with advertisers reducing spend

Aram: will we see regulatory effects that will change how you estimate the impacts?

Garrett: good question
... post-cookie world will largely be a largely GDPR compliant world
... may be some consent
... if regulators squeeze on opt-in
... will bring levels higher than we see in status quo

Charlie: great talk, really informative
... question about studies published
... my understanding is some studies only removed cookies for personalization
... for targeting
... wonder about studies looking at cookies for relative value for products
... is there research for value of something just for measurement

<kris_chapman> +q

Charlie: strip out the personalization piece
... and look only at measurement

Garrett: FB studies speaks to loss of personalization

<Alan> [103 people in the meeting - WOW]

Garrett: question of measurement has not been addressed
... hard to view one without the other
... interesting question
... we don't have data on that
... If you have more data about a user can you do better prediction of clicks
... thanks, Charlie

dkwestbr: thanks, Garrett
... one, do you perceive potential
... where ten dollars might go...dynamics with walled gardens and premium publishers in contextual context
... Steve brought up, part of differentiation of digital ads is differentiaion
... maybe go back to linear TV
... go back to status quo
... studies done on how things work today
... only potential is how these solutions look inside of Apple
... Scadnetwork
... been out for a few years; perhaps can give us a better picture of where things are headed

Garrett: Apple is looking at this
... maybe burning down not best choice until there is a replacement
... differentiation with @
... growth of digital ad spending is eyeballs
... spending more time on devices than newspapers and television
... optimization...
... if you lose that I expect movement towards other things
... other things is advertisers move $ from display into search
... might be one adjustment
... we will continue to spend more time on devices
... this may obfuscate the shrinking pie; great question

Mike_Pisula: I represent buyers; part of agency holding company
... I think we are ignoring a problem
... Safari - no third party cookies
... advertisers want to reach consumers but we cannot do that
... the first-party workarounds, the fingerprinting so bad
... privacy sandbox looks like a more scalable way
... my advertisers want to spend money
... studies are binary; no cookies, no targeting
... but doesn't have to be way it is today
... we have a problem so let's create a better solution

Garrett: with some humility
... extent to which Firefox and Safari...defy spending
... identifiers use is a bit counter to this
... that you can target on Chrome sustains the barrage
... why not seeing more to invest in tech for post cookie world where some of world is already there
... good to hear that advertisers want to spend the money

arnaud_blanchard: I really liked presentation and the pros and cons of the arguments
... ad revenue would shrink is important statement
... agree with most of what you said
... unless we try it not sure
... second one
... very few arguments; it will be divided more unequally by design
... if you look at everything done around cohorts
... targeting business
... unequal access to the ad ecosystem
... some publishers are obfuscated
... unequal access between small and big publishers
... that you have data...small and large publishers
... data is growing at square root of data set
... privacy sandbox increases the inequality
... this is really something that should be noted
... and undisputed and discussed as a topic
... thank you for bringing that to the whole community's attention

Garrett: I laid out tensions in my summary

<Zakim> weiler, you wanted to discuss confirm how email addrs are being used now

Garrett: Arnaud's points are important for understanding what these mean at scale

Sam: you mentioned that email addresses would be share more in a post-ID world

Garrett: industry better to speak to it
... large companies like TradeDesk and LiveRamp
... looking to scale to publishers
... we see more sign-in walls because they want this identity
... that's all I can say

dialtone: There is third party or first party cookie ends up in map where key is the email address
... and inverse
... can look up first party cookie of new publisher and jump to user profile

Sam; how widespread?

Valentino: going on for decades
... a low scale
... advertisers don't force people to login
... but it's a viable infrastructure
... if industry is forced
... together we find privacy solutions
... industry looking at back stop using identity trying to make it as privacy forward as possible
... does not use an unreliable third party cookie
... not to benefit of large advertisers or buyers
... that is discussion going on right now
... not a good solutions for a solution for old problems
... maybe trading off a set of old problems
... follow-up question
... Largely I agree with...isolation of bigger players that have first party relationships
... if open web did not monetize as well
... there will be less content, and less content means less revenue from search
... so Google will suffer
... and FB has articles shared through other sources
... I see the one-eyed blind man metaphor
... that is my POV

Garrett: i agree with that
... it is sensible
... from my vantage point
... we have this engagement and set of solutions
... especially because the Chrome team cares about the open web
... that is why we are here, even with our own business interests

<Zakim> kleber_, you wanted to ask about the overhead of the non-cookie identity solutions

Wendy: The W3C is critically concerned with keeping the web open for all

Michael: I think you are exactly right why we at Chrome are interested in the long-term health of the web
... I am the bird keeper
... listening to discussions
... Steve pointed out Prebid User ID Sub-Modules 19 ways to get ID

<Jukka> I see a market for one-time email addresses (like Apple ID) springing up :)

Michael: but not satisfying for many reasons
... any way to get overhead
... on loss of ID solutions; how much that costs
... if cost due to dropping third party cookies if the minus 50-70%
... then "the bird" proposals have a lot to make up
... how much of that loss must we regain
... where we use third party cookie workarounds?

Garrett: the more the merrier
... the comments I would ad to that
... from outside of industry
... if you replace things with email
... my naive perspective is you are putting same data through pipes
... where it's interesting is extent to which publishers and adtech will share across each identifier
... if that is the case, we expect the benefit of cookies...will be less so
... that makes "the birds" task easier if you are sharing for these first parties
... vs. cookies with all these problems

Wendy: We are nearly at the break
... a couple more on queue

Kris: Respond to Sam's question on volume of email identifiers
... Google and FB support sending audiences to them using PII
... Salesforce does that and has partnerships with both of them

<kleber_> Thanks again, Garrett. Also please restart your copy of Chrome. That red "update" arrow in the top-right corner makes me uncomfortable

Kris: not using cookies but uploading emails
... and translate to own cookie space
... programs support other PIIs such as phones and addresses
... we have seen it is 100% off the email addresses the things that match

<Garrett_Johnson_> Thanks @mkleber_ , didn't notice that! Will update.

Kris: large walled garden players cannot identify those; very focused on the emails

bleparmentier: we sign in
... will be easy and superior
... will publishers start to bother users to get emails
... because the birds are too hard
... TD and Sparrow will succeed if we can go to a site

bleparmentier: I hope it will not be GDPR
... click and give address, email, blood type, wife's name before you go to a web site
... there will be some workaround
... if we do good work on the birds
... if we don't do a good job
... there will be the 50% drop; if we get back 40% good; if only 10 we lose
... say that question is how much a publisher will ask
... and impose on a user

Garrett: are we at time, Wendy?

Wendy: yes, we are
... We have a 30-minute break
... people are welcome to stay on the line for some socialization
... if you want to take a rest, get some food, please rejoin us at top of the hour
... for topics of optimization

<wseltzer> [break]

<wseltzer> THANK YOU KAREN!

<kleber_> @Karen you're awesome

/me blushes; tries to keep up but some people talk really fast :)

<wseltzer> [return at 1700 UTC]

<AramZS> For those interested in the user impact, experience and feedback around ad tech methods, we will def be digging into that in the upcoming session.

<kleber_> FYI, the increase in users clicking X to complain about the ads they see is from the final paragraph of the Google no-cookies A/B experiment whitepaper https://services.google.com/fh/files/misc/disabling_third-party_cookies_publisher_revenue.pdf

<kleber_> Garrett: The speaker is Wendell Baker from Verizon

<weiler> scribenick: weiler

Privacy and Optimization

<kleber_> @hober sorry I missed what you said during the break — bad time to run out to get coffee. Did you signal support for any particular use cases?

aram: we've had conversations re: parts of ad stack, use cases. all of us have some degree to which we want to say we represent users
... but different perspective.
... recent Q's re: optimizations and priv in ad tech.
... We've missed some perspectives on root causes. what users are seeing, perspectives of general community.
... I've invited people who ahv been doing research and reporting on this.
... Harriet Kingby, former Moz fellow, chairs of Conscious Advertising Network. completed report on @4.
... Aaron Sankin, The Markup.
... Shoshana Wodinsky, Gizmodo

harriet: I am cochair of conscious ad network. voluntary coaltion, ~90 orgs

<wseltzer> https://www.consciousadnetwork.com/

harriet: mission statement of helping ethics catch up with the tech. we believe adv. is negatively impacting society. six manifestos re:

<AramZS> Hi all, here is a list of our speakers, some of the work they have done and additional resources - https://docs.google.com/document/d/1BZrAXjydtzzufDWOaPAS1oDKYsAHmic5juNUxKVnW4s/edit?usp=sharing

<wseltzer> CAN Manifestos

harriet: @7. Advertising inadverently funding @8.
... six areas of concern. want to take conversation beyond privacy. real world impacts... want to reframe issues.
... What happens when you include two forms of AI in ads? ML - since currently incorporated. and emotional recognition - on the cusp of being involved.
... @9 used to identify peopel for advertising.
... Run this through @10 to take a consumer perspective to impact how this might be impacting trust in adv.
... Severn harms identitied. discrimination, restriction of choice,
... lack of consumer agency - can't chose vendors processing datas.
... looked at scams - how could make scams more targetred and in theory more @11.
... and looked at @12 [people not online?]
... and environmental impacts. and hate speech.
... tech to infer secual orientation from face.
... Report makes recommendations for cross-functional forums like CAN.

wendy: would love links to these materials.

aram: links in the doc I shared.
... add more in IRC.

<wseltzer> Aram's overview doc with links

aaron sankin: reporter with The Markup, a nonprofit investigative news org, launched feb 2020. do data-based

aaron: reporting on privacy issues, algorithmic accountablity.
... looked at how @13 are psreading their message. Now adtech.
... Project "Blacklight" ... you're talking re: attribution... blacklight looks at data collection. loooks at where data ia going - give users a chance to understand what is happening.
... @14 built this.
... enter URL, blacklight does tests as a headless browser, looks a t attempts to set cookies, what domains get data.
... looks for canvas FP, keylogging.
... links via duckduckgo.
... w3.org came up very clean. jetblue.com does much tracking.
... used this at scale to scan 80k of top 100k urls on tranco list.
... most have 3rd p tracking, most google
... I used blacklight to scan sites serviing privacy-sensitve groups, e.g. undoc immigrations, abuse surfaces, sex workers, LGBTQ people.
... govt sites re: reporting child abuse.
... Big take-away: many sites didn't know what their site was collecting. things came in as presets from site that built site.
... Showed me site ops are just as confused by ad tech infrastructure as end users.
... These aren't sites highly invested in ad tech. most not doing major user tracking. no big ad campaigns.
... showed me that current adtech instructure, which informs how web is set up, to encorage much tracking, in ways these site ops not comfortable with
... they had to make a devil's bargain - send data in order tohave robust, functional site.
... site ops uncomfortable w collection; not particularly savvy. from users, generally creeped out.
... could put in plug-ins, but breaks websites, gives you less relevant ads.
... users are asking "what can we do" and "how do we keep ourselves safe". answers not very fulfilling.
... Orgs that scan own websites uncomfortable - they don't see benefits to themselves.
... Lots of confusion. from users and site ops.

<dialtone> oh, ok

aaron: Also looked at how sites interacted w/ people in bad ways. e.g. targetting people interested in pseudo science w/ ten-foil hat ads.
... happened to me.

aram: q's at end.

shoshana: I report on priv at gizmodo.
... In trade press, don't talk much to consumers - use panels, surveys. many issues boil to transparency + choice.
... Current ecosystem provides neither, even if we tell them we do.,
... When I was an adtech reporter, this came up every day.
... WSJ in Feb: cell location provider had a side biz selling data to feds for immigration enforcement.

<kleber_> (whew, I was worried that was *my* computer with a low battery for a minute...)

shoshana: Police could bypass warrants, since it was commercial data.
... Editor asked if we could figure out what apps people were using at border. Gravy Analytics - the location provider - they worked with liveramp and other DMP's.
... after a month, told editor - fundamentally impossible to say which aps are sharing data. that's terrifying.

<wseltzer> [DOOH=Digital out-of-home]

shoshana: I reported: I found was being targetted w/ Instagram ads re: my AD&D. tried to find out where it came from: GoodRx. They were sharing details with Braze (CRM povider). med, dosage, pahrmacy.
... ultimately, not sure where it came from.
... folsk on consumer side don't understand how convoluted this landscape is.
... because dig ad is so unregulated. this is completely legal. and as a result you have people targeted w/ ads based on mental health, and the border issues. Law is behind
... People like targetd ads but what if that data is shared with the feds? employers? or involves sensitive topics.
... As for choice: chrome "bug" deleted data EXCEPT from gogole sites.
... Pattern of bad practices. [see slide]
... Left with getting targetted ads, not by choice, even if telling them otherwise.
... Hard to know who's doing bad stuff. Is there a way to totally opt out? A: no. fundamentally impossible. could be tripped out by ad id; data in bidstream.
... very fragile system. because I used to write re: adtech and couldn't opt out, end user w/ no fundamental understanding .... it's impossible. even if you say there's choice, if you make a choice impossible, no choice.
... People want to know where data is going. they want opt-out to be meaningful, and they want to be spoken to like actual human beings. need to bridge dividie between industry and public.

swodinsky@gizmodo.com

<wseltzer> scribenick: wseltzer

<weiler> aram: this came together quickly. blame re: irregularities

<Zakim> kleber_, you wanted to ask Aaron about Google Analytics first-party-identity and "tracking technology"

kleber_: thanks to all three of you
... to Aaron, we've spent lots of time distinguishing between first and third parties
... I was surprised in the Blacklight report to see Google Analytics lumped with Ads
... since Analytics doesn't cross site boundaries
... it creates ad audiences on a fraction of sites that use remarketing
... the extra 24% of "google tracking technology" is single-site, I think
... I'd be interested in your thoughts on usefulness of that distinction

Aaron: I don't think people distinguish between first and third parties
... or understand cookies

<jrosewell> +1 to Aaron that people don't understand difference between first and third party cookie

Aaron: The way we set up our system does lump all Google things together
... building on Duck Duck Go's tracker radar

<kris_chapman> GA does also drop the third-party doubleclick cookie too to support those ad use cases

Aaron: good point re where GA fits into larger ecosystem
... GA is siloed, but if enable advertising freatures, drops DoubleClick cookies
... even for me took a long time to understand
... Complicated for site operators
... e.g. talked to ProtonMail, who didn't want to let anyone onto their site, rolled their own analytics system
... other examples
... internal policy controls at Google as to where data goes; can be difficult for the public to understand

jrosewell: I'm reminded I'm a statutory director of a UK company, with GDPR fines
... to Aaron, is there any difference between European sites and other jurisdictions?

Aaron: Blacklight is a headless browser, doesn't navigate GDPR consent forms
... if we hit a site, and no cookies are dropped, comes up clean
... suggested lots of instances where cookies aren't firing
... Some examples, HuffPost.com vs HuffPost.co.uk. US site drops cookies, UK site clean
... Talked to people who do consulting, including a few places with "cosmetic" GDPR consent buttons
... but many with functional consent buttons. larger question what consent actually means

jrosewell: thanks for the observation of some effect of regulation

<Zakim> dialtone, you wanted to Why are those sensitive sites putting up trackers?

dialtone: thanks all.
... Agree with the points re transparency, sensitive sites.
... As a company, we try to steer away from sensitive topics and ads
... for the societal impact.
... How do you square the theory of what people say they like/are freaked out by,
... with the majority of Hulu users who don't pay $4/month more not to see ads
... and GoodRX is an affiliate business, clearly funded through ads
... How do you square those perspectives?

<weiler> [a value exchange that you just admitted they likely don't understand]

Shoshana: GoodRX users may be needing discounts on drugs they need to survive
... privacy may be a luxury that not everyone can afford
... Some people are ok with that tradeoff; some don't like society being stratified
... Re disclosure to Facebook, people may not know how pixel works

Aaron: lots of Internet companies premised on giving things away
... sets expectations
... the same may be happening to site operators, who don't know what happens with free social sharing buttons

Harriet: tradeoffs between cheap hardware and privacy; we found patents for TVs that would use emotion recognition technology to tailor ads
... when we look at trust in advertising
... lots of components

dialtone: value exchange. There's a comparable product that lets you pay and offers more privacy
... at the end of the day, building services isn't free. Someone has to pay
... the cost is often filled by advertising, affiliate marketing, etc.
... Not clear we're asking users the right questions

Shoshana: I agree. A failure of communication not just from reporters, also from industry.
... that people don't understand what's going on

<jrosewell> Agree with Shoshana that people don't understand privacy or the value exchange.

Shoshana: no way to know that if you live at the US-Mexico border, a game app may share data on your family with authorities

Garrett_Johnson_: re Shoshana's reporting, sharing this sensitive data with third parties is problematic
... but targeting sensitive categories based on context is something we should be open to
... on-site, vs following to another website

<kris_chapman> regardless of advertising, I think the public needs to change their belief that the internet is free - or that anything they read online is true, too

<Garrett_Johnson_> Research on how GDPR affected 3rd party domain interactions on websites. Twitter thread: https://twitter.com/garjoh_canuck/status/1189530013567205377?s=20. Talk tomorrow info here: https://sites.google.com/view/macci-epos-virtual-io-seminar. Videos of past talks here: https://www.garjoh.com/research

Garrett_Johnson_: and post-GDPR, ^

AramZS: Deborah had a brief note ...
... and to Harriet, report on how personalization encouraged discrimination. Can you share examples?

Deborah_Carver: worked on agency and local news org sides
... Minneapolis has strong local news
... concern wrt ad tech that often our news orgs don't really understand the tech they're implementing on their own sites
... big gap between how news ads are sold and how digital
... and news orgs often end up placing whatever tracking tech ad sellers offer
... concerning, e.g., following Black Lives Matter protests
... lack of transparency around tech and middlemen

Harriet: Discrimination through ad system happens in a number of ways
... algorithmic tailoring is more problematic when relates to e.g. educational opportunities, housing
... if I'm not shown the same educational or housing choices someone from a different background gets, that can lower aspirations, discriminate

<AramZS> @wseltzer are we ok to complete the queue even if we are over time?

Harriet: Discrimination relating to content. Blocklisting of safe LGBTQ content, "Muslim"; so media for those communities is difficult to fund
... Demonetizing hard news about coronavirus just as we need it most
... need to do risk assessment about what kind of discrimination is inherent or could be created

<jrosewell> Harriet's point about transparency around automated decision making and use of AI probably the biggest tech challenge for this decade. Does the W3C have a group working on this?

arnaud_blanchard: along with how and to whom data flows, how much of the data
... variants of data and how personal it is
... explaining how much data is going where and for what reason is important
... Blocklisting is problematic because it forces content-producers to tailor content to what advertisers need
... minimizing targeting will increase this effect, I fear
... I don't think characterizing me based on what I read is fair to me or to the publisher

Harriet: Some organizations do place advertising on specialized content, as a counter to blocklisting
... my suspicion is that we don't need all this data all the time for tailoring
... checkmyads, we advocate for supply chain accountability
... because the supply chain is so complex, there's no accountability when something goes wrong
... CAN advocates, make better inclusion lists
... don't always rely on tech to solve the problem
... advertiser responsibility for where ads are going

AramZS: Thanks to our guests today, and apologies for going over

weiler: thanks Aram, and Harriet, Aaron, Shoshana
... to Valentino, your question about affiliate links, and lack of understanding...

valentino: some people are well aware and happy to save

<jrosewell> Shoshana and Harriet raised some good examples of unacceptable privacy violations. Drug prescriptions details, turning on cameras to recognise faces, location data, etc.

<jrosewell> Aaron explained that GDPR does show promise in addressing bad actions by bad actors. We can now clearly see that not all answers to privacy problems are to be found in tech.

AramZS: Thanks again!

<jrosewell> Sharing more directly identifiable personal information with a smaller number of larger companies isn’t a remedy to the privacy issues raised. Garrett recognised this in Q&A.

<jrosewell> Whilst reducing the number of participants in the market might make it easier to trace data that is passed between different entities it will obscure data processing in practice and reduce consumer choice.

<jrosewell> Focusing on strengthening the audit (supply chain accountability) and sanctions around data transfer and processing increasingly looks like the demonstrably best method of “Improving Web Advertising” which is the objective of this group.

AramZS: feel free to reach out to participants for follow-up, and I'd love to return to this conversation

<jrosewell> That means working on solutions that aren’t 100% about tech, but also about laws and policies.

<weiler> [it's hard to accept shifting that burden to individuals, esp. if they've already been victimized in other ways]

<dialtone> who should make the call though if not them?

<dialtone> we can't possibly claim power to decide if $2 is the right value for a GoodRx affiliate conversion for everyone

Magazine Media Mode

<dialtone> frankly, if some aspects of advertising are too sensitive then the regulatory body should step in and just forbid it outright

<weiler> [acknowledging economic network effects, it's more of 'why should they be forced to give up the social value of a big network (e.g. FB) just because they need to hide from someone. it's an imperfect question, but it highlights shifting the burden to the individual]

<jeff_> scribenick: jeff_

Wendell: WE talked about Media Mode
... added the word Magazine

<dialtone> afaik FB has privacy controls that prevent all targeted ads, but one isn't forced to use FB, I certainly don't use it, although I'm a sample of 1

<wseltzer> Wendell's slides

Wendell: TL;DR... simplification is the order of the day
... We've been doing this for a few weeks
... browsers; ad-tech crew
... we need to finish this up
... where will this end up in 2 years
... in my house: denial? money will come to us? doomsday models - 60-70% of the spend will disappear
... nuanced business models
... we thought about "Private by Default Web"
... what would be 2023/4 be like?
... we spun up after SD workshop
... been at this for 2 years
... trade, activism, browser people working this for many more years
... lots of expertise
... but what's the bigger perspective?
... there are things that will never change
... Here are some observations
... browsers are beyond huge
... need lots of money
... like semiconductors
... lots of people try to influence
... in semis - huge battle about putting an ID in silicon
... the activist side does not understand what this does
... the people who run the tech don't know
... system behavior is an emergent property
... you need experience to understand
... but the browser has been the thing that did not change
... once was superlative
... that's no longer true
... many use cases
... safety, regulatory concerns
... bad people breaking in
... look at CVE list
... media trades people are spending time working around the rule
... many levels of stack
... Aleecia MacDonald said that the stack is corrupt at all levels
... so the system is not working
... people are working around the rules
... for us to all get something happening
... need to simplify stack and look at some use cases
... focus on Magazine use case
... let's clarify the problem
... many studies (academic, etc.) do not effect what we do in business
... but some things do have an effect (later in presentation)
... dithering causes people to get stuck, so you need to move around a bit
... [baseball analogy]
... Our focus is lifestyle magazines with personalization
... need to think like publisher
... even people who run the sites don't understand
... like the Editor in Chief does not care how the printing press works
... so we call it the "HTML industries"
... how do you drive SEO?
... work your business into search engines
... attract and recirculate
... we had thought that the browser was an inert user agents
... for many years it was true
... now that has changed
... so you have a publisher mindset with servers
... you have clients
... and you have complexity
... many hands in the middle
... need measurement and metering to make it work
... "red" box is difficult stuff
... "green" is what we are doing
... Now let's look at other kinds of media
... there are others
... each are different
... more proprietary

<jrosewell> Business logic suggests the "bit in the middle" generates more revenue for publishers than not having the "bit in the middle", otherwise it wouldn't be used.

Wendell: don't always have groups like this
... the purple boxes near the ears of the client make all the difference
... once the inert boxes become less inert
... it doesn't work
... that's what happened in the last 2 years
... from 2 vendors
... inside of Verizon Media we say that the Ad use cases is a wonderful document
... that we laid out advertising needs and people acknowledge that they need to be solved
... and what's not on the list won't happen
... there is debate about the machinery will be allowed
... most oriented towards continuity with pre-2020
... but long list of attractions
... manipulation, moral attribution
... I look at whether they are real
... some are names of essays
... "content is king"
... means the audience has to want to come
... other industries don't have this problem
... they have primary stakeholders and others
... we have encryption and DRM on the web
... the rest are distracting
... we understand that this is years away from reality
... no code yet
... difficult to make plans for the next 2-4 years
... other issues: rights, consent, jurisdictionary specified UIx
... imagine - lawyers are involved
... confidential computing - can it be commercially viable
... not well matched with the current industry
... not shown yet
... only a few companies with sufficient PhD staff to make it work
... and SOX compliant
... most of us have seen the birth of the web
... but some engineers have not
... i.e. we are getting older
... the target of magazines are someplace else
... bundles, packages, applets
... newsletters are big deals
... changes the dynamic; traffic acquisition; identification
... so we are moving to PII; building arrangements
... gives you contract, legitimate interest
... audio and video beyond the web
... lots of time being spent there
... ad stuff will move there
... wrap up: 2 years ago we did user consent and permissions
... surprising interest on regulation side
... lots of conversation with browsers
... we want a general browser engine
... can do media
... can talk about machine learning
... someone says that browser does not know what is happening at layer 8 and 9
... so we need to shut it down
... here is a plea to build a browser that understands this
... tailor towards that
... final point: metering and measurement
... need to model around consent and permissions

James_Rosewell: Thanks for an excellent pitch for Gopher

<weiler> jrosewell++

James_Rosewell: it answers a question I've had since June
... this helped me understand your thinking
... what is the purpose of the web browser
... you highlighted an existential threat to the open web
... money will go elsewhere

<weiler> [seems like some interesting overloading of the term "open"]

James_Rosewell: I'll think about it. Tx.

Kris_Chapman: Thanks Wendell

<jrosewell> Gopher spec : https://tools.ietf.org/html/rfc1436

Kris_Chapman: +1 that browsers need to understand level 8 and 9
... difference between end user tracking v prescription for a particular drug
... need a signal what might be acceptable
... then system functions well.

Wendell: Penn and Teller talk (sp?) pointed out that people are turning up their arguments

<jrosewell> BTW Turn up to "11" was Spinal Tap

Wendell: a medical site might use data in some way hence can build the technology

<weiler> "

<weiler> ;

Wendell: in our shop we look at various things including jurisdictional
... if tech can't figure it out we need different tech

Aram_Zucker-Scharff: In terms of the moral dilemma
... the problem is not what shows up in the press
... but when we see something we can assume there are others
... should not dismiss single example
... should think about our business needs
... we serve our advertisers but also our readers
... if we continue to generate bad press as ad technologists
... that will have a non-quantifiable but relevant impact on our business.

Wendell: Hard to disagre
... I'm in the business
... I see it.
... Need to be legal, but then comes a gotcha
... What kind of apps do we want?
... we want to be in lifestlye magazines
... auto, finance, etc.
... end-terminal equipment not working for them right now

<wseltzer> https://github.com/w3c/web-advertising/blob/master/meetings/TPAC2020.md

Wendy: With an empty q, ^^ is agenda for tomorrow
... multi party computing for measurement tools
... measurement reporting API proposals
... progress, incubation
... gatekeeper proposals
... browser/server tradeoffs
... certification
... disaggregate privacy preserving tech
... today had great overviews
... think about next steps
... shared understanding of problems, landscape, research
... appreciate all of the participation
... presenters, questioners, thoughtful sharers within your orgs
... look forward to bring back for more discussion today and tomorrow
... thanks to Karen, Sam, and the other scribe for picking up the pen

Joshua: I enjoyed the presentation
... I struggle to come up with a tech that cannot be misused
... we don't remove rocks because they are thrown
... are there techs that never cause harm?

Aram: We need seat belts in cars
... but also people in cars understand the risks
... with ad tech we are seeing here, most consumers are not capable of making that car
... we want to (a) make technology safe and
... (b) we must give users the ability to understand the risk
... people understand cars
... in a Tesla and you use autopilot because you don't understand - the problem is the tech

<Zakim> weiler, you wanted to critique the analogy

Sam: Take apart analogy about rocks
... I don't keep my guns at the street

<Garrett_Johnson_> FYI, I posted a twitter thread on my earlier remarks: https://twitter.com/garjoh_canuck/status/1318989352534478851?s=20

Sam: so I don't give my tools to the street to abuse them
... (Something about building codes)

Michael: We have items for tomorrow to address this
... multiparty computation; gatekeepers
... answer how do we build tech that does not cause harm
... not a guarantee
... but it asks what better we can do
... active discussion
... also must acknowledge the policy component
... technology does not get us the whole way
... end up with safer collection of tools

Paul: A step further from Michael
... we are thinking about cross-site tracking
... when we hear Shoshana and Harriet talk about concerns
... these harms will still exist in the future
... may exacerbate them
... advertising will be less effective
... strengthen the tech giants

Kris: We talked about how the general public does not understand
... ad tech has poorly explained
... not taken privacy seriously
... data privacy is important
... needs to be education for general public
... Internet is not free
... not everything online is truthful
... people need to be skeptical and understand the economics.
... market that to public.

Brad: Better analogy is leaded gas
... improved tech with same benefits
... also when we deprecated plug-in APIs
... plugins were causing unintended harms
... instead we built out the capability of the web
... build new APIs without use cases

<AramZS> I think there are lesser and greater vulnerabilities we are opening people up to, and removing cross site targeting can lessen that vulnerabilities. I wrote briefly about how the problems of ad tech are of scale and specificity, not a perfectly zero-able problem - https://twitter.com/Chronotope/status/1270508875989561345

<AramZS> Thank you to Wendy for moderating!

Wendy: Great place to wrap-up
... tomorrow we focus on the APIs
... improvements to ecosystem
... looking forward to seeing you

<wseltzer> [adjourned]

Wendy: Thanks very much

This is scribe.perl Revision of Date 
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/Holkower/Hulkower/
Succeeded: s/@/what's changing is our/
Succeeded: s/Arnaud/bleparmentier/
Succeeded: s/@/elsewhere/
Succeeded: s/with targeting/without targeting/
Succeeded: s/@/emails, which are PII/
Succeeded: s/shape/Shave/
Succeeded: s/@/Quaker Oats/
Succeeded: s/tion/tino/
Succeeded: s/data@@/data-poor/
Succeeded: s/DDPR/GDPR/
WARNING: Bad s/// command: s/blind
WARNING: Bad s/// command: s/informative
Succeeded: s/@/linear/
Succeeded: s/@/cohorts/
Succeeded: s/sending to/sending audiences to/
Succeeded: s/@/bother users to get emails/
FAILED: s/too @/too hard/
Succeeded: s/bid.@.../Prebid User ID Sub-Modules/
Succeeded: i|Chair: Wendy|Topic: Intros
Succeeded: i|Ready to go, Steve?|Topic: An attempt to demystify advertising
Succeeded: i|Garrett's slides|Topic: Economics of Digital Ad Identity
Succeeded: i|many of the questions that came up will be addressed|Topic: Economics of Digital Ad Identity
Succeeded: i|means these contextual solutions|scribenick: Karen
Succeeded: s/optimiations/optimizations/
Succeeded: s/@3/Conscious Advertising Network/
Succeeded: s/@5/The Markup/
Succeeded: s/@6/Shoshana Wodinsky, Gizmodo/
Succeeded: s/sec workers/sex workers/
Succeeded: s/ment/men/
Succeeded: s/yeard/years/
Succeeded: s/pitch/pitch for Gopher/
Succeeded: s/Pen and Tyler/Penn and Teller/
Succeeded: s/causing unintended/plugins were causing unintended/
Succeeded: s/scribe+/scribenick: jeff_/
Succeeded: s/Improving Web Advertising BG/Improving Web Advertising BG vF2F, day 1/
Succeeded: s/Alicia/Aleecia/
Succeeded: s/ birds are too bother/ birds are too hard/
Succeeded: s/start @/start to bother users/
Succeeded: s/too hard users to get/too hard to get/
Present: bleparmentier dialtone ionel sharkey-cafemedia mlerra wseltzer Karen kris_chapman pbannist wbaker_ mjv blassey deepak joshua_koran shigeki kevinG [100people]
Found Scribe: Karen
Inferring ScribeNick: Karen
Found Scribe: weiler
Found ScribeNick: Karen
Found ScribeNick: weiler
Found ScribeNick: wseltzer
Found ScribeNick: jeff_
Scribes: Karen, weiler
ScribeNicks: Karen, weiler, wseltzer, jeff_

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)