Meeting minutes
<kleber> Happy New Year, everyone
<gendler> Thank you, happy New Year to you as well
Wendy: We'll start by looking over the agenda
… agenda curation and introductions
… the Augury API and Scaup API that Gang introduced us to last time; we'll get to discussion
… perhaps a visit to our dashboard for highlights
… any other business?
<Gang_Wang> Wendy, Ardian from Google will talk about Augury API first. Then I can discuss the Scaup API
Wendy: thoughts to share in coming weeks; as we launch into new calendar year
… we made lots of progress last year; hope this year will help us to identify more promising areas to take into the standards process
… perhaps to charter new working groups, or take some of work to propose to existing working groups where it might be in scope
… Do we have any introductions? Anyone joining us to introduce themselves?
Augury API and Scaup API
Wendy: Let's move on to agendum 2
… Gang, you have us an introduction last time; any questions you have seen in the mean time to raise?
Gang: thanks, Wendy. Happy New Year everyone.
… I have not seen anyone filing Github issues for Augury or Scaup
… we could talk about more details with Augury API
Ardian: I am co-author of Augury API
… share screen?
Ardian: thank you, happy new year; I'm from Google Ads
… first, the object of proposing this Augury API
… to help TD adoptions
… with cookies going away, it will take time
… for businesses to adopt
… a lot of companies from buy and sell side
<wseltzer> https://
Ardian: we want to simplify the implementations
… we will discuss a little later
… I want to highlight, this is not meant to be a final solution but a temporary solution to cope with current business as good as possible
… a lot of proposals are lying around
… could be something different
… or a @ of the Augury API if adopted
… My first slides was objective of API
… we want to get feedback
… introduce basic form of API
… we use basic form to understand what is capable
… and second is to get industry feedback
… whether other SSPs and DSPs are interested in API
… and should Google invest more in profiling this API
… and if others are willing to provide similar framework
… and second thing, if this simple API has deficiencies, then what type of functionalities and enhancements are needed
… background on why we started Augury API
… last January we started in Google Ads
… it is hard to implement Turtledove
… looking at best ways to do things
… some aspects are hard to implement; require a lot of compromises
… the first is bidding; there are a lot of proposals floated about bidding; still it is a challenge
… look at NextRoll or RTB House proposals, accurate bidding is important
… ML in browser would be expensive
… if we don't ship @ Mozilla browser [didn't catch]
… second thing is the enforcement
… have not seen this being discussed a lot
… finally time for advertisers to @ ad in the site
… a lot of advertisers...placements daily
… exposure of publishers, advertisers' information
… everything implemented in JS is exposed to browser
… we can try obfuscation but there is a limit
… proposal by NextRoll worries about this issue
… if model is downloaded in browser
… different requests, Interest groups
… see what model is doing
… number four is about billing and auction integrity; also IP
… final bid is computed as JS in browser
… it would be hard to ensure that the bid is limited
… there are billions of browsers in the world; don't know which is getting compromised
… and be beneficiary of @ some network
… we need assurances
… other challenges with experimentation
… Here is proposal of diagram for the Augury API
… instead of computing the bid in the brower, what we do
… is try to move a lot of this computation
… back to the server
… essentially, idea here is
… during ad request
… profile should come up with IG
… with particular IG
… can do what we have
… context informaiton...so we can do a lot of things
<kleber> w/with IG/with predicted IG/
Ardian: we can compute a bid
… can apply rules
… bid cannot be less than some amount
… and here in step five
… SSP sends to the browser, list of ads
… from this flow it is very clear
… that we may have some mispredictions, some false positives, some false negatives
… not sure if any candidate from IG can appear in the browser
… we can see a lot of complications on the logics
… and applying
… SSP and the systems
… Last slide
… when we proposed stuff, some questions
… how to do publisher/advertiser combinations
… profile doesn't want to have limit
… do a lot of things; do exploration
… one thing that we came up with
… one bit that can cover multiple advertisers is one possibility
… also similarly with how ML does
… ML model with rules
… with @ rules impossible to cover every advertiser and publisher
… a lot of them, rely on some other features to compute
… the bid
… can be done similarly
… ML model
… fall back to higher representations
… @ necessary bits if needed
… modifications
… bidding may be one of less ones; lots of proposals there
… without bidding the API will still be useful
… maybe a @ of IP protections; we may be able to offer that as well
… Let me stop here for questions
Wendy: Thank you Ardian
Arnaud: thank you for presentation; my main question is what you plan to measure through this test?
… not something that will be implemented for a long period of time
… if a test, what is outcome and how should it influence design of proposals
Ardian: Good question
… the Augury API was built to be compatible with TD
… if we somehow put aside the uses of third party server
… it still has a lot of issues that I described before
… Augury API may be useful for those variations
… questions for how to measure success
… we want to see if other buyers and sellers
… are interested in that
… we are thinking that we may
… if we have enough interest; to offer API for testing
… let others use it
… there may some changes needed
… based on experimentation we can have better idea of solutions
Arnaud: If I understand correctly, you see that a way to support TD
… I guess something else to consider, it is very far from TD proposal
… not much happening in the browser
… not have delays.
… Consideration of performance and resources dedicated to advertising
… could be taken has a result
… help flexibility of TD proposal
… I would be cautious not to run that as TD because it's not; it's something else
… that looks like it
… it's far from TD and Sparrow
… Happy to see how you guys
… something I would be happy about is to see how we can use these tests to improve the solutions, whether TD or other
Ardian: you are exactly right
… we use TD as the main...
… was being proposed; a lot of things concepts are not in TD
… we are not saying that we are committed to TD
… from Google Ads POV, this is one way to implement TD
<btsavage> Is there a link to an explainer?
Ardian: not saying that TD is being implemented
Brian: A very interesting adjunct to TD proposal
<wseltzer> Augury explainer
Brian: curious about expectations about number of Augury bids that might be returned to browser
… and how much effort to put into points in the chain and doing selection from that list
Ardian: a good question we have discussed
… there are some limits about this size
… first is from seller side
… @ limits
… maybe not as important as with Augury API
… maybe put some KB limits
… and limits from browser; may be more important
… ignoring modification possibilities
… when sell side processes Augury bits
… we only need to send...the highest bid
… we cannot release all the numbers; but numbers not that many
… we should be able to handle a couple of kilobytes
… I think it's less than ten KB
… we build on TD
… number of Augury bits not that expensive
… maybe we can handle hundreds of thousands of bids
… find whether any of Augury bits are inside user browser
… JS or computation that was proposed in TD one
Kleber: thank you for this proposal
… I think I'm going to start by disagreeing with Arnaud who described this as "not TD" as a wholly separate proposal like others
… it seems to me that this proposal is a way for DSPs and SSps to use the mechanisms already inside TD
… strip TD into its basics, browser holds onto the IGs, and what ads those IGs would show
… then some ondevice computation for the bids
… this Augury proposal is about the servers
… making a prediction about which present in browser
… and sending pre-computed bids
… browser knows what IGs a user is in
… opportunity to create bids that get ranked and chosen
… fact that those bids are guesses returned by server
… the fact that a given IG on the browser makes it bidding not by a complicated ML model
… but looking up if in a list of IGs guest server side
… seems like a reasonable way to use TD
… and doesn't require substantial changes
… or separate out DSP and SSP world
… how many guesses can be sent back
… guess how many IG a browser is in, are the right kinds of questions
… sending back thousands of bids that take up a few K worth on network
… would not strain anyone's resources
… seems like a great way to move computations server side, if good enough to predict
Wendy: Aram
Aram: I guess I'm curious about how the bid is determined
… course matching keys to eliminate the scale issues
… a lot of controls publishers could put on
… might limit those controls and move to SSP
… curious how this impacts those capacities
Ardian: very good question
… with coarse bid
… you are right
… one of things we are thinking about
… if the bid is coarse
… ads would have similar controls
… other than we mean it to move back some of logic back to browser
… type of feedback we are looking for
… if we have more discussions on this it will be clear
Aram: we have fine-tune controls over the size of the bids; we want to be able to continue to have those fine-tuned controls
Erik: thanks for presentation and proposal
… I have two questions
… one, seems the biggest difference from TD is the contextual and the Augury candidates are delivered together
… so there is a little contextual info that goes into Augury candidates like brand safety
… as Mike pointed out, some IGs, and some contextual info, maybe the domain name it's valid for
… make sure I'm understanding that correctly
… go ahead
Ardian: thanks for question; you are exactly right
… we have some contextual info to do modeling
… having information
… can have info flow through from context...
… more about moving computations server side to force @
Erik: but browser not only validates user is within the IG
… but that it's within the IG and that the bid for that IG is valid on some context
… you need slightly more info for that to work
Ardian: I don't think so
… in context world, we have what's needed and we have the IG
… it should be @...if done correctly
Erik: a bid for an IG would be valid on any web site if I tell browser
… any site where that bid wins is valid
… and we are saying this should only be valid for social.web site
<kleber> Erik: I think the point here is that the Augury predicted bids are just used on a request-by-request bases — there is no caching or reuse
Erik: on some other publisher.example, the foo wins
… has not been validated for that context and we don't want it to win
… so slightly more validation
Ardian: validation does not happen in the browser; the Augury bits sent by server
… inside any of bits inside server
Erik: bids don't persist as they would inside TD proposal
Ardian: right
Erik: you only send thousands....contextual and Augury bids arrive at same time, you have to download the ad asests
… as soon as that happens you have a pretty clear timing attack
… I know that this user is downloading this ad and thus is in this IG
Ardian: this Augury proposal is trying to implement the bidding logic of TD
… assumes you are using TD framework
<joshua_koran> Ardian: I believe the point is bidding logic requires a feedback loop as to what is working relatively better
Ardian: in TD refresh in browser before contextual ads; using same principal and framework
… already have the ad; that's the idea
… some Augury bits...what does not match we don't care
… who is actual winner, we already have ad bundle
… follow TD mechanism and render in browser, no link
Erik: ok, that clarifies
Wendy: We have a long queue, so ask people to try to keep Q&A concise
Arnaud: if proposals are similar, do you expect same kind of performance? I would expect not to be similar
… if you include that it's most important
Kleber: I see them as different types of things
… TD as platform, Augury as one way to possibly show ads
… all different things that ad networks could choose to do
… I think that TD is infrastucture and Augury is more like piece of engineering built on top of TD
Arnaud: does Augury... add performance?
Kleber: browser would not know that Augury is happening
… no way that browser knows or cares about Augury as way of using TD from some other way of doing bids
… using server side prediction for how on-device bids are created
… all the infrastructure of TD stays the same
… where SSps and DSPs adopt Augury as a way of showing bids
Basile: just a small remark
… I feel that in Augury you expect to predict IGs that are present
… if you can do that, contextual is already
… I have impression is one issue I have with Augury
… it will help places that do need user data, but not user content
… go on @ web site
… very hard to do
… web site which needs advertising
… if it is very low content
… web site will need a little less than the one that does need it
… information is already @
… will help more web sites to miniplace
Ardian: add here
… from idea we may be able to understand hundreds of thousands of bids
… some signals may be there; proposal like FLOC
… tens of thousands of users, should be able to narrow down as well
Basile: thank you
… even so, I think it's quite complicated; not benign
… would need computation on server
… sending one bid...or one thousand
Ben: thanks for the proposal
… question for Charlie
… explainer says suggestion for how to use the IGs is to use the aggregate reporting API
… that makes sense, but it's not a capability I understand how to use
… my understanding might be wrong
… do a write only endpoint?
… can Charlie sketch out what SK code might look like...to collect stats on top @ on a web site
Charlie: a good question
… I think the basic idea
… I don't think we have a concrete proposal for how to do this in JS
… the generic aggregate reporting is something we can give as a new web platform capability in some of these fenced environments
… of IGs but not exposing all that info to the page
… not 100 percent as familiar with the Augury proposal; maybe Ardian and Michael can chime in
… as long as you have some context...have access to this data
… reasonble for you to collect some aggregated measurement to get that data on your side
Ben: reasonable and sensible to enable
… write data that you cannot read in browesr
… intersting to sketch out how a late bound
… assign key value to IG...proposal for what the JS code could look like would advance the discussion
Charlie: similar to the SPURFOWL idea with write only storage
… and cannot communicate with outside page but help to craft reports
Ben: related concept in writing data
… there you have info you knpw; you know a click or impression happened
… this is different; browser has secret device info you cannot read
… assignment of unknown variable
Michael: Like SPURFOWL's trails, the browser does know all the information, just not all at the same time. At one time the advertiser joins an interest group, at another time...
… browser knows what page user is on; figure out how to join them up for reporting
… philosophically feels like use case for infrastructure and SPURFOWL seems like a plausible approach, but not only one
Joel: One comment
… building upon what Basile said
… to put some numbers to it
… a DSP might have millions of advertisers and each might have a dozen campaigns with a few dozen creatives
… and each of creatives has millions of products
… seems like a large space to guess what might be in the browser for popular sites, like weather sites
… i think for popular sites it would be difficult to hit the IG
… something like a hundred thousand bids return, you would not have a good chance of hitting
… you might receive a FLOC IG that could help us predict
… do you think there are other browser signals beyond publisher stuff that could help with this predictive task
Ardian: Browser signals...ask Michael to answer this
Michael: how good a job you do server to predict IGs is the most important question; using FLOC or coarse screen are valuable things to think about
… not sure what other types of signals are being discussed
… others are difficult because they add to the finger printing
Joel: I agree with that
… but to make this predictive, some additional signals may be required
Michael: for product based TD proposal
… exact products user has seen you would not have to predict
… a coarser grained IG with products placed inside of it
Joel: that is fair; might be a calculation
… talking abou millions
Michael: agree; one of challenges in using Augury
Wendy: we are getting to end of our time
Joel: understand what ad is most valuable; TD makes that hard; only have access to half that info
… Augury is interesting; helps us to price, helps publisher know what is most valuable
… send back bids that are higher than contextual
… is there a case when you might send zero
… might be a hack for pricing
… a mechanism
… would you see world where only these bids are eligible if @ present
Ardian: one second
Wendy: Joel, are you still with us?
<joelmeyer_> sorry - lost internet
Jonasz: one brief feedback from RTB House
… we care about bidding accuracy
… we don't see a way to retain accuracy of bidding under Augury
… if Augury is one way to use TD, if someone willing to go extra mile and implement bidding logic, then still possible to use Augury API
Ardian: thanks for the feedback
… as I mentioned in beginning, one way to get feedback
… useful, my understanding that
… most critical is user signals
… some way to handle that
… second thing about other ways
… that is based on SSP
… building a @
… seems to be a hard problem
… If SSP cannot guarantee an option is....
… support general API
… computing bids in browser
… SSP support rather than question of API itself
… SSP support for multiple bids
… then it's possible
Wendy: seems Joel lost his connection
… maybe we can get back to that
… sounds like you have lots of potential issues and comments for people to raise in Github
… Invite people to keep up the discussion online in the interim
… Gang, did not get to Scaup
… we maybe we can bring that back in the future as well
… Anyone is invited to send additional agenda requests and proposals, and other items for future consideration
… heard a suggestion for talking about a roundup of where different proposals are in their development
… discuss how to best represent that with our tools
… and determine which should come back for another round of dicussion
… Thanks again for another full meeting
… See you again on the 12th of January
… We are adjourned
<wseltzer> [adjourned]
<wseltzer> karen++