<peterswire> 404 area code is peterswire

<WaltM_Comcast> 215-480 iw alt michel

<Yianni> thanks

<kulick> yes

<npdoty> scribenick: susanisrael

<aleecia> (my regrets this week -- buried in other work.)

<scribe> scribenick: susanisrael

peter swire: will do administrative matters until Mattias joins

Peterswire: planning to have meeting april 10 re: user interface. Have not yet seen proposed agenda items for this. Please post to list in advance of call

<Chris_IAB> I believe Alan Chapell has something for that agenda Peterwire, but he's not on the call yet.

Peterswire: if you want something on agenda post it by last date of march. We'll do things within the scope of things posted then.
... questions or comments? [none]

<npdoty> one of the Microsoft background numbers was providing static, hence muting; let us know if you want to speak

Peterswire: next point, no. 6, on agenda follows up and provides more language on item raised last week...will go through....
... need to recognize interdependcies in compliance spec. Many people are reluctant to say issues are truly closed because they are afraid they'll be permanently agreeing.
... difficult to create new category of things, but we will create subcategory. At times, we have talked through something as much as we need to now, and text is stable pending whole package, we will call that .....

<scribe> ....pending review - Stable. we would then add a note to issue tracker categorizing it as such. we might have one text or alternatives that are pending review-stable, meaning we won't discuss them more now....

UNKNOWN_SPEAKER: similar to re-opening a closed issue. This requires real concrete text supported by more than one person.

rvaneijk: where would be the transparency of visibility into status of all items? Maybe it would be good to highlight it within the agenda too......
... if the agenda comes out a day before meeting, we may overlook these items.

<npdoty> scribenick: npdoty

<susanisrael> peterswire: editors of compliance text are now working on updating the bare bones compliance text, partly to alert people where text is stable.

peterswire: next thing Rob is asking, the day before (for agenda) is not a lot of time
... trying to assign things well ahead of time, encouraging people to post their language further ahead of time

<rvaneijk> ok, thanks

peterswire: in my own work, trying to find ways to see what's coming

<susanisrael> *nick, sorry I can pick up again

<scribe> scribenick: susanisrael

Peterswire: financial auditing slides and audience measurement text went around to list
... do we have the right people? [richard weaver and george say yes]

audience measurement

peterswire: can someone paste into the list the audience measurement text

<npdoty> http://lists.w3.org/Archives/Public/public-tracking/2013Mar/0335.html

peterswire: very large players, the biggest companies in this space have contributed.....i think that's positive.
... there is talk about the expansion of self-regulatory efforts, and this is a very serious professional document that was given to us.
... will convey some concerns i have heard then open to questions

<justin> +q

david stark: we looked at definitions of pseudonymous and didn't like [ ] definition

<Wileys> s/[ ]/ICO

<rvaneijk> comment......

scribe: given our understanding of de-identified in dnt, we thought that would be confusing.

<rvaneijk> please want to react....

<Wileys> +q (for Rob)

<jmayer> +q

<Wileys> -1

<Wileys> -q

<rvaneijk> peter, queue please

<rigo> ack (for, Rob)

<jchester2> key question, thanks Peter

peterswire: a related question, from cambridge, you can have cookies, do those need to be scrubbed out for pseudonymized.

<jchester2> But you know device

richard: we're talking about cookie id. we are not looking within other cookies for audience measurement. That's irrelevant.

<moneill2> +q

<johnsimpson> you're putting unique identifiers on a k ow device

<justin> Presumably it would be a requirement that the research firms couldn't put email addresses in cookies. There's a related question if the referer urls include unique identifiers like UDIDs. There could be a requirement to have a process to strip those out in short order.

<efelten> +q

peterswire: up to now, dnt standard has not had 3 stages. had de-identified and not. So one difference is this intermediate stage, pseudonymized. some people favor this and some don't. I favor.

<npdoty> if you use my social security number as my cookie ID but don't attach my name....

peterswire: another question was length of time, 53 weeks. we had a presentation from mrc that talked about the similar length of time for auditing. They also said at that time that they have given privacy waivers....

<npdoty> the definition as I read it would include device IDs as pseudonymous, justin

for shorter times, such as 60 or 90 days. Is this linked to MRC code? or from a different set of considerations.

richard weaver: MRC in US, but many different auditing bodies. 53 weeks is global standard.

<justin> npdoty, Not sure I understand your point. Yes, a random cookie is pseudonymous. SSN or email address or name would not be.

<Chris_IAB> peterwire, I think there is some confusion about this idea of MRC "privacy waivers"-- I'd like to clarify

<rigo> Chris_IAB: there are many people wanting to clarify many things here :)

peterswire: most campaigns are 90 days or less, though some are last valentines day or last christmas. How easy or hard would it be to have a presumption based on shorter time....

<Chris_IAB> rigo, that's why you see me on the que

<rigo> :)

peterswire: with different approach for subset of campaigns that are year over year audience measurement.

richard weaver: I actually would not say that most campaigns are shorter. could you explain different approach for longer?

peterswire: idea is to compare in some cases to annual holiday runs in previous year.

<npdoty> justin, what makes an identifier pseudonymous and not identifying? my device's UDID can correlate traffic, and for some people it can lead you back to my name, but for others it would appear to be a random pseudonym

peterswire: if we are trying to show we have done good measurement, we might imagine that for shorter campaigns/comparisons, we might have shorter retentions, but not for all.
... for shorter campaigns, the 53 week retention seems much longer than the length of the campaign. Is it necessary for calibration?

<Wileys> Nick, that is a key test for pseudonyms - the holder does not have the ability (technical, policy, process, etc.) to link the identifier to personally identifying information

richard weaver. initial thoughts: auditing might not take place immediately after campaign, and you might also compare several short campaigns.

<justin> npdoty, You could put in a requirement that the pseudonym be party-specific. Though not sure what you do with IP addresses then . . .

<peterswire> we will go to the Q once I do one more topic for question

george: premise, i guess of most campaigns are shorter....i am kind of aware of cookie data being used for both.

<efelten> WileyS, actually the proposed definition for "pseudonymization" says they *do* have the ability to link to an individual.

george: I'm not sure about the numbers, i.e. which is higher value, regardless of number of projects that may be categorized as short-term vs. year over year.

<npdoty> right, WileyS, justin, I was just pointing out that the proposed definition in Kathy's email doesn't have those restrictions

<justin> npdoty, Yes, I was just assuming that would change :)

<Wileys> "identifiably individual" vs. "unique browser" - meaning I know the identifier is consistent for some period of time but I don't the identity of the individual the identifier is linked to

peterswire: last point has to do with opt out. In your approach it provides you with an opportunity for opt out of data collection.
... we understand that today an independent opt out makes a lot of sense, in a world with no dnt running at scale.

<efelten> WileyS, the definitions says "attaching a coded reference to a record to allow the data to

<efelten> be associated with a particular device or individual"

peterswire: if dnt does get adopted and scaled, would you still need an independent opt out?

<jchester2> If you look at Yahoo advertising blog, it's clear that they know the identity of the individual to move them through the funnel/transaction.

george: audience measurement research in this context has typically worked with some control in hand of user or research subject. but we understand that internet is different from other modalities we measure....

<johnsimpson> It sounds to me like you want to offer an opt-out that nobody will use ..

<rigo> Wileys, the key is not only to know the name, but also to be able to single out. "Don't want to know your name" is not a possible way out as long as you can easily get back to a person (or discriminate this person)

we kind of conceive of what we are doing as core to Internet being viable medium for a lot of commercial purposes. We try to strike balance between giving user ultimate power and still wanting to exercise...

exclusion from even most basic counting to [general research practices?]

peterswire: mattias now on, and we have a speaker at 1. How long do you need mattias?

schunter: 20 min?

<Wileys> Rigo, that is not the goal of psuedonyms. The goal is to find the middle ground between "personally identifiable" and "de-identified". A psuedonym allows recognition of a unique ID associated with a browser (and therfore perhaps a person, Ed) for individual treatment - but in such a way as I don't know who that person really is in the world.

peterswire: i am inclined to have longer speaker queue comment but not have responses right now (due to time considerations).

rvaneijk: I like text that says you need to process data before statistical analysis. 2nd. I don't see how oob consent description fits into this text proposal. You can't see them separately, can't have both.

<Wileys> Jeff, we do not know the "identity" of the user but we do know a unique ID stored in a cookie is seen both at ad impression and at further states through a purchase funnel. Again - not knowing the true identity is the goal.

rvaneijk: 3. If OOB consent description continues to play out, it's important to know where group stands on that. For me, it's important to see and focus on de-id possibility and relation of oob consent/permitted uses.

<rigo> Wileys, in that sense all web log data is already pseudonymous, so nothing to do in any way, no processing needed. Isn't that a bit short to get into Rob's "yellow" area

rvaneijk: this definition of pseudonymous data is old, out of date. more recently, article 29 wp has said that pseudonymization should not unduly remove data from definition of "personal" data.

justin: i had a question re: processing before aggregation. I don't understand the point. It seems to me more logical to collect the data in pseudonymous form. Maybe i don't understand what you're trying to do

jmayer: there is a longstanding debate in group about whether there should be some objection to pseudonymization, which in many spaces is already the norm.
... many of us are concerned that it could be re-identified. some object on policy, some challenge comp sci research, but many of us find these practices objectionable. I am not certain why

<rvaneijk> no definition of personal data, it will create legal uncertainty since the definitions are part of the process towards a Regulation. Any concept/defintion on pseudonymous data has to be fully consistent with the defintion of personal data and that it does not lead to unduly removing certain categories of dat from the scope of the Regulation, in particular in cases where it is not clear whether the data has indeed been fully anonimised/de-identified/pseudon[CUT]

<Wileys> Rigo, to Rob's "yellow zone", that is part of the de-identification process where unique identifiers have been de-identified (no longer linkable to a unique browser) but that the key used to faciliate the process still exists (has not yet been destoryed). You only make it to "green" once the key is destroyed. Rob - is this a fair recollection?

jmayer: industry specific exceptions would be acceptable. Many services could be provided in privacy preserving way. I would love to help companies implement measurement without pseudonymous data.

<rvaneijk> For the minutes, to me the discussion on out of band consent can not be seen seperate from a call for a permitted use.

<rvaneijk> You can not have both

dan auerbach: will quickly reiterate points. at a high level, just preferring to continue to track these users shouldn't be enough to permit this permitted uses. I prefer 2 states of data, no pseudonymous.

<rigo> Wileys: imagine the keys would be given to the auditor. For the moment, everything is in one hand and we are back to pure usage control and a raw store for 53 weeks, which raises eyebrows

<justin> To be clear, my comment was not my only concern --- but the "pseudonymization before processing" was just a point I did not understand.

<hefferjr> For the minutes, I disagree that OOBC is necessarily tied to market research exemptions.

scribe: there is an assumption that honoring dnt will bias studies. why not measure dnt:1 requests now, before they are being honored to see if that would create bias.

mike o'neil: i also object to pseudonymization. this is how people are tracked over time now.

<rigo> so pseudonymization is also that not every information to single out a person/browser is in one hand. So personal data is if an ID permits to re-identify and single out, but not for that one actor

<Wileys> Rigo, there is risk that an outside force would legally compell an organization to release a key (wouldn't allow for direct reverse engineering but would allow for a dictionary attack). That is a risk an organization bears for holding the key (risk-based de-identification approach)

scribe: so i think this intermediate stage should be taken out. DNT is about stopping people's web history from being collected over time. this data should not be retained and associated with
... same device or individual.

peterswire: pseudonymization before analysis is more rigorous than retaining identifiable data

<Wileys> Rigo, as we reviewed in the HIPPA discussions, there is always some level of risk but its up to organizations to decide to what level of risk they are willing to bear with respect to their de-identification process. If they fail, they are held accountable.

ed felten: i don't think definition is meaningful. And it's backwards. Should be in terms of what can no longer be done with data. I don't see how it would comfort user.

<rigo> Wileys, sure, this is the democratic risk, and we can only mitigate that risk by shorter retentions (and 53 weeks is long for that)

chris_iab: my comment goes back to MRC requirement and idea of privacy waivers. MRC has legal obligation to review audit and attempt accreditation when asked.

<justin> WileyS, How are you accountable if you are legally compelled by an outside force?

<rigo> but the other risk is that all data in one hand is just so easy to abuse. If the key is in a different hands, you need more eyes to abuse the data and higher chances that it will blow

chris_iab: they compare to industry standards or what's available. George ivie said they must review any audit, but in some cases they audit against the standard the party has offered, not indutry

<justin> (To be clear, I'm open to a middle state for pseudonymous data, but not sure this is the right place, and certainly not in this way!)

chris_iab: or mrc standard. But might not accredit. that's not a privacy pass

<Wileys> Justin, I believe private organizations have demonstrated their ability to resist legal requests where they feel this endangers the privacy protections their users expect or have been promised

kathy joe: coming back to 53 weeks. These standards are drawn up by stakeholders in industry bodies that determine measurement elements and retention times.....

scribe: but 53 weeks is industry global standard. No meaningful data.

ronan heffernan: oob consent is not tied to market research. they are different.

<rvaneijk> @Ronan: why are they two different things...

<npdoty> hefferjr: non-real-time out-of-band consent is really a different thing from market research permitted use

<rvaneijk> it is about panel data..

lou mastria: this is not a preference. this is how ad funded content works. This has the objective of funding content.

<npdoty> rvaneijk, I think this particular permitted use is *not* about panel data

<rvaneijk> Nick, yes agree, but audience measurement depends on panel data

scribe: i also want to put a push pin in idea of whether comp sci is correct or not. we are not questioning that. But some people do not want to take into account additional administrative safeguards....

<rvaneijk> the exemption cannot be seen seperate from the OOBC discussion !!!

scribe: I don't think that is getting enough credence in this group.

TPE

schunter: sorry for being late. OOB consent discussed in IRC channel. background as follows:

during tpe discussion people also said there is also oob consent for things otherwise not permitted by dnt:1. Should be registered. we agreed that this exists and will continue, for example....

by contract. led by ronan we now have a discussion about how this actually works in practice and how privacy can be preserved in such a setting.

who is speaking?

<Wileys> +q

<Wileys> Rob, DNT is one type of a consent mechanism but is not the only one - I believe that is the point. There are other methods to obtain user consent - not only DNT.

rvaneik: I am concerned that oob consent could undermine consent mechanism and global considerations work. could just result in use limitations framework. Limited.

<npdoty> rvaneijk, is your concern about the short-term collection? we had currently proposed an exception for short-term data, for example to de-identify data you collected

<rvaneijk> yes agree that there is OOBC

<jmayer> +q

schunter: would like to push time gap out a bit. where i think we have agreement is that there is something like out of band consent. dnt: 0 is not only means to gather consent. do we have agreement?

<hefferjr> yes

<npdoty> I think there is agreement that it's compliant to get consent out-of-band

schunter: please offer comments on this topic

<fielding> we already have a "time gap" for existing permitted uses, so I don't see how that makes any difference -- DNT doesn't mean the UA is invisible.

jmayer: my understanding of group history to explain why i think agreeing on this is"sort of right"

<schunter> topic is "out of band consent in general" (not yet implementation details)

jmayer: initially we focused exclusively on oob consent, then we developed interest in browser based api. there would still be background of oob consent but maybe something like a "should."
... now it seems group has shifted back and people want to have both modes that will co-exist. so I agree oob consent would EXIST. But your suggestion that this is a normal mode of operation, is something...
... where i think there will be much less comfort. to say that you can use it instead of API where API is available. I and others are concerned about race to bottom, with web sites stretching definition....
... of consent. There are 2 checks on this. 1 = policy check, and 2= procedural, moving consent into browser on notion (which i know is controversial) that browsers have better incentives and capacity to deliver meaningful consumer choice..

<fielding> It hardly matters what comfort there is in the group -- it is impossible to make use of an API which has not been implemented anywhere and is unlikely to be implemented correctly for a long time (if ever). Hence, OOB consent is the only option right now and will remain the only option for a very very long time.

schunter: so if i understood, your view is if there is api consent use that, and only go to oob consent if api is not available.

jmayer: anything that gives users meaningful choice and prevents race to bottom is ok with me

shane: i am with jonathan re: oob consent being necessary. It already exists by law and we have agreed standard will not modify law. so any agreement i have with user under law has to be honored.

<jmayer> fielding, Perhaps you misunderstood. I'd be more fine with using out-of-band consent where in-band consent isn't implemented by a browser.

but that said, [.....] i look at this from browser questioning consent point of view. I am afraid if they do this, they will create a race to bottom from browser perspective that will requiore oob consent to maintain....

balance in the ecosystem.

mike o'neil: I don't see point of oob consent. easy enough to get browser based consent.

rigo: for me signaling oob consent or "c", shane, is like signaling "d" for "dismissal."

<justin> As I've said repeatedly, I don't care whether it's in-band or out-of-band --- let the marketplace work it out. But there has to be transparency if any claimed exception wants to trump DNT:1.

<David_MacMillan> zakim ann is David_MacMillan

rigo: what shane is fearing, and I agree, is that if browsers create a reaction to dnt signal. .....and dnt is real communication mechanism....then c just means "give me the data, because

<David_MacMillan> zakim aann is David_MacMillan

rigo: i promise that i have some reason you should open up."

<jmayer> Re: Shane's point, contract formation has been greatly watered down by some courts in the U.S. I would not link the Do Not Track consent standard to vague, diverse, and widely-criticized contract formation standards.

rigo: for now we just send people to pointer [link?] where we explain.

<David_MacMillan> np - thanks

rigo: this "c" as "d" would be race to bottom. But should be discouraged because we are only specifying that browser can make meaningufl choices.

<Wileys> Jonathan, I would leave it up to courts to decide what is a meaningful contract and not this working group

adrianba: i think we are drifting slightly off topic with discussion of whether there should or shouldn't be oob consent and how it should be used. we have discussed a lot in past....

<jmayer> Shane, consumer protection law exists to, among other things, protect consumers from the excesses of contract law. Why would we devolve our responsibility to users?

adrianba: a use case was if a service wants to roam across devices. exception api should only be called when user wants to authenticate for exception. I don't think we should discuss if oob consent is necessary....
... it is. on strict issue of issue 252, this should be a may not should or must, shouldn't require particular ui implementation

<jmayer> Moreover, we already lack consensus on whether a contract should trump Do Not Track. We had that conversation in the context of service providers.

schunter: what we have to do on oob consent is discuss reasons, implemtnation, but no more time for this today.

peterswire: next week the whole meeting will be on compliance. Introducing rena mears and time davis re: financial auditing permitted use

Financial Auditing

important questions are: how long do you need data for different uses? differeint in us and europe? specific text that would capture this permitted use?

<Wileys> Jonathan, I'm not suggesting we devolve our responsibilities to users (and you know that) - I'm saying that the concepts of what represents a valid, enforceable contract is a matter of law and this group shouldn't attempt to solve that for the courts. As you said, this is already a sticky situation but with respect to the FTC Sears Consent Decree, I believe industry has good guidance on how best

<Wileys> to address these situations from a consumer protection perspective.

rena mears has impressive background on privacy auditing, for deloitte and touche, and tim davis has been an expert on online advertising for them.

rena: it would be helpful if you ask questions. we will do slides quickly.

<npdoty> http://lists.w3.org/Archives/Public/public-tracking/2013Mar/att-0320/mears.financial_accounting_use.pdf

<npdoty> (direct link to PDF of slides) ^

rena: this addresses accounting and auditing standards, which are 2 different bodies of standards.
... questions may apply differently to accounting and auditing.

tim: to frame discussion of what requirements there are for financial auditing, need to discuss what the auditing is and the frameworks against which it is conducted.
... these are important but some audits may have greater need for that informatoin.

<npdoty> (page 3 of 7)

rena: next slide discusses some of your questions, peter.

not on irc, can't see questions....

financial audits are only one set of audits that company is subject to....soc 1 and soc 2 have now replaced sas 70s, or regulatory audit, AT 101 type literature, industry audits, and others.

scribe: financial audits governed by many standards. country by country. ifrs is ongoing discussion to bring commonality to international discussion.

guidance from fasb, sec and pcob for public company, accounting requires keeping records in reasonable detail to support transactions that are merely economic events within companies.

aipc gives guidance that records should be kept for not shorter than 5 years.

scribe: pcob says 7 years....

record retention differences apparent in different standards, and companies may produce records in different forms.
...ex: GAAP based statements and regulatory statements

<justin> Say, log files?

<npdoty> "economic event"

peterswire: what triggers 5 year or 7 year requirement

rena: it depends, but it depends on economic event definition, and is changing [this is for data retention]

tim: predominant types of audits: about accuracy of financial statements and internal controls.

<jchester2> Clearly the economic event should be documented in a truly privacy respectful way, esp in the real-time Big Data targeting environment.

tim: in terms of detail needed, if it affects revenue of entities, basic elements, is there contractual arrangement in place, and if so what is it.....
... auditors typically understand how systems work. may rely on how those systems work/report ....rely on reports at level of aggregation above pii

<npdoty> "all they want to see is some evidence that that ad was delivered"

<justin> "some evidence that ad was delivered" . . .

tim: rarely is financial auditor concerned with how info collected. just want to confirm that ad was delivered

<npdoty> financial statement auditor is less concerned about the quality [apologies, trying to track phrases I know we'll have to come back to]

rarely is auditor concerned with quality (targeting) of delivery. wants to confirm payment for ad delivery....

scribe: typically not concerned with level of detail that is subject of dnt....

audit work of iab re: standards for impression membership, audience measurement, MRC as independent auditor using those standards.....

<jchester2> But isn't it so that auditing is also concerned with the quality of the delivery (interactions, for ex), even on a per user basis (given growing payment systems by brands related to performance)

want to confirm that impressions are legitimate and valid, (human initiated).....

so they need more detail re: quality of delivery.

scribe: other concern advertisers have is brand protection. advertisers want to know where their ads are showing up online......

<npdoty> MRC/IAB auditors more concerned with, for example, whether it was a human that viewed that ad

scribe: not places that would harm brand...

<justin> Basic question: do sites need to retain cookie data/IP address/referer url for 5-7 years? Is retention of referer url necessary?

*NICK: are you scribing, or just supplementing?

<Wileys> Contracts now included elements of "quality" of delivery, as Tim just mentioned, whereas we are being asked to prove impression by real human (not a bot), in the location expected (not low quality sites), and within the dimensions of targeting (geolocation for example).

<moneill2> +q

peterswire: just to clarify, it sounds like you said mrc, is closer to quality

<justin> WileyS, so retention of referer url necessary?

*npdoty, tx

*again, tx

tim: financial statements are at levels of aggregation, not referring to dnt data itself.

<Wileys> Justin, yes - for site quality requirements (where possible - often obscurred via iFrame)

tim: might refer to auditors working papers, necessary support for financial statement.

<justin> WileyS, thanks, that's what I thought.

tim: nothing says explicitly that the dnt level detail must be retained. judgment required.....

<Wileys> Justin, a real-world use case, advertiser states they do not want their ads to appear on "adult oriented" websites - how do we prove that we met that contractual requirement?

tim: in my experience many companies don't keep data just because there is so much, so they provide aggregated reports.

<justin> WIleyS, no I get it, just wanted to be clear. But it doesn't sound like you're required to retain that for seven years --- speakers just said that companies often aggregate at some point.

tim: less focused on is the back end data bases being collected for these reasons. these secondary repositories may be where the risk is.

rena: i want to be cautious. i would be concerned that someone walks away thinking there is or is not a definite need for specific information.
... there are accounting requirements, and there are auditing requirements. Back end info that time is concerned about, granularity, is used in ACCOUNTING (not auditing) information....

<rigo> Wileys, you have to keep that as long as the prescription runs. There is a german company for toll roads called Toll collect that has a well developed concept for data deletion. And we need some kind of agreement there. IMHO this is a role for DAA to get to industry practices. Because this would be overkill for the TPWG

<Wileys> Justin, agreed - at some point aggregation is acceptable - question is when. Some argue a few years - some argue something longer. Its a corporate risk dimension - what level of financial risk do you take on by aggregating data too soon?

rena: and that trend will probably continue. transaction acctg is just economic events....

<jchester2> Can you identify Best Auditing Practices emerging from this field to address AdEx', predictive optimization, geo-targeting, etc?

accting is method of recording. what systems are in and out, which are financial, will affect what time just said.....

<rigo> Wileys, exactly, you need a common practices, and those will be localized as they are closely tied to local law

scribe: that same level of discussion (re browswers) goes on re: revenue recognition, what is transaction, what is cost, all requires judgment.....

it's an art and requires audit evidence to determine validity....

<jchester2> +q

where there is risk you will need more info....we are talking about controls. if there are good controls and you are comfortable with that....

you may require less substantive auditing. More of this when risk that controls are not adequate.

<peterswire> about Q-- priority to get explanation of the slides, given limited time; sorry on that!

<justin> WileyS, Yikes. If you're saying companies need to keep at cookie/IP/referer individualized level for at least three years . . .

scribe: so controls will be analyzed at higher level, but if not good enough you need more granular data.....

how long do you keep? AICPA has list of how long you may need accounting records......

scribe: depends on which acocunting schedule you fall under....

peterswire: so lots of ways data may be swept in.....

tim: marked trends underway that buyers of behavioral advertising are increasingly looking for assurance on quality of the product they are buying, so more demand for auditing.....
... are ads being targeted as advertised. i.e. moms age 30-35, are you delivering to that demographic. today not much assurance but this is where it is headed.....

so i anticipate the need for a lot of retention of this information....

peterswire: is it simplistically said that as this becomes more important economically for companies auditing will be more rigorous....
... as info from browser more restricted....lots of pressure on companies to justify what they do...advertisers understand.....and want assurances that they should pay a premium...

rena: one trend with impact on this is what is financial system, and what is auditing target (may be database with pii)...

<npdoty> tim: harder for us to be confident that this user is the same user as last week, or fits into that particular demographic (for which the advertiser is paying a premium)

rena: as targeting becomes more mainstream that affects materiality , so requires more examination....

moneilll: same issue as for audience research. re: unlinkability, how long do you need to retain connection to a particular user or device?

<Zakim> rigo, you wanted to ask if the data is still needed after audit

rigo: my qu is on retention....there is clear accounting data that for legal reasons you have to retain.....
... but raw data to justify fulfillment of contract, must this also be held beyond the audit? when you then stand for the accuracy of data? and how long do you take to audit campaing?

jeff chester: given flood of big data in digital advertising system that is now material can you describe debate in privacy audit community re: best practices

brooks dobbs: can you clarify that what you are talking about is not simply re: oba but for all online advertising; the measure of where economic event occurred

<npdoty> Brooks, that was my understanding as well

dan auerbach: if i am an ad network that wants to delete raw log data after 2 months, rena urged caution. Might that change?

rena: divide between true financial and other audit

<Brooks> Nick, I just think it is important that this is well understood, not that it should be particularly contraversial

tim: how long does underlying detail have to be retained. Judgment for company and audit based on risk that auditor received
... there is also judgment in interpretation of legal requirements too, so no precise answer.....

so as a practical matter, i have seen companies disposing of data after some time, but i advise they speak to stakeholders, attorneys, auditors, customers (who may have audit rights in contract)....

rena: may seem obvious but whatever standard you adopt must be incorporated into a corporate policy document because you will talking about policy and compliance....

peterswire: one question is what are trend and why....heard about increasing materiality

rena: my concern, which also addresses my cautioin note, is that since this is judgment call, and i am on advisory board of oba company, you see this moving to mainstream company....

<Chris_IAB> ultimately, an audit is only as good as the market perceives it to be rigorous, fair and accountable and impartial

when you audit you look at main chunks of revenue and assign more risk there.....

you end up with auditors having to recognize major business model change which is a risk, along with amount, when it becomes material.....

need to understand requirements given those trends......

<justin> And "Do Not Track" is only useful insofar as it's perceived as a meaningful limitation on data collection.

tim: we have seen major corporations getting into this business but also a tremendous amount of fragmentation in this business with many intermediaries in the value chain......

<Wileys> Justin, I believe the focus on "collection" versus "use" is still murky. While all agree on "use" not as many agree on "collection".

but if you are small to medium player you will not do everything yourself but will rely on third parties, and complexities makes it hard for intermediaries to determine who is responsible and has data and for how long.....

<jchester2> what about emerging ethical best practices from the auditng field? Are they working on it?

peterswire: how tied to financial auditing....

tim....goes back to popularity of soc 1 and soc 2 reports, may ahve to get representations from other auditors....

<npdoty> +1, yes, many thanks

peterswire: thank you , we look forward to possibly continue the conversation