Privacy Enhancing Technology
Guidelines and Testing Methodology
W3C/QA Position Paper

Tara M. Swaminatha
tms@cigital.com

Privacy continues to gain ground as a mainstream business and consumer issue. Organizations and individuals increasingly turn to privacy enhancing technologies (PETs) as a solution to protect personal information. As well, the market has seen an increasing flood of products that claim to be privacy enhancing. Beyond the marketing literature little has been done to set any baseline criteria to objectively test and report on privacy enhancing technologies. Most products are in their first iterations. Neither have they stood the test of time nor have they been held to accepted criteria.

In March 2001, an ad hoc working group chaired by Mike Gurski of the Ontario Information and Privacy Commission (IPC) met after hours at the Computers, Freedom, and Privacy (CFP) Conference and established its purpose to develop a set of widely accepted testing criteria necessary to test privacy enhancing technologies (both software and hardware) in a laboratory setting. In March 2001, the Ontario Information and Privacy Commission convened an ad hoc group to develop a set of widely accepted testing criteria necessary to test privacy enhancing technologies. The group is still determining how best to proceed, and is considering the possibility of bringing their proposal to a standards organization or other group that could potentially host this activity. The W3C is one such organization that might be an appropriate place for such an activity. This short paper discusses how W3C might approach this activity and why it is relevant to the W3C QA effort. It will be important for this effort to be successful that the group in charge of developing criteria for privacy tools should not also develop the testing scheme. This would certainly represent a conflict of interest and jeopardize the credibility of the criteria and testing methodology. As the ad hoc group proceeds in its direction of the development of tools criteria and a testing methodology, multiple groups will be involved.

Two efforts to this end should ensue.

1. Minimal requirements for PETs should establish a baseline of required functionality for PETs. This could be designed as a parallel to the Web Accessibility Initiative (WAI) currently being undertaken by the W3C or QA group. In order for privacy tools to be given the credibility they deserve so they may afford consumers an appropriate level of protection, a formal body of the W3C should be tasked with developing requirements criteria, to lend credibility to the criteria. In addition to PET requirements, there could be user agent requirements for privacy, electronic business requirements for privacy and any other requirements that affect consumer privacy.
2. A testing methodology should be established so that there are established guidelines for testing the privacy enhancing technologies against the criteria established for their creation.

Current efforts

A few notable efforts have been undertaken to test privacy enhancing tools. Efforts in the United States, Canada, Germany, and the Netherlands were all attempted but met with significant obstacles.

In Canada, the Quebec Information and Privacy Commission set up a lab and hired staff, but found the resource requirements prohibitive and halted the initiative. In Germany, Oliver Bethold, Hannes Federrath and Marit Kohntopp tested a number of "anonymizers" using various attack methods (Project Anonymity and Unobservability in the Internet see http ://www.inf.tu-dresden.de/~hf2/publ/2000/BeFK2000cfp2000/). Similarly, the Netherlands Data Protection Commission has undertaken a number of studies and initiatives (see http://w ww.registratiekamer.nl/bis/top_1_5_35_3.html#335 and http://www .registratiekamer.nl/bis/top_1_5_21.html#396), the latter in partnership with the Ontario Information & Privacy Commission. In addition, Herbert Burkert has provided valuable written insight on Privacy Enhancing Technologies. ¹

In the U.S., Professor Lance Hoffman directed a team of graduate students in the George Washington University Computer Science department in testing and evaluating several privacy tools. Two other such efforts in the U.S. were also conducted near the end of 2000. One was an effort commissioned by U.S. Senator Orrin Hatch, the result of which was the creation of a report, "Know the Rules, Use the Tools." The report was designed to educate citizens on Internet privacy risks and to introduce users to appropriate browser settings and a small sample of privacy enhancing tools. The other project was a brief overview of the various online privacy protection technological tools conducted by Dr. Lorrie Cranor (see http://www.research.att.com/~lorrie/pubs/privacy-tools-sept2000.html ).

Tara Swaminatha, the author of this position paper, was the project manager for Dr. Hoffman's privacy tools testing project and found notable obstacles to successful completion of the project. The products were divided into three categories: anonymizers, blocking products and choice products. Several of the products tested in each category were beta versions and functioned erratically. Some of the products, such as the Platform for Privacy Preferences (P3P), were in fact formal efforts, while others were operations run out of a concerned citizen's garage. Many of the products did not function as intended. Some products crashed systems and rendered IP applications altogether useless until a reboot was performed. The project was not funded, therefore any tools requiring money were not evaluated. Several tools piqued the interests of the project leads but were unable to be tested due to time, testing environment, financial and personnel resources limitations. An important group of products omitted because of these limitations were privacy tools designed by credit card companies (e.g. American Express) to bolster the privacy of users making online purchases with that company's credit card. In the course of a semester project, the students were unable to gather sufficient information and test the products according to appropriate software testing practices. With more time and resources, the effort could have been more successful.

At the outset, the students were asked to determine a set of criteria against which the tools would be evaluated. This proved an insurmountable task in the time allotted. There was not enough coherent information about the set of products to do so. The effort does serve as a launching pad, however, for more formalized efforts, as its findings both in terms of product evaluations and suggestions for better testing processes can provide a valuable baseline for future endeavors.

Another effort underway is being conducted by the P3P Specification Working Group. This group is developing a test suite for the P3P specification (this is noted in the exit criteria at the beginning of the latest P3P specification), and has already developed a P3P-enabled web site validation tool (see http://www.w3.org/P3P/va lidator/20001215).

As is evident, a number of worthwhile initiatives begin to provide a foundation regarding analyzing and testing Privacy Enhancing Technologies. However, what is still needed is an ongoing testing and reporting process that involves state of the art lab and testing facilities and is based on widely accepted testing criteria developed by a separate body. This appears to be an effort that would fall easily under the W3C QA framework or a separate W3C initiative. Initial interest on the part of IBM labs and ICSA labs to conduct the necessary testing provides a window of opportunity assuming that the necessary documentation and criteria can be developed and provided to the labs to allow them to test the various technologies under a pre-determined testing methodology.

Similarity to W3C QA Efforts

The QA efforts span the breadth of the W3C spectrum. They help ensure the veracity and consistency of regulations, guidelines and suggestions established by various groups in the W3C realm. Privacy is an issue that should be considered in development of user agents, web host criteria, privacy enhancing tool development as well as other web access points. Whether or not privacy efforts fall directly in the scope of the QA area, they could have considerable impact on all QA issues in the W3C.

Similarity to W3C Technology & Society Efforts

W3C's Technology & Society domain currently manages its work on privacy. The Technology & Society domain is collaborating with multiple organizations, institutions and experts internationally. Working groups in this domain are developing technical specifications for P3P, organizing interoperability sessions and developing education material.

Since the Technology & Society domain states that it "understands the social impact of the web and reaches out to affected communities," the development of criteria for privacy tools and creation of a corollary testing methodology could benefit from the research conducted in this domain as well as could have implications for its output. The Technology & Society domain readily recognizes the impact on various sectors of society who have access to new technologies. The invocation of privacy tools in many groups of web users will have a significant impact on current privacy conditions.

It may be deemed appropriate to include privacy tool guidelines and testing methodology in the Technology & Society domain however the privacy guildelines issue appears to supersede its domain. It may seem appropriate to include efforts to initiate privacy tools criteria and testing methodology across more W3C sectors than simply Technology & Society.

Similarity to W3C Accessibility Efforts

The Web Accessibility Initiative (WAI) was started because web accessibility is a critical issue. Five specific reasons are enumerated supporting its importance and relevance as a W3C initiative:

1. Use of the web is spreading rapidly into all areas of society.
2. There are barriers on the web for many types of disabilities.
3. Millions of people have disabilities that affect access to the web.
4. Web accessibility has carry-over benefits for other users.
5. Some web sites are required to be accessible.

Different disabilities result in reduced benefit from non-accessible web options. Those individuals who have visual, auditory, physical, cognitive or neurological disabilities require certain accommodations be made in order to gain equal access to web resources as their peers. The W3C formed the WAI in order to address these needs and develop three sets of guidelines: web content accessibility; web authoring tool accessibility; and user agent accessibility. It is my belief that such guidelines are equally necessary to protecting the privacy of all consumers who use the web.

The W3C defines its mission, "to lead the World Wide Web to its full potential by developing common protocols that promote its evolution and ensure its interoperability." Privacy fits well into the W3C mission in that it is an issue that affects every individual who browses the web without exception and has the opportunity to push the web towards its full potential. Without proper criteria for privacy enhancing tools, however, privacy could become an issue that thwarts the growth of the web. Privacy is an issue that is threaded throughout all W3C domains and one that should be built into other W3C guidelines. It affects users internationally and locally. Most countries aside from the United States, in fact, have more strict privacy legislation and place higher importance on its inclusion in all aspects of life. Privacy officers are becoming common in European nations in the business sector as well.

In a parallel to the WAI, a privacy initiative would enable "stakeholders" in privacy market sectors and private sectors to come together to develop elements & requirements for privacy guidelines. A wide variety of organizations should contribute to the development of the guidelines so they may hold the most value. International members should come from private and public sectors, industry, military, government, research organizations, commercial developers and advocacy groups alike. The forum under which the guidelines should be developed will follow the W3C Process for "consensus-based development of work."

The privacy intitative, if owned by the W3C, could have a substantial positive effect on all web users. Privacy issues can no longer be ignored as more and more consumers are demanding rights as outlined by privacy advocates. Concise justification for the inclusion of privacy as an iniative undertaken by some portion of the W3C is as follows:

1. Privacy affects all users who use the web whether or not they participate in e-commerce.
2. Privacy policies by companies can be deceptive and are often subject to change without notice.
3. There does not exist an established set of criteria for which users can seek to evaluate privacy enhancing technologies.
4. Since consumers should not trust their privacy is protected on the web, and by extension should not trust privacy tools without evaluation, a trusted body needs to establish criteria for consumers to use.
5. Privacy tool criteria will help web host owners tailor their policies to respect, accommodate and actively protect consumer privacy.

Proposed Actions

Following is an outline of suggested actions that can evolve from this position paper. The suggestions are intended to guide how the W3C might approach this project. The aim of the suggestions are to spur discussion and cultivate ideas that will contribute to the goals at hand.

First, it is suggested the W3C adopt this initiative in the appropriate realm and assemble working groups. The working group could oversee the efforts of the initiative and provide appropriate guidance, directives and resources when necessary. The appropriate parties should be invited to collaborate on the effort and should provide input on the impending process.

The group that will own this privacy initiative could first develop a more detailed taxonomy of privacy enhancing technologies in order to better identify salient features and necessary criteria for PETs. For example, anonymizing products can be divided into anonymous browsers and anonymous re- or e-mailers. Further, anonymous browsers could be broken down into categories based on their proxy or cookie-handling systems. Web browsing functionality will be restricted in some cases with the use of anonymous browsers and this should be reflected in a comprehensive taxonomy.

A subsequent effort should ensue once the taxonomy is finalized. This effort will be to define guidelines for privacy similar to those developed by the WAI for accessibility. This effort should develop criteria, checklists, suggested implementations and techniques and other pertinent material for privacy tools. If deemed necessary, this effort could include similar documents pertaining to user agents, e-commerce or other areas affecting consumer privacy.

Once guidelines are revised and finalized according to appropriate W3C courses of action, efforts to develop a testing methodology could commence. It is suggested that during this phase, software testing experts be consulted and accepted software testing practices be closely adhered to in developing an appropriate methodology for testing privacy tools against the established criteria. The testing methodology should serve as concomitant guidelines to the organizations performing testing for evaluating compliance with criteria defined. The combination of the guidelines and methodology will spur organizations willing to dedicate time and resources (e.g. IBM and ICSA testing labs which have already volunteered, as well as other testing labs as they are available) to testing privacy tools against the criteria. This effort should be monitored by the W3C and its privacy initiative.

An ad hoc group of individuals, chaired by Mike Gurski of the Ontario Information and Privacy Commission, has begun this initiative in its conceptual phase. These individuals could be considered the experts to be consulted in determining the specific steps of this process and should have considerable input on the requirements for the guidelines established by a W3C privacy initiative. Dr. Lance Hoffman is the Advisory Committee representative from The George Washington University and can advocate for this group if necessary at AC meetings. Dr. Lorrie Cranor is the Advisory Committee representative from AT&T and chair of the P3P Specification Working Group, and is also involved in this effort. Drs. Cranor and Hoffman are both soliciting advice, research efforts and support from any interested parties.

The W3C would be remiss in its mission if it did not adopt this privacy initiative as important and timely. Its development of guidelines for privacy tools, testing of tools and for web hosts to afford privacy to their consumers will greatly further the cause it sets out to champion. Those individuals involved in the efforts across the globe thus far are willing to provide valuable insight into the obstacles met in their commencement of these efforts so far. Their experience and research can serve as a baseline and should the W3C establish a privacy initiative, it will have substantial background from which to move forward.

Notes
¹ "Privacy-Enhancing Technologies: Typology, Critique, Vision" in Technology and Privacy: The New Landscape, ed. Philip Agre and Marc Rotenberg, MIT Press: 1997.

____________________________________________________________
About the Author
Tara M. Swaminatha (tms@cigital.com) is currently a Software Security Consultant with Cigital
and a student at George Washington University studying under Dr. Lance Hoffman.