Tara M. Swaminatha - Privacy Enhancing Technology Testing -
Position Paper
Privacy Enhancing Technology
Guidelines and Testing Methodology
W3C/QA Position Paper
Tara M. Swaminatha
tms@cigital.com
Privacy continues to gain ground as a mainstream business and consumer
issue. Organizations and individuals increasingly turn to privacy
enhancing technologies (PETs) as a solution to protect personal
information. As well, the market has seen an increasing flood of
products that claim to be privacy enhancing. Beyond the marketing
literature little has been done to set any baseline criteria to
objectively test and report on privacy enhancing technologies. Most
products are in their first iterations. Neither have they stood the
test of time nor have they been held to accepted criteria.
In March 2001, an ad hoc working group chaired by Mike Gurski of
the Ontario Information and Privacy Commission (IPC) met after hours at
the Computers, Freedom, and Privacy (CFP) Conference and established
its purpose to develop a set of widely accepted testing criteria
necessary to test privacy enhancing technologies (both software and
hardware) in a laboratory setting. In March 2001, the Ontario
Information and Privacy Commission convened an ad hoc group to develop
a set of widely accepted testing criteria necessary to test privacy
enhancing technologies. The group is still determining how best to
proceed, and is considering the possibility of bringing their proposal
to a standards organization or other group that could potentially host
this activity. The W3C is one such organization that might be an
appropriate place for such an activity. This short paper discusses how
W3C might approach this activity and why it is relevant to the W3C QA
effort. It will be important for this effort to be successful that the
group in charge of developing criteria for privacy tools should not
also develop the testing scheme. This would certainly represent a
conflict of interest and jeopardize the credibility of the criteria and
testing methodology. As the ad hoc group proceeds in its direction of
the development of tools criteria and a testing methodology, multiple
groups will be involved.
Two efforts to this end should ensue.
- Minimal requirements for PETs should establish a baseline of
required functionality for PETs. This could be designed as a parallel
to the Web Accessibility Initiative (WAI) currently being undertaken by
the W3C or QA group. In order for privacy tools to be given the
credibility they deserve so they may afford consumers an appropriate
level of protection, a formal body of the W3C should be tasked with
developing requirements criteria, to lend credibility to the criteria.
In addition to PET requirements, there could be user agent requirements
for privacy, electronic business requirements for privacy and any other
requirements that affect consumer privacy.
- A testing methodology should be established so that there are
established guidelines for testing the privacy enhancing technologies
against the criteria established for their creation.
Current efforts
A few notable efforts have been undertaken to test privacy
enhancing tools. Efforts in the United States, Canada, Germany, and
the Netherlands were all attempted but met with significant obstacles.
In Canada, the Quebec Information and Privacy Commission set up a
lab and hired staff, but found the resource requirements prohibitive
and halted the initiative. In Germany, Oliver Bethold, Hannes Federrath
and Marit Kohntopp tested a number of "anonymizers" using
various attack methods (Project Anonymity and Unobservability in the
Internet see http
://www.inf.tu-dresden.de/~hf2/publ/2000/BeFK2000cfp2000/).
Similarly, the Netherlands Data Protection Commission has undertaken a
number of studies and initiatives (see http://w
ww.registratiekamer.nl/bis/top_1_5_35_3.html#335 and http://www
.registratiekamer.nl/bis/top_1_5_21.html#396), the latter in
partnership with the Ontario Information & Privacy Commission. In
addition, Herbert Burkert has provided valuable written insight on
Privacy Enhancing Technologies. 1
In the U.S., Professor Lance Hoffman directed a team of graduate
students in the George Washington University Computer Science
department in testing and evaluating several privacy tools. Two other
such efforts in the U.S. were also conducted near the end of 2000. One
was an effort commissioned by U.S. Senator Orrin Hatch, the result of
which was the creation of a report, "Know the Rules, Use the
Tools." The report was designed to educate citizens on Internet
privacy risks and to introduce users to appropriate browser settings
and a small sample of privacy enhancing tools. The other project was a
brief overview of the various online privacy protection technological
tools conducted by Dr. Lorrie Cranor (see http://www.research.att.com/~lorrie/pubs/privacy-tools-sept2000.html
).
Tara Swaminatha, the author of this position paper, was the project
manager for Dr. Hoffman's privacy tools testing project and found
notable obstacles to successful completion of the project. The
products were divided into three categories: anonymizers, blocking
products and choice products. Several of the products tested in each
category were beta versions and functioned erratically. Some of the
products, such as the Platform for Privacy Preferences (P3P), were in
fact formal efforts, while others were operations run out of a
concerned citizen's garage. Many of the products did not function as
intended. Some products crashed systems and rendered IP applications
altogether useless until a reboot was performed. The project was not
funded, therefore any tools requiring money were not evaluated.
Several tools piqued the interests of the project leads but were unable
to be tested due to time, testing environment, financial and personnel
resources limitations. An important group of products omitted because
of these limitations were privacy tools designed by credit card
companies (e.g. American Express) to bolster the privacy of users
making online purchases with that company's credit card. In the
course of a semester project, the students were unable to gather
sufficient information and test the products according to appropriate
software testing practices. With more time and resources, the effort
could have been more successful.
At the outset, the students were asked to determine a set of
criteria against which the tools would be evaluated. This proved an
insurmountable task in the time allotted. There was not enough
coherent information about the set of products to do so. The effort
does serve as a launching pad, however, for more formalized efforts, as
its findings both in terms of product evaluations and suggestions for
better testing processes can provide a valuable baseline for future
endeavors.
Another effort underway is being conducted by the P3P Specification
Working Group. This group is developing a test suite for the P3P
specification (this is noted in the exit criteria at the beginning of
the latest P3P specification), and has already developed a P3P-enabled
web site validation tool (see http://www.w3.org/P3P/va
lidator/20001215).
As is evident, a number of worthwhile initiatives begin to provide
a foundation regarding analyzing and testing Privacy Enhancing
Technologies. However, what is still needed is an ongoing testing and
reporting process that involves state of the art lab and testing
facilities and is based on widely accepted testing criteria developed
by a separate body. This appears to be an effort that would fall easily
under the W3C QA framework or a separate W3C initiative. Initial
interest on the part of IBM labs and ICSA labs to conduct the necessary
testing provides a window of opportunity assuming that the necessary
documentation and criteria can be developed and provided to the labs to
allow them to test the various technologies under a pre-determined
testing methodology.
Similarity to W3C QA Efforts
The QA efforts span the breadth of the W3C spectrum. They help
ensure the veracity and consistency of regulations, guidelines and
suggestions established by various groups in the W3C realm. Privacy is
an issue that should be considered in development of user agents, web
host criteria, privacy enhancing tool development as well as other web
access points. Whether or not privacy efforts fall directly in the
scope of the QA area, they could have considerable impact on all QA
issues in the W3C.
Similarity to W3C Technology & Society Efforts
W3C's Technology & Society domain currently manages its work
on privacy. The Technology & Society domain is collaborating with
multiple organizations, institutions and experts internationally.
Working groups in this domain are developing technical specifications
for P3P, organizing interoperability sessions and developing education
material.
Since the Technology & Society domain states that it
"understands the social impact of the web and reaches out to
affected communities," the development of criteria for privacy
tools and creation of a corollary testing methodology could benefit
from the research conducted in this domain as well as could have
implications for its output. The Technology & Society domain
readily recognizes the impact on various sectors of society who have
access to new technologies. The invocation of privacy tools in many
groups of web users will have a significant impact on current privacy
conditions.
It may be deemed appropriate to include privacy tool guidelines and
testing methodology in the Technology & Society domain however the
privacy guildelines issue appears to supersede its domain. It may seem
appropriate to include efforts to initiate privacy tools criteria and
testing methodology across more W3C sectors than simply Technology
& Society.
Similarity to W3C Accessibility Efforts
The Web Accessibility Initiative (WAI) was started because web
accessibility is a critical issue. Five specific reasons are
enumerated supporting its importance and relevance as a W3C initiative:
- Use of the web is spreading rapidly into all areas of society.
- There are barriers on the web for many types of disabilities.
- Millions of people have disabilities that affect access to the web.
- Web accessibility has carry-over benefits for other users.
- Some web sites are required to be accessible.
Different disabilities result in reduced benefit from non-accessible
web options. Those individuals who have visual, auditory, physical,
cognitive or neurological disabilities require certain accommodations
be made in order to gain equal access to web resources as their peers.
The W3C formed the WAI in order to address these needs and develop
three sets of guidelines: web content accessibility; web authoring tool
accessibility; and user agent accessibility. It is my belief that such
guidelines are equally necessary to protecting the privacy of all
consumers who use the web.
The W3C defines its mission, "to lead the World Wide Web to
its full potential by developing common protocols that promote its
evolution and ensure its interoperability." Privacy fits well
into the W3C mission in that it is an issue that affects every
individual who browses the web without exception and has the
opportunity to push the web towards its full potential. Without proper
criteria for privacy enhancing tools, however, privacy could become an
issue that thwarts the growth of the web. Privacy is an issue that is
threaded throughout all W3C domains and one that should be built into
other W3C guidelines. It affects users internationally and locally.
Most countries aside from the United States, in fact, have more strict
privacy legislation and place higher importance on its inclusion in all
aspects of life. Privacy officers are becoming common in European
nations in the business sector as well.
In a parallel to the WAI, a privacy initiative would enable
"stakeholders" in privacy market sectors and private sectors
to come together to develop elements & requirements for privacy
guidelines. A wide variety of organizations should contribute to the
development of the guidelines so they may hold the most value.
International members should come from private and public sectors,
industry, military, government, research organizations, commercial
developers and advocacy groups alike. The forum under which the
guidelines should be developed will follow the W3C Process for
"consensus-based development of work."
The privacy intitative, if owned by the W3C, could have a
substantial positive effect on all web users. Privacy issues can no
longer be ignored as more and more consumers are demanding rights as
outlined by privacy advocates. Concise justification for the inclusion
of privacy as an iniative undertaken by some portion of the W3C is as
follows:
- Privacy affects all users who use the web whether or not they
participate in e-commerce.
- Privacy policies by companies can be deceptive and are often
subject to change without notice.
- There does not exist an established set of criteria for which users
can seek to evaluate privacy enhancing technologies.
- Since consumers should not trust their privacy is protected on the
web, and by extension should not trust privacy tools without
evaluation, a trusted body needs to establish criteria for consumers to
use.
- Privacy tool criteria will help web host owners tailor their
policies to respect, accommodate and actively protect consumer
privacy.
Proposed Actions
Following is an outline of suggested actions that can evolve from this
position paper. The suggestions are intended to guide how the W3C
might approach this project. The aim of the suggestions are to spur
discussion and cultivate ideas that will contribute to the goals at
hand.
First, it is suggested the W3C adopt this initiative in the
appropriate realm and assemble working groups. The working group could
oversee the efforts of the initiative and provide appropriate guidance,
directives and resources when necessary. The appropriate parties
should be invited to collaborate on the effort and should provide input
on the impending process.
The group that will own this privacy initiative could first develop
a more detailed taxonomy of privacy enhancing technologies in order to
better identify salient features and necessary criteria for PETs. For
example, anonymizing products can be divided into anonymous browsers
and anonymous re- or e-mailers. Further, anonymous browsers could be
broken down into categories based on their proxy or cookie-handling
systems. Web browsing functionality will be restricted in some cases
with the use of anonymous browsers and this should be reflected in a
comprehensive taxonomy.
A subsequent effort should ensue once the taxonomy is finalized.
This effort will be to define guidelines for privacy similar to those
developed by the WAI for accessibility. This effort should develop
criteria, checklists, suggested implementations and techniques and
other pertinent material for privacy tools. If deemed necessary, this
effort could include similar documents pertaining to user agents,
e-commerce or other areas affecting consumer privacy.
Once guidelines are revised and finalized according to appropriate
W3C courses of action, efforts to develop a testing methodology could
commence. It is suggested that during this phase, software testing
experts be consulted and accepted software testing practices be closely
adhered to in developing an appropriate methodology for testing privacy
tools against the established criteria. The testing methodology should
serve as concomitant guidelines to the organizations performing testing
for evaluating compliance with criteria defined. The combination of
the guidelines and methodology will spur organizations willing to
dedicate time and resources (e.g. IBM and ICSA testing labs which have
already volunteered, as well as other testing labs as they are
available) to testing privacy tools against the criteria. This effort
should be monitored by the W3C and its privacy initiative.
An ad hoc group of individuals, chaired by Mike Gurski of the
Ontario Information and Privacy Commission, has begun this initiative
in its conceptual phase. These individuals could be considered the
experts to be consulted in determining the specific steps of this
process and should have considerable input on the requirements for the
guidelines established by a W3C privacy initiative. Dr. Lance Hoffman
is the Advisory Committee representative from The George Washington
University and can advocate for this group if necessary at AC meetings.
Dr. Lorrie Cranor is the Advisory Committee representative from AT&T
and chair of the P3P Specification Working Group, and is also involved
in this effort. Drs. Cranor and Hoffman are both soliciting advice,
research efforts and support from any interested parties.
The W3C would be remiss in its mission if it did not adopt this
privacy initiative as important and timely. Its development of
guidelines for privacy tools, testing of tools and for web hosts to
afford privacy to their consumers will greatly further the cause it
sets out to champion. Those individuals involved in the efforts across
the globe thus far are willing to provide valuable insight into the
obstacles met in their commencement of these efforts so far. Their
experience and research can serve as a baseline and should the W3C
establish a privacy initiative, it will have substantial background
from which to move forward.
Notes
1 "Privacy-Enhancing
Technologies: Typology, Critique, Vision" in Technology and
Privacy: The New Landscape, ed. Philip Agre and Marc Rotenberg, MIT
Press: 1997.
____________________________________________________________
About the Author
Tara M. Swaminatha (tms@cigital.com)
is currently a Software Security Consultant with Cigital
and a student at George Washington University studying under Dr. Lance
Hoffman.