Abstract
The decentralized nature of DIDs has inspired a wide range of
DID methods, as different innovators document novel approaches
for using DIDs with different Verifiable Data Registries. At last count,
there were 226 methods listed in the W3C extensions registry
and countless others in use.
This means that implementers are faced with the
challenge of choosing which methods to support. This is true
not only for components like DID Resolvers, which necessarily
process method-specific features, but also for issuers and
verifiers, who must decide independently with method meets
their business needs for features like revocation, provenance,
and cost.
How do implementers decide which methods are suitable for
their use?
Applying the DID Method Rubric is one answer to that question.
This document defines a coherent way to evaluate multiple DID
methods on a feature-to-feature basis, tailored to the
business context addressed by implementers.
The rubric presents a set of criteria which an Evaluator can
apply to any DID Method based on the use cases most relevant
to them. Rather than reducing this Evaluation to a single
number, the rubric retains the sensibility that requirements
for use tend to be multidimensional and many of the possible
responses are not necessarily good or bad. It is up to the
Evaluator to understand how each response in each criteria
might illuminate favorable or unfavorable consequences for
their needs.
While this rubric assists in the evaluation of many aspects of
a DID Method, it is not intended to be exhaustive. Evaluators
are encouraged to extend the framework to incorporate their
own criteria (and hopefully share those criteria back to the
community when possible).
This section is non-normative.
A rubric is a tool used in academia to communicate
expectations and evaluate performance. It consists of a set
of criteria to be evaluated, possible responses for each
criteria, and a scoring guide explaining how to both choose
and interpret each response. The act of evaluating a rubric,
which we call an Evaluation, provides basis for
self-evaluation, procurement decisions, or even marketing
points. Written records of an evaluation, which we'll call
an Evaluation Report, document how a particular
subject is measured against the criteria. For students, a
rubric helps to clarify how their work will be evaluated by
others. For Evaluators, a rubric provides a
consistent framework for investigating and documenting the
performance of a subject against a particular set of
criteria.
We were inspired to develop a rubric for decentralization
when discussions about the requirements for decentralized
identifiers, aka DIDs, led to intractable disagreement. It
became clear that no single definition of "decentralized"
would suffice for all of the motivations that inspired
developers of DID Methods to work on a new decentralized
approach to identity. Despite this challenge, two facts
remained clear:
-
the people invested in this work shared a common goal of
reversing the problems with centralized identity systems
and
-
they also had numerous, distinct reasons for doing so.
Rather than attempt to force a definition of "decentralized"
that might work for many but would alienate others, the
group set out to capture the measurable factors that could
enable Evaluators to judge the decentralization of
DID Methods based on their own independent requirements,
with minimal embedded bias.
This approach identified several useful criteria about
methods that have nothing to do with decentralization.
Rather than create a separate rubric for these criteria, the
DID WG extended the scope of the rubric to include any
criteria that might be useful for evaluators.
Pick the most important criteria for your use, ask each
question, and select the most appropriate response. Do this
for all of the DID Methods under consideration.
Each Evaluation should start with an explicit framing
of the use under consideration. Are you evaluating the
Method for use in Internet-of-Things (IoT)? For school
childrens' extra-curricular activities? For international
travel? The use, or set of uses, will directly affect how
some of the questions are answered.
Where a given Method offers variability, such as multiple
networks for the same Method, then evaluate each variant.
For example, did:ethr supports Ethereum mainnet, multiple
testnets and permissioned EVM-compliant networks such as
Quorum. To apply a criteria to did:ethr, you will evaluate
it against all the variations that matter to you.
Each variation should get its own Assessment. This
applies to Level 2 Networks that can operate on multiple
Level 1 Networks as well as DID Methods that directly offer
support for multiple underlying DID registries.
When creating an Evaluation Report, we recommend
noting both the Evaluator and the date of the
Evaluation. Many of the criteria are subjective and
all of them may evolve. Tracking who made the
Evaluation and when they made it will help readers
better understand any biases or timeliness issues that may
affect the applicability of the Evaluation.
Be selective and acknowledge the subjective.
Evaluations do not need to be exhaustive. There is no
requirement to answer all the questions. Simply answer the
ones most relevant to the use contemplated. Similarly,
understand that any recorded Evaluation is going to
represent the biases of the Evaluator in the context
of their target use. Even the same Evaluator,
evaluating the same Method for a different use, may come up
with slightly different answers—for example, that
which is economically accessible for small businesses might
not be cost-effective for refugees, and that could affect
how well-suited a given Method is for a specific use.
Finally, note that the rubric is not just about
decentralization. There are security, privacy, and economic
concerns that are considered. However, no such list of
criteria could ever be exhaustive and none should be
considered "complete". We encourage Evaluators to use
this rubric as a starting point for their own work rather
than the final say in the merit of any given Method.
In short, use this rubric to help understand if a given DID
Method is appropriate for your needs, whatever your use
cases may be.
To record and report an Evaluation, we recommend two
possible formats, either comprehensive or comparative.
A comprehensive Evaluation applies a single set of
criteria to just one Method. This set is chosen by the
Evaluator; it need not be all possible criteria, but
it is all relevant criteria as judged by the
Evaluator.
A comparative Evaluation includes multiple Methods in
the same table to easily compare and contrast two or more
different Methods. This may include any number of criteria.
In addition to the selected criteria, we recommend each
report specify
-
The Method(s) being evaluated
-
A link to the evaluated Methods' specification(s)
-
The Evaluator(s)
-
The date of the Evaluation
-
A description of the use case(s) for which the Method is
being evaluated
-
The rubric used for the Evaluation, along with
reference to the specific version.
-
Optionally, a URL for retrieving the report.
We have grouped our criteria into several categories:
-
Rulemaking
-
Design
-
Operations
-
Enforcement
-
Adoption & Diversity
-
Security
-
Privacy
Evaluators should consider criteria from all groups, as
best fits your use cases.
Three categories cover how a given Method is governed:
Rulemaking, Operations, and Enforcement. Our approach
parallels the same separation of authority embodied in the
tripartite governments.
Rulemaking addresses who makes the rules and
how. (This is the legislative branch.)
Operations addresses how those rules are
executed and how everyone knows that they are carried out.
(This is the executive branch.)
Enforcement addresses how we find and
respond to rule breaking. (This is the judicial branch in
the US.)
This mental model is key to understanding the criteria of
each section as well as why we included some criteria and
not others.
The remaining categories each covers different areas worth
considering when evaluating DID Methods.
Design addresses the method as
designed. In other words, the output of the rulemaking:
what rules apply to this DID method?
Adoption & Diversity covers questions
related to how widely a DID Method is used.
Security influences overall trust in the
ecosystem. Different DID methods offer different security
guarantees, or guarantees of different strengths.
Privacy addresses the ability of a DID
method to ensure various privacy mechanisms. When DIDs are
used as identifiers for people, it becomes important to
consider what tools a DID method offers to operate at
different levels of privacy.
When evaluating the governance of DID Methods, three
potentially independent layers should be considered: the
specification, the network, and the registry.
-
The specification is the governing
document for the Method that outlines how that particular
Method implements the required and any optional components
of the DID Core specification [DID-CORE].
-
The network is the underlying
communications layer, i.e., how users of the Method
communicate with others to invoke the operations of the
Method.
-
The registry is a given instance of
recorded state changes, managed according to the
specification, using the communications channel.
For Rulemaking, the criteria should be evaluated against all
three of the above layers.
For Operations, the criteria should be evaluated against the
network and the registry. The specification is taken as a
given (it is the core output of Rulemaking).
For Adoption, the criteria should be evaluated for each major
software component: wallet, resolver, and registry.
For the examples in the rest of this document we refer to a
set of Methods that are familiar to the authors and exhibit
interesting characteristics for Evaluation. See the
tables below.
The following sources provided example evaluations for this
rubric. These are not presented as objective fact, but
rather as attempts by contributors to illustrate noteworthy
differences using their subjective judgement. Additional
contributions that flow into the registry should include a
section that documents their source with an additional row
in the table.
This is not a comprehensive list (see the DID
Specification Registries for a current list of registered
Methods). Rather, this section aggregates the methods used in
Example Assessments of current criteria for easy reference.
The term "criteria" is often treated as both a singular and
a plural noun. In the singular, we say "The most important
criteria is the buyer's age". This singular use of criteria
has been in use for over half a century https://www.merriam-webster.com/dictionary/criterion#usage-1
In the plural we say "That proposal doesn't meet all of the
criteria." Sometimes people use both in the same sentence:
"Select one criteria from the list of criteria."
However, for formal use, "criteria" is more broadly accepted
as plural while "criterion" is singular. By this style rule,
the last sentence in the previous paragraph should
be "Select one criterion from the list of criteria."
As editors of a Note published by the World Wide Web
Consortium, we are torn. We would prefer to use rigorous
grammar and be consistent in doing so. At the same time, we
find attempts to enforce the formal rule sometimes leads
people to using the inverse, such as "Select a criteria from
the list of criterion." This is the exact opposite of our
desired outcome.
In our own work with implementers and developers of the DID
Core specification, we have found that the singular
"criteria" is readily accepted and understood and leads to
no confusion when used, even alongside plural usage. Since
the DID Method Rubric is, at its core, a set of criteria
with ample reason to refer to, for example, "Criteria #23",
we find the combined singular and plural use is cleaner
(just stick with "criteria"), less confusing, and more
aligned with common usage among our audience.
As such, throughout the DID Method Rubric, we use the term
"criteria" to refer to both singular instances of criteria
and plural sets of criteria.
This registry — the DID Method Rubric Registry —
provides a public vehicle for publishing updated DID Method
Rubric criteria. In order to add or update a criteria, a
submitter MUST submit a modification request for this
registry, as a pull request on the repository where this
registry is hosted, where the modification request adheres to
the following policies:
-
If there are copyright, trademark, or intellectual
property rights concerns, the addition and use MUST be
authorized in writing by the intellectual property rights
holder under a F/RAND license. For example, criteria that
use trademarked brand names, use the titles or excerpts
from copyrighted works, or require patented technology to
perform the evaluation.
-
Additions MUST NOT create unreasonable legal, security,
moral, or privacy issues that may result in direct or
indirect harm to others, including violations of the W3C
Code Of Ethics And Professional Conduct
https://www.w3.org/Consortium/cepc/
Examples of unacceptable additions include any containing
hate speech, unprofessional or unethical attacks,
proprietary information, and any personal data or
personally identifiable information (excepting
identification details of the submitter).
-
All criteria MUST be uniquely identified and versioned to
ensure permanent linkability with version trackability.
Those identifiers MUST be present as an
href anchor in
the HTML for the criteria itself. See the sections below
on identifiers and versioning. Subcomponents, such as
questions, responses, relevance, and examples, are not
separately versioned; any change to a subcomponent
triggers an appropriate version change to its primary
component.
-
Methods, and evaluations cited within a criteria MUST
include a local link to a full citation in the relevant
citation section: Methods Considered and Evaluations Cited,
respectively.
-
Rubric criteria and associated metadata in the DID Method
Rubric Registry MAY be updated or removed at the editors'
discretion, including the addition or removal of
categories of metadata such as Evaluations Cited.
-
Removed criteria SHOULD be listed as Retired, e.g.,
placed in the
retired.js file, with a permalink to the last
version of the registry where each criteria was active.
The primary components managed by this registry are criteria
for evaluating DID Methods, with as many as eight
subcomponents: name, id, version, question,
assessmentTemplate, response, relevance, exampleAssessments,
and, optionally, a source. In addition, the DID Method
Rubric maintains a list of cited DID methods and
evaluations. The criteria are independently identified and
versioned (see those sections for details) while references
cited in the criteria themselves link to their full citation
in the appropriate list.
-
Updates to criteria MUST follow the following versioning
guidelines.
-
Accepted criteria MUST have at least
three example assessments, and generally SHOULD have no
more example assessments than there are possible
responses to its question. Each example should
distinctly illustrate one possible response to the
criteria's question.
-
DID Method Rubric criteria can be proposed by submitting
a pull request that defines the criteria in a file under
the
rubric/criteria/ directory. The file
name MUST be the criteria's number (from its
id, e.g., 1 for criteria-1) followed by a
kebab-case slug of the criteria name and the
.json file extension (e.g.,
1-open-contribution-participation.json).
-
The pull request MUST also add the criteria's
id to the appropriate category in
rubric/rubric-outline.json. The rubric
outline determines the organization and ordering of
criteria within the rendered document. A criteria file
that is not referenced in the outline will not appear
in the Rubric.
- To be considered, criteria MUST have the following
subcomponents:
-
Name
-
Identifier
-
Version
-
Question
-
Assessment Template
-
Response
-
Relevance
-
Example Assessments
-
Additionally, criteria MAY have the following optional
subcomponents:
-
Source
The following is an example of a complete criteria
JSON file:
{
"name": "Offline creation",
"id": "https://www.w3.org/TR/did-rubric#criteria-42",
"version": "1.0.0",
"source": "https://didcriteria.com/criteria/7",
"question": {
"question": "Does the Method require network communications to create a DID?"
},
"response": {
"type": "multipleChoice",
"possibleResponses": [
{
"label": "A",
"meaning": "No. Creation is expected to be off-line."
},
{
"label": "B",
"meaning": "Yes. Creation requires network coordination with a single party."
},
{
"label": "C",
"meaning": "Yes. Creation requires network coordination with multiple parties."
}
]
},
"assessmentTemplate": {
"columns": [
{
"heading": "Method",
"type": "method",
"propertyRef": "method"
},
{
"heading": "Spec.",
"type": "enhancedLetter",
"propertyRef": "spec"
},
{
"heading": "Notes",
"type": "note",
"propertyRef": "note"
}
]
},
"relevance": "Communication is costly, with increasing costs the more parties are involved.",
"exampleAssessments": [
{
"id": "a-29",
"method": "did:v1.testnet",
"spec": "A",
"note": "Veres One DID creation is a local cryptographic process.",
"evaluationCitation": "#eval-3"
},
{
"id": "a-31",
"method": "did:web",
"spec": "B+",
"note": "The DID controller must be able to communicate with the web server.",
"evaluationCitation": "#eval-4"
}
]
}
Each criteria needs a human-friendly, short, descriptive
name that captures the essence of the criteria as
concisely as possible.
The criteria id must be explicit, unique, and persistent.
See the section on Identifiers for details. Criteria ids
under the DID Method Rubric namespace will be assigned by
the rubric maintainers.
The criteria version must increment, as appropriate, any
time the criteria, or any of its subcomponents, is
updated. See the section on Versioning for details.
The key question for evaluating the criteria. This
property is REQUIRED.
Any additional instructions for how the evaluator should
interpret or apply the question when making their
assessment. This property is OPTIONAL.
An ordered array of column definitions. Each column
specifies a heading, a type, and a property reference.
This property is REQUIRED. The columns array MUST
include a column with a type of method as
the first entry, which identifies the specific DID
method being assessed.
The display label for the column in the assessment
table. This property is REQUIRED.
The type of data expected in this column. This
property is REQUIRED. The following types are
currently defined, but additional types MAY be
introduced as needed:
-
method
-
The DID method being assessed. Every assessment
template MUST include a column of this type.
-
enhancedLetter
-
A letter response from the possible responses,
optionally enhanced with modifiers (e.g., "A-",
"C/D").
-
multipleChoice
-
A response selected from the defined possible
responses.
-
note
-
Free-text notes providing context for the
assessment.
The property name used in example assessments to
provide the value for this column. This property
is REQUIRED.
This subcomponent defines the options an evaluator has for
responding to the criteria's question.
The type of response expected from evaluators. This
property is REQUIRED. Currently defined types include:
-
multipleChoice
-
The evaluator selects from a predefined set of
labeled responses.
An ordered array of the possible responses available to
evaluators. This property is REQUIRED when the response
type is multipleChoice.
A short identifier for the response. Labels MUST start
with "A" and progress sequentially through the English
alphabet. This property is REQUIRED.
A description of what the label represents. This
property is REQUIRED.
This subcomponent explains why this criteria is useful.
Readers should be able to understand the extent to which
this particular criteria is applicable to their situation.
This subcomponent lists example assessments of different
DID methods against this criteria. Each example assessment
is an evaluation of a specific DID method, structured
according to the criteria's assessmentTemplate.
The set of example assessments is presented as a table for
easy comparison across methods.
-
Example assessments for a given criteria SHOULD
highlight distinct responses. That is, each assessment
should have a different set of responses from the other
example assessments. The point of the examples is to
illustrate how different DID methods score differently.
-
The editors will curate the list of proposed examples
to provide a best effort illustration of each criteria,
with consideration to highlighting a broad corpus of
methods.
- Criteria SHOULD have no more example assessments than
there are possible responses.
This OPTIONAL subcomponent tracks the provenance of the
criteria. Criteria in the Rubric may be independently
versioned and developed externally under their own
identifiers. When such iterations are folded back into the
Rubric, the source MUST point to the external origin so
that the versioning history is preserved.
The value MUST be either:
-
"."
-
The criteria originated in this document.
-
A URL
-
The criteria was developed externally. The URL
SHOULD point to the specific criteria at its
external source.
If no source is provided, the default value is
".".
Each criteria must be explicitly, uniquely, and persistently
identified using incremental numbers. New criteria should
use the next available increment based on the highest
numbered identifier in the current publication.
Previous numbered identifiers MUST NOT be re-used, even if
the criteria so identified are no longer published; this
maintains the persistent linkability of previously published
versions. Such "retired" criteria MAY be listed in a Prior
Criteria section linking directly to the latest published
Rubric containing the retired criteria; this Prior Criteria
section MUST also contain an anchor with the canonical id.
The registry shall maintain an entry with the "next
available" number. Additions should use the "next available"
number to construct an identifier for the new criteria, and
update the "next available" entry at the same time. Editors
will manage any sequencing errors when accepting PRs.
Identifiers must be constructed by appending that numerical
value to the string "criteria-" and incorporated in the ID
of the heading element for that criteria, which when placed
in the Rubric appears as something like (note the version in
its own span after the permalink):
<h4 id="criteria-1">Open contribution (participation)</h4>
<a href="#criteria-1">https://www.w3.org/TR/did-rubric#criteria-1</a> <span>1.0.0</span>
This allows permanent links to citations based on the
publication date of a given Rubric:
https://www.w3.org/TR/2021/NOTE-did-rubric-20210826/#criteria-32
as well as version-free links that resolve to the latest
version of the criteria, and if retired, to an entry in the
Prior Criteria list which itself links to the last valid
presentation of the criteria. So that, for example, https://www.w3.org/TR/did-rubric/#criteria-32
points to the current criteria-32 in the latest published
document. When that criteria is retired, that same URL
points to the entry in the Prior Criteria list, which links
to its last versioned publication, e.g.,
https://www.w3.org/TR/2021/NOTE-did-rubric-20210826/#criteria-32
Criteria versions allow for incremental improvements while
retaining long-term referenceability.
-
Versions contain a Major, Minor, and Patch number, based
on SemVer 2.0
-
Every new criteria has a version of 1.0.0.
-
Updates to criteria must increment the appropriate version
number based on the scope of change:
-
Major number MUST increment if,
and only if, the criteria is fundamentally
changed in a way that invalidates existing evaluations.
This could be a material change to any subcomponent of
the criteria, including removing or changing Possible
Responses.
-
Minor number MUST increment if,
and only if, new responses are added, without
invalidating existing responses. It's understood that
an evaluator of an existing evaluation may desire to
choose the new response if they were evaluating the
criteria anew; however, as long as prior evaluations
remain valid, a Minor increment SHOULD be applied.
-
Patch number MUST increment if,
and only if, the change to the question is
"editorial", defined by clarifying the original intent
of the criteria WITHOUT invalidating existing
evaluations.
-
As long as a given criteria is an evolution of essentially
the same consideration, it retains the original
identifier, even as its version is updated throughout its
lifetime.
-
Criteria that address fundamentally different
considerations MUST be assigned a new criteria number
rather than iterating an existing version number.
The Editors of the DID Method Rubric Registry MUST consider
all of the policies above when reviewing additions to the
registry and MUST reject registry entries if they violate
any of the policies in this section. Entities registering
additions can challenge rejections first with the W3C DID
Working Group and then, if they are not satisfied with the
outcome, with the W3C Staff. W3C Staff need not be consulted
on changes to the DID Method Rubric Registry, but do have
the final authority on registry contents. This is to ensure
that W3C can adequately respond to time sensitive legal,
privacy, security, moral, or other pressing concerns without
putting an undue operational burden on W3C Staff.
Submissions to the registry that meet all the criteria
listed above will be considered for inclusion, however the
editors retain the responsibility of curating the content to
ensure the broadest applicability of the DID Method Rubric
with the goal of enabling effective evaluations of any DID
Method for any legitimate use case.