Meeting minutes
<swcurran> W3C one
<Wip> transcriber-bot, resume
Agenda Review
<pchampin> transcriber-bot, connect
Will Abramson: shoes, if...
… If we get to it
… Um, mm-hmm. I'll just ping Jenna
… Uh, okay
… Let's go into the Diplomatic Charter
DID WG Charter - CID Spec
<swcurran> Should we call the discussion "Resolution Client" instead of "DID URL Dereferencing"?
Will Abramson: Good. Yeah, great, um
<Wip> w3c/
Will Abramson: So this is just to
… update folks of this PR that's live, um, uh, out for review. I don't know if, Manu, you want to speak to it, but it's basically just to say
… We, the DID Working Group, are about to recharter. One of the things that we've discussed multiple times is bringing the SIDS back under the DID Working Groups. uh
… authority, if you like. Um… I think the discussions, maybe I'll let Mano get to it, is about whether this is a maintenance mode
… Or not for this. This document. I think we've moved that we are, but I'll let Manu. Here we go
Manu Sporny: Sure thing, um...
… One second, I'm trying to get the pull request up. So, right, exactly what Will said. The pull request updates the charter. It lists the SID spec as a deliverable
… Um, it initially stated that it would be maintenance only. Um, Ivan's input was, yeah, but
… we may want to rearrange the spec. Uh, his primary concern was around, like
… We jammed multi-base and multi-key into the SID spec. It wasn't the ideal place for it. We just did it for expediency's sake, and we needed to define it somewhere in a way that was shared between multiple specs, and it's just where it landed. Long term, it's the wrong place to put it. The multi-formats, multi-base, multi-key should be their own
specs
… Um, and they should probably be registries. So, uh, I added language to say that the working group may choose to reorganize the specification, the SID specification into multiple specifications and or create new W3C registries for
… portions of the technologies defined by the spec. Um, Yvonne said that addresses his concern
… Uh, yesterday on the call, Brent Zundel, uh, one of the co-chairs of the VC working group, uh, raised the question around Class 3 and Class 4 changes. Um, he noted that if the DID working group is going to take this spec over, why don't you, like, really take it over and
… the, you know, develop it more. Um, uh, I noted towards the end that the expectation was that the DID working group charter was going to be a maintenance charter in a full rework of SID or massive, you know, being able to, like, really change it significantly
… Um, would then call into question on whether or not the did work… the did core spec, uh, should, uh, be, you know, actively, uh, improved upon, which would then bring in the question, like, are we actually a maintenance, uh, working group or not?
… Um, I suggested as a compromise that we, uh, we say that we'll work on Class 3 and Class 4 changes
… that are… we may work on them, so it's up to the working group to decide whether or not they want to work on it, and we will… the cutoff is, uh, those issues, uh, and PRs need to exist before the charter goes into effect
… So it's not a blanket license to change SID drastically. Um, uh, DID would still… DID and DID resolution would still be maintenance mode, uh, and SID would largely be maintenance mode, except for a very small exception on Class 3 and Class 4 changes
… So, uh, that is the current proposed text. I'm waiting to see if that would address Brent's concern, and I'm waiting to see if this group would have any concerns about that, um, uh, extra language
… Uh, being, uh, added. Um, and I will put the link
… to the pull request in the concrete language there. That's it
<manu> https://
Will Abramson: Great. Thanks for that, Manu. Yes, this is just a request if you have comments today or just please do take some time to review that PR. I think I see PA on there...
Pierre-Antoine Champin: Yeah, one small comment. There is a middle ground between keeping the multi-base, multi-key as they are in SID and forking them into a completely separate registry...
… the process has this notion of embedded registry, so we keep it in the same REC, but the REC contains the description of registry tables with how to update them. So it might be a lighter weight option if we
Will Abramson: It's mine...
Pierre-Antoine Champin: If we don't want to take over the work of spinning off a complete distinct deliverable...
Manu Sporny: Yeah, thank you for the option, PA. Yes, we could choose to do that. I feel like we've taken half measures on these specs for 5 years now, and we, you know, are to the point where we should probably...
… refactor them, noting that it is up to the working group to determine how they publish specifications like whether or not they split up specifications. So this wouldn't be significant normative changes. It would just be editorial refactoring into multiple specifications
… um, publishing as, you know, uh, an FPWD all the way, uh, through to a REC. And it is, again, um, in the charter, it's up to the working group to decide how much work to do there, if any, right? Um, I think the big concern with the DID
… maintenance group is that, um, a number of our collective attention is going to shift to DID method standardization, um, and or other verifiable credential stuff, and, um
… we will not have as much, uh, as many people in this group being actively participating, uh, as we have over the past, um, two and a half years. Um
… So all that to say, you know, my personal preference is keep it as largely maintenance mode, defer to the working group on the level of work that they want to do based on the number of people that show up to the new working group. And, uh
… And I think, you know, PA, we don't have to mention
Will Abramson: Right. Okay...
Manu Sporny: that option, it's just there for us if we want to take it, right, if the working group decides to go that way. Um...
Will Abramson: Thanks, Manny. Um...
TPAC
Manu Sporny: That's it...
Will Abramson: Right, let's move on to the next topic, which is TPAC. Um… Oh, sorry...
Manu Sporny: Sorry, um, a question to PA. Should… what is… what is your expectation around merging the PR? Um, I… I want to wait for Brent to respond, but if you are… if you need to get this in front of the membership for a vote...
Pierre-Antoine Champin: Uh, well, the, the...
Manu Sporny: Um, do you have a timeline that you want us to hit?...
Pierre-Antoine Champin: At the end of the day, we would like to start the votes...
… at least 28 days before TPAC starts. So, uh… and… I would not feel entirely comfortable in sending it to vote the day after this is merged
… I think it… I mean, the pull request being out there is already available for review, obviously, but some people might only look at what's already merged, so… Uh, so let me take my, uh, my calendar, and, uh, basically
… We need to start the vote, uh
… Before the 4th of October. And so, um
… So ideally, I would like this merge before the… before the… well, 4th of October is a Sunday, but yeah, okay, before the 28th of September
… Yep. Absolutely, yeah
Manu Sporny: I am pinging Brent right now to get his thoughts in on that Class 3, Class 4 change stuff, and then it would be good to have a number of people in this group either thumbs up it, or object to it, or something...
Pierre-Antoine Champin: Thanks...
Will Abramson: Yep, great. So I think that's just a request to everyone in this group. If you've got a moment, please review this PR. It's about the Git working group charter, ideally by next week, so we can get it merged. Perfect...
<Wip> w3c-cg/
Will Abramson: Okay, now briefly on TPAC, I know we discussed this last week, but
… Again, so this… we want to discuss this issue, which is raised by the SOLID folks, about the TPAC agenda, specifically around identity on the web. It's happening on Friday, which I think will overlap with the BC working group. Um, for just, uh
… An hour and 15 minutes, like
… I guess I just wanted to socialize this. I mean, I think probably I'll be at TPAC, and I'll try and go to this, even if it does overlap with the VCWG stuff, to represent bids. Is there anything else that we wanna
… think about as a group around how we might show up for this. Manu?
Manu Sporny: There's a pretty big cross group meeting between verifiable credentials, web payments, EMVCO, FIDO Alliance, Federated Identity, you know, the DC API stuff...
Will Abramson: Uh, yeah, I think that's fine. I think we can just meet for the first… this is at the end of the day, right, Manu? Like, this is...
Manu Sporny: Um, that is going to be in the final hour of our meeting on Tuesday, the DID Working Group meeting on Tuesday. I'm just noting that. I'm going to have to go to that other meeting. Um… I don't know if anybody else is going to, and I'm just flagging that as a conflict...
Will Abramson: Yep...
Manu Sporny: It's, it's, uh, it is, it overlaps with the group meeting time on the calendar. It is at the end of the day, yes, but there's...
Will Abramson: Yeah, I think we were just imagining that we would have all the sessions and probably end a little early and go to that one. I guess if there are critical things that we need to discuss and people want to continue to talk about. talk, then maybe we can, but I think I expect a lot of people are going to go to that session, so...
… My expectation was we would have three sessions of the working group
… Okay, any other comments from folks about T5?
Joe Andrieu: So...
Will Abramson: Not seeing anyone on the queue. So yeah, let's move on. I just want to remind. Oh...
Joe Andrieu: Sorry, Will, I did have a question. Do we have someplace that we're creating a draft agenda for what we're going to talk about there?...
… Very good. Thanks
Will Abramson: Uh, no, but I do need to get that set up, so me and Otto will take that...
<ottomorac> good catch Joe
Will Abramson: Yeah, I'll do that. And I did say I'll try and set up something so folks can let us know if they're attending. It should be great to start learning about. I'll do that next week. Okay. Let's move on
DID Rubric
Will Abramson: Uh, so next, we want to talk about the DIG rubric, and, you know, this is based on a request from Manu around, like, a bit more of a deep dive on what changes have we made to the DIG rubric, um, before we ask or request for horizontal review. So, you know, in our charter, in fact, we did say that we wanted to turn this into a registry, that
rubric into a registry, and we have even voted on that, I believe
… Uh, have a resolution to that effect, but now we wanna
… I mean, me and Joe have been doing quite a lot of work on it. I think maybe Joe has something to present around the changes that are in this V2 edition
… But we just want to start the horizontal review process, right? That's not to say that a DID rubric is final, it's just to start this process, because we don't really know what horizontal review looks like for a registry track document. There's not really a separate process, it's just treating it as a rec track document, so it's either going to
be
… fast, or it's going to be difficult. And we really want to just start that process and learn how it happens. uh
… Joe, do you have something to share, or… Okay
Joe Andrieu: I… I don't have a formal presentation, um...
… the… I mean, we could… we could certainly talk through it. What… what I'm curious about… so there were some back and forths with Pierre Antoine that was very clarifying for us. Um
… And so let me just say this out loud, and I think PA has confirmed this in email. The next thing we need to do is simply change the document in the repo
… Um, such that it is a draft registry document, and add the language that magically invokes the registration language. Like, we have to, um… Uh, I've… it's not a coordinator, uh
… the… we have to identify the person who's in charge using the language that's in the process, um, and then we have to identify the tables. There… we have described all this in the rules of the rubric, but we didn't quite use the language that's in process, so we need to make those changes, we need to change the status in, uh, RespecConfig,
and then I think it'd be
… it becomes a draft registry. My understanding, PA, is we are okay to do that. There's nothing procedurally we need to do further than that. Once we have that, um, then, um
… Uh, I think we need to come back to the group, um, when we are ready to go for AC review and basically make it a candidate rec. Um
… But we need some engagement with the group before we get to candidate rec. I don't know how much engagement we'll wanna have before we ask for horizontal review. Seems like we're ready for that. But I'll leave that up to the group to figure out, you know, do we do we want a more deep dive before we even start horizontal review?
… Or should you just schedule one deep dive before we go to candidate rec?
Will Abramson: Yeah, thanks, Joe. And I'm so happy to just take 10 minutes and go over the repo so people can orient themselves in it if they want to look at it themselves today. I'm interested to hear from Pierre and maybe you, Manu, who had those last time. Yeah?...
Pierre-Antoine Champin: Yes, just one point of clarification. It's not entirely up to us. We need approval from the team to transition this document from note to registry. So there is a...
… There is a process step, but apart from that, we continue working on the same document, it will be published on the same short name, but there is this approval by the team, because as any transition
… When we transition to a first public working draft, then to a candidate recommendation. So that's a transition. As any transition, it needs this approval. But on our, except for that, and I can take care of that whenever you tell me this is ready, this now can become a registry draft
… I can do that. But apart from that, I mean, there's no specific thing that we as a group need to do
Will Abramson: Great. Thanks, Pierre...
Pierre-Antoine Champin: Except, of course, turning the document to the proper shape...
Will Abramson: Manu?...
Manu Sporny: Yeah, on the question of when do we want the group to review it, I suggest before it goes out to horizontal review, primarily because, you know, this is something that we're kind of saying, like, this is what the group, you know, would like to do...
… Um, and I think, uh, you know, a decent bit has changed since the last time we really took a look at it. Uh, I think it would be good for it to be on the record that the group did take a look at this and gave it a once-over and a, you know, more or less thumbs up that it should proceed
… Uh, before it goes to HR. So that's all I'm, you know, asking for on the call today, is to just be brought up to speed about, like, has anything changed? What is the, you know, expectation here? Like, I'm looking at the document, I'm like, there's a registration process, okay
… That establishes the registry, but there's also a huge section on criteria. Are those separate registry entries? How is that being managed? Is it just creating the initial registry? Like, there are a lot of questions around how this thing's going to be managed. Um, uh, that I have, and so just
… hearing what the proposal is from Joe and Will would be good, as well as the delta of, you know, what has changed, and then what your expectations are for how involved the group needs to be, you know, with the registration process and
… you know, all that kind of stuff. Um, those are the details I'm looking for on the call today. Um, that's it
Will Abramson: Yeah, that's great. And so, I mean, we did ask a few weeks ago for people to take a review, but obviously maybe that's a bit daunting and we don't have time. So maybe I'll take 10 minutes and I'll review...
… the repo and the spec, and I think Joe also has some analysis of, like, exact things that change, so if I miss things, then… Maybe he can jump in. But. Oh yeah, sure
Joe Andrieu: Well, could I jump in before you go? Um… I just want to touch on some of Manu's things, I think...
… uh… structurally, we should just… you should just ask those, and we should talk about them. Um… the
… In general, the rules that are in the registry governing the criteria are not going to change. They have not changed since we first created it, which is there are some editors. Those editors are curating a set of criteria
… And that's just moving over into registry language So we talk about those as tables
… Uh, and so this, this document is just being enshrining how we've been doing it, um, so that we can do it under the registry rubric, uh, registry approach. That's it. Go ahead, Will
Will Abramson: Yeah, right. Uh, okay. Let's do this up...
… I'm going to share my screen. Maybe someone could just track the queue for me. um… Yeah, so
… I mean, this is the repo, right? Like, this is the spec, I mean, it surely should say V2 now
… Uh, three times. What I wanted to start is, in this repo, the main thing I think that we've done is just refactored the existing criteria
… into a set of JSON files, right? So now every criteria is defined in its own JSON that is then rendered in a very similar approach to how the DID methods
… list is rendered, right? Like, there's some JavaScript in here
… In this tooling folder, I mean, I really just copied what was going on in the… in the. Did
… thing, right? Well, the point is we're rendering a bunch of criteria. There are some other things that are additional that we define also in JSON files. So we have different evaluations that are cited, and evaluations cited are where these criteria originated
<JoeAndrieu> language from process we are currently missing: custodian and registry tables need to be specified as the editors and the specific tables in the document that are managed as registry tables.
Will Abramson: Um, and then methods considered. So every criteria has to have a bunch of example assessments, and those example assessments are assessments of the criteria for a specific method, and every method that is assessed, at least once, must be. in the document
… And there's just a way to hopefully, you know, when we do move to the registry. We're making managing these changes or like submissions to the registry much easier because those submissions must be through a pull request. Those pull requests should either add, delete, or change existing JSON files
… That really should be the compass of what the registry is managing, as opposed to, like, the rec track document
… Um, there is a few other changes. I don't know why mine does this, it's kind of annoying, this black thing. Anyway, uh
… There are a few other changes. I mean, the… probably the most important one, or, like, kind of important one, is this retired criteria
… Uh, Manu, I know you had a comment to, like, maybe we should remove this. The reason we have retired criteria, so these are criteria that either
… The editors have decided we should remove, or in often, often cases, they have been, like, iterated upon, but in such a way that has led us to create an entirely new criteria, and these are, like, derivative, or, like, source criteria from. The reason to have retired criteria in here is so that. This… this… the link to Criteria 3
… The LINCS criteria 3… Like, can stay as a permalink
… Right. And then you go to like, okay, this criteria was actually defined in like an older version of the work. Like we want criteria, you know, criteria, if you add a criteria to this registry, it should always exist, but then obviously we might remove it from like the set that we're curating
… Um, but we want that permalink to continue to work, if it's referenced externally
… Uh, some other tidying up stuff we did is, like, we removed a category. And
… Yeah, we've really changed a few things. Mostly what we've changed is, like, the example assessments. Before, the example assessments were often coming from a set that I think Joe and Daniel did in the past, and it wasn't that diverse in the different methods that we're choosing. Now, this registry is not about picking winners of, you know,
these are the methods we're trying to
… assess. Instead, what we've done is tried to select some methods that exercise the different. Um
… responses of the criteria, right? So like we want to try and pick methods that do exercise these different kinds of responses and show them in the criteria
… Uh, and then there are a few sort of minor changes, like, to the wording, like, just improvements
… I don't know if folks have got any questions, that's kind of my understanding of
Manu Sporny: There is a queue, Will, just to...
Will Abramson: Oh, yeah, okay...
Manu Sporny: And...
Will Abramson: Yeah, so can someone just manage that for me? Or… I'll have to chop change...
Joe Andrieu: Sure, go ahead. I think, Manu, you're on the queue...
Will Abramson: Mmhm...
Manu Sporny: Yeah, so this is great. I love the breakout into individual JSON files, though I didn't quite understand the, you know, I need to just take a look at what's in the JSON file and that there's a build process and that it ends up rendering in the spec in this way I think is great. Um, uh, the...
… A question around a process that I had was, like, okay, if somebody does an evaluation on a DID method, and it's raised as a PR, is, I guess, the… what is the job of the registry maintainers? Do they
… look at the evaluation to see if it's correct or not, um, or do they just, like, merge it in? Clearly, there are problems both ways. Um, thoughts?
<Zakim> JoeAndrieu, you wanted to answer evaluation question
Will Abramson: Yeah, I mean, probably Joe's thoughts, but I — oh, wait, Joe, you can just answer that. You'll answer it better than me...
Joe Andrieu: Yeah. So the only evaluations that we have in here are those that are cited in example assessments. So if someone wanted to add an example assessment that used an evaluation that isn't listed in here because maybe they came up with one...
… Then they would both cite their evaluation and then cite the add the assessment to that criteria. And that PR would be considered together So if it's a good assessment and it helps under explain how that particular
… um, uh, criteria, um, can be evaluated, then we would add the evaluation, and we would add the criteria. Um, there is not a
… check on whether or not any given evaluation is quote unquote good. That's not the role of this situation to sort of rank or evaluate evaluations. Evaluations are anchored by their author
… So, the evaluations we have here, one of them I did, one of them Daniel did, one of them was me and Eric
… In fact, the evaluation one, we should have some more names on actually will. That's a good note to us because we've lost the other co editors of that first paper that we did. But the point is the evaluations are solely the judgment of the evaluator
Will Abramson: Great...
<Zakim> swcurran, you wanted to say can there be multiple rubric evals for a single did method --
Joe Andrieu: Um, and it's not our role to decide if that was good or bad...
Will Abramson: Thanks. Stephen?...
Stephen Curran: So, I'm assuming that there can be multiple evals per DID method? Um, and...
… Presumably, um, given what you just said, presumably they could, um, contradict and battle with each other, which is probably fine
… One other thing to raise is that we've actually had a conflict in the in the
<JoeAndrieu> yes. the focus is on a good set of assessments that illustrate the criteria, not on ensuring the evaluations themselves
Stephen Curran: extensions registry about registering the same did method by multiple parties. And we haven't evaluated that. But that would
… complicate this just a little bit, um, because there could actually, um, it's looking like there could actually be multiple DIDs using the same method. But, um
… The direct question was, I assume there could be multiple, evals per method
Will Abramson: Yeah, I mean, I can speak to that a bit. I promise I'll give the similar thing...
… Right, the… I think the, like, thesis is, or the hope is, right, this is a set of criteria that the registry is defining and curating, but then people who want to evaluate DID methods are going to use this as, like, a
… you know, they want to create an evaluation for, like, did ESA… they're going to look at the criteria and select some criteria that are meaningful to them, based on the things that they care about, and then they're going to do that evaluation. And
… Maybe that's where it stops, right? The registry does not need to be in the loop for that thing. This is a bit of source material to help people do good evaluation. But if the evaluation causes them to, you know, maybe they have a really interesting evaluation of a specific criteria and they think that would be good as an example in the
registry, they can submit that to the registry
Markus Sabadello: Yeah, I'm not sure...
Will Abramson: Or maybe they think the criteria isn't quite right, and they want to change the question slightly, or they want to add a kind of response. That also would be an entry, like, a submission into the registry to, like, version the criteria. And then the last case is, like, maybe they define an entirely new criteria that is useful for
their. Evaluation, that they haven't...
Markus Sabadello: I don't know...
Will Abramson: That isn't defined yet. And then they also might submit that to the registry. There's no requirement to do that, but the idea is the registry is kind of like performing this curation role that is helpful for people wanting to do evaluation...
… That's my understanding based on working with Joe. Manu
Manu Sporny: Yeah, that's great. That addresses my concern that it would be an enormous amount of work on the maintainers to figure out whether or not to pull something in. Potentially, you know, that process could be automated...
… or semi-automated through some LLM usage. There are some concerns I have about, like, abuse of the registry and people submitting, you know, spamming things, but I think the maintainers are there to kind of
… Try and protect the registry from that sort of
… Uh, behavior, um, I also noted, uh, you know, there's like a did V1 evaluation in here, uh, that I think probably needs to be removed. Sadly, that was a failed experiment, and, um, I think it's interesting for historical reasons, but
… You know, I'm concerned about, you know, things that are out of date. And then finally, I'm wondering if the DID methods working group should have doing an evaluation as one of the exit criteria for any DID methods standardized or W3C in general should take that position
… Um, uh, because I think it would be good if you're gonna go and you're gonna standardize a DID method really anywhere, um, officially, um, or through some process. Um, I think, you know, maybe, maybe as a group we should say that, like, there's an expectation that you do an evaluation
… um, and put it into the, um, uh, into the, into the, uh, registry. So, um
… But all of this is pointing in a good direction, so I feel very comfortable saying, like, yes, this is a good direction. We should send it out to horizontal review. I don't think I have any further questions
Will Abramson: Great. Thanks, Matt. Stephen?...
Stephen Curran: Just a quick question. Your answer confused me. There are. Anyone can submit an evaluation, and it goes into the registry. for a DID method...
… And then there's curation on the ones that are floated up to be examples, and these… those are intended simply to improve the understanding of what the criteria is. Is that right? Uh, you sort of… Yep
Will Abramson: uh...
Stephen Curran: Okay...
Will Abramson: Not quite. I think it's, you know, anyone can complete evaluations...
… they only submit them if they think within those evaluations there are good examples that they want to be included in the registry. So this list of evaluations cited is the set of evaluations cited
… That are evaluations for, you know, that are used in the, in this, in this, in this sort of curated set
Stephen Curran: Really?...
Joe Andrieu: Will, can I jump to queuing?...
Stephen Curran: Wow...
Will Abramson: Like, it's not… if you've done an eval… if you've done an evaluation of some criteria that has no, you know, and you don't think that the registry needs to change, then I think what you do with that evaluation is kind of out of scope of this...
… Guess what? That's mine. Sure
Joe Andrieu: Respond. So there's a misconception that were in the last two questions. The rubric is not a registry of evaluations...
… It is… one of the main changes that we did in this run was to get rid of the six sets of DID methods that were dominating all of the
… the criteria because this is not a place that should be promoting any one method over any other. So the point here is to curate the set of criteria, not to curate the set of evaluations. So, um, contrary to what you had proposed, Manu
<Zakim> JoeAndrieu, you wanted to speak to curation not exhaustion as spam mitigation
Joe Andrieu: We should not be forcing evaluations, whether they're W3C sponsored DID methods or not, to get into here. This is not the place where we should be keeping a registry of all the good evaluations out there that we like and we want to promote. That would be a role for a different place
Will Abramson: Thanks. Act teacher, I don't know if that was all of your comments...
Joe Andrieu: Yeah, that was basically what I was getting on about...
Will Abramson: Okay. Uh, Marcus?...
Markus Sabadello: Um, I actually wanted to say that I thought what Manu proposed was a good idea to somehow build these evaluations into a process of submitting or standardizing...
… did methods. I mean, I I don't feel strongly about it, but it sounded like a way of of potentially improving the quality of submissions of opted methods. In in deep. We've done something similar right in in deep. We have a
… process for recommended DID methods, and as one of the steps for proposing a DID method, we added
… a requirement to fill out the DIT trades entry for a DIT method, so I thought it could be a good idea to
<swcurran> Doing a DID Rubric could still be done -- just not in this repo.
Markus Sabadello: To add, uh… Maybe the… the evaluation… to to the methods and
… And this also takes me to a question that I have, which is about the TID trades. At some point, there was some discussion about maybe aligning TID trades with TID rubric. I think the author of the TID trades, JC, is not active anymore, so maybe
… This isn't relevant anymore, but I'm just curious if there have been any developments regarding this topic
Will Abramson: Yeah, great. I could talk to a bit of both of those. I mean, certainly, the idea was to integrate the traits. There's an open issue about it. I even maybe submit a PR to the legendary thing that explored doing that. I mean, I think that's the problem, where we haven't had anyone...
… wanting to step up and do the work, like, we're definitely very keen to… I think it makes sense, right? They're kind of interesting, different kinds of criteria, maybe with yes or no answers, but there's a lot of work just to translate them into the format that we just have to go through
… So it's doing it. And then I wanted to speak to, uh, the idea of evaluations from, you know, as part of standardizing DID methods. I mean, I think that is a good idea. Like, you know, like, the DID methods working group, or whatever, would select the criteria that they care about
… and then perform evaluations based on those criteria, right? The DID… the working group could do those evaluations of specific DID methods. And then I think it's up to the working group where they… where they publish those, you know, that could be a note. Of the working group. But like, I think the point Joe's getting to, I agree with is
that's not the registry's role in this
… You know, and then separately, you know, if you develop a really cool DID method on your own, and you want to make a business out of it, or whatever, maybe you do an evaluation, and you publish it on your website, right? Like. I think there's lots of places where these things could be published that make sense
… But the registry is not one of them. Uh
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): I'm concerned that...
Will Abramson: Ted...
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): What the rubric was designed to do and to be is not the way that people seem to be talking about it today...
… Oh
… As a did method author
… I should not be evaluating my DID method according to the rubric, except as an ego boost for myself. Um
… The point of the rubric was to help
… people who need to use DIDS for some reason or other to evaluate
… Some number of methods, comparatively, against the criteria that matter to them
… There are some people for whom
<Wip> transcriber-bot, resume
TallTed: DID Traits are whether or not you have this thing in your DID Method, could be part of a rubric eval, but limited result.
TallTed: You ahve this or not, on or off, those are traits.
TallTed: Rubric is shades of gray eval, how much does this method handle this feature w/in rubric list and how much does this feature matter to me... 0%, 50%, 90%...
TallTed: Doesn't matter necessarily to anyone else, but that's not what the Rubric is for... unless we're really re-designing it.
<Zakim> JoeAndrieu, you wanted to suggest the did method registry might want to ask for evaluations, but the rubric is the template for evaluations, not where we store evaluations
JoeAndrieu: +1 to the spirit of what you said, that's right, the point of the approach of the rubric is that evaluations are subjective, you need to map use cases, sensisbility to conclusion. Nuance that I would add, segue into DID traits, did method creator doing their own evaluation is a form of marketing.
JoeAndrieu: But it won't be as high quality as a Consumer Reports style eval, or a consultant eval of 200+ DID Methods and see what it maps to... intended to be a subjective tool so someone can figure it out. Segue into DID Traits, halfway there to supporting them, first thing we completed to make it straightforward was to JSON-ify the registry.
JoeAndrieu: Unfortunately, DID Traits don't have all criterian necessary for registry, there is a hope that DID Traits are objective, but I look at some of those evals and disagree with finding.
<Wip> fyi you can see my initial attempt at integrating DID traits here - https://
JoeAndrieu: It would be useful, talked about architecture of two different registries, DID Methods registry, DID Rubric, haven't gotten into registry process w/ DID Methods, it would be appropriate for DID Method to register an evaluation, but ask from DID Traits (eval is nice) what we really want is an index/dictionary for specific traits so we can search in DID Methods registry... if you are looking for particular trait, you could filter them to top of
search.
JoeAndrieu: Rubric is about criteria, there is a refactoring to support Traits.
JoeAndrieu: Integrating traits into this is a bit of extra work to make DID Method registry more usable.
Wip: The vision, where we'd like to get to, there are ecosystems that define set of critiera that they care about... they have to eval based on that... what they care about, we're a long way from that.
Wip: Do we feel ready to get horizontal review on this?
<ottomorac> transcriber-bot, connect
manu: +1 on requesting horizontal review
… We are not to CR with this. Plenty of time to change things
Manu Sporny: view. You know, I mean, we're not even to CR or whatever with this. There's plenty of time to change things that we feel are of concern. You know, I hear what people are saying. You know, there's still a little bit of a work in progress going here. But I think the way that the work's been split apart is, is...
… manageable. Um, I didn't hear anything that was, uh, problematic. Um, the other I mentioned about, like, did traits and the did methods, um, and maybe, um, putting an evaluation, uh, in the did methods registry, uh, plus one to that. You know, we've been wanting to do that for many years now. I think this helps move that work forward
Will Abramson: Great. Thank you...
Manu Sporny: And that's it. Plus one to sending this out for horizontal review. That's it...
Will Abramson: Okay, anyone else have any comments? Or, I mean, should we run a proposal for this? I think that's probably the easiest way for us to, um… Just point at something to tag...
… Anyone opposed to us running a proposal? Ted
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): Um, just...
… A small follow-up. An ecosystem or a
… Group a consortium of organizations that care about the same criteria and
… running their team evaluation using the rubric, perfectly fine. It's no different from an individual running that evaluation, in my mind
… But they're not doing it for the general public. They're not doing it for the world as a Consumer Reports
… evaluation of the rubric against a number of DID methods. They're doing it for their own purposes, and with their own importance and their own criteria. It's not
… The rubric is not a Consumer Reports basis of evaluation, and
… it's never going to be based on how I understood it to have been created and evolved
… And if it's going to go in that direction, then what we have thus far needs to be really thrown out and started again. Because it's not that. That's it
Will Abramson: Okay, thanks, Chad. Uh, Manu?...
Manu Sporny: Yeah, I guess, Ted, I'm confused. You seem...
… Semi frustrated in arguing
… For something. But I don't
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): No, uh, just...
Manu Sporny: I don't think… I think the… I think someone said Consumer Reports, and maybe you latched onto that, and now you're kind of like, we're trying to do a Consumer Reports thing, but I think Joe and multiple other people said, it's not that. We're… we're...
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): Right, Joe said that it's not that, and that's where it stuck in my brain as something useful to use that comparison. Will is the one who just now, in transitioning between speakers...
Manu Sporny: Oh, okay...
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): Tried to put a counterpoint to what I said. Except that that's… again...
… that's still not what it is, and Will trying to guide it in that direction makes me want still to say, throw it out and start again, or do something different, because the rubric is, as most rubrics in the world that I understand to be there are
… is a list of criteria and how to understand them in this space, in this field of expertise, and how you, who are new to this field of expertise. Might want to consider evaluating
… some DID methods because you know you want a DID method in whatever application, but you don't know which ones are good for your application. So you need to look at what might the criteria be and say, oh, centralization, yes, that's important. Oh, anonymous, that's important. Oh, long lived without a
… a custodian, that's important. Whatever the criteria are that are important to you, and this is a good way to evaluate those criteria in the methods that you are considering. How you're choosing that list might be based, at least in part, upon this list of
… did method traits, because they are checkbox items. And not every method is going to have every trait, because some of the traits are in direct opposition to each other, so if you have a green in both of them, then something went wrong somewhere. That's what a rubric is all about
… And Consumer Reports doesn't exist in the world of rubrics because it's not a generic evaluation. It is subjective
… To the individual or enterprise or consortium. That is working together
Will Abramson: Okay...
TallTed // Ted (he/him) Thibodeau Jr (OpenLinkSw.com): To do a thing. I hope that was clearer...
Will Abramson: Yep, thanks, Ted. I mean, I don't know whether I agree with what you said. So sorry if I cause confusion...
… Okay, I'm wondering if we can run a proposal then, just so we get something on the record that we can point to for tag?
… Not hearing anyone opposed to that
Joe Andrieu: Yeah, let's do that...
Will Abramson: uh...
… Proposal text, though, I do not have that ready
… Well, I moved to horizontal review for the
Joe Andrieu: Well, ask… ask for a horizontal review...
… It's not a transition state, really
Will Abramson: Mm-hmm...
… Voila
<ottomorac> transcriber-bot, pause
Will Abramson: This one from Manning. Hey
<TallTed> fwiw, +1 to ask for horizontal review. It could have been good 6 months or longer ago, too.
<smccown> +1
<ottomorac> +1
<pdl-ASU> +1
<Wip> PROPOSAL: Ask for Horizontal Review for the the DID Method Rubric v2.0 https://
<manu> +1
<JoeAndrieu> +1
<Wip> +1
<pchampin> +1
<markus_sabadello2> +1
<pdl-ASU> +1
<JennieM> +1
<ottomorac> +1
<TallTed> +1
<swcurran> +1
<smccown> +1
RESOLUTION: Ask for Horizontal Review for the the DID Method Rubric v2.0 https://
<ottomorac> transcriber-bot, resume
Will Abramson: Okay, uh, wonderful. See all plus ones, so we are resolved. Thanks for that, uh, it's a good discussion today. Um, yeah, like I said, this is just the start, right? Like, I mean, maybe we need to turn the transcriber back on...
… Great. Yeah, so this is just the start, right? We still have a long way to go to get this to CR, but I think it's moving in the right direction
… I see we're pretty much at time, so we'll close there, and we'll resume the resolution client object URL that you're referencing this session next week. Thanks, folks. Have a good weekend