DID WG Telco — Minutes

Date: 2021-03-23

See also the Agenda and the IRC Log

Attendees

Present: Ted Thibodeau Jr., Charles Lehner, Joakim Soderberg, Shigeya Suzuki, Adrian Gropper, Brent Zundel, Daniel Burnett, Kyle Den Hartog, Manu Sporny, Orie Steele, Drummond Reed

Regrets:

Guests:

Chair: Brent Zundel

Scribe(s): Charles Lehner, Manu Sporny

Content:

1. Special Topic Call

Brent Zundel: starting with agenda review
… plan for it to be a test suite working session. if you have been assigned normative statements to turn into tests
… starting about the special topic call,
… giving brief overview of our timeline, where we are in the process of creating a recommendation, …
… and process moving forward
… then talk about resolutions made during the did spec registries
… then talk about testing of the did spec - as primary topic of the day
… Does anyone have any questions or items to add to agenda?
… Minutes in IRC serve as minutes for the meeting; want to have them correctly scribed.
… Special topic call is this coming Thursday at I believe noon eastern time.

Drummond Reed: No, looks really good

Brent Zundel: Now we are on to introductions and reintroductions

2. (Re-)introductions

Brent Zundel: asking joakim

Joakim Soderberg: R&D, machine learning engineer,
… working in car signal and related analytics

Brent Zundel: Welcome
… for reintros, asking Orie

Orie Steele: I’m Orie Steel, CTO at Transmute. I work on DIDs, VCs and supply chain traceability

Brent Zundel: Thank you, Orie. Next topic…

Manu Sporny: we are caught up.

Brent Zundel: Thank you. Apologies for the delay
… Please come if you are contributing to the test suite, especially if you have been assigned to write tests for normative statements
… Any questions on the special topic call before we move forward?

3. Timeline - where we are

Brent Zundel: https://docs.google.com/presentation/d/1nSLk3cwJ8CanDoMLsO_JS3-ltBEeM8HZVXSsAZbrIl4/edit#slide=id.p

Brent Zundel: This is a single slide. Anyone can view it.
… which I am sharing on my screen, which is this … where we are, where red circle is. Entered CR as of March 2021
… Working backwards from end of WG, need to have a rec by September of this year
… which means our proposed recommendation needs to be in place by August 2021
… That means that if we are going to go through another CR phase, the last opportunity we will have to do that happens in June of this year
… So if there are substantive changes that need to be changed in the spec to address feedback from implementers of the test suite, need to have them in by June
… This means we must have the test suite done as soon as we can.
… Please add yourself to Q if you have questions. We have time for a second CR if necessary, but only have a couple of months to get there

Orie Steele: please review https://github.com/w3c/did-test-suite/pulls

Manu Sporny: This past weekend I was able to implement all core properties tests, all conforming producer tests, JSON-LD production and consumption tests. I think that is the majority of the core of the specification now testable; added a did:key implementation and will add two more implementations. Good news: got through all tests without needing a second CR. Things could have gone really wrong and they didn’t. now biggest sections concerned about are the DID resolution and DID URL dereferencing sections.

Brent Zundel: Thank you for writing tests.

4. Process moving forward

Brent Zundel: Our process moving forward will be similar to as thus far. we are going to respond to issues with PRs. As those PRs come in, we are going to determine through consensus whether we are going to merge them in or now. Slight change: as PRs come in, we need to determine whether the changes they propose are substantive changes, = one that would require an implementer to change their implementation. It may involve normative statements.

Daniel Burnett: we are trying to restrict ourselves to nonsubstantive changes. something that would break the spec if we don’t change it.
… not talking about something not ideal, but if something fundamentally broken, we must fix it. Could happen if we discover we did not put in a distinguish needed to implement, or something that cannot be implemented that way
… Our goal is not to do that unless it is critical
… Even if we have a second CR, the smaller such changes the better

5. DID Spec Registries resolutions

Brent Zundel: During a special topic call, a number of resolutions regarding DID Spec Registries were made. This is primarily an informative step here

Orie Steele: also this big cleanup PR should be merged asap: https://github.com/w3c/did-spec-registries/pull/269

Orie Steele: since it will cause lots of merge confs if it lingers

Brent Zundel: but our process does allow for … statements

Brent Zundel: https://www.w3.org/2019/did-wg/Meetings/Minutes/resolutions

Brent Zundel: Here is the link to resolutions we made

Brent Zundel: https://www.w3.org/2019/did-wg/Meetings/Minutes/resolutions#resolutions-taken-at-working-group-topic-calls

Brent Zundel: The resolutions we are specifically talking about right now is further down the page ^
… We have resolution 1 2 and 3 at top of 2.1.2021

Charles Lehner: Resolution 1 …

Brent Zundel: change provisional to “Provisional. + YYYY-MM” so that date of registration can be used to sort

Brent Zundel: First resolution was this ^
… Second resolution was somewhat related…

Brent Zundel: Update the registration table to include, status, first registered date, last reviewed date

Brent Zundel: so that in addition to having a status, we would not when the DID method showed up in the registry and the last time it was reviewd

Kyle Den Hartog: do we want that only for a DID method?
… I’m fine to add all these things for everything registered

Brent Zundel: I think that is an excellent question, … to discuss in issues. These resolutions open the way for that to happen, but not necessarily. These resolutions are around DID method registration

Manu Sporny: Some of the things don’t necessarily apply to some of the other entries in the registries

Brent Zundel: third resolution: create a new table for 1.0 conformant methods registered after the spec is published

Manu Sporny: The biggest thing was for DID methods. We could talk about other things in a future call

Kyle Den Hartog: +1 agree Manu for case by case basis. Let’s start with did methods and I want to consider adding to others after

Brent Zundel: Once we have a spec, DID methods would need to either re-register, or at that point register to be added to a list that is methods conformant to the recommendation
… Because the current list of registered DID methods - we don’t have assurance that those are or will be conformant to the recommendation once its published

Drummond Reed: There is the issue 266 that has been started ^
… I wrote several things right after that call, just to bring to folks’ attention. This process to go beyond provisional is written up…
… I’ve written up a set of ideas and thoughts coming out of that special topic call discussion. Ivan has added some notes from the minutes of that meeting
… I invite any member of the WG. It’s pretty well-spelled out

Ted Thibodeau Jr.: When we made that first resolution, I was thinking the table was dynamically sortable, which it seems it is not
… Given the second resolution … probably don’t need that
… Is it possible to make that table a sortable thing? For someone coming to the page, being able to sort would be helpful

Justin Richer: I have not looked at what is generating that page, but there is a fantastic JavaScript library known as DataTables, set up to do exactly that kind of thing, sortable tables. I have used it on a number of projects, and recommend looking into it

Ted Thibodeau Jr.: Key question is whether Respec can do it

Brent Zundel: I’ll make a note

Justin Richer: I didn’t realize we were using Respec; apologies

Brent Zundel: Made note to ask Ivan about tools for dynamically ordering tables
… I agree the first resolution is somewhat subsumed by the second.

6. Testing the DID Spec

Brent Zundel: I think I put the wrong link in the agenda

Manu Sporny: https://github.com/w3c/did-test-suite/pulls

Manu Sporny: On the PRs: I’ve got a question for the group
… The question is how quickly can we merge these things in
… We have tests and certain implementations. If we fall back to the way that we work with the spec, it could take a long time. I would like us to iterate faster
… Meaning that hopefully it becomes very clear that there is an issue on a particular test
… I wonder if we can use Issues to iterate quickly, vs. standards and consensus-making process

Kyle Den Hartog: +1 for using GH issues to move test suite

Manu Sporny: Difficult on a test suite - it has a lot of code, difficult to follow if don’t read code
… Can we merge when at least 2-3 implementers have looked at, and have all tests passing? Or are we expected to wait for objections?

Markus Sabadello: I think it’s fine to manage the process through GitHub and make progress and merge things without necessarily having to discuss everything on a call

Brent Zundel: +1 to managing the process through github

Markus Sabadello: I would suggest we have a minimum time before merging
… Would not like to see something merged after 2 days if just 2 people approved it… that would be too fast I think
… In previous test suite work I have had some problems, misunderstandings

Daniel Burnett: +2 to managing the process through GitHub. Suggest that editors propose something

Markus Sabadello: Maybe we do it like that with a … 7 day period or so

Daniel Burnett: I’ve seen this done in a number of groups. The spec statements are normative; the tests are not
… Obviously we are using the tests to demonstrate the spec is implementable. Assumption of correctness. But most groups do not go through an excruciatingly detailed process around tests
… Usually the problem is to get people to write tests, and look at them
… The chairs … you guys need to agree on something
… If there is something grossly wrong, that it is not ignored. Most of the group will not be reviewing tests in any detail

Brent Zundel: I agree
… +1 to that

Manu Sporny: +1 to that
… don’t want things people disagree with to stick around. 7 day wit period for tests… what I was not looking for … not conducive … drag out forever. we have a tight turnaround
… We will not exit CR if people disagree about tests
… will not able to address CR unless we address … issues
… My suggestion is either get 3 people implementing to give a thumbs up on test, then can merge;
… Or after 7 days with the same amount of support
… to be clearer: 3 implementers +1 means gets merged, minimum 2 days
… if don’t have that, as long as Orie approves, at 7 days can merge

Brent Zundel: In the past, with test suites, it’s when an implementation is submitted and a test either passes or doesn’t pass (that the implementer expects to go the other way) - using the test suite is often the process where errors may emerge
… Agree that PRs being merged more quickly than not - doesn’t mean that errors will not be fixed
… Often the best people to judge are the editors and implementers

Manu Sporny: Curious to hear from markus_sabadello

Brent Zundel: would you opposed to the process manu outlined?

Markus Sabadello: two days is very short. there is a difference between adding a substantial new test suite and fixing smaller things
… Previously I looked at production and consumption, and I disagreed with how the tests were written
… Felt like if tests are merged after 2 days and something wrong, would have been better to discuss more
… But don’t want to block things and hold things up
… I’d rather have more time, but …

Shigeya Suzuki: How about if we are going to extend the review period more than 2 days if anyone lays concern
… If something difficult, someone thinks needs more review, can extend more days for reviewing it

Brent Zundel: +1 to Shigeya’s suggestion

Markus Sabadello: Sounds good

Shigeya Suzuki: We can add whether the PR is complex or not enough, can look at it and quickly determine whether need more time

Daniel Burnett: +1 to Shigeya’s suggestion (as rephrased by Orie)

Ted Thibodeau Jr.: Merge after a rolling 2 days without objection (i.e., since last change to the test PR), seems reasonable.

Brent Zundel: Good suggestion
… Any more questions and comments?

Manu Sporny: https://github.com/w3c/did-test-suite/pulls

Brent Zundel: Manu, want to go through PRs individually?

Manu Sporny: https://github.com/w3c/did-test-suite/pull/34

Manu Sporny: here is the URL
… This is a call to review or object
… Charles’ PR. out there for 15 days
… Meant to fix a number of items in the test suite having to do with camel case, versionTime, etc. Has two positive +1s
… Good example of something to merge immediately. No one voted against it. Orie and I think it should be merged
… Let’s try with what we were talking about - this would have been merged. Would anyone object to it being merged? #34

Markus Sabadello: +1 to merging 34

Drummond Reed: +1

Manu Sporny: We’re seeing if anyone has any issues with it

Brent Zundel: I am hearing no objections

Manu Sporny: https://github.com/w3c/did-test-suite/pull/37

Manu Sporny: That is going to be merged when the editors get around to it
… Next is #37. Raised by Joel. New tests for metadata properties. Also fixed a couple issues with the test suite - contentType, DID Resolution metadata, stuff like this. Two positive +1s by Orie and myself. Out there for 4 days.
… What did we say… sit for 7 days and then merge?

Brent Zundel: At least one more reviewer giving a +1 and we would be able to merge it

Manu Sporny: If you want to review, this is it. There we go, Ted, thank you, approved the changes. This will go in
… Three positive reviews. Any objections? If get objection, would hold off

Brent Zundel: I’m hearing no objections

Daniel Burnett: to be clear, it is no objections for 2 days. But it has been out there for a while and I think it is fine to ask on the call as you are doing

Manu Sporny: https://github.com/w3c/did-test-suite/pull/38

Manu Sporny: Agreed. This process is a little shaky since we just agreed to it
… Next: PR 38. A big PR.
… I would hold off for a little while before merging this, because it tests all of the DID Core process. There are 40-50 tests in here. Might be good for people to take a look at it. Honestly, the way people would see stuff go wrong is they go add theiri DID method and it fails… “Wait a second, I should be passing section … DID Controller” Then they see … didn’t implement correctly
… Typically when people discover issues with the tests, they discover it when they go to add their DID method
… I would hold off on merging this until the end of this week and 3 people approved
… Not going to ask for objections; people couldn’t reasonably review it on the call
… A lot of questions on this PR that I raised for Orie. Might be good to share your thinking on how this work. And Ted.

Orie Steele: for the record I agree with Ted and am grateful for the recommendation and feedback.

Manu Sporny: Previously, Orie had said these are vendor tests, Ted said is about implementations. I believe the test suite is now implementation-based
… One thing tried out is specify the implementation…

Manu Sporny: Implementations can be specified like so: “did:key - 2018 Ed25519 cryptosuite - did-key-js - Digital Bazaar”

Manu Sporny: I did an implemntation of did:key with 2018 Ed25519 cryptosuite for the did-key-js library by Digital Bazaar. That is how you tell people that’s the DID method that you are testing. did:key is an interesting example of that because we have multiple did:key implementations.
… Orie, did you see any issues with doing that; Ted, does that address your concerns?D

Orie Steele: I don’t, assuming it gets reviewed positively. I agree with what Ted said about implementations; we are starting to have multiple implementations of the same … by different vendors
… Think it is important to test multiple implementations by different folks for the same DID method… however we structure it.
… Generally +1 to your PR manu, we’ll see if it gets merged

Ted Thibodeau Jr.: I think the general motion is going in the right direction. This boiler plate needs to be translated to a boilerplate so it is clear what is in the columns. That is product you are producing as Digital Bazaar?

Manu Sporny: It’s a library we are producing.

Ted Thibodeau Jr.: the product name. Seems like one library … product

Orie Steele: +1 there needs to be a way to tell differences

Ted Thibodeau Jr.: Seems like need a way to differentiate implementations by a vendor… or all the complementary implementations of a given method

Manu Sporny: +1, need to be able to differentiate implementations; even multiple implementations within the same product within the same vendor
… think we have a path there with this proposal

Orie Steele: a better solution might be to formalize metadata, like github repo, company website, etc…

Kyle Den Hartog: What’s our take on when we are reusing libraries? Think will be common with the new Indy DID method
… For us, we will reuse implementations if they are out there. Do we consider it a different implementation?
… Or if because that library is conformant, we are also conformant?

Manu Sporny: Nuance: if all you are doing is taking someone else’s library of the shelf and reusing it, you are not a separate implementations. Those cannot be counted as independent implementations. But if you are taking someone’s library and doing something significant (hand-wavey) then it may be considered an independent implementation
… We have to be careful. This is about DID Document: if you are not doing anything different to the way the document is generated, it would be hard to argue it is an independent implementation if you are reusing that code path.
… We should be careful. People are going to scrutinize the implementation.

Orie Steele: Yeah, I’ve come across this in a couple other areas, in particular student competitions - for code reuse, students will use a straight-up copy to win a competition
… Typically, folks who manage the registration of the implementation will at regular intervals review the source behind implementations; often flag or disable entries too close to others
… Starcraft … most folks publish code with license forbidding submitting shallow forks without consent
… The burden is on folks reviewing submissions to look at the code if available. If available and a very shallow fork, to flag in some way
… But for code that is proprietary, absolutely know way to tell. Need to be transparent to evaluate, folks asserting what library they are using
… At end of day what we are doing is submitting test fixtures. You can totally cheat - I expect people will

Brent Zundel: All the tested implementations may not be truly independent implementations. Even if not fraud, it can be multiple implementations… it’s complicated, but I think we will be able to clearly see that everything has been implemented the way it needs to

Kyle Den Hartog: I was going to bring up the proprietary factor. We are that our customers are getting standards based implementations.
… whether or not we should submit … Sounds like … I’ll let you guys know

Kyle Den Hartog: thanks

Manu Sporny: When in doubt, ask questions: always a good strategy
… Other things from that PR: when you write a test, put the section number and title before each test, so people testing know what they are passing
… i.e. in your test text, don’t put “the value must be a string, …” - that doesn’t tell people where. Instead put “5.1.1 DID Subject … normative language”. Examples already there, trying to set up best practices there.
… Everything else is stuff we can clean up in the test suite
… I believe we want to make it easy for people to write/submit DID documents. Currently there is this deeply nested structure

Orie Steele: The original structure of the tests was based on the idea that you would be providing some structured data in the same format, and then the tests would iterate that structured nested data and find the parts that need to be tested.
… I shamelessly copied the integration tests from our did:key implementation, that uses a structure to allow us do testing with that… there were no objections
… Whatever solutions we pick, that problem will have to be implemented
… I suspect there will be some dependency on the structure of the fixtures. You will need to bind your input to a fixture structure in order to pass the tests.
… You’ll encounter these problems are you start to write tests

Manu Sporny: I think the structure is fine. it didn’t take a lot of effort. I didn’t read any documentation; felt self-explanatory. 30 minutes of frustration until figured it out, then fairly easy. Think it’s fine and fairly workable. Only other thing…
… The tests are weird and repetitive - because we are testing statements - effectively writing a linter - or JSON Schema
… Because we are trying to tell people exactly what part of the specification their DID method does not conform to … something to keep in mind
… “Why not just use JSON Schema” is a valid thought… Welcome to testing specifications instead of implementations

Drummond Reed: @kyle - good suggestion on the thread on DID method specification registration proposals

Drummond Reed: I’m responding now