22:04:23 RRSAgent has joined #did-topic 22:04:23 logging to https://www.w3.org/2021/04/13-did-topic-irc 22:04:24 present+ 22:04:30 present+ 22:04:31 present+ 22:04:47 present+ 22:04:49 zakim, this is did-topic 22:04:49 got it, brent 22:04:51 scribe: cel 22:04:57 chair: brent 22:05:14 present+ juan 22:05:17 meeting: DID WG Special Topic Call - DID Test Suite 22:05:36 rrsagent, make logs public 22:05:50 brent: Getting started 22:06:02 present+ geun-hyung 22:06:12 q+ 22:06:17 ... This is a meeting to discuss issues and problems with the test suite, as well as to work together to resolve them 22:06:25 ack manu 22:06:50 manu: I thought we could go over the two PRs, just one outstanding, need input from Orie and Charles. 22:06:57 ... The other one, need confirmation from Markus 22:07:12 ... Then may handle challenges in tests 22:07:26 sounds good 22:07:30 ... Thoughts about discussion. Changes? 22:07:43 present+ 22:07:55 ... Jumping in. Screen sharing. 22:08:04 ... First item: 22:08:13 ... 2020 did:key method 22:08:17 q+ 22:08:18 ... Waiting for folks to approve 22:08:30 ack markus_sabadello 22:08:46 Orie has joined #did-topic 22:08:52 markus_sabadello: Thought it was okay. But wondering if we want to merge implementation reports at this point or after. But either way is okay 22:09:08 manu: You were saying we aren't committing implementations yet? 22:09:21 markus_sabadello: yes, formats may change ... not finalized 22:09:50 manu: Given I have been restructuring, am happy to keep these up to date with changes 22:09:55 ... Will go ahead and merge 22:10:27 https://github.com/w3c/did-test-suite/pull/55 22:10:34 manu: Addressing feedback 22:10:50 ... Could change representation -> didDocumentStream 22:11:04 Orie: want didDocumentStream. But it could be separate PR 22:11:08 manu: will update this PR 22:11:10 q+ 22:11:18 ack markus_sabadello 22:11:43 manu: what do folks want for the key? 22:11:51 markus_sabadello: representation more accurate 22:11:57 Orie: should update DID Core then 22:12:12 manu: We are about to get a bunch of implementations 22:12:22 Orie: do what the spec uses. Why use a different term? 22:12:37 manu: fine changing it, sorry markus_sabadello, like representation better but... any objections? 22:12:56 markus_sabadello: section talking about didDocumentStream is only in resolution 22:13:02 ... other sections talk about representation 22:13:11 Orie: sad. why do we have didDocumentStream in spec at all? 22:13:26 Orie: when I look at this, I expect it to be about resolution, therefore expect it to match 22:13:42 manu: it is not, it is associated with the ADM 22:13:48 Orie: could change spec ... 22:13:53 manu: that would be normative change 22:14:03 q+ 22:14:21 manu: would change implementations? or maybe wouldn't. 22:14:32 Orie: is the section on resolution normative? 22:14:39 manu: yes, it's left hand value 22:14:58 Orie: I would expect document that is a DID document stream to have value didDocumentStream 22:15:07 manu: but it's not resolution 22:15:17 Orie: is it a document encoded as a byte string? 22:15:29 manu: section 6 - representations. 22:15:45 ack cel 22:15:46 Orie: I guess this is why better to write tests than specs 22:15:49 scribe+ 22:16:02 cel: I don't have strong opinons 22:16:03 q+ 22:16:11 ... wasn't aware of the discrepancy 22:16:25 manu: happy to flip vote 22:16:29 Orie: what is this section used for? 22:16:46 manu: representation-specific entries, ... data model core properties tests 22:17:16 ack markus_sabadello 22:17:40 markus_sabadello: there is also did document metadata and resolution metadata in this fixture. is it needed there? 22:17:51 manu: It might not be needed anymore. There are some tests using DID resolution metadata 22:17:59 markus_sabadello: These tests should not need that 22:18:16 manu: Some parts of the specification say must signal the media type in some way... there are two or three tests that use that 22:19:12 ... Can change later ... search and replace. Would anyone object to that? 22:19:24 Orie: Would be fine with raising an issue 22:19:44 manu: The other one... https://github.com/w3c/did-test-suite/pull/55#discussion_r611645272 22:20:01 ... I considered that when writing the tests, but unsure if people want the same data model to apply to different .... 22:20:04 q+ 22:20:10 ... representations or DID document streams or whatever you want to call it 22:20:19 Don't cross the representation streams 22:20:47 manu: data model properties and representation-specific entries... Charles is proposing moving it up 22:20:48 q+ 22:21:07 ... but I was unsure if people would want that. This allows us to have different data model representations per serialization (?) 22:21:12 ... This mechanism is more flexible 22:21:25 cel: can you clarify? 22:21:50 manu: did+ld+json. representation-specific entries with context. in did+json, might have that but not expecting it to be preserved 22:22:09 ... If we were to take this and move it up, then all of the sudden, our representation-specific entries would have other entries 22:22:37 ... Raises questions. Whereas this... we have representation + representation-specific entries, and expect this result 22:22:51 ... Concern, in the future will have other things 22:24:12 ... representation-specific entries are meant to be for the representation being produced. I don't think we have consensus on what should be in there, but spec should say what to preserve 22:24:23 by_caballero_ has joined #did-topic 22:24:24 cel: ok, i think it's fine 22:24:36 present+ 22:24:42 ack Orie 22:25:02 Orie: Don't know about others, but for JSON should preserve. But I would put them outside, and show in our implementation that we preserve ... 22:25:27 ack markus_sabadello 22:25:42 ... Each implementation can still show that they preserve whether inside or outside... but inside has more duplication 22:25:59 markus_sabadello: Should be outside, because they are representation-specific entries 22:26:00 q+ to note perfect being the enemy of the good. 22:26:30 q? 22:26:33 ... It is called representation-specific entries so should be in the block. But the dmProperties should be outside 22:27:07 manu: I want to point out that doing these kinds of changes isn't actually changing the outcome of the test suite. Doesn't need to be perfect, just need to make sure the statements are tested 22:27:38 ack manu 22:27:38 manu, you wanted to note perfect being the enemy of the good. 22:28:15 q+ 22:28:19 manu: . Is this correct? 22:28:28 ack by_caballero_ 22:28:55 by_caballero_: I hesitate to have opinions on this because it is very complex. I think many people are on mute because it boils down to the translation issue that is confusing to a lot of people 22:29:12 manu: Going to implement that. Heard Orie and Markus agree 22:29:23 ... Then will merge the PR. 22:29:45 ... markus_sabadello: do you want to talk about the tests that are challenging next? 22:29:59 markus_sabadello: Yes. 22:30:27 markus_sabadello: I implemented tests for DID resolution and dereferencing. There are some statements I am not testing yet. 22:30:38 ... Summarizing in an open issue. Some reference other parts of the specificatioon 22:30:49 ... Such as production and consumption that we just talked about 22:30:57 https://github.com/w3c/did-test-suite/issues/53 22:31:27 ... Something I haven't implemented, because thought I would be able to use code that has now been merged... I will probably try to do another pass on the DID resolution where I try to reuse some of the other code 22:31:47 ... I could test the resolve function, and what comes out of it could then run through production and consumption code to effectively test this statement in the resolution section 22:32:04 ... That is my idea, why I didn't initially implement some of these statements 22:32:28 ... There are two or three of those. There is also one that says that one of the return values of the resolution function is DID document metadata, must be a valid metadata structure 22:32:40 ... There are other tests now testing whether something is a valid metadata structure 22:32:49 ... I will do another pass on the resolution tests to see if I can complete some of the missing ones here. 22:33:04 markus_sabadello: There are also a few statements I don't know how to test them. 22:33:07 q? 22:33:15 ... Related to equivalentId and canonicalId 22:33:24 ... about what a DID method specification must do 22:33:30 ... I don't know how to write a test for it 22:33:35 q+ 22:33:59 dbuc: I think this goes down to a deeper point about so many parts of the spec. Of course we all accept, I hope, that DID methods have to do things that are not outwardly testable. 22:34:18 ... How does it find the right keys? That is a method-specific question. Replete throughout the spec, there are countless such questions. 22:34:25 ... Service endpoints tracking 22:34:35 ... I don't see this as any different 22:34:41 ack dbuc 22:34:41 q+ 22:34:45 ack manu 22:35:03 manu: I agree with that, dbuc, I think it's unfortunate that we put normative statements here for these two. 22:35:22 ... But the thing that sells it for me is that it refers to DID method specifications. 22:35:27 ... That is not machine-testable 22:35:47 dbuc: One thing that is testable, is you can write tests, obviously contrived... 22:36:02 ... You can write a database in memory that updates a column with canonical value 22:36:13 ... I can write a test that does the correct thing. 22:36:28 ... One might use MongoDB, or MySQL. Queries will look slightly different 22:36:57 manu: markus_sabadello: Are you okay with this? 22:37:06 manu: Yes, don't need to write tests for these items. 22:37:27 markus_sabadello: Maybe I will mention this. 22:37:39 you can make the statement as .skip and add a comment in it pointing to the meeting minutes for the call. 22:37:43 ... This one, Charles had a suggestion in the thread on how to test this, which I think will work 22:38:03 ... This one, and this one, ... I will try to somehow test them using test code from other sections 22:38:28 ... This one ... must return in conformant representation ... I think I know how to test it as well. 22:38:44 ... That is the whole list; I will do another pass and report back on the issue 22:39:04 manu: Charles, are there outstanding tests you had any challenges with? 22:39:08 cel: I don't think so 22:39:11 q+ 22:39:15 manu: Orie 22:39:18 Orie: I don't think so 22:39:22 dbuc: I was, a little 22:39:28 manu: Any of the other ones? 22:39:49 I am in a holding pattern for my assigned issues 22:39:50 ... shigeya I think there was one? Were you able to address it or should we talk about it? 22:40:11 shigeya: I was observing the discussion on how to phrase the method-specific representation parameter thing earlier in this call. 22:40:24 ... I am wondering how we can adapt for the did-spec-registry thing in my context 22:40:34 ... Let me think about it and ask questions later. I think we can find a better way. 22:40:58 manu: dbuc, you have the JSON tests assigned to you. Let me share ... 22:41:07 ... https://github.com/w3c/did-test-suite/issues/26 22:41:20 ... JSON Production and Consumption ... There are examples of what these tests look like 22:41:21 ack dbuc 22:41:37 dbuc: I looked at the did-jsonld-production file ... 22:41:57 ... Inbound on that function there is an object with resolutionResult ... don't see same structure 22:42:00 manu: It is old 22:42:13 ... You can use JSON-LD production as a starting point 22:42:26 dbuc: Ok, I can do that 22:42:46 manu: For consumption... I guess we haven't had a case where we needed consumption tests. 22:43:06 dbuc: I also had a sortof general question about the JSON-only version in these tests. In the production I didn't really see anything super-specific 22:43:18 ... Other than it ... has this media type 22:43:39 ... Consumption, I wasn't seeing about ... bouncing values, even though justin_r on previous call said ... need to bucketize ... 22:43:57 manu: Consumption is basically JSON.parse with making sure you are testing all these values as input. That's really all consumption is. 22:44:08 Orie: You'll just struggle with numbers and dates, because there aren't really any examples 22:44:26 ... I don't think anyone has shown a DID document that encoded a number, or null; it's just not a thing 22:44:47 manu: But there is ... But the vast majority is JSON parse and stringify. 22:45:11 dbuc: The context value.... do I have to care? No but you have to do something to show you are doing something with it? 22:45:16 manu: That's sortof already there. 22:45:30 ... The best thing to do is to take a shot at writing the tests. 22:45:39 ... People will complain if they feel it is not right 22:45:49 ... Let us know how we can help. 22:46:20 ... That shoul be all of the tests, I believe. Every one we have been contemplating - modulo CBOR Production and Consumption which johnnycrunch has on his plate to do. 22:46:32 ... Orie, the visualization thing, I think we are close 22:46:42 Orie: Will kick it... until more to visualize 22:46:49 manu: There are three 22:47:04 Orie: did:example I don't really count as real. 22:47:14 manu: two did:key implementations 22:47:28 ... Assigning it 22:47:53 ... Other thing to bring up... dbuc since you are here, we ask people who are implementating equivalentId and canonicalId , didn't get anyone except.... 22:47:58 dbuc: Troy, did:orb, ... 22:48:03 manu: We just need two 22:48:09 ... ion 22:48:16 dbuc: Don't know... SecureKey... 22:48:30 manu: We just need two independent implementations 22:48:39 ... I think that's it. Is there anything else test suite related 22:48:46 ... that folks want to talk about? 22:49:00 ... Reminder on roadmap and timeframing: we need tests done by end of this month 22:49:07 ... Please do as soon as you can 22:49:17 ... End of April all tests in, and of May all implementations in 22:49:22 brent: That's the goal, yes 22:49:29 dbuc: Are there more things on the agenda? 22:49:32 manu: Not from me 22:49:38 dbuc: Can I ask one thing about the relative path stuff... 22:50:01 ... It wasn't immediately clear.... we were using a blank string because populated by JSON-LD and seemed fine... 22:50:08 ... Is using just a pound sign okay? 22:50:11 q+ 22:50:21 ... Could use fully qualified URLs. But ... is ambigious ... 22:50:25 ack Orie 22:50:40 Orie: You should switch to using fully-qualified URLs; everyone will thank you 22:50:46 ... and you are probably already gzipping 22:51:03 Orie: We messed up when we allowed relative refs in the DID document. 22:51:20 dbuc: Can switch it... We have people asking... It's still in the spec. What is even allowed? 22:51:37 ... On the web, pound sign if not populated doesn't refer to. 22:51:51 Orie: It's the same, but then there are other normative statements 22:52:00 ... Like about controller property, must be a DID 22:52:07 https://w3c.github.io/did-core/#did-controller 22:52:15 manu: To answer directly: nothing will change in the spec unless some raises a PR or issu 22:52:30 ... We are locked in. If you feel additional language needed, can... But I suggest let's not go there 22:52:39 brent: Anything else? 22:52:54 ... Thank you all for coming. 22:53:08 ... Special call to be held weekly until test suites done. 22:53:15 ... Let's keep moving forward 22:53:19 thx all 22:53:43 You are welcome 22:53:49 zakim, end the meeting 22:53:49 As of this point the attendees have been manu, brent, cel, dbuc, juan, geun-hyung, shigeya, by_caballero_ 22:53:51 RRSAgent, please draft minutes 22:53:51 I have made the request to generate https://www.w3.org/2021/04/13-did-topic-minutes.html Zakim 22:53:54 I am happy to have been of service, brent; please remember to excuse RRSAgent. Goodbye 22:53:58 Zakim has left #did-topic 22:54:02 rrsagent, bye 22:54:02 I see no action items