DID WG Topic Call on Property preservation rules for production and consumption — Minutes
Date: 2020-10-13
See also the Agenda and the IRC Log
Attendees
Present: Shigeya Suzuki, Justin Richer, Brent Zundel, Manu Sporny, Tobias Looker, Kyle Den Hartog, Jonathan Holt, Dave Longley, Orie Steele, Dmitri Zagidulin, Drummond Reed
Regrets:
Guests:
Chair: Brent Zundel
Scribe(s): Manu Sporny, Tobias Looker, Dave Longley
Content:
1. Unregistered properties / Unknown properties / production consumption rules
Brent Zundel: Would anyone like to kick it off?
… We are looking for concrete PROPOSALS so we can make RESOLUTIONS to bring us to consensus.
Jonathan Holt: Wondering if anyone has taken a look at CDDL? Conversations around CDDL — very impressed by it. Manu and I talked about it, not an abstract data model, but there are some people, like Google that are writing the entire specification in CDDL. It gets us to a pseudo abstract data model if that’s a thing.
… There is some beneficial occupancy to getting it pushed forward. In core spec, put forward things we’re missing. Numbers are really a pain, the whole IEEE754 Bignum/Bigfloat and as Manu and I were discussion, there was a loss of precision when converting between JSON and CBOR… JSON can only do 53-bits of precision, whereas CBOR can do full 64-bits.
Orie Steele: I’ve seen that pull request, appreciative of Jonathan working on DID Spec registries and aligning this stuff. DID Spec Registries have been pushed to the side as a place to go to register properties. The registries were created with the hope of interop, but we haven’t had much of a contribution to it outside of Jonathan.
… There is a thing that’s happening with ADM is happening that doesn’t have to do w/ production, but there is a desire codifying normative statements in libraries, produce YAML/CBOR — this is highlighted by difference between DID Method produces representation on YAML and then someone does translation.
… When you ask DID Method to do that, you get VDR representation - DID Method intention… but when you do translation, you get some version of conversion of those properties, and we have language in producing/consuming — interesting to note, DID spec today w/ producer/consumer overrides DID Method author. ADM is now responsible for DID Method stuff — so even if I as a DID Method want to produce something, ADM might override my intent — I didn’t sign up to do how external software converts DID Documents. Security issues related to this ADM consumer/production language overrides intention of DID Method author.
Jonathan Holt: +1 to Orie, agree there is a potential security vulnerability.
Orie Steele: When you convert a DID Method from one representation to another, what you’re doing isn’t production, it’s a form of translation — you’re authorized to do that, but that process — I’m concerned about how much time we’re spending on discussing that because it doesn’t have to do with DID Method intent.
Orie Steele: here is the thread: https://github.com/w3c/did-core/issues/417
Markus Sabadello: Yes, Orie and I have been discussing this on an issue w/ Accept input metadata property — we’ve been using a fictional YAML implementation of representation — useful thought experiment, easy to do examples, easy to talk about what it would look like, at same time it’s not JSON, so you avoid some details about JSON and JSON-LD.
… On Orie’s point, one of my comments, I identified two conflicting objectives, on one hand, the DID Method author should have the freedom to specify and define what goes into a DID Document in a specific representation. If you define a DID Method and you want your JSON-LD to look a certain way, you want to have that freedom. Same thing for JSON or YAML.
… Make use of features of those specific representations…
… If you want to produce in JSON, you should be able to use whatever JSON allows you to do — at same time, you don’t want DID Methods to support every represenation… should be possible to support new representations w/o updating 65+ DID Method specs.
… it must work, get from ADM to every representation w/o requiring DID Method author to specify for each representations - those are the two conflicting needs.
Manu Sporny: We had a joint meeting with the WOT WG this morning
Manu Sporny: https://www.w3.org/TR/wot-thing-description/
Manu Sporny: this WG took a hard turn when we started looking at the ADM, and I wanted to point out other WG’s have taken a pretty different direction to this, the WOT thing description is an example of this which took JSON as their core representation format
… which can also be interpreted as JSON-LD
… I think fundamentally one of the questions we have to answer is do we allow @context in application/json and or do we allow @context to be injected
… where the injected proposal I would argue against because of its inherent danger
Dave Longley: +1 to determining how what we say will affect users of the technology … and to prioritize them first (in terms of simplicity)
Justin Richer: Keep in mind that when you say “let’s make things simple” — it’s relative to assumptions in the ecosystem, what may seem simple for you, may be limiting to protocol as a whole.
… It should be as simple as possible, but no simpler, that’s the line that’s really hard to define.
… As for the @context question, that’s not the question we need to be asking ourselves… The question isn’t about @context allowed in plain JSON. The question is whether @context is allowed to be a property in the abstract data model, and what do we do with that value when we do “translation”… I wouldn’t call it that personally, but not worth splitting that hair right now.
Orie Steele: I feel translation is actually out of scope for the working group.
Jonathan Holt: I think explore the dangers - injecting it is a big vulnerability headache, change meaning of document.
… So, I think that’s a non-starter — my conclusion was that you may include an @context and it’s there to aid in interpretation of meaning of attributes.
Brent Zundel: q
Jonathan Holt: The challenge is exploring security vulnerabilities - more thoroughly, what is the danger?
… What is the danger of misinterpreting the meaning of the document should there be an @context that you don’t understand? What might limit full potential, full limitations of JSON-LD in this context.
Kyle Den Hartog: Yes, what I’ve seen with @context — injection is definitely a bad idea, I can see ways we could do it easily… that could limit dangers of it, by saying “You can only inject if you’re doing so @context pointing to registry, versioning concerns, etc.” I’d rather we don’t do injection.
Markus Sabadello: big +1 to Kyle..
@contextis a JSON-LD artifact, not a property
Kyle Den Hartog: I would think that @context property should be considered not a property, but rather a JSON-LD artifact, describing something about document itself, not about DID Document itself… no injection, that property is not defined, but in JSON-LD representation have a JSON-LD @context that is commonly used.
Orie Steele: I recall a while back whether DID Resolution was in scope for the WG - we spend a tremendous amount of time to define function signature, part of the signature we’re having trouble with this, we never really got agreement with how that works, so everything that comes after that is not defined and is not working.
… Is this WG allowed to define software processes defined for DID Document data model after DID Document constructed after DID Method… like how a resolver translates a DID Document… ultimately that’s what Justin and Markus are advocating that we cover.
Justin Richer: q
Orie Steele: There is desire to do that — questioning whether or not we’re allowed to work on that — software process that only happens in DID Resolver… different from what a DID Method is doing when they produce a representation.
Markus Sabadello: I don’t know what it means to inject @context — what software, at what stage? The question is always “How does the representation get produced?”
… DID Method author should be able to produce documents in different representations.
… @context is JSON-LD specific context… why would it show up in any other representation?
… We have some XML specific artefacts, version number, tags also version numbers, would that show up in JSON or any other representation? I think the original intention — introducing registry in first place was to make sure we can produce both JSON-LD and plain JSON - plain JSON doesn’t contain @context.
… I don’t see why @context would show up in things that are not JSON-LD
Orie Steele: +1 to making
@contextrequired again :)
Justin Richer: To note in WoT Things definition, it is defining a JSON-LD data model where @context is required — if we did that w/ a DID Document, a lot of this rabbit holing about @context would go away with defined required values, this group doesn’t want to do that. I want to second all of Markus’ points, XML Directive, XML Include, that exist in other representations — so do those need to be carried through in some way, or can a producer create
Manu Sporny: those when creating representation?
Dave Longley: +1 to making
@contextrequired or, alternatively, define a default value for it :)
Justin Richer: I agree with Orie about us questioning what exactly we are defining, that’s where a lot of the confusion is coming from — a lot of what Markus, Drummond, and I are proposing for a narrow set of things, you have one thing and you’re going to another thing.
… That’s all that this was trying to define — well I’m reading it off of a registry, I’m handing it over — I’m translating it from one thing to another — you’re creating a new DID Document. A new representation out of same abstract data model, a new DID Document.
… It’s important that we need to be really precise about what we’re doing, we’ve tried to be precise about the narrow space we cover and what it doesn’t cover. There are a lot of things that people want to do with DID Documents that are not these functions.
… Everyone seems to apply these labels for something else they’re doing and it doesn’t match.
Jonathan Holt: https://tools.ietf.org/html/rfc7049#section-3.3.3
Jonathan Holt: https://tools.ietf.org/html/rfc7049#appendix-C
Jonathan Holt: In my research into CBOR section, came across two tidbits - how to handle unknown properties or unknown additional information values.
… The reason why you drop them on the floor is the security vulnerability - Indefinite Lengths list, buffer overflow, crash resolver — potential for malicious activity.
… Well known and valid representations — this is what it means to be well formed/valid.
… I’m ok for JSON-LD, don’t want to require JSON-LD processing, direction of WoT, with LD Signatures, deterministic ordering of RDF, so that’s a challenge if you don’t get it right.
… even with BBS+ signatures, JSON-LD in that is unexplored area of security vulnerabilities, you can reorder the interpretation, don’t know what those security vulnerabilities are, have reservations about it.
Kyle Den Hartog: Wanted to answer Markus’ question - around injection - if it’s a DID Method that wants multiple transactions, put DID Document in, DID controller resolver get context, resolver itself, which is consuming DID Document, adds a context in a way that it wasn’t put there by DID Controller, that’s injection, if DID Controller didn’t put it there, or didn’t claim that as context of did document, it’s injected.
Manu Sporny: Two things, first one is to answer, why would ever have @context in a JSON doc, during the last call we outlined that there are existing deployments that do this, such as WOT WG Thing description that is JSON but includes @context, Veres One did method also operates in a similar way
… that we are going to keep @context out of the application/json ship has sailed long ago
… Two proposals, we polled pretty strongly for the second one last time, another proposal is to include @context as a property in the ADM as Justin suggested
Orie Steele: I wanted to point out that @context will never be present in any DID Document if the DID Method doesn’t allow it to be produced… but if you produce YAML, and 3rd party context that wants to produce JSON-LD from that, you will have to inject @context or you won’t be able to produce JSON-LD. I like where Kyle was going, let’s define what injection means — some software other that DID Method or controller is injecting properties, and I think
Manu Sporny: that’s a security issue.
Orie Steele: If we are going to see proposals for injection to be normatively required, then you’re going to see objections from me on that.
… We can define injection as a thing that’s an attack.
Markus Sabadello: Regarding having @context in application/json — I understand that’s legal, but I think all production systems that do this don’t distinguish between JSON-LD and JSON at all, which is fine and legitimate, but we say that there is a difference in DID Core.
… The understanding was JSON representation of DID Document wouldn’t contain @context.
… I’d be fine w/ allowing that, but I don’t think we will have different representations, one representation that has @context.
… I think it’ll be bad if we have application/json and some DID Methods would add @context and others wouldn’t.
Justin Richer: +1 for markus’s point
Markus Sabadello: I don’t think the ADM works if we do that.
Jonathan Holt: the @context helps semantic interop, in essence the DID Document is a representation of key material, use registered attributes in registry and is non-issue and more simple approach, point to existing registries of JWK, algorithms, this is a thing that a public key could be.
… Could see things be simplified — verification methods, better at modelling those to what they actually do — in DID Document, easier closed world model, harder to constrain VC model since it’s more open world.
Justin Richer: So, two parts
… First, it seems that all of the arguments for preserving @context come from when there is a well formed context that means what you want it to mean and it gets carried through, but what if I create a plain JSON document and I create @context: 7 — and I need to serialize it into other formats… and then I serialize it into JSON-LD.
… This is where translation is a slightly different process - there is something there to be captured.
… Second, We need to look at what @context field does — if we’re really to understand it — if you look at all production rules, you’ll see that translating a field in a representation into a property of ADM is not simply looking for a string and copying it over — you need to take name of field and value of field and translate both and therein lies a very important thing, in plain JSON, translating the field name, plain JSON member object
Manu Sporny: name - map string exactly to string property name of registered property and go on with your day.
Justin Richer: @context defines root namespace for these properties to live in just like XML Namespacing.
… @context functionally defines, what semantic mapping for what JSON object member names is, which means, within jSON-LD consumption, what you’re doing is not just processing member names and looking for strings, you’re processing them in the @context of @context fields applied… DID Documents have a certain set.
Justin Richer: What this means is that we can have valid JSON-LD representation that does not use DID Core root context — and it would functionally still be able to map, because upon consumption of the serialized format, you would take into account these values when you look at all property names, CBOR would do something similar w/ registered numbers and things like that.
… You are always allowed to short circuit this, you see a value that you already know and you have a quicker way, go do that, have a grand old time.
… semantically, we are defining the model, taking something that is an item in a typed data map and placing it into a typed data map and placing it into another typed data map.
Drummond Reed: Has it been discussed at all at tail end of previous call — add category of processing properties to ADM so information that is relevant to process specific representation is available?
Dave Longley: That was a proposal that was emoted right before you joined.
Drummond Reed: “emoted”?
Dave Longley: I wonder if we can revisit rules for JSON representation, if it’s present, it must be w/ did @context value is first in array.
Orie Steele: +1 to what dave is proposing.
Orie Steele: its also better alignment with WoT
Dave Longley: if we do that, that guarantees all properties in registry would follow same processing rules for something else.
Manu Sporny: I am going to try to run the current proposals that have been put out here
Justin Richer: A potential way forward — at end of day, this specification will be served best by it being honest about what it is and what it expects of implementaters, claiming it’s JSON but requiring JSON-LD fields to be there is being sneaky.
Orie Steele: +1 to justin_r …. claiming that its JSON but requiring MOSTLY RDF…. it disingenuous.
Justin Richer: I’m not proposing this because I’m in favor of it, but it’s a possible direction the group could go, define @context as core data model property.
Drummond Reed: Are you saying it’s required?
Justin Richer: Yes, required.
Orie Steele: its not just
@context… its the RDF triple structure of the entire spec.
Dave Longley: we can define a default instead.
Justin Richer: The Web of Things - it claims to be JSON, but requires @context as mandatory property.
Drummond Reed: oh, that’s JSON-LD
Justin Richer: I know
… if this is a JSON-LD spec, so be it, just call it that.
Orie Steele: +1 to being honest about this being a JSON-LD spec.
Manu Sporny: Right so the thing that I am hesitating on is that we relive the F2F meeting
… I think there might be a compromise in what Dave is proposing
… I think it is attempting to strike a delicate balance
… which proposal do we want to run first?
… I’m concerned depending on the order we may not get consensus
Proposed resolution: Add processing properties as a new category of properties to the ADM (Drummond Reed)
Manu Sporny: lets try Dave’s one then Markus’ one
… unless there are objections
Drummond Reed: I just added a third
Drummond Reed: +1
Justin Richer: +1
Manu Sporny: lets run yours first drummond +1 -1
Manu Sporny: -1 (adds complexity that we don’t need)
Orie Steele: -1 confusing
Brent Zundel: +0
Dmitri Zagidulin: -0
Shigeya Suzuki: +0
Kyle Den Hartog: -1 another can of worms
Dave Longley: -0.5 would only support this as a last resort to make things work for implementers
Jonathan Holt: 0 not sure what processing properties as a new category of properties are
Jonathan Holt: -0.5 really
Manu Sporny: lets see in light of that
… markus lets take up your proposal next
Proposed resolution:
@context,<?xml version="1.0"?>,%YAML 1.2, and other representation-specific constructs are NOT properties of the DID subject. (Manu Sporny)
Justin Richer: +1
Drummond Reed: +1
Orie Steele: -1 logical attack
Kyle Den Hartog: +1
Markus Sabadello: +1
Manu Sporny: -1, wrong question —
@contexthelps you understand DID subject properties.
Brent Zundel: +0
Jonathan Holt: 0 not sure of the implications
Tobias Looker: 0
Dave Longley: +0 its true but perhaps misleading
Dmitri Zagidulin: -1 that list of properties is apples & oranges
Shigeya Suzuki: 0 not sure
Brent Zundel: no clear consensus here
Dmitri Zagidulin: the equivalent JSON-LD property is not @context
Manu Sporny: right lets run dave’s proposal next
Dmitri Zagidulin: it’s whatever the version 1.1 is
Proposed resolution: If
@contextis present in an application/json representation, define a specific constraints on its value (i.e., an array that begins with the DID core context). It does not have to be present in an application/json representation. (Manu Sporny)
Drummond Reed: +1
Dave Longley: +1
Orie Steele: +1
Justin Richer: -1 (implies context would be carried into the ADM as a property of the subject — remove that concern and I’m fine)
Manu Sporny: +1
Dmitri Zagidulin: +1
Kyle Den Hartog: +.5 could live with this
Tobias Looker: +1
Brent Zundel: +1
Drummond Reed: -1 if it implies
@contextis in the ADM as a property of the subject
Markus Sabadello: -1, since with the current ADM design,
@contextshouldn’t end up in application/json in the first place
Shigeya Suzuki: +.5
Jonathan Holt: +0.5 could live with this, but doesn’t require Json-ld processing
Brent Zundel: two minus -1s with the same concern
Manu Sporny: let me try another one that may resolve the concerns
Justin Richer: manu, this is what drummond was saying that you -1’d
Manu Sporny: if we change what the ADM is about is about instead of it being strictly about the did subject
Brent Zundel: there is already a proposal on the table
Justin Richer: +1 to exactly this point
Justin Richer: the fact that they’re in the same JSON is an artifact
Proposed resolution: Clarify that ADM isn’t specifically about properties about the DID Subject and may contain properties that aid in generation of representations. (Manu Sporny)
Justin Richer: stop encoding artifacts
Manu Sporny: +1
Justin Richer: -1
Orie Steele: +1
Dmitri Zagidulin: +1
Orie Steele: if you don’t agree with this proposal, you only believe in RDF
Drummond Reed: -1 if it is not specified that it is a processing property, +1 otherwise
Jonathan Holt: -1 to my prior point, don’t think it belongs in ADM
Markus Sabadello: +0
Dave Longley: +0.5 just want to find the middle ground that will get the previous proposal through
Shigeya Suzuki: +0
Kyle Den Hartog: +0
Brent Zundel: +0
Manu Sporny: and then we can pickup markus’ proposal at the very end
Proposed resolution: Remove JSON representation and allow plain JSON processing of the JSON-LD representation? (Manu Sporny)
Manu Sporny: +1
Brent Zundel: no clear consensus here
Justin Richer: -0
Dave Longley: +1
Kyle Den Hartog: +1
Orie Steele: +1
Drummond Reed: -1
Dmitri Zagidulin: +1
Markus Sabadello: +1
Brent Zundel: +0
Orie Steele: this looks closer than I would have thought
Jonathan Holt: 0, seems reasonable
Shigeya Suzuki: +0
Manu Sporny: that one seems to have the closest consensus
Brent Zundel: I look forward to the reasons for drummond’s minus one if these conversations can be continued in issues please do so
Jonathan Holt: not sure what plain JSON processing of the JSON-LD representation means
Justin Richer: jonathan_holt: it’s basically what dlongley said