JSON-LD Working Group F2F in Fukuoka — First day — Minutes

Date: 2019-09-19

See also the Agenda and the IRC Log

Attendees

Present: Gregg Kellogg, Benjamin Young, Pierre-Antoine Champin, Ivan Herman, David I. Lehn, Manu Sporny, Ruben Taelman, Simon Steyskal, Dan Brickley

Regrets: Ivan Herman

Guests: Duy-Tung Luu, Sanders Tolk, Bert Bos, Charles Neville, Daniel Burnett, Addison Phillips, Richard Ishida, Hadley Beeman, Dan Brickley, David Clarke

Chair: Rob Sanderson, Benjamin Young

Scribe(s): Gregg Kellogg, Rob Sanderson, Benjamin Young, Manu Sporny, Charles Neville

Content:

1. Introductions

Rob Sanderson: I’m Rob, co-chair. Architect at J Paul Getty Trust

Gregg Kellogg: Gregg Kellogg, chief editor for the spec, and invited expert to the group

Ivan Herman: Ivan Herman, W3C staff contact, mostly in France

Benjamin Young: Benjamin Young, co-chair, work for John Wiley & Sons, Inc.

Duy-Tung Luu: Duy-Tung Luu, in data science and ML. I did a PhD in statistics. In my company, there is a web semantics project, so I came here ot lear.
… Canton Consulting

Sanders Tolk: Sanders Stolk, I work for Semmtech.
… We’re using LD for our clients to help in infrastructure to share information on buildings from design, building to maintanence.
… In NL near Schiphol

Pierre-Antoine Champin: Pierre-Antoine Champin, an assoiate professor at Univ. Lyon.
… I’ve been on the WG for a year. My research is in SemWeb, also in RDF WG
… Also co-editor of the syntax doc.

2. Review of Accomplishments

Benjamin Young: Virtual click-through slide deck

Rob Sanderson: We’re close to CR. There are three open issues.
… Three issues to consider, two this morning.

Benjamin Young: This afternoon, we want to talk about string direction, which may bring in more outside visitors.
… Its of keen interest to a wide group; we thought it would be solved in RDF, but it may be on our shoulders.

Ivan Herman: Formally speaking, it may be worth adding an explicit issue, or reopening the old issue. #11

Benjamin Young: version triggering is something to consider.

3. Open issues

3.1. Use of null value with JSON literals

Rob Sanderson: See Syntax Issue #258

Gregg Kellogg: in most statements we do not include the nulls. one exception is JSON literals
… but null is a valid JSON value, so there are reasons to keep them
… pchampin pointed out a similar situation with strings, but that was an implementation issue
… but it did help us find this
… you can keep the null value currently by using the @type: @json
… but in other cases we take it out
… so we’re at least inconsistent
… it implies that there’s some missing algorithm text
… for preserving a key/value with a value of null
… there are various options for preserving them
… we could also decide not to
… but I do think we can
… if you expand it, and then re-expand it the null goes away…even when using the @json value type
… it’s an editorial issue once we figure out how we want to handle it
… because it’s a lack of clarity
… should be the work of an hour or two

Rob Sanderson: the question then is should the null be treated as a JSON literal null or should we remove the property

Benjamin Young: Apache CouchDB and MongoDB (iirc) will remove keys set to null

Rob Sanderson: that’s what this does in every other case…except the new @json type
… I think it’s more surprising for the null to go away if the type is @json than for it to stay around
… but…do we want to still provide a way to remove the key/value?

Ivan Herman: why would we want to remove key/values?

Benjamin Young: databases do it

Rob Sanderson: but this would be full statements

Gregg Kellogg: so most of the testing I’ve done was with numeric values and objects

Benjamin Young: is null on it’s own a valid JSON value/document?
… what would JSON.parse do?

Gregg Kellogg: it would keep it

Pierre-Antoine Champin: I’m playing around with the code right now, and gkellogg mentioned that re-expansion would get rid of it
… I added the @version: 1.1
… otherwise it doesn’t work at all
… and when I do it, it looks like we get an invalid value

Gregg Kellogg: part of the use of null is to eliminate contexts
… you can certainly handle these by setting a 1.1 at the processor level–if the document itself doesn’t set it

Rob Sanderson: azaroth tests JSON.parse("null") in chrome, and the result is a real null

Gregg Kellogg: but you’re right it could be restricted to 1.0-based processing
… so we may need to update tests to use 1.1 mode
… and test with expanded values
… so setting the processing mode is probably the way to do it

Proposed resolution: WG believes that it is least surprising to process null as a JSON literal in this case, and not to remove the property in the internal representation, and thus the null should be preserved. (Rob Sanderson)

Rob Sanderson: +1

Ivan Herman: +1

Benjamin Young: +1

Gregg Kellogg: +1

Pierre-Antoine Champin: +1

Resolution #1: WG believes that it is least surprising to process null as a JSON literal in this case, and not to remove the property in the internal representation, and thus the null should be preserved.

David I. Lehn: +1

3.2. keyword alias for @id and @type cannot be singly @protected

Benjamin Young: See Syntax issue 246

Rob Sanderson: if you alias @type to type it can still be a set
… however, using @protected on an object-based term definition will not protect the id aliasing
… so my proposal is that we change @protected to be a legal keyword for keyword aliases

Gregg Kellogg: does @protected protect the @type based aliasing?

Rob Sanderson: [adds more examples to 246]

Ivan Herman: the examples don’t show any reason for the @protected

Rob Sanderson: [fixes examples]
… [reads text from 4.1.5]
… [reads text from 4.3.3]

Gregg Kellogg: looks like the API is correct

Rob Sanderson: ah, so they’re out of sync

Gregg Kellogg: we do need more @protected text in this API section

Rob Sanderson: so the syntax fix is editorial

Proposed resolution: Allow the use of @protected as a valid keyword in term definitions which are aliases of keywords or where the term is @type. (Rob Sanderson)

Rob Sanderson: +1

Gregg Kellogg: +1

Benjamin Young: +1

Pierre-Antoine Champin: +1

Ivan Herman: +1

Resolution #2: Allow the use of @protected as a valid keyword in term definitions which are aliases of keywords or where the term is @type

3.3. editorial problems

Gregg Kellogg: before we go on to the next issue, there are some visual things maybe we can solve
… I’ve never liked the green stuff
… for new items
… since we already have some active JS in there
… maybe we can toggle highlight changes in 1.1

group: [discusses various CSS related tweaks…decides to leave things as is]

Gregg Kellogg: but then there’s “[Exposed=Window]”

Ivan Herman: why is that even needed?
… let’s just display: none it
… it doesn’t apply to us
… since our use of WebIDL is not language specific or browser specific, there’s no reason for this exposed window object thing

Rob Sanderson: there are a few other issues we should chip away at

3.4. 220 in syntax; move 9.3 and “default object” to framing only?

Benjamin Young: See Syntax issue 246

Gregg Kellogg: so things show up when they’ve been referenced
… and the terminology here does make this confusing
… we introduce these concepts, because many people who use these don’t look at API documents
… the frame is more of a hybrid document of syntax and api
… we might just be being to explicit here
… we even describe a frame object here for that matter…

Rob Sanderson: this doesn’t really add much…and I’d suggest just removing 9.3

Gregg Kellogg: and there is an earlier reference to framing earlier

Rob Sanderson: so…yeah

Gregg Kellogg: and farther down, we introduce expansion, compaction, etc.
… and there’s a framed document form

Benjamin Young: off topic, but I’d love to have expanded form explained earlier
… too big a change right now…but maybe in the primer

Ivan Herman: it helps me too to think about expanded form

Gregg Kellogg: but back to this specific issue

Ivan Herman: right. frames are valid 1.1 documents now
… so the syntax document has to call this out

Rob Sanderson: right, 5.4. explains it, and you can’t understand 5.4 without 9.3

Gregg Kellogg: this issue can capture why its there

3.5. regarding expanded form earlier in the doc

Gregg Kellogg: maybe we just add a new section that talks a bit more about expanded form in the intro
… that highlights that various forms exist and how they work

Ivan Herman: it’s true that these other forms are helpful

Benjamin Young: do we need an action for this?

Gregg Kellogg: maybe we can put a basic overview of just expanded form and its use in the intro/basic concepts section
… and then to note that semantically the compacted + expanded are equivalent
… even though syntactically they look different

Action #1: write a couple paragraphs + example(s) for explaining expanded form and its use (Gregg Kellogg)

4. implementations

group: [discussing the implementations available and needed]
… primarily C++ and Java need the most attention

Gregg Kellogg: http://drobilla.net/pages/contact.html

5. Text directions, Part I, versioning

Rob Sanderson:

 {
    "@context": {
        "@vocab": "https://example.com/",
        "schema": "https://schema.org/",
        "name": "schema:name"
    },
    "name": {
        "@value": "fish",
        "@language": "en",
        "@direction": "ltr"
    }
}

Pierre-Antoine Champin: +1

Pierre-Antoine Champin: { “@context”: [“http://schema.org/”, { “@direction”: null }], “tag:p”: { “@value”: “foo”, “@language”: “en”, “@direction”: “ltr” } }

Manu Sporny: Discussion around use of @direction and how it affects JSON-LD 1.0 processors.

Manu Sporny: Seems that there is an issue wrt. the use of @vocab and @direction… processors are choking.

Manu Sporny: A bug exists in 1.0 and 1.1 that doesn’t flag the use of @direction and @vocab.

Rob Sanderson: Issue created: https://github.com/w3c/json-ld-syntax/issues/259

Manu Sporny: Current 1.0 processors are going to ignore use of @direction in various places.

Pierre-Antoine Champin: {"@version": null} :-D

Rob Sanderson: Moving on to whether or terms can start with @ symbol.
… Currently, the spec is not clear wrt. SHOULD NOT vs. MUST NOT.

Benjamin Young: This is why HTML is not versioned.
… I’m less concerned about two documents processing two documents differently… we don’t want to touch old data, but we don’t want to prevent accomodation of the future at the same time.

Gregg Kellogg: If HTML pages had something that wasn’t interpreted, but then it would… like span, or blink, or marquee … then there could be issues.

Benjamin Young: When I use a newer processor, I’m aware that it’s going to look different… I don’t touch old data.
… I’d suggest you use a switching mechanism in processor… nuclear option is to use 1.1.
… If I point towards the past… I provide HTML5 document to HTML 3 processor, you don’t lose data
… If you give @version to 1.0 processor, it explodes.

Manu Sporny: Yes, but the only consumers are schema.org and they’ll just do 1.1.

Benjamin Young: yes, the evergreen browser argument

Pierre-Antoine Champin: Changing 1.0 parsers, if they don’t fail on 1.1 parsers, they’ll break on other things

Benjamin Young: See Feature Detection issue

Gregg Kellogg: We have to do something intelligent when we hit @container: subway – 1.0 will throw a syntax error, it should throw errors when it sees an issue. My feeling is that we should remove the explicit version announcement, and do feature detection when things are discovered and throw errors when they don’t understand.

Ivan Herman: Another reason why I’m in favor of incremental stuff, in a weaker way than what you say, is because we have to look ahead, we’ll have 1.1 now, but there’ll be a different discussion on how to proceed when this WG stops… listening to presentation from yesterday, next charter whatever 2.0 - essentially, admin and names put aside, we’d be in a position to put one feature that we want to add to the language like direction.
… If in a year from now, we add @direction, what do we do – version 1.1.1 or version 1.2?
… We get into versioning/naming issue… it’s difficult to maintain… sometimes they release 5.3, they do incremental changes, then implementations catch up… we might have the same sort of future ahead of us, we should be careful to not make things too difficult… 1.1 as a standard adds new features, not necessarily a thing called a 1.1 processor, we hope we have processors that implement everything, that’s the goal.

Manu Sporny: so. the original @version proposal was a list of features
… so if you were to process this document, you must support those features
… if a processor can’t support those, then it just blows up
… and 1.1 was just an alias for a bundle of features
… the issue we saw was that there might be a feature…well…
… one, announcing them doesn’t mean you’ll be using them
… or it might just get forgotten that you’ve left the “opt-in” around when you ended up not using it after all
… so, the feature detection works best when it’s also BC friendly
… and that turns out to be rather hard
… we’d have to go through each feature and determine how detectable they are
… but the reason against that is that it’s a huge amount of spec text
… and we weren’t sure that every new feature weren’t able to figure those out
… and they might determined it differently
… which is why we were ultimately in favor of 1.1

Ivan Herman: so does that mean if we go into continuous development

Manu Sporny: …yeah. I’m against that also
… I was opposed to this 1.1 work earlier…the 4 year gap was helpful for the stability of the data and community

Ivan Herman: k…but if we keep shipping, then we’d just keep shipping these version number things

Manu Sporny: yeah…but that gets complicated again
… and if you look at HTML5, feature detection is insane
… the earlier issues were about feature announcement not detection

Gregg Kellogg: the point of 1.1 was not so we’d have a 1.2
… it was so we’d be locking things down
… and that’s principally done in context processing
… and we did not do that properly in 1.0
… so, virtually, I’d say 90% of features are discovered in @context processing
… The original version was 1.1, was not to do 1.2, it was so we could detect things that should not be there in 1.0… if 1.2 added @direction, a 1.1 processor would fail if it saw it because it was not in the whitelist. 1.2 is not necessary to say @version 1.2 to do the right thing… virtually, 90% of feature issues are discovered in context processing, the exception is when processing expanded documents where there are some features used.
… the others show up in expansion
… We do look for those in 1.1 mode and 1.0 mode… frame processing mode…
… I think where we are detecting these things in expansion processing, maybe more needed for forward compatability, everything else is pretty much be in the @context processing phase.

Ivan Herman: If we had it today, @direction wouldn’t be detected in @context.

Gregg Kellogg: If we had it in the context it would be.

Ivan Herman: If I just use it as part of the value, it’s a 1.1 feature that I don’t see in the @context.

Gregg Kellogg: Anything that looks like a keyword and the processor doesn’t understand it, it should abort.
… I disagree w/ manu, I think we can do feature detection.

Manu Sporny: I don’t think I said we’d use 1.1 when doing JSON-LD 5.0

Ivan Herman: isn’t this an issue for stream processor?

Gregg Kellogg: Well, you could throw an error if you hit something…
… I think we’d be in a better position if we don’t use @version.

rob: We need to break for lunch soon.

Sanders Tolk: In some cases, it’s important to break down with newer features used… it’s a conscious choice to move from a 1.0 to a 1.1 interpreter…
… There is something to be said to an interpreter breaking on a feature it doesn’t understand, vs. a feature thing… parser, or implementation…
… Depending on level of confidence, you use one or the other… this is something that people should be aware of anyway.

Gregg Kellogg: In some cases, in order to get behavior, you should be in validation mode.

Benjamin Young: I want as much of data out of top of page, don’t care about reviews, give me what you can give me.

Sanders Tolk: To be able to post a snippet in a web page, does this have any unknown features for 1.1… so people can know, am I going outside of that…

Gregg Kellogg: It is using features that are defined in 1.1… if your parser dies, sometimes it helps if you have it all gathered.

6. Text directions, Part II

Rob Sanderson: The issue that we want to talk about is notion of internationalization and feature of text direction of strings, which is not possible in underlying RDF data model. Proposal that came out of discussion was that it would be better for the infrastructure overall if JSON-LD did text direction. Absence of it already existing at RDF concepts layer… it would be better back port it into RDF from JSON-LD.

Addison Phillips: We move the upper stories of the Eiffel tower first :)
… We are concerned that there is a window opened right now that so many specs are dependent on JSON-LD, we would like to make sure common serialization form for handling this, if we don’t have something, we might have non-interop solutions for this.
… We are interested in seeing if we can do something at this level, see if we can move this down to RDF level.

Rob Sanderson: One question that would be good to answer, why isn’t language sufficient to do this?
… Why can’t we just say English is ltr, arabic is rtl, and be done…

Addison Phillips: You can infer base direction, but not the same as base direction information, like when you don’t have accurate language metadata, or applying base direction isn’t applicable.

Richard Ishida: See Article on Base direction and language tag

Addison Phillips: Rather than doing coercion, we should have metadata slot rather than interfering… that’s the basis of it. Key aspect is that it has to do w/ mixed directional text… text that flows in one direction isn’t a problem, text that starts or ends with an opposite direction, bring multiple strings together, you have spillover effects that are desirable. That’s a challenge that we see as fairly fundamental.

Charles Neville: There is another problem in RDF in particular, which is that RDF doesn’t know anything about the language… lang=ar, you know it’s rtl… the only modern language that you can’t detect is azeri.

Richard Ishida: No, not true by a long shot.

Charles Neville: RDF doesn’t understand ar-kw ar-?? is all arabic

Richard Ishida: And new subtags are added all the time…

Charles Neville: You’d have to parse RDF language tags to get this partial solution

Richard Ishida: There was a point where we thought it would work… the URL that I just posed in IRC is the result of that.
… The conclusion we came to was that language wouldn’t do the job… wrote two articles on Kashmiri - the language tagging that I used in both documents was ks, didn’t put script info because I didn’t need to… there is a lot of human stuff going on there. You don’t know if it’s rtl or ltr. Even I didn’t feel like I needed to put subscript and relying on language.

Ivan Herman: Coming back to what Addison said, whatever we do, there are cases that we cannot handle directly, and we shouldn’t even go there. If we have text that internally mixes language, the only way we could handle that is to fall back on HTML encoding of content.
… metadata only on the one string.
… When you have mixed text, we can’t do that.

Addison Phillips: We just want base direction of whole string… there are bidi things that can happen on whole string. Supplied in form of HTML… set the base here… we’re not talking about internal changes

Dan Brickley: What RDF understands of the language structure… a lot of stuff you just pass through RDF… if you want to be able to write SPARQL queries, you’d have to have advanced knowledge of those language fields, you push the solution into the application.

Addison Phillips: Language tag mapping, don’t know what the subtags mean… SPARQL just does string matching, if you say ‘bs’ which stands for “bosnian”, you need a “bs range” to match.
… You still don’t need to know what the tags and subtags mean… the hyphens tell you what the boundaries mean… it’s deliberate, keep away from needing to read registry, use complex logic.
… If you are doing complex matching, you need application logic.

Dan Brickley: We don’t need to address fancy SPARQL query.

Sandera Stolk: I was going to talk about lang matches, doesn’t fix solution for direction… you’re probably aware of, if a solution is found for direction in JSON-LD, there may be multiple other types of language interpretation attributes to help with something that goes beyond direction… if there is a solution found in JSON-LD, it should also be a solution that should allow for other attributes as time goes by… it can be placed into the same construct.

Charles Neville: If we follow this plan, what breaks?

Ivan Herman: What’s the plan? Add syntax for @direction and add a new property

Charles Neville: See the proposed json-ld syntax

Manu Sporny: Group looks at directions in rdf-dir-literal document.

Ivan Herman: Note that the document we are referring to (link above) describes a range of possible solutions. We suggested implementing the json-ld syntax in json-ld as a starting point, because it doesn’t yet have a representation in RDF semantics
… We ended up saying this, if I put RDF hat on, there are downsides in practice, there are duplicated ways to represent literals in RDF… literal w/ language that RDF has today, in parallel we have another approach, might break some deployed OWL mechanisms, object properties, data properties, not a good solution, but possible solution…

Gregg Kellogg: We also discussed that this was a transitional solution.

Ivan Herman: Hope is in time, there is a more general mechanism in RDF, don’t know if this will happen. Current proposed approach, maps nicely to JSON-LD, covers what sander wants.
… What we discussed today, yes, this is doable, but what happens w/ existing JSON-LD data out there, and what are the effects… and answer is not that simple…
… Adding the feature is easy, what happens to the process is not a big deal….

Manu Sporny: I really pushed to fix this, for verifiable credentials to be able to move on…
… we rushed in to getting good ideas on the table. I am going to check whether this is really the right course.
… seems like language tag does 95% of what we want, a lot of times.
… so it is true that people extract some text direction out of the language tag

Addison Phillips: mostly right now it isn’t used…
… most people without a base direction appy the unicode algorithm.

Charles Neville: anecdata suggests that in practice, people are really implementing stuff withthe language tag and accept that as generally good enough for customers.

Manu Sporny: There is a reality on the ground that people are using the lang tag. All of these specs getting changed might not be as good as working out how to do this through teh lang tag.
… we could deal with it in the lang tag and not have to change the RDF layer

Benjamin Young: the mentioned list of specs https://w3c.github.io/rdf-dir-literal/draft-charter.html#normative

Addison Phillips: One thing is that by design implemnentations don’t need to know about the content of the language tag - and most of them don’t.
… there is a massive number of subtags.

Addison Phillips: One of the things I mentioned earlier, by design, implementations don’t need to know about subtags/language tags… we can maybe smuggle serialization schemes… the direction metadata doesn’t belong in the language tag, it’s semiorthogonal not totally unrelated, separate piece of data, I’ll admit that people have struggled… we failed to provide a mechanism to do this, we should have had this conversation before the paint was dry.
… we have looked, in https://w3c.github.io/rdf-dir-literal at ways to “smuggle” the information metadata in via the lanugage tag. It is semi-orthogonal, being a separate bit of data, that we have to be able to serialise and deserialise. There are a lot of places we suffer because we haven’t yet managed to solve the problem, since we didn’t do it from the start.

Richard Ishida: r12a’s slides start here: https://www.w3.org/International/talks/1810-paris/#metadata

Addison Phillips: People who have data catalogs who have base direction metadata, when people have this affordance, it helps to be able to do this… does it work a lot of the time, yes, but corner cases are really not that unusual…
… people with data catalogues who have the metadata have an easy time if they can set the direction directly, Carrying around lists of subtag matching works a lot of the time but is pretty ugly, and the corner cases are not that unusual.
… we launched a site in an RTL language, and hit a pile of bugs. I don’t always have convenient mechanisms to manage this, so I am interested in a way to provide direction metadata in JSON-LD
… Where I’m headed to here, I am not a purist, if we had an impure serialization scheme that had greater compat, that wouldn’t be bad…
… Talking w/ RDF community, we’re creating an interesting hack, doesn’t speak to extensibility… don’t know if it’s a hack… I’d be open to a serialization, looks hacky relative to other alternatives… maybe we can be compatible w/ existing implementation things.

Dan Brickley: I liked the idea of doing something in JSON-LD… if we could backport for 1.0, we could get something happening in 1.0, Google might get involved.
… We’ve encouraged people do to good solid JSON-LD 1.0 files, there are millions and millions of this….
… The smallest little thing we can do is important, if we can do some surface hackery, we can parse this stuff…

Gregg Kellogg: Yes, some transitional mechanism.

Ivan Herman: We are having a discussion about versioning discussion…

Gregg Kellogg: parameterize RDF algorithms stuff…

Dan Brickley: Parser being invoked w/ arguments, people run RDFa parser in certain mode.

Rob Sanderson: There are two separate questions… the first one, do we feel comfortable adding @direction to JSON-LD for @value objects regardless of how it transforms to RDF.
… There are things that don’t round trip in JSON-LD, such as indexes, there is precendent for non-roundtrippable stuff in JSON-LD… if cost to RDF layer is high, we can defer the serialization through into RDF… the proposal is 1) do we want to do this, to 2) how do we want to do this.

Charles Neville: I think we want to do this, happy to give up round trippability… what’s going to break?
… If that’s a fast path, this is a good solution.
… Forcing things that don’t parse language to parse language, that’s a big ecosystem thing.

Manu Sporny: I think the vast majority of appications don’t pay attention to the language tags, or direction.

Ivan Herman: In browsers, they don’t establish direction with language they do it with direction metadata.

Richard Ishida: https://github.com/w3c/string-meta/wiki/Base-direction-and-language-tags

Richard Ishida: The normal way of detecting language is to look at first strong character heuristics, for most things in strings, that’s the way to do it… far more efficient to look at first character… that works for 95% of time time, for the other 5%, that’s the solution we’re looking for, not having language, not having direction, and when direction on character is wrong…

Manu Sporny: I just wanted to make sure we are making an educated decision.

Addison Phillips: I think the right thing to have done is split the two – language and dir must be separated.

Richard Ishida: Yes, they’re not doing the same thing

sander: Pinpoint something wrt RDF - our company uses multiple language tags to discuss the same term, we find it’s difficult to find proper name for same term.
… We would be more reluctant if we adopt this that this is a different language encoding to …

Ivan Herman: If you look at this thing in RDF version, if there is no @direction, it maps against a literal and language tag… it’s some bnode with properties, it’s very different.

sander: I wanted to point out that if capturing such information would require another pattern, then such adoption for orgs to adopt direction, if that causes structural thing, that would hinder even looking for these things.
… smuggling direction in language might help adoption because it’s easier to do… let’s not make this have additional barriers.

Gregg Kellogg: To clarify, are we done at 3?

Rob Sanderson: This is something we want to do…

Gregg Kellogg: We get together we have an hour, we do the same discussion, we should put a stake in the ground. Something is better than nothing.

Manu Sporny: I’m committed and convinced to do @direction

Rob Sanderson: Is it possible for you to be back after break, danbri?

Dan Brickley: What about the future? When you have arabic, french, in Japanese?
… Are we defining something where future direction happens…?

Charles Neville: What about text orientation and stuff… LTR and RTL…

Addison Phillips: There is a high degree of confidence on number of text directions…

Rob Sanderson: on the Agenda is continue until exhaustion
… in favor of progress
… I think that is contentious to add the surface syntax
… so reopen that original issue
… and then look at RDF round tripping after that’s resolved
… the worst scenario is to go to CR with no solution
… it is better to go CR with @direction and without a mapping into RDF, than to not put in @direction

Gregg Kellogg: curious about danbri’s thoughts

Dan Brickley: that’s up to Google (Bing, etc), but we can encourage users of Schema.org to publish it

Rob Sanderson: (better to also have consensus about the RDF transitional reflection, but that seems more complicated)

Dan Brickley: but can’t promise SEO bots will consume it

Ivan Herman: if I put in a @language into my web site data, what happens

Dan Brickley: we certainly have the notion of languages of pages, etc.
… but the more narrow @language usage would be up to the various products/consumers of that data
… and I can’t speak for what they do
… I think it’s likely that we’re not doing a great job with @language–let alone @direction, etc.
… so I don’t think our tooling will catch fire, etc.
… it would be nice if there were an appendix or something that said even if you’re publishing in 1.0, that you can still put @direction in and hope for the best

Gregg Kellogg: there have been some calls in Schema.org to add some properties to say
… things like breadcrumbs might make use of the @list capability
… or strings that used a map structure
… where there would be a @context to understand those
… whereas at Google they’re just checking directly
… might you consider a similar pattern for @direction

Dan Brickley: I think you’re designing hacks for a system neither of us know much about :)

Rob Sanderson: let’s project some examples

Dan Brickley: I was thinking whatever Voodoo you work up–so in 1.1 it’s these little @ things…

Gregg Kellogg: so, yes, we have @language and soon, we think, @direction
… I can set an @language for the whole document or a certain object
… and when it gets expanded, you have that language data as well as the value

Dan Brickley: so each publishing web site would make use of its own context?

Benjamin Young: let’s make sure we’re all talking using the same terms
… I don’t we’re all sure what expanded form is, etc.

Rob Sanderson: [projects some examples]

Dan Brickley: for examples of google’s use of json-ld (and schema.org), see Search Gallery at https://developers.google.com/search/docs/guides/search-gallery

Manu Sporny: so I feel we’re overusing an unbuilt feature
… how often to we expect people to use the @direction feature actually?
… most authors won’t want to grow to great pains to express it

Addison Phillips: no, but most machines written to care will
… so people with direction metadata will want to consistently use it all over the place

Dan Brickley: Can’t we just use the dir attribute on a script tag?
… You can pick it up from the HTML dir element…

Benjamin Young: No, we’re not allowed to do that.
… You can send it into processing algorithms for JSON-LD…

Ivan Herman: We looked at this, HTML5 spec says to not use lang for script tags.

Dan Brickley: https://html.spec.whatwg.org/multipage/scripting.html#data-block
… is there a way to do this at the top of the file?

Ivan Herman: no

Gregg Kellogg: you can at the top of the file for @language
… and presumably we would for @direction

Ivan Herman: so, yes, it could be there, and on each value
… it is an open question whether the container mapping thing would also be used for language direction
… basically, yesterday’s proposal was to mimic @language, but for the @direction data

Benjamin Young: clearer? or worse?

Dan Brickley: but what if context’s aren’t used?

Dan Brickley: eg https://gist.github.com/danbri/9627452ec7dcbe63af9cb5a631acfdf8#file-gistfile1-txt

Manu Sporny: so you’d just use @language and @value directly as an object for the value of the property–i.e. a value object

Dan Brickley: so, here’s an example we’ve been playing with at Google
… we don’t want to do this on every field–as you can see it’d be painful and verbose

Gregg Kellogg: yeah, we would let you set for the whole document–as with @language

Manu Sporny: I feel like we’re handing people a foot gun
… everything would become ltr or rtl–even if it’s not supposed to

Gregg Kellogg: well, it depends on how they visualize the output to know if they’ve done it wrong

Manu Sporny: people don’t visualize it

Benjamin Young: they just put the JSON into text areas and call it done

Manu Sporny: yeah…most of them don’t even care about @language

Dan Brickley: so do it only down at the property level

Addison Phillips: I can see doing it there

Dan Brickley: would it make sense to have it in the schema languages?
… like certainly fields are “directable”?
… so gsin’s or isbn’s are not directable or language-able

Addison Phillips: isbn’s are a great example
… they have to be a string, includes letters, hyphens, etc.
… but it’s not in a language
… it’s a string literal
… and the world is few of these
… but it’s also full of language-bearing strings which have both language and direction
… in some cases that can be inferred, but in some cases we can’t
… so we do have direction fields in our databases to track that

Manu Sporny: so, one option is to allow @direction at the top and have it cascade to the entire document
… the other extreme/option is to put it on only the values

Ivan Herman: so it is up to Schema.org to decide if they want to promote @direction at the top
… we are here talking about the standard
… and I don’t see why we would disallow this

Addison Phillips: the reason you do this with @language is because you don’t want to repeat it

Ivan Herman: the same applies to @direction

Gregg Kellogg: we have various things to apply @language across the whole document
… I can say that ISBN is no language
… by setting @language: null on just that value
… for symmetries sake it would be odd for us to do this only for @language, but not @direction
… even though we might not have obvious use cases, there is an orthogonality argument to make to do it for the whole document

Ivan Herman: I agree

Addison Phillips: I agree

Gregg Kellogg: something I can see us doing is that if there’s a default @direction, then it could be limited to only things that bear a @language

Ivan Herman: no, ISBN is a good example
… they always go “rtl”

Addison Phillips: MAC addresses are a great example
… they draw backwards if they flow the wrong way

Gregg Kellogg: so, we should do this in all the same places we allow @language

group: [general positive responses]

Gregg Kellogg: I presume it doesn’t make since to have @direction maps?

Addison Phillips: @language maps are very useful
… the question then is how to do both if you use @direction

Gregg Kellogg: we could do nested objects inside nested objects

group: [growns]

tung: so we define only the language, and guess the direction?

Gregg Kellogg: you could leave the direction off
… you could provide both
… or the string could just use the default direction

tung: if the user provides a direction, but if the user does not state a direction it falls back to language?

Addison Phillips: the language map is the space in which you’d define a language
… but on each value

Duy-Tung Luu: what about vertical text?

Addison Phillips: vertical text is a presentational concern

Manu Sporny: I think we keep breaking our own principles here
… now we’re saying we’re using language for text direction when it’s not available?

Addison Phillips: they’re not completely independent

Manu Sporny: yeah…so the fact that this was thoroughly confusing to a bunch of developers…
… the stuff presented earlier about not guessing direction from language…but now we are

uests: David_Clarke

David Clarke: Yiddish can be in Romain characters or Hebrew characters
… so declaring the direction is still important

Manu Sporny: page-wide dir on HTML was maybe a mistake

Addison Phillips: no, there’s a reason to have it on the outside of the container for a bunch of reasons
… there is some linkage between language and direction
… and what we want is to give people the things they need to reach beyond what language provides
… if you ever watch folks attempting to develop or write this stuff, it’s really really hard
… we’re trying to provide enough affordance to take a string out of a page and have it present properly
… without having to introspect, etc.
… we’d like for folks to be able to express their data in all the various standards without having to rewrite all the things
… that’s why having direction is important here
… because it handles all the other bits beyond the text–bullet points, etc.
… works in html, etc.

*Proposed resolution: (1/) We add @direction as a keyword to the JSON-LD Syntax to assert the base text direction of a literal string** (Rob Sanderson)

Manu Sporny: +1

Gregg Kellogg: +1

Rob Sanderson: +1

Benjamin Young: it’s really similar to RDFa using a document level dir for all the things
… and being overridden elsewhere

Benjamin Young: +1

Simon Steyskal: +1

Ivan Herman: +1

Pierre-Antoine Champin: +1

Dan Brickley: +1

Resolution #3: (1/) We add @direction as a keyword to the JSON-LD Syntax to assert the base text direction of a literal string*

*Proposed resolution: (2/) @direction is legal as part of a value object in the json-ld internal model but may not be used if @type is present** (Rob Sanderson)

Manu Sporny: +1

Pierre-Antoine Champin: +1

Rob Sanderson: +1

Benjamin Young: +1

Ivan Herman: +1

Gregg Kellogg: +1

Simon Steyskal: +1

Resolution #4: (2/) @direction is legal as part of a value object in the json-ld internal model but may not be used if @type is present*

Duy-Tung Luu: why no @direction with @type?

Gregg Kellogg: because you can’t have @language with @type

tung: but what if the @type has a language?

Gregg Kellogg: that’s a different case

tung: can we use a class to define language?

Gregg Kellogg: that was considered

*Proposed resolution: (3/) @direction is legal in an expanded term definition if you could have used @language in that definition and applies to all values of that term in the same way as @language** (Rob Sanderson)

Rob Sanderson: I think we’re back in solution space, and should get back to resolutions

Gregg Kellogg: +1

Manu Sporny: +1

Benjamin Young: +1

Pierre-Antoine Champin: +1

Ivan Herman: +1

eg: {"label": {"@id": "rdfs:label", "@direction": "rtl"}}

Resolution #5: (3/) @direction is legal in an expanded term definition if you could have used @language in that definition and applies to all values of that term in the same way as @language*

*Proposed resolution: (4/) The value space of @direction is one of “ltr” and “rtl” (“auto” would have been the same as not asserting the keyword)** (Rob Sanderson)

Rob Sanderson: +1

Gregg Kellogg: +1

Ivan Herman: +1

Simon Steyskal: +1

Manu Sporny: +1

Benjamin Young: +1

Pierre-Antoine Champin: +1

Resolution #6: (4/) The value space of @direction is one of “ltr” and “rtl” (“auto” would have been the same as not asserting the keyword)*

*Proposed resolution: (5/) We expand the possible values of language maps to be either a string (as per today) or a value object with direction and without language or type** (Rob Sanderson)

Rob Sanderson: +1

Gregg Kellogg: +1

Pierre-Antoine Champin: +1

Ivan Herman: +1

Benjamin Young: +1

Manu Sporny: +1

eg: {“en”: {“@value”: “Rob”, “@direction”: “ltr”}}

Resolution #7: (5/) We expand the possible values of language maps to be either a string (as per today) or a value object with direction and without language or type*

*Proposed resolution: (6/) @direction can be present in a context node, and sets in the active context that the default direction for all string literals** (Rob Sanderson)

Rob Sanderson: +1

Gregg Kellogg: +1

Ivan Herman: +1

Manu Sporny: +1

Simon Steyskal: +1

Benjamin Young: +

Pierre-Antoine Champin: +1

Benjamin Young: +1

Resolution #8: (6/) @direction can be present in a context node, and sets in the active context that the default direction for all string literals*

Proposed resolution: (7 agrees that @container: @direction is not a useful feature for JSON-LD 1.1 (unlike @container: @language) (Rob Sanderson)

Rob Sanderson: +1

Manu Sporny: +1

Dan Brickley: +1

Pierre-Antoine Champin: +1

Benjamin Young: +1

Simon Steyskal: +1

Resolution #9: (7/) WG agrees that @container: @direction is not a useful feature for JSON-LD 1.1 (unlike @container: @language)*

Ivan Herman: +1

7. Versioning issues

Dan Brickley: https://w3.org/2018/03/jsonld-wg-charter.html “All changes must preserve backward compatibility for JSON-LD 1.0 documents. This means that, when processing existing JSON-LD documents, JSON-LD 1.1 processors generate the same expanded output, unless that output is subject to errata in JSON-LD 1.0 or is otherwise an unspecified implementation detail.”

Manu Sporny: who will this cost?
… when they upgrade

Dan Brickley: I don’t think our processor do language…or value object or whatever…processing

Gregg Kellogg: so, your developers are happy, but folks who use the JSON-LD labeled tooling, will get a different result
… and for those you’d have to say @version: 1.1

Dan Brickley: we’ve felt at Google and Schema.org for a long time that JSON-LD 1.0 had finally gotten good momentum
… and then the 1.1 work showed up
… and that’s why we pushed against it a bit
… and you all are deep into the @context space
… but we’ve known from the start that the public Web JSON-LD space was actually processed differently
… and without the @context
… the promise made by the WG that the new parsers would still successfully parse the old JSON-LD
… is that still true?

group: yes

Rob Sanderson: but we should probably talk about version triggering…as that’s goes a bit farther
… to opt-in to the new features, you have to include @version: 1.1
… and if you do, that in a context, the old processors–using the @context file–will choke

Dan Brickley: most of the others (Bing, etc) aren’t doing JSON-LD processing using the @context
… and we aren’t either
… so who does this break?

Rob Sanderson: those folks processing schema.org based data with JSON-LD 1.0 processors

Manu Sporny: so, what we could do is use the current versioning mechanism
… but schema.org could promise to not upgrade for 2-3 years
… but still encourage folks to use the @language and @direction stuff now

Dan Brickley: or we could do the opposite and put 1.1 in the context–and see who cries?

Gregg Kellogg: so the way this works is a 1.0 processor is supposed to die when it sees the @version: 1.1, so it doesn’t consume the wrong data

Rob Sanderson: right, and some of those features can allow spoofing of terms…which could have security concerns–depending on what that data is used for

Manu Sporny: so the compromise proposal is to signal with the @version: 1.1 … but what we could do also is that any 1.1 processor could do feature detection … i.e. choose to upgrade for other reasons than just the @version: 1.1`

Gregg Kellogg: so no need for the version announcement?

Manu Sporny: I think it should still be there

Gregg Kellogg: so I’ve come to feel that the @version announcement is such a big bearier
… that people will avoid it

Manu Sporny: so over time if we see that the @version isn’t being used, then we can take it out of the spec
… so it may go away in a future JSON-LD

Gregg Kellogg: so Schema.org could avoid @version?

Manu Sporny: yes, they can use @direction now and that could trigger the upgraded processing

Dan Brickley: what I tried to get across last year was that there is JSON-LD “stuff” that isn’t intended currently to be processed as JSON-LD
… and there are other concerns arising elsewhere…and I’m sorry I’ve not been involved more to help you avoid these…but they still need addressing

Pierre-Antoine Champin: things will still break…just later…so the @version could be avoided
… but we can recommend avoidance of other items also

Rob Sanderson: so if you are concerned that there are 1.1 features that you MUST have processed, then you signal with @version
… otherwise, you just let the processor figure it out from the stuff in the document
… if you have scoped properties or whatever in your context, but your document doesn’t use them…then things would be fine

Ivan Herman: we have a close dependency right now
… which is the publication manifest and we’re at the end…and we had our story with i18n exactly for this reason
… with these resolutions, I would have to add @direction to the manifest
… that manifest is built up from schema.org for most of it’s vocabulary
… and adds a few more that we need that schema.org doesn’t have
… so what happens if I produce a book and don’t put @version in it?
… google or bing or whoever processes it

Dan Brickley: we don’t have a common code base…so we have no idea how these things are processed

Benjamin Young: Rob, the chances that you have that playground up w/ @version? Oh, there it is…. no it isn’t.
… This is what the Publishing WG could do… define it as a term… “direction” set to “@direction”

Gregg Kellogg: I think schema.org has a ‘direction’ term?

Benjamin Young: and you can’t do, in context, @direction: direction?…

Manu Sporny: Group works through example on JSON-LD Playground…

Benjamin Young: A v1.0 processor is going to choke, whatever we do.

Dan Brickley: What if we do an errata?

Benjamin Young: yes. errata could work
… main thing is getting people to support it so it doesn’t choke
… The bulk of JSON-LD code is out of this code base, could “fix” it… could put it in 1.1.

Manu Sporny: ivan just have Publishing WG use 1.1

Ivan Herman: If we introduce @direction…

Gregg Kellogg: 1.0 will die

Manu Sporny: ivan and gkellogg brawl! Chairs thrown, walls torn apart! :P

Benjamin Young: We have stuff that have multiple languages, multiple directions… contents beyond are english… we need a way to do both, in our data, we’d ship @direction and use something in @context.

Gregg Kellogg: You don’t need @version…

Benjamin Young: I don’t want other JSON-LD processors to break.

Ivan Herman: If we find a way where we are not required to use @version… we have an option to use that, but a lot of data that used that stuff, it wouldn’t use it anyway… we do redefine @direction to direction.

Dan Brickley: java https://github.com/jsonld-java/jsonld-java , rdfjs https://github.com/rdfjs-base/parser-jsonld , php https://github.com/lanthaler/JsonLD , dotNet https://github.com/linked-data-dotnet/json-ld.net … do we know which of these toolkits are likely to be upgraded? is there a test harness that can run them and see if they all barf on @direction ?

Pierre-Antoine Champin: The difference between the expressivity using context and data – the benefits of the compromise is that you can publish a context that is not using any 1.1 feature… such as schema.org and still use 1.1 features in the data… yes, 1.0 processors will choke on the data, but they won’t die on the context.

Manu Sporny: that’s the exact point I wanted to make
… if make the @version mandatory
… then you can force it into 1.1 mode
… but at the cost of the current processors
… but we should leave it around until we know we can take it out and still accomplish the same goal

Ivan Herman: Is it possible to express the choice as a feature at risk?

Dan Brickley: For the case where the JSON-LD is in the web page, late in the 1.0 process, Gregg helped spec the script tag, application/ld+json… what happens w/ the new version iterations.

Benjamin Young: all version detection happens in body, the new version could get a new media type… it could… or a nother parameter to the media type.
… in that case, SEO bots could distinguish/care… devs could be told, to opt in, add semicolon v=2.
… different way to signal that…

Dan Brickley: Can we put some wiggle room in so future groups can do stuff?

Rob Sanderson: Yes, we can do this effectively for free.

Benjamin Young: That’s not an insane thing to say, as long as we have enough access to know media type expressed falls back.

Proposed resolution: Mark the requirement for @version at-risk for CR, with the intent to make it optional to include but mandatory to process if present, allowing systems that want to require a particular processing mode to do so, without preventing feature detection based systems to upgrade over time and as needed (Rob Sanderson)

Rob Sanderson: +1

Manu Sporny: +1

Gregg Kellogg: +1

Benjamin Young: +1

Ruben Taelman: +1

Pierre-Antoine Champin: +1

Ivan Herman: +1

Resolution #10: Mark the requirement for @version at-risk for CR, with the intent to make it optional to include but mandatory to process if present, allowing systems that want to require a particular processing mode to do so, without preventing feature detection based systems to upgrade over time and as needed

Ivan Herman: We’ll need to be very clear about how they’re going to detect features, most likley in the text…

Gregg Kellogg: It’s worth running this by Ralph and getting some thoughts.

Benjamin Young: I’d like to talk about media type parameter, to signal that the content is JSON-LD 1.1… when does schema.org encourage its community to do that?

Dan Brickley: The companies build what they build… we can encourage people to look at it… JSON-LD, for a long time Google was using it… Microsoft picked it up, facebook… wobbly for us to say preferred format JSON-LD… we’re a project about schemas…

Gregg Kellogg: My feeling is that SEO community has moved over to JSON-LD…

Dan Brickley: large part was Google picking it up.

Manu Sporny: having an extra signalling mech complicates things

Gregg Kellogg: I think this makes script tags ugly and error prone
… there could be other stuff in the media type to signal processing mode

Benjamin Young: The only way it would add value is if there is a declaration if people are using features, put it in safe box, people start consuming it.
… this is not an evolutionary choice to start over… whereas other feature… DINNER!

8. Resolutions

9. Action Items