IRC log of annotation on 2016-03-25

Timestamps are in UTC.

14:40:56 [RRSAgent]
RRSAgent has joined #annotation
14:40:56 [RRSAgent]
logging to
14:40:58 [trackbot]
RRSAgent, make logs public
14:41:00 [trackbot]
Zakim, this will be 2666
14:41:00 [Zakim]
I do not see a conference matching that name scheduled within the next hour, trackbot
14:41:01 [trackbot]
Meeting: Web Annotation Working Group Teleconference
14:41:01 [trackbot]
Date: 25 March 2016
14:41:10 [ivan]
Chair: Tim Cole
14:41:30 [ivan]
14:42:09 [ivan]
rrsagent, draft minutes
14:42:09 [RRSAgent]
I have made the request to generate ivan
14:50:22 [ivan]
ivan has joined #annotation
14:54:39 [TimCole]
TimCole has joined #annotation
14:57:48 [Jacob]
Jacob has joined #annotation
14:58:14 [TimCole]
14:59:18 [Kyrce]
Kyrce has joined #annotation
15:00:30 [Jacob]
Present+ Jacob_Jett
15:00:36 [ShaneM]
Present+ shepazu
15:00:41 [ShaneM]
present+ ShaneM
15:00:43 [ivan]
ivan has joined #annotation
15:00:43 [Kyrce]
Present+ Kyrce_Swenson
15:00:47 [TimCole]
Present+ Tim Cole
15:00:50 [bigbluehat]
Present+ Benjamin_Young
15:00:58 [ivan]
Present+ Ivan
15:01:34 [ShaneM]
present- shepazu
15:01:37 [tilgovi]
tilgovi has joined #annotation
15:02:08 [bjdmeest]
bjdmeest has joined #annotation
15:02:16 [bjdmeest]
Present+ Ben_De_Meester
15:03:15 [ivan]
ivan has changed the topic to: Agenda:
15:04:05 [ivan]
Regrets: Nick, Dan_Whaley
15:04:09 [tbdinesh]
tbdinesh has joined #Annotation
15:04:13 [tilgovi]
Present+ Randall_Leeds
15:04:33 [TimCole]
scribenick: bjdmeest
15:04:58 [TimCole]
Topic: Acceptance of Minutes:
15:05:11 [TimCole]
PROPOSED RESOULTION: Minutes of the previous call are approved:
15:05:23 [bjdmeest]
TimCole: any concerns?
15:05:27 [ivan]
15:05:36 [takeshi]
takeshi has joined #annotation
15:05:43 [bjdmeest]
RESOLUTION: Minutes of the previous call are approved:
15:05:45 [PaoloCiccarese]
PaoloCiccarese has joined #annotation
15:06:05 [TimCole]
Topic: Results of the CFC
15:06:27 [bjdmeest]
... CFC was sent out last week
15:06:40 [bjdmeest]
... 13 plus ones since a couple of minutes ago
15:06:44 [bjdmeest]
... no 0 or -1s
15:06:49 [bjdmeest]
... so CFC is accepted
15:06:51 [ivan]
15:06:59 [bjdmeest]
... any concerns on the call?
15:07:04 [tbdinesh]
Present+ TB_Dinesh
15:07:13 [PaoloCiccarese]
Present+ Paolo_Ciccarese
15:07:21 [TimCole]
15:07:33 [bjdmeest]
... the CFC was under the assumption that the minor editorial issues would be addressed before publishing
15:07:47 [bjdmeest]
... everything done except for one
15:07:49 [takeshi]
Present+ Takeshi_Kanai
15:08:01 [bjdmeest]
ivan: that's fine now, remaining is for the future
15:08:23 [TimCole]
ack ivan
15:08:42 [bjdmeest]
... if we decide to make a FPWD, there are two (minor) consequences
15:09:00 [bjdmeest]
... first: history of the splitting up is lost
15:09:17 [bjdmeest]
... second: patent policy would require to start from scratch for that document
15:09:30 [bjdmeest]
... so at least 6 months needed between FPWD and REC
15:09:32 [bjdmeest]
... that's not ideal
15:09:37 [bjdmeest]
... I discussed
15:09:40 [bjdmeest]
... result:
15:10:00 [bjdmeest]
... Vocab doc is published as FPWD, and previous version is the Model doc
15:10:12 [bjdmeest]
... so consequences are resolved
15:10:39 [bjdmeest]
... practical consequence is a small editorial change
15:11:22 [TimCole]
Proposed Resolution: publish all 3 as Working Draft, with previous draft for Vocabulary being the earlier Data Model draft
15:11:43 [bjdmeest]
Resolution: publish all 3 as Working Draft, with previous draft for Vocabulary being the earlier Data Model draft
15:12:09 [bjdmeest]
ivan: question is: are the documents as they are today final and ready to be published?
15:12:15 [bjdmeest]
bigbluehat: yes
15:12:35 [bjdmeest]
ivan: also: have they been checked by link checker and html checker etc.?
15:12:46 [bjdmeest]
paolociccarese: We can do it again
15:13:20 [bjdmeest]
ivan: I will have to check it to be safe, but if you guys could do that by Monday, I can do the rest of the admin on Monday.
15:13:47 [bjdmeest]
... PaoloCiccarese, also change the previous version of the Vocab doc to the Model doc, as discussed
15:13:53 [bjdmeest]
... I will pick it up from there
15:14:24 [bjdmeest]
ShaneM: Ivan, should SoTD be updated to say this is a split?
15:14:38 [bjdmeest]
ivan: Paolo can do it as well, but yes!
15:15:06 [bjdmeest]
... in the status of the Vocab document, there should be an extra sentence that this is a split from the Model doc
15:15:24 [TimCole]
Topic: Welcome ShaneM
15:15:59 [bjdmeest]
ShaneM: I've been with W3C since '97
15:16:09 [bjdmeest]
... I;m with spec ops, doing standards related work
15:16:19 [bjdmeest]
... Shepazu contacted me about testing
15:16:26 [bjdmeest]
... I have a ton of questions
15:16:44 [bjdmeest]
... I've been doing standards work since '85, and testing since '87
15:16:45 [ivan]
15:17:17 [TimCole]
ack ivan
15:17:26 [bjdmeest]
ivan: Shane is modest. He was one of the main editors for the RDFa spec
15:17:32 [bjdmeest]
... which might be useful
15:17:41 [bjdmeest]
... he also co-maintains respec
15:18:10 [bjdmeest]
TimCole: other anouncements?
15:18:30 [tantek]
tantek has joined #annotation
15:18:42 [ShaneM]
It is the section entitled "Status of this document"
15:18:54 [TimCole]
Topic: Testing
15:19:11 [TimCole]
15:19:12 [bjdmeest]
TimCole: There are some notes in last week's minutes
15:19:47 [bjdmeest]
... we have to look into our documents, find the features that are described, and provide for a strategy to test these features
15:19:58 [bjdmeest]
... and make sure they are unambiguous and implementable
15:20:01 [bjdmeest]
... I welcome help
15:20:19 [bjdmeest]
... Identifying the features is important, implementing them also
15:20:39 [bjdmeest]
... particularly the selector approach might not be implemented in every system
15:20:48 [bjdmeest]
... we have to have at least two implementations for each feature
15:21:06 [bjdmeest]
... how do we go on with this?
15:21:11 [ivan]
15:21:18 [TimCole]
ack ivan
15:21:47 [bjdmeest]
ivan: the real question (for the model) is: what is really what we want to test?
15:22:04 [bjdmeest]
... it is a vocabulary for a specific usage, we have to identify what we want to test
15:22:21 [bjdmeest]
... the current direction is that we define specific scenarios
15:22:40 [bjdmeest]
... an implementation should show that these scenarios can be mapped on the correct annotation structures
15:23:09 [bjdmeest]
... and maybe also the other way around: annotation structures should be understood by implementations and in some way tell us what they would do with these annotations
15:23:42 [bjdmeest]
... questions are: does that provide a reasonable way of testing the spec, and can this be translated into proper tools?
15:24:07 [bjdmeest]
... we have to be very careful that we are not testing implementations, but we are testing the spec
15:24:25 [ShaneM]
q+ to disagree a little with ivan about what must be tested...
15:24:58 [TimCole]
ack sha
15:24:58 [Zakim]
ShaneM, you wanted to disagree a little with ivan about what must be tested...
15:25:01 [bjdmeest]
TimCole: about the scenarios: a suggestion was to have sample resources to be tested, to illustrate the features
15:25:26 [bjdmeest]
ShaneM: we're not testing implementations was said...
15:25:36 [bjdmeest]
... but each feature must be implemented
15:25:46 [bjdmeest]
... so we test the implementations implementing the features
15:26:10 [aaronpk]
aaronpk has joined #annotation
15:26:16 [bjdmeest]
ivan: if my goal would be to test the implementation, I would use automatic tests
15:26:54 [bjdmeest]
... in this case, we test the specifications, we can ask implementers to send a report, without knowing how they tested their implementation
15:27:39 [bjdmeest]
ShaneM: In the Web annotation spec, there are requirements on a lot of different actors
15:27:44 [bjdmeest]
... data structure requirements
15:27:48 [bjdmeest]
... client requirements
15:27:54 [bjdmeest]
... interpretation requirements
15:27:58 [bjdmeest]
... server requirements
15:28:15 [bjdmeest]
... I assume you want to test all of them?
15:28:19 [bjdmeest]
TimCole: I think so
15:29:10 [bjdmeest]
... question was: what does an implementation have to do with an existing annotation to make sure it interprets it correctly?
15:29:29 [bjdmeest]
ShaneM: you don't have behaviour requirements
15:29:40 [bjdmeest]
... or UI requirements
15:29:52 [bjdmeest]
... so that makes your testing burden lower
15:30:13 [bjdmeest]
,,, here, you just have to ensure that consumers of the data format receive the data format intact
15:30:27 [bjdmeest]
... you can ignore SHOULDs for testing purposes
15:30:41 [bjdmeest]
... I would focus on the MUSTs and MUST NOTs
15:31:27 [tbdinesh]
+1 for Tim
15:31:41 [bjdmeest]
... Talking about MUSTs
15:31:59 [bjdmeest]
... there's some data structural requirements, which come for free via JSON-LD
15:32:16 [bjdmeest]
... so testing conforming output is probably kind of manual
15:32:34 [bjdmeest]
... e.g. these N selection annotations needs to be tested
15:32:51 [bjdmeest]
... you don't want to test if the region makes sense or the CSS selector is correct etc.
15:33:35 [bjdmeest]
ivan: Let's say that we describe a scenario: here's an SVG file, we want to put an annotation on this circle on the upperhand corner
15:33:52 [bjdmeest]
... the resulting annotation structure should correspond with an annotation put on that corner
15:34:09 [bjdmeest]
... in the output, we assume an SVGSelector to the correct corner
15:34:26 [bjdmeest]
... so we need to check for correct JSON-LD, and correct to our spec (i.e., that it's an SVGSelector)
15:34:43 [bjdmeest]
... but we don't have to check that the SVGSelector actually selects the correct target?
15:34:58 [bjdmeest]
ShaneM: you go into that dept, but I'm not sure it's required
15:35:11 [bjdmeest]
... because there are a lot of ways of specifying that region
15:35:14 [TimCole]
15:35:36 [bjdmeest]
... suppose you have a server, he's going to analyze that annotation, but it's hard to analyze every detail
15:35:44 [bjdmeest]
... you would need an SVGrenderer
15:35:56 [bjdmeest]
... you could do that manually, but that very consuming
15:36:10 [bjdmeest]
ivan: I was considering human testing, but that's very time consuming
15:36:15 [TimCole]
15:36:29 [bjdmeest]
ShaneM: I always look at this as: what can we automate?
15:36:37 [bjdmeest]
ivan: the only advantage is that the model is relatively simple
15:36:48 [bjdmeest]
... we don't have a huge number things to test
15:36:52 [TimCole]
ack ti
15:37:05 [bjdmeest]
ShaneM: there are some combinatorial things that I have noted
15:37:51 [bjdmeest]
TimeCole: Manual can be very expensive, we thought about: I have a specific scenario: this is the image, this is the exact circle to annotate, and that should limit the number of ways to do it
15:38:10 [bjdmeest]
... e.g., textQuoteSelectors doesn't have that many ways
15:38:26 [bjdmeest]
ShaneM: depends on what kind of constraints you want to put on the client
15:38:42 [bjdmeest]
... talking about textRangeSelection
15:38:49 [bjdmeest]
... you allow for a number of ways to express
15:38:55 [bjdmeest]
... not all clients will implement all ways
15:39:10 [bjdmeest]
... And I assume the client decides which expression will be the right way
15:39:22 [bjdmeest]
... depending on the context
15:39:54 [bjdmeest]
... do you want to require for test X that the client gives a CSS selector, and test Y gives a textRangeSelector
15:39:58 [bjdmeest]
ivan: that doesn't sound right
15:40:19 [bjdmeest]
ShaneM: another way would be: here's a sample document, select these 5 words
15:40:39 [bjdmeest]
... the server-side should check: which expression does it use, and is that correct?
15:40:55 [bjdmeest]
... that way, you simplify the test matrix, without testing every possible combination
15:41:07 [bjdmeest]
... you can say: the textselector works
15:41:16 [bjdmeest]
ivan: it can happen that one of the selectors is never used
15:41:24 [mete_pinar]
mete_pinar has joined #annotation
15:41:29 [mete_pinar]
mete_pinar has left #annotation
15:41:37 [bjdmeest]
... because another combination of selectors always works for the given scenarios
15:41:41 [mete_pinar]
mete_pinar has joined #annotation
15:42:03 [bjdmeest]
... does that mean that that selector shouldn't be in the spec
15:42:17 [bjdmeest]
ShaneM: you could say it becomes a feature at risk
15:42:25 [bjdmeest]
... or, it could be part of the testing cycle
15:42:57 [bjdmeest]
... e.g., are there ways to require the feature
15:43:30 [bjdmeest]
TimCole: one extra thing before the end of the call
15:43:47 [bjdmeest]
... who on the WG could help to build the list of features?
15:44:08 [bjdmeest]
... someone (pref. not the editors) should identify the features in the docs
15:44:16 [bjdmeest]
... certainly the MUSTs, maybe the SHOULDs
15:44:33 [bjdmeest]
ivan: we have to be careful to minimize people's time
15:44:47 [ShaneM]
q+ to say that you should also identify if requirements are on a repository vs. a generator vs. a consumer
15:44:53 [bjdmeest]
... we have to know in what format to make these scenarios
15:45:05 [TimCole]
ack Sh
15:45:05 [Zakim]
ShaneM, you wanted to say that you should also identify if requirements are on a repository vs. a generator vs. a consumer
15:45:49 [bjdmeest]
ShaneM: it's useful, I've been doing it, I think one of the critical pieces are checking whether MUSTs and SHOULDs are really MUSTs and SHOULDs
15:46:00 [bjdmeest]
... and also, what parts of the system these features are one
15:46:09 [bjdmeest]
... we need to compartimentalize by
15:46:16 [bjdmeest]
... repository vs. a generator vs. a consumer
15:46:43 [bjdmeest]
... in terms of interoperability, we should make sure any generator can talk to any repository
15:46:58 [bjdmeest]
ivan: you have implementations not bound to a specific server
15:47:05 [bjdmeest]
... there are 2 directions of testing:
15:47:14 [bjdmeest]
... scenario-based
15:47:39 [bjdmeest]
... and from annotation-structure to interpretation of the implementation
15:48:23 [bjdmeest]
TimCole: I don't know whether the compartiments are already named very well
15:48:50 [bjdmeest]
PaoloCiccarese: I don't have specific testing-ideas
15:48:59 [bjdmeest]
... I didn't have to test for different servers or clients
15:49:48 [bjdmeest]
... we changed the specs over time, in my case, we used SPARQL validators for the model
15:49:57 [bjdmeest]
... and adapted over time
15:49:59 [ivan]
15:50:41 [bjdmeest]
TimCole: The Annotation Community Group identified 55(?) MUSTs and SHOULDs, and validated semantically using a set of SPARQL queries
15:50:47 [bjdmeest]
... two for every must and should
15:51:06 [bjdmeest]
... one to check whether a feature applied, a second to validate
15:51:18 [bjdmeest]
... but the spec has changed since then
15:51:30 [bjdmeest]
... and not only semantic anymore
15:51:48 [bjdmeest]
... there must be three components, but there not defined yet
15:51:51 [PaoloCiccarese]
15:51:56 [PaoloCiccarese]
15:52:04 [TimCole]
ack ivan
15:52:04 [bjdmeest]
ShaneM: I agree, and I would structure the tests that way
15:52:33 [bjdmeest]
ivan: I would be very helpful if Paolo could give a small description of how his implementation was tested
15:52:42 [bjdmeest]
... it would help me, but maybe also Shane
15:53:06 [bjdmeest]
... maybe I could ask Takeshi to do something similar
15:53:56 [bjdmeest]
takeshi: I have some, but I was thinking about modifying the testing files to test for internationalitation
15:54:09 [ShaneM]
Every little bit helps
15:54:44 [bjdmeest]
ivan: having a good feeling of what is currently being done would be very helpful
15:54:57 [bjdmeest]
ShaneM: every little piece will come together in the whole
15:55:09 [bjdmeest]
... tests should be as discrete as possible
15:55:21 [bjdmeest]
... big (i.e., integration of system) tests exist
15:55:55 [bjdmeest]
... but the small tests show where something goes down
15:56:10 [TimCole]
ack paolo
15:56:11 [bjdmeest]
... e.g., the same scenario needs to be tested for 11 different variables
15:56:39 [bjdmeest]
PaoloCiccarese: critical point would be the republishing of annotations
15:57:04 [bjdmeest]
... I'm not sure we have the solution to that
15:57:09 [bjdmeest]
... it will be interesting to test
15:57:15 [bjdmeest]
... it will be cross-system testing
15:57:34 [bjdmeest]
... test system a, then system b, then send from a to b, and have certain expectations
15:57:43 [bjdmeest]
... it's one of the most delicate points
15:57:59 [bjdmeest]
... duplications of annotations will make things go out of control
15:58:04 [bjdmeest]
... but it's the web, so it will happen
15:58:25 [bjdmeest]
... about my current testing: I mostly do code testing, very tailored, lots of RDF
15:58:41 [bjdmeest]
... so testing is many roundtrips to the triple store
15:59:18 [uskudarli]
uskudarli has joined #annotation
15:59:28 [bjdmeest]
TimCole: There's a need to talk about what we think are the compartiments (generator, consumer, repository)
15:59:45 [bjdmeest]
... then, we need to talk about scenarios and features
16:00:10 [bjdmeest]
... to make some progress before F2F, for next week we might also talk about this testing topic
16:00:11 [ivan]
ivan has joined #annotation
16:00:27 [bjdmeest]
ivan: email discussion is fine at the moment
16:00:55 [bjdmeest]
ShaneM: I'll put issues or send questions to the mailing list
16:01:44 [bjdmeest]
ivan: will you join the WG?
16:01:51 [bjdmeest]
ShaneM: I'll ask, I know the drill
16:02:05 [bjdmeest]
TimCole: [adjourn]
16:02:13 [ivan]
rrsagent, draft minutes
16:02:13 [RRSAgent]
I have made the request to generate ivan
16:02:17 [bjdmeest]
... next week: F2F, conformance
16:02:21 [bjdmeest]
... bye!
16:02:38 [ivan]
trackbot, end telcon
16:02:38 [trackbot]
Zakim, list attendees
16:02:38 [Zakim]
As of this point the attendees have been Ivan, Frederick_Hirsch, Rob_Sandersion, Rob_Sanderson, Tim_Cole, Benjamin_Young, Jacob_Jett, shepazu, davis_salisbury, Paolo_Ciccarese,
16:02:42 [Zakim]
... Ben_De_Meester, Chris_Birk, TB_Dinesh, Takeshi_Kanai, Randall_Leeds, Dan_Whaley, Susan, Uskudarli, !, Nick_Stenning, Suzan_Uskudarli, 0, Kyrce_Swenson, ShaneM, Cole
16:02:46 [trackbot]
RRSAgent, please draft minutes
16:02:46 [RRSAgent]
I have made the request to generate trackbot
16:02:47 [trackbot]
RRSAgent, bye
16:02:47 [RRSAgent]
I see no action items
16:02:47 [TimCole]
TimCole has left #annotation