14:52:48 <RRSAgent> RRSAgent has joined #annotation
14:52:48 <RRSAgent> logging to http://www.w3.org/2016/08/12-annotation-irc
14:52:50 <trackbot> RRSAgent, make logs public
14:52:50 <Zakim> Zakim has joined #annotation
14:52:52 <trackbot> Zakim, this will be 2666
14:52:52 <Zakim> ok, trackbot
14:52:53 <trackbot> Meeting: Web Annotation Working Group Teleconference
14:52:53 <trackbot> Date: 12 August 2016
14:54:03 <ivan> Hm. Indeed, it seems that both Tim and Rob forgot to send it out, although we did have a discussion on the agenda content
14:54:05 <ivan> Sigh
14:54:11 <TimCole> TimCole has joined #annotation
14:54:14 <azaroth> Just sent it
14:54:18 <azaroth> :blush:
14:54:57 <azaroth> azaroth has changed the topic to: Agenda: https://lists.w3.org/Archives/Public/public-annotation/2016Aug/0117.html
14:55:28 <ShaneM> ack
14:55:56 <ivan> Agenda: https://lists.w3.org/Archives/Public/public-annotation/2016Aug/0117.html
14:56:31 <azaroth> Chair: Rob_Sanderson, Tim_Cole
14:56:35 <azaroth> Present+ Rob_Sanderson
14:56:56 <TimCole> Present+ Tim_Cole
14:58:54 <Jacob> Jacob has joined #annotation
14:59:07 <nickstenn> Present+ Nick_Stenning
14:59:09 <Jacob> Present+ Jacob_Jett
14:59:13 <ivan> Present+ Ivan
15:00:43 <dwhly> Present+ Dan_Whaley
15:02:30 <azaroth> TOPIC:  Scribe Selection, Agenda Review
15:03:08 <azaroth> scribenick: Tim_Cole
15:03:30 <TimCole> Topic: Agenda review
15:03:49 <TimCole> azaroth: minutes, announcements, internationalization issues (brief), testing
15:04:00 <TimCole> ... any other topics for today?
15:04:07 <bjdmeest> bjdmeest has joined #annotation
15:04:11 <azaroth> TOPIC: Minutes Approval
15:04:25 <bjdmeest> Present+ Ben_De_Meester
15:04:25 <azaroth> PROPOSED RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/08/05-annotation-minutes.html
15:04:30 <ivan> +1
15:04:34 <azaroth> +1
15:04:39 <TimCole> +1
15:04:43 <Jacob> +1
15:04:46 <ivan> Present+ Ben_De_Meester
15:04:47 <TimCole> RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/08/05-annotation-minutes.html
15:05:04 <TimCole> Topic: Announcements?
15:05:17 <TimCole> scribenick: nickstenn
15:05:24 <azaroth> TOPIC: Issues
15:05:41 <bigbluehat> Present+ Benjamin_Young
15:05:56 <nickstenn> azaroth: The discussion around I18N has continued. We now have six (6) open issues around this topic.
15:06:08 <nickstenn> ... To summarise:
15:06:22 <nickstenn> ... #335: WONTFIX -- long thread with social web WG
15:06:29 <ivan> q+
15:06:41 <nickstenn> ... last state was I18N folks were discussing with Activity Streams folks and would get back to us
15:07:52 <nickstenn> ... #342: A suggestion from Sergiu to add a note to say if dc:language is specified and processingLanguage is not, the latter should be assumed to be the former
15:07:57 <nickstenn> ... seems like a good editorial note
15:08:21 <nickstenn> ... Similarly with #343.
15:08:45 <nickstenn> ... which is about whether processingLanguage should be required to be a language in dc:language
15:09:09 <nickstenn> ... Haven't had much of a chance to look at #345, also about processingLanguage
15:09:41 <azaroth> ack ivan
15:09:59 <nickstenn> ... #341 also about processingLanguage for multilingual resources, and we've decided to postpone
15:12:40 <TimCole> q?
15:12:41 <ivan> Issue #345 is an attempt from Richard to close an issue and discussion with gsergiu via an editorial change proposal
15:12:52 <azaroth> q?
15:12:57 <TimCole> ack sees
15:13:38 <nickstenn> azaroth: suggest we spend the time on the call discussing testing rather than going into the details on some of these I18N issues
15:13:44 <nickstenn> ... any problems?
15:13:48 <azaroth> TOPIC: Testing
15:14:17 <nickstenn> azaroth: Two parts of testing -- 1) model, and 2) protocol
15:14:38 <nickstenn> ... TimCole and ShaneM have been working on this. Update:
15:15:12 <nickstenn> TimCole: regarding the creation of the underlying schemas for the tests -- we've captured everything 3.1, 3.2, and 3.3 excepting agents, and most of section 4
15:15:25 <nickstenn> ... these schemas are in the "definitions" folder
15:15:37 <nickstenn> ... and are referenced by the schemas we intend to use for assertions
15:16:05 <TimCole> http://testdev.spec-ops.io:8000/tools/runner/index.html
15:16:12 <nickstenn> ... I've just started working on the test scripts. Have been hashing out with ShaneM what those look like.
15:16:23 <nickstenn> ... This [^] is a test environment ShaneM has set up.
15:16:51 <nickstenn> ... You can use this with "Run tests under path" using the following path:
15:17:05 <ShaneM> http://testdev.spec-ops.io:8000/tools/runner/index.html?path=/annotation-model should work too
15:17:23 <nickstenn> ... "/annotation-model"
15:18:04 <nickstenn> ... you can paste JSON[-LD] in and run through through the test suite
15:18:28 <nickstenn> ... that's all working.
15:19:13 <nickstenn> ... Having some small issues for SHOULD requirements, where we're not necessarily expecting you to pass the test. Currently if you *do* pass the requirement, the test fails.
15:19:37 <ShaneM> try going to this URI now:
15:19:39 <ShaneM> http://testdev.spec-ops.io:8000/annotation-model/annotations/3.1-model-musts-v3-manual.html
15:20:43 <azaroth> Example annotations to play with: https://github.com/w3c/web-annotation/tree/gh-pages/model/wd2/examples/correct
15:21:16 <nickstenn> ... ShaneM: [walking us through how to use the test tool at the link above to test annotations]
15:23:03 <azaroth> q+ to ask about display:none in HTML ?
15:23:09 <ShaneM> Errors: data should have required property '@context'; expected true got false
15:23:15 <nickstenn> ShaneM: Question: do we want to suppress the output above ^
15:23:16 <azaroth> q?
15:23:21 <azaroth> q- sits
15:23:24 <azaroth> ack azaroth
15:23:24 <Zakim> azaroth, you wanted to ask about display:none in HTML ?
15:23:53 <nickstenn> azaroth: Would it be possible to have the AJV stack trace be in a display:none; area with a button to reveal it or similar?
15:24:16 <nickstenn> ShaneM: I don't think so. We just report data back to the test harness, which is responsible for the display.
15:25:05 <nickstenn> azaroth: the "data should have required property @context" is particularly useful to understand what's going on
15:25:19 <nickstenn> ShaneM: I'll leave it there, then.
15:26:13 <nickstenn> ivan: currently can't rerun with updated JSON
15:26:19 <nickstenn> ShaneM: I'll see if we can fix that
15:26:24 <TimCole> q+
15:26:29 <azaroth> ack TimCole
15:26:34 <TimCole> http://testdev.spec-ops.io:8000/annotation-model/bodiesTargets/3.2-model-manual.html
15:28:11 <nickstenn> TimCole: if you put in a test annotation that *passes* the SHOULD requirements, the output is a little harder to interpret
15:28:18 <nickstenn> ... not sure quite how to improve the output in that case
15:30:00 <azaroth> I used: https://github.com/w3c/web-annotation/blob/gh-pages/model/wd2/examples/correct/anno41.json
15:30:02 <nickstenn> ivan: [proceeds to find a bug in one of the testing schemas while on the call]
15:32:20 <nickstenn> TimCole: [paraphrasing heavily] currently for the SHOULD assertions, we expect non-conformance, which means that if it actually is conformant, the test fails
15:32:38 <azaroth> Pass if it's not used, with a success message that it SHOULD be there?
15:32:47 <nickstenn> ShaneM: no way of doing "warning" in the framework
15:33:09 <nickstenn> ... but we might be able to use "testType" to distinguish between MUST and SHOULD assertion types
15:33:09 <azaroth> Result: Pass Message: WARNING: Format SHOULD be included for bodies, if known
15:34:22 <ShaneM> use assertionType of must, may, or shold
15:34:25 <nickstenn> TimCole: we have other scenarios where we say "SHOULD have 1, MAY have more than 1 X"
15:34:34 <ShaneM> testType has to do with automation.
15:35:36 <nickstenn> ... there are other cases where you MUST NOT have more than 1
15:35:49 <nickstenn> ... so in these cases we can have multiple cases for the different cardinalities
15:36:03 <nickstenn> s/multiple cases/multiple assertions/
15:36:16 <ShaneM> http://testdev.spec-ops.io:8000/annotation-model/bodiesTargets/3.2-model-manual.html
15:36:30 <nickstenn> ShaneM: If you look at the top of [^] you'll notice that the page fills in as the page loads.
15:37:18 <nickstenn> ... In the description at the top of the page, there's a list of things the test will check
15:37:35 <nickstenn> ... do we want to include SHOULD/MUST/MAY information in these descriptions?
15:37:44 <nickstenn> [noises of general agreement]
15:38:31 <azaroth> q?
15:38:48 <azaroth> q+ to ask about fails for example 41
15:39:23 <nickstenn> TimCole: going to spend the next little while cleaning this up with a view to sharing it more widely
15:39:31 <nickstenn> ... how are the test outputs recorded?
15:39:54 <nickstenn> ShaneM: anyone with an implementation can record their JSON output and add it to a git repository which contains all the results
15:40:08 <nickstenn> s/JSON output/JSON test output/
15:41:08 <nickstenn> TimCole: how do people want to break up the various test for bodies/targets/optional keys/etc.?
15:41:43 <nickstenn> ShaneM: speaking as "not an implementer" -- the smallest number of manual tests that get us the information we need is probably a reasonable guideline
15:42:03 <nickstenn> TimCole: probably a discussion for the mailing list
15:42:57 <azaroth> https://github.com/w3c/web-annotation/blob/gh-pages/model/wd2/examples/correct/anno41.json
15:43:09 <nickstenn> azaroth: example 41 [^] is a completely contrived example at the end of the spec
15:43:23 <nickstenn> ... it seems unlikely (in the short term at least) that any client would generate such an annotation
15:43:30 <nickstenn> ... but perhaps not outside the realms of possibility
15:43:48 <nickstenn> ... putting in the 3.2 set of tests, it passes 8 but fails 5
15:44:15 <nickstenn> ... wondering what those are: problems with the test harness, the SHOULD problem, or problems with the data?
15:44:58 <nickstenn> TimCole: would need to have a look, but it's probably the SHOULD issue with multiple formats
15:46:46 <azaroth> q?
15:46:47 <azaroth> ack azaroth
15:46:47 <Zakim> azaroth, you wanted to ask about fails for example 41
15:47:05 <nickstenn> azaroth: we should probably spend some time talking about protocol testing
15:47:10 <azaroth> TOPIC: Protocol Testing
15:48:06 <nickstenn> bigbluehat: have mostly passed the work I've done onto ShaneM
15:48:35 <nickstenn> ShaneM: the server tests bigbluehat are awesome, but let's talk about client tests for a second
15:48:51 <nickstenn> ... the server runs in the WPT environment
15:49:29 <azaroth> ( issue https://github.com/w3c/web-annotation/issues/344 )
15:49:45 <nickstenn> ... the way annotations work is that an annotation collection lives at a IRI, and thus the server needs to serve at some named route within WPT
15:50:14 <bigbluehat> PUT to overwrite
15:50:24 <nickstenn> ... but we need to work out how create/update/destroy operations work in the test server
15:51:00 <nickstenn> ... in particular because we don't want ephemeral data created on the test server
15:52:00 <nickstenn> ... so we're going to be arranging things such that data created by clients is destroyed as soon as it is read back from the server
15:52:33 <nickstenn> ivan: how is that going to work if you want to prepare a bunch of data and then run a load of tests against that?
15:53:20 <nickstenn> ShaneM: in those cases the client will access some static collection of annotations rather than data they created
15:53:34 <nickstenn> q+
15:53:47 <azaroth> ack nickstenn
15:53:52 <azaroth> scribenick: azaroth
15:54:18 <azaroth> nickstenn: Question about how this is going to work -- the protocol spec doesn't say what the server is supposed to do with the data that you give it, even reasonable things
15:54:35 <azaroth> ... for example, in a distributed annotation system, you POST to create it, but you might not be able to read it back again straight away
15:54:50 <azaroth> ... want us to be careful that we're not testing that you can read something straight away
15:55:02 <azaroth> bigbluehat: The protocol says that it comes back with the full representation
15:55:08 <azaroth> ivan: Comes back with an id
15:55:49 <azaroth> nickstenn: That's a different point though. Could return it straight away, but the server doesn't necessarily have state beyond that
15:56:04 <azaroth> ... intuitively reasonable assumptions are fine, but that's not in the spec
15:56:29 <azaroth> ... need to be careful to not write tests on our understanding of the spec, but what the spec actually says
15:56:43 <azaroth> bigbluehat: e.g. there's no guarantee that you'll be able to get the annotation back after you create it
15:57:02 <azaroth> ShaneM: Wondering if the server tests you wrote rely on creating and then immediately retrieving it
15:57:28 <azaroth> bigbluehat: would need to go through the tests to see if it requests the same ones later, I don't think so
15:57:39 <azaroth> ... it puts stuff in but I don't think it checks again later
15:57:50 <azaroth> ShaneM: If that's the case, then we're good
15:58:09 <bigbluehat> this is the thing ShaneM's been mentioning btw: https://github.com/BigBlueHat/web-annotation-protocol-tester
15:58:13 <azaroth> scribenick: nickstenn
15:58:33 <nickstenn> ShaneM: in our last 2m, let's just agree on what we think is going to happen over the next week
15:58:56 <nickstenn> ... I have a couple of next actions from this conversation and will get on those straight away
15:59:06 <nickstenn> ... I have another to ensure people know how to run tests and upload results
15:59:15 <nickstenn> ... also working on getting the protocol stuff implemented
15:59:32 <nickstenn> TimCole: going to fill in a few more schemas to cover sections 3 and 4
15:59:41 <ivan> q+
16:00:03 <azaroth> ack ivan
16:00:36 <nickstenn> TimCole: if people could help make some invalid annotations to help test the failure cases, that would be helpful
16:00:44 <nickstenn> azaroth: I can do some of that
16:00:59 <nickstenn> ivan: When do you folks think we can begin to pester implementers to provide reports?
16:01:33 <nickstenn> TimCole: I think we need to do a reality check next Friday before we start inviting implementers -- maybe the week after that?
16:02:19 <azaroth> TOPIC: Adjourn
16:02:38 <ivan> rrsagent, draft minutes
16:02:38 <RRSAgent> I have made the request to generate http://www.w3.org/2016/08/12-annotation-minutes.html ivan
16:03:03 <ivan> trackbot, end telcon
16:03:03 <trackbot> Zakim, list attendees
16:03:03 <Zakim> As of this point the attendees have been Rob_Sanderson, Tim_Cole, Nick_Stenning, Jacob_Jett, Ivan, Dan_Whaley, Ben_De_Meester, Benjamin_Young
16:03:11 <trackbot> RRSAgent, please draft minutes
16:03:11 <RRSAgent> I have made the request to generate http://www.w3.org/2016/08/12-annotation-minutes.html trackbot
16:03:12 <trackbot> RRSAgent, bye
16:03:12 <RRSAgent> I see no action items