IRC log of annotation on 2016-08-12

Timestamps are in UTC.

14:52:48 [RRSAgent]
RRSAgent has joined #annotation
14:52:48 [RRSAgent]
logging to
14:52:50 [trackbot]
RRSAgent, make logs public
14:52:50 [Zakim]
Zakim has joined #annotation
14:52:52 [trackbot]
Zakim, this will be 2666
14:52:52 [Zakim]
ok, trackbot
14:52:53 [trackbot]
Meeting: Web Annotation Working Group Teleconference
14:52:53 [trackbot]
Date: 12 August 2016
14:54:03 [ivan]
Hm. Indeed, it seems that both Tim and Rob forgot to send it out, although we did have a discussion on the agenda content
14:54:05 [ivan]
14:54:11 [TimCole]
TimCole has joined #annotation
14:54:14 [azaroth]
Just sent it
14:54:18 [azaroth]
14:54:57 [azaroth]
azaroth has changed the topic to: Agenda:
14:55:28 [ShaneM]
14:55:56 [ivan]
14:56:31 [azaroth]
Chair: Rob_Sanderson, Tim_Cole
14:56:35 [azaroth]
Present+ Rob_Sanderson
14:56:56 [TimCole]
Present+ Tim_Cole
14:58:54 [Jacob]
Jacob has joined #annotation
14:59:07 [nickstenn]
Present+ Nick_Stenning
14:59:09 [Jacob]
Present+ Jacob_Jett
14:59:13 [ivan]
Present+ Ivan
15:00:43 [dwhly]
Present+ Dan_Whaley
15:02:30 [azaroth]
TOPIC: Scribe Selection, Agenda Review
15:03:08 [azaroth]
scribenick: Tim_Cole
15:03:30 [TimCole]
Topic: Agenda review
15:03:49 [TimCole]
azaroth: minutes, announcements, internationalization issues (brief), testing
15:04:00 [TimCole]
... any other topics for today?
15:04:07 [bjdmeest]
bjdmeest has joined #annotation
15:04:11 [azaroth]
TOPIC: Minutes Approval
15:04:25 [bjdmeest]
Present+ Ben_De_Meester
15:04:25 [azaroth]
PROPOSED RESOLUTION: Minutes of the previous call are approved:
15:04:30 [ivan]
15:04:34 [azaroth]
15:04:39 [TimCole]
15:04:43 [Jacob]
15:04:46 [ivan]
Present+ Ben_De_Meester
15:04:47 [TimCole]
RESOLUTION: Minutes of the previous call are approved:
15:05:04 [TimCole]
Topic: Announcements?
15:05:17 [TimCole]
scribenick: nickstenn
15:05:24 [azaroth]
TOPIC: Issues
15:05:41 [bigbluehat]
Present+ Benjamin_Young
15:05:56 [nickstenn]
azaroth: The discussion around I18N has continued. We now have six (6) open issues around this topic.
15:06:08 [nickstenn]
... To summarise:
15:06:22 [nickstenn]
... #335: WONTFIX -- long thread with social web WG
15:06:29 [ivan]
15:06:41 [nickstenn]
... last state was I18N folks were discussing with Activity Streams folks and would get back to us
15:07:52 [nickstenn]
... #342: A suggestion from Sergiu to add a note to say if dc:language is specified and processingLanguage is not, the latter should be assumed to be the former
15:07:57 [nickstenn]
... seems like a good editorial note
15:08:21 [nickstenn]
... Similarly with #343.
15:08:45 [nickstenn]
... which is about whether processingLanguage should be required to be a language in dc:language
15:09:09 [nickstenn]
... Haven't had much of a chance to look at #345, also about processingLanguage
15:09:41 [azaroth]
ack ivan
15:09:59 [nickstenn]
... #341 also about processingLanguage for multilingual resources, and we've decided to postpone
15:12:40 [TimCole]
15:12:41 [ivan]
Issue #345 is an attempt from Richard to close an issue and discussion with gsergiu via an editorial change proposal
15:12:52 [azaroth]
15:12:57 [TimCole]
ack sees
15:13:38 [nickstenn]
azaroth: suggest we spend the time on the call discussing testing rather than going into the details on some of these I18N issues
15:13:44 [nickstenn]
... any problems?
15:13:48 [azaroth]
TOPIC: Testing
15:14:17 [nickstenn]
azaroth: Two parts of testing -- 1) model, and 2) protocol
15:14:38 [nickstenn]
... TimCole and ShaneM have been working on this. Update:
15:15:12 [nickstenn]
TimCole: regarding the creation of the underlying schemas for the tests -- we've captured everything 3.1, 3.2, and 3.3 excepting agents, and most of section 4
15:15:25 [nickstenn]
... these schemas are in the "definitions" folder
15:15:37 [nickstenn]
... and are referenced by the schemas we intend to use for assertions
15:16:05 [TimCole]
15:16:12 [nickstenn]
... I've just started working on the test scripts. Have been hashing out with ShaneM what those look like.
15:16:23 [nickstenn]
... This [^] is a test environment ShaneM has set up.
15:16:51 [nickstenn]
... You can use this with "Run tests under path" using the following path:
15:17:05 [ShaneM] should work too
15:17:23 [nickstenn]
... "/annotation-model"
15:18:04 [nickstenn]
... you can paste JSON[-LD] in and run through through the test suite
15:18:28 [nickstenn]
... that's all working.
15:19:13 [nickstenn]
... Having some small issues for SHOULD requirements, where we're not necessarily expecting you to pass the test. Currently if you *do* pass the requirement, the test fails.
15:19:37 [ShaneM]
try going to this URI now:
15:19:39 [ShaneM]
15:20:43 [azaroth]
Example annotations to play with:
15:21:16 [nickstenn]
... ShaneM: [walking us through how to use the test tool at the link above to test annotations]
15:23:03 [azaroth]
q+ to ask about display:none in HTML ?
15:23:09 [ShaneM]
Errors: data should have required property '@context'; expected true got false
15:23:15 [nickstenn]
ShaneM: Question: do we want to suppress the output above ^
15:23:16 [azaroth]
15:23:21 [azaroth]
q- sits
15:23:24 [azaroth]
ack azaroth
15:23:24 [Zakim]
azaroth, you wanted to ask about display:none in HTML ?
15:23:53 [nickstenn]
azaroth: Would it be possible to have the AJV stack trace be in a display:none; area with a button to reveal it or similar?
15:24:16 [nickstenn]
ShaneM: I don't think so. We just report data back to the test harness, which is responsible for the display.
15:25:05 [nickstenn]
azaroth: the "data should have required property @context" is particularly useful to understand what's going on
15:25:19 [nickstenn]
ShaneM: I'll leave it there, then.
15:26:13 [nickstenn]
ivan: currently can't rerun with updated JSON
15:26:19 [nickstenn]
ShaneM: I'll see if we can fix that
15:26:24 [TimCole]
15:26:29 [azaroth]
ack TimCole
15:26:34 [TimCole]
15:28:11 [nickstenn]
TimCole: if you put in a test annotation that *passes* the SHOULD requirements, the output is a little harder to interpret
15:28:18 [nickstenn]
... not sure quite how to improve the output in that case
15:30:00 [azaroth]
I used:
15:30:02 [nickstenn]
ivan: [proceeds to find a bug in one of the testing schemas while on the call]
15:32:20 [nickstenn]
TimCole: [paraphrasing heavily] currently for the SHOULD assertions, we expect non-conformance, which means that if it actually is conformant, the test fails
15:32:38 [azaroth]
Pass if it's not used, with a success message that it SHOULD be there?
15:32:47 [nickstenn]
ShaneM: no way of doing "warning" in the framework
15:33:09 [nickstenn]
... but we might be able to use "testType" to distinguish between MUST and SHOULD assertion types
15:33:09 [azaroth]
Result: Pass Message: WARNING: Format SHOULD be included for bodies, if known
15:34:22 [ShaneM]
use assertionType of must, may, or shold
15:34:25 [nickstenn]
TimCole: we have other scenarios where we say "SHOULD have 1, MAY have more than 1 X"
15:34:34 [ShaneM]
testType has to do with automation.
15:35:36 [nickstenn]
... there are other cases where you MUST NOT have more than 1
15:35:49 [nickstenn]
... so in these cases we can have multiple cases for the different cardinalities
15:36:03 [nickstenn]
s/multiple cases/multiple assertions/
15:36:16 [ShaneM]
15:36:30 [nickstenn]
ShaneM: If you look at the top of [^] you'll notice that the page fills in as the page loads.
15:37:18 [nickstenn]
... In the description at the top of the page, there's a list of things the test will check
15:37:35 [nickstenn]
... do we want to include SHOULD/MUST/MAY information in these descriptions?
15:37:44 [nickstenn]
[noises of general agreement]
15:38:31 [azaroth]
15:38:48 [azaroth]
q+ to ask about fails for example 41
15:39:23 [nickstenn]
TimCole: going to spend the next little while cleaning this up with a view to sharing it more widely
15:39:31 [nickstenn]
... how are the test outputs recorded?
15:39:54 [nickstenn]
ShaneM: anyone with an implementation can record their JSON output and add it to a git repository which contains all the results
15:40:08 [nickstenn]
s/JSON output/JSON test output/
15:41:08 [nickstenn]
TimCole: how do people want to break up the various test for bodies/targets/optional keys/etc.?
15:41:43 [nickstenn]
ShaneM: speaking as "not an implementer" -- the smallest number of manual tests that get us the information we need is probably a reasonable guideline
15:42:03 [nickstenn]
TimCole: probably a discussion for the mailing list
15:42:57 [azaroth]
15:43:09 [nickstenn]
azaroth: example 41 [^] is a completely contrived example at the end of the spec
15:43:23 [nickstenn]
... it seems unlikely (in the short term at least) that any client would generate such an annotation
15:43:30 [nickstenn]
... but perhaps not outside the realms of possibility
15:43:48 [nickstenn]
... putting in the 3.2 set of tests, it passes 8 but fails 5
15:44:15 [nickstenn]
... wondering what those are: problems with the test harness, the SHOULD problem, or problems with the data?
15:44:58 [nickstenn]
TimCole: would need to have a look, but it's probably the SHOULD issue with multiple formats
15:46:46 [azaroth]
15:46:47 [azaroth]
ack azaroth
15:46:47 [Zakim]
azaroth, you wanted to ask about fails for example 41
15:47:05 [nickstenn]
azaroth: we should probably spend some time talking about protocol testing
15:47:10 [azaroth]
TOPIC: Protocol Testing
15:48:06 [nickstenn]
bigbluehat: have mostly passed the work I've done onto ShaneM
15:48:35 [nickstenn]
ShaneM: the server tests bigbluehat are awesome, but let's talk about client tests for a second
15:48:51 [nickstenn]
... the server runs in the WPT environment
15:49:29 [azaroth]
( issue )
15:49:45 [nickstenn]
... the way annotations work is that an annotation collection lives at a IRI, and thus the server needs to serve at some named route within WPT
15:50:14 [bigbluehat]
PUT to overwrite
15:50:24 [nickstenn]
... but we need to work out how create/update/destroy operations work in the test server
15:51:00 [nickstenn]
... in particular because we don't want ephemeral data created on the test server
15:52:00 [nickstenn]
... so we're going to be arranging things such that data created by clients is destroyed as soon as it is read back from the server
15:52:33 [nickstenn]
ivan: how is that going to work if you want to prepare a bunch of data and then run a load of tests against that?
15:53:20 [nickstenn]
ShaneM: in those cases the client will access some static collection of annotations rather than data they created
15:53:34 [nickstenn]
15:53:47 [azaroth]
ack nickstenn
15:53:52 [azaroth]
scribenick: azaroth
15:54:18 [azaroth]
nickstenn: Question about how this is going to work -- the protocol spec doesn't say what the server is supposed to do with the data that you give it, even reasonable things
15:54:35 [azaroth]
... for example, in a distributed annotation system, you POST to create it, but you might not be able to read it back again straight away
15:54:50 [azaroth]
... want us to be careful that we're not testing that you can read something straight away
15:55:02 [azaroth]
bigbluehat: The protocol says that it comes back with the full representation
15:55:08 [azaroth]
ivan: Comes back with an id
15:55:49 [azaroth]
nickstenn: That's a different point though. Could return it straight away, but the server doesn't necessarily have state beyond that
15:56:04 [azaroth]
... intuitively reasonable assumptions are fine, but that's not in the spec
15:56:29 [azaroth]
... need to be careful to not write tests on our understanding of the spec, but what the spec actually says
15:56:43 [azaroth]
bigbluehat: e.g. there's no guarantee that you'll be able to get the annotation back after you create it
15:57:02 [azaroth]
ShaneM: Wondering if the server tests you wrote rely on creating and then immediately retrieving it
15:57:28 [azaroth]
bigbluehat: would need to go through the tests to see if it requests the same ones later, I don't think so
15:57:39 [azaroth]
... it puts stuff in but I don't think it checks again later
15:57:50 [azaroth]
ShaneM: If that's the case, then we're good
15:58:09 [bigbluehat]
this is the thing ShaneM's been mentioning btw:
15:58:13 [azaroth]
scribenick: nickstenn
15:58:33 [nickstenn]
ShaneM: in our last 2m, let's just agree on what we think is going to happen over the next week
15:58:56 [nickstenn]
... I have a couple of next actions from this conversation and will get on those straight away
15:59:06 [nickstenn]
... I have another to ensure people know how to run tests and upload results
15:59:15 [nickstenn]
... also working on getting the protocol stuff implemented
15:59:32 [nickstenn]
TimCole: going to fill in a few more schemas to cover sections 3 and 4
15:59:41 [ivan]
16:00:03 [azaroth]
ack ivan
16:00:36 [nickstenn]
TimCole: if people could help make some invalid annotations to help test the failure cases, that would be helpful
16:00:44 [nickstenn]
azaroth: I can do some of that
16:00:59 [nickstenn]
ivan: When do you folks think we can begin to pester implementers to provide reports?
16:01:33 [nickstenn]
TimCole: I think we need to do a reality check next Friday before we start inviting implementers -- maybe the week after that?
16:02:19 [azaroth]
TOPIC: Adjourn
16:02:38 [ivan]
rrsagent, draft minutes
16:02:38 [RRSAgent]
I have made the request to generate ivan
16:03:03 [ivan]
trackbot, end telcon
16:03:03 [trackbot]
Zakim, list attendees
16:03:03 [Zakim]
As of this point the attendees have been Rob_Sanderson, Tim_Cole, Nick_Stenning, Jacob_Jett, Ivan, Dan_Whaley, Ben_De_Meester, Benjamin_Young
16:03:11 [trackbot]
RRSAgent, please draft minutes
16:03:11 [RRSAgent]
I have made the request to generate trackbot
16:03:12 [trackbot]
RRSAgent, bye
16:03:12 [RRSAgent]
I see no action items