Web Annotation Working Group Teleconference

1. Scribe selection, Agenda review, Announcements?

azaroth: also talk (quickly) about CFC
... if no large outstanding issues, let's issue a CFC within one week
... any announcements?

timcole: meeting next week?

<TimCole> I will be present

<dwhly> I will be

I can be present

<takeshi> I will be present

TimCole: is there any specific topic to work on?

azaroth: testing would be the big one

TimCole: I'm happy to host a meeting about that next week
... how about I'll do an e-mail about that call?

azaroth: perfec
... t

Minutes approval

<azaroth> PROPOSED RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/03/11-annotation-minutes.html

RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/03/11-annotation-minutes.html

Issue CFC for publication of Model, Vocab and Protocol

azaroth: we would like to have a CFC to publish the docs
... especially vocab needs more text, and diagrams to be done
... but to get feedback about the technicalities, more visibility would be good
... we can do a one-week CFC via email
... any concerns about that?
... any issues that need to be addressed before? except for ivan's comments

timcole: we don't have a vocab doc on W3C yet, I think

ivan: that was an index mistake, I changed that

<ivan> http://w3c.github.io/web-annotation/model/wd2/

<ivan> http://w3c.github.io/web-annotation/vocab/wd/

<ivan> http://w3c.github.io/web-annotation/protocol/wd/

ivan: (these are the urls used for the CFC)
... these dates are close to this date
... these three documents would become the next versions of the /TR WD's

timcole: draft should be updated quickly

ivan: it's very timely that we have these things published

azaroth: we don't have a shortname for the vocab yet

ivan: we need to have that
... in the CFC, best, we propose the shortname as well
... the final resolution e-mail should also have that shortname

azaroth: is annotation-vocab ok?

<TimCole> +1

<nickstenn__> +1

azaroth: seems consistent with the other shortnames

<ivan> +1

+1

ivan: on timing: the restriction I have, is that I am around the week of the 28th of March, the week after that, I am away for two consecutive weeks
... I propose we try to get this published on Thursday the 31st

<azaroth> +1 to 2016-03-31

ivan: the editor should prepare by going through all checkers (i.e., link checkers)
... so there won't be any last-minute editorial problems

azaroth: I did it before and I'll do it again
... the model and protocol doc point to the vocab spec, but there is not publication yet

ivan: you can have a local bibliography in respec
... the local bibliography should include all three documents
... I'll send you an example

azaroth: by the time we get through CR, we don't need a local bib?

ivan: I think these version should always be dated URIs, I think..
... it always puts the date of the publication there, but that would be wrong..
... we have to update that until the REC version

azaroth: ok
... any other thoughts?

Testing

azaroth: last week, we talked about what we could ask the clients and servers to conform to
... i.e., core model vs support support for different types of selectors

shepazu: so the profiles-thing?

azaroth: yes, also syntactic vs semantic vs UI testing
... also about the tools to test with
... and about the W3C rules about what needs to be tested

shepazu: usually testing depends on conformance

ivan: question is: for the three things (vocab, model, protocol): what do we want to test?
... profiles etc. is separate
... how was the LDP protocol tested?

azaroth: LDP had a test-kit that you could download and test against your implementation
... it would do the various methods
... and generate a report

ivan: so a separate program (client-side) that tested the server?

azaroth: yes, rather than a module incorporated into the server
... I can ask our developer that used it about his experience

ivan: for the protocol, that seems reasonable to me

shepazu: Chris and I talked about testing the model
... a validator would not be sufficient
... simply saying 'this incarnation validates to the model' is not enough, that's testing some content
... we talked about creating a test page, i.e., the target of the annotation
... the annotation client does the steps to create an annotation (manual or automatic)
... that annotation is sent to a toy annotation server
... downloaded again into the client, and validated there, to check whether the structure fits with the model
... that would be 1) reusable, and 2) actually test the client, not only validate

timcole: that makes a lot sense
... but it conflates the testing of the protocol and the model

shepazu: it doesn't matter how the annotation is published to the website, it does matter that it is published
... e.g., in CSS, they make a baseline
... if the client could simply give the annotation to the other client-side code, that would also work, but would violate the principle that the client would be inaccessible to other clients

<azaroth> (LDP Implementation Report: https://dvcs.w3.org/hg/ldpwg/raw-file/default/tests/reports/ldp.html )

timcole: do we need something like: how does a server react to the protocol: a report
... for the client: can it create and take correct annotation and use them
... so, can the client also recognize incorrect annotations?

<azaroth> And the test suite: http://w3c.github.io/ldp-testsuite/

timcole: I'm worried not having a method specified for checking the generated annotations
... the process of sending it somewhere could miss errors

ivan: two things:
... first: not all implementations that we may want to test are such that you can set up a web page that easily
... if the goal is to annotate data on the web, then I don't see what the web page is
... I wouldn't restrict to only web page based implementations
... second: let's say we have that, the annotation system produces the relevant structure, and we have to test whether the structure is right, or makes mistakes
... what we did for RDFa, is that something happens, and produce a clear structure
... for each of the tasks, we have the pattern that must be generated
... and an automatic procedure that could compare results
... that requires that the annotation system can dump the structure into an external file
... if we can't do automatic testing, we give the annotation system some tasks, and the outputted structure we compare those with what we expect that should happen
... I don't know whether we can do that automatically

shepazu: manual testing is more time-consuming, and doesn't work well within our testing framework, but it might be inevitable
... i.e., the W3C testing framework

<Zakim> azaroth, you wanted to ask Dan about AAK as potential audience

ivan: that's not for all types of working groups

azaroth: Dan: about the AAK as potential audience: what would be valuable, as testing implementations?

dwhly: it's a little premature for now
... coalition is currently group of publishers, not software writers
... I think that's most useful for the upcoming F2F: what are the use cases for annotation, and how do those use cases articulate in an interoperable annotation layer?
... the technical people can triage the use cases to see what works with waht the W3C is doing, and what not

shepazu: in similar situations, W3C sees that validators are useful if other people want to use W3C annotation stuff
... seeing that they are doing the output correctly
... a validator would be a necessary component
... if other people want to use the Web Annotation model

timcole: So, single page won't be enough: shouldn't we identify a set of annotation tests
... and seeing whether a client can generate a file which can be validated and checked for errors
... I would like to see whether we can identify all test cases that are in the model

shepazu: each test would have his own page, and that test would also contain the passing criteria

timcole: is it feasible, given that we have different kinds of objects and different kinds of implements, and some implementations would only pass a part of the annotation tests

shepazu: that's about different conformance classes
... if your annotation client would only work with, e.g., image objects, we test that client only for the relevant test cases
... W3C doesn't actually test implementations, it tests the implementability of the specifications
... i.e., this feature of this spec was implemented interoperably by two or more user agents
... if that feature does not have two passing implementations, it is marked at risk and possibly removed
... until we have two passing implementations, or we move the spec forward without that feature

timcole: I expect not to find two implementations that implement all features

shepazu: you don't need that, it could be some kind of combination of different clients

timcole: good, my first question was: can we find all test cases

ivan: [about CSV working group]: each test had a scenario, data file
... implementation had to produce something (JSON, metadata, etc.)
... each of these tests were run separately
... each implementation had to validate itself and return the results in some accepted format
... about 350 different use cases
... to cover the various features
... if we have a structure like that, we need a certain number of scenarios

<Zakim> azaroth, you wanted to +1 tasks on sample resources

azaroth: I also like the idea of setting up a series of defined tasks
... and possibly downloaded, tested, and uploaded again, or done online if possible
... question is that we would a group of people to implement the testing framework

nickstenn: what would the tests for the model look like?
... actually, this means processing annotation data like clients would do it in the real world?
... but the model is about the semantics of the model, not about implementations

<azaroth> +1 to not testing UA behavior on consumption of an annotation

nickstenn: how could we possibly create a testing framework that would work for all user agents?
... I think about giving a set of annotations, and asking to implementers: can you client interpret it correctly?

timcole: the thing is that someone somewhere could implement a tool that could generate, e.g., a selector
... secondly, could our testing framework distinguish between correct and incorrect usage of a feature
... can a tool recognize the difference between a correct and an incorrect annotation?

ivan: let's say we set up an HTML page
... and describe in human terms: this is the annotation the user is supposed to do: select and comment
... the implementer would have to perform this task, and internally, you would have to build up the annotation via the annotation model
... and the implementation needs to dump the model in, e.g., a JSON file
... then, the implementation shows that the model can describe that action
... the other direction is that we provide annotation structures, and see whether these annotation structures can be understood by implementations
... it would be interesting to understand how current annotation clients are tested

nickstenn: hypothesis tests on a granular level

ivan: this is a question for other implementers as well
... we need a feeling of what is realistic

azaroth: we only test syntax, not semantics
... we don't test the interaction
... if there are any comments about testing: put it on the mailing list
... and continue next week

<nickstenn__> sounds good to me

azaroth: adjourn

<azaroth> Thanks to bjdmeest for scribing! :)

<ivan> trackbot, end telcon