See also: IRC log
azaroth: also talk (quickly) about CFC
... if no large outstanding issues, let's issue a CFC within one week
... any announcements?
timcole: meeting next week?
<TimCole> I will be present
<dwhly> I will be
I can be present
<takeshi> I will be present
TimCole: is there any specific topic to work on?
azaroth: testing would be the big one
TimCole: I'm happy to host a meeting about that next week
... how about I'll do an e-mail about that call?
azaroth: perfec
... t
<azaroth> PROPOSED RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/03/11-annotation-minutes.html
RESOLUTION: Minutes of the previous call are approved: https://www.w3.org/2016/03/11-annotation-minutes.html
azaroth: we would like to have a CFC to publish the docs
... especially vocab needs more text, and diagrams to be done
... but to get feedback about the technicalities, more visibility would
be good
... we can do a one-week CFC via email
... any concerns about that?
... any issues that need to be addressed before? except for ivan's
comments
timcole: we don't have a vocab doc on W3C yet, I think
ivan: that was an index mistake, I changed that
<ivan> http://w3c.github.io/web-annotation/model/wd2/
<ivan> http://w3c.github.io/web-annotation/vocab/wd/
<ivan> http://w3c.github.io/web-annotation/protocol/wd/
ivan: (these are the urls used for the CFC)
... these dates are close to this date
... these three documents would become the next versions of the /TR WD's
timcole: draft should be updated quickly
ivan: it's very timely that we have these things published
azaroth: we don't have a shortname for the vocab yet
ivan: we need to have that
... in the CFC, best, we propose the shortname as well
... the final resolution e-mail should also have that shortname
azaroth: is annotation-vocab ok?
<TimCole> +1
<nickstenn__> +1
azaroth: seems consistent with the other shortnames
<ivan> +1
+1
ivan: on timing: the restriction I have, is that I am
around the week of the 28th of March, the week after that, I am away for
two consecutive weeks
... I propose we try to get this published on Thursday the 31st
<azaroth> +1 to 2016-03-31
ivan: the editor should prepare by going through all
checkers (i.e., link checkers)
... so there won't be any last-minute editorial problems
azaroth: I did it before and I'll do it again
... the model and protocol doc point to the vocab spec, but there is not
publication yet
ivan: you can have a local bibliography in respec
... the local bibliography should include all three documents
... I'll send you an example
azaroth: by the time we get through CR, we don't need a local bib?
ivan: I think these version should always be dated URIs, I
think..
... it always puts the date of the publication there, but that would be
wrong..
... we have to update that until the REC version
azaroth: ok
... any other thoughts?
azaroth: last week, we talked about what we could ask the
clients and servers to conform to
... i.e., core model vs support support for different types of selectors
shepazu: so the profiles-thing?
azaroth: yes, also syntactic vs semantic vs UI testing
... also about the tools to test with
... and about the W3C rules about what needs to be tested
shepazu: usually testing depends on conformance
ivan: question is: for the three things (vocab, model,
protocol): what do we want to test?
... profiles etc. is separate
... how was the LDP protocol tested?
azaroth: LDP had a test-kit that you could download and
test against your implementation
... it would do the various methods
... and generate a report
ivan: so a separate program (client-side) that tested the server?
azaroth: yes, rather than a module incorporated into the
server
... I can ask our developer that used it about his experience
ivan: for the protocol, that seems reasonable to me
shepazu: Chris and I talked about testing the model
... a validator would not be sufficient
... simply saying 'this incarnation validates to the model' is not
enough, that's testing some content
... we talked about creating a test page, i.e., the target of the
annotation
... the annotation client does the steps to create an annotation (manual
or automatic)
... that annotation is sent to a toy annotation server
... downloaded again into the client, and validated there, to check
whether the structure fits with the model
... that would be 1) reusable, and 2) actually test the client, not only
validate
timcole: that makes a lot sense
... but it conflates the testing of the protocol and the model
shepazu: it doesn't matter how the annotation is published
to the website, it does matter that it is published
... e.g., in CSS, they make a baseline
... if the client could simply give the annotation to the other
client-side code, that would also work, but would violate the principle
that the client would be inaccessible to other clients
<azaroth> (LDP Implementation Report: https://dvcs.w3.org/hg/ldpwg/raw-file/default/tests/reports/ldp.html )
timcole: do we need something like: how does a server react
to the protocol: a report
... for the client: can it create and take correct annotation and use
them
... so, can the client also recognize incorrect annotations?
<azaroth> And the test suite: http://w3c.github.io/ldp-testsuite/
timcole: I'm worried not having a method specified for
checking the generated annotations
... the process of sending it somewhere could miss errors
ivan: two things:
... first: not all implementations that we may want to test are such
that you can set up a web page that easily
... if the goal is to annotate data on the web, then I don't see what
the web page is
... I wouldn't restrict to only web page based implementations
... second: let's say we have that, the annotation system produces the
relevant structure, and we have to test whether the structure is right,
or makes mistakes
... what we did for RDFa, is that something happens, and produce a clear
structure
... for each of the tasks, we have the pattern that must be generated
... and an automatic procedure that could compare results
... that requires that the annotation system can dump the structure into
an external file
... if we can't do automatic testing, we give the annotation system some
tasks, and the outputted structure we compare those with what we expect
that should happen
... I don't know whether we can do that automatically
shepazu: manual testing is more time-consuming, and doesn't
work well within our testing framework, but it might be inevitable
... i.e., the W3C testing framework
<Zakim> azaroth, you wanted to ask Dan about AAK as potential audience
ivan: that's not for all types of working groups
azaroth: Dan: about the AAK as potential audience: what would be valuable, as testing implementations?
dwhly: it's a little premature for now
... coalition is currently group of publishers, not software writers
... I think that's most useful for the upcoming F2F: what are the use
cases for annotation, and how do those use cases articulate in an
interoperable annotation layer?
... the technical people can triage the use cases to see what works with
waht the W3C is doing, and what not
shepazu: in similar situations, W3C sees that validators
are useful if other people want to use W3C annotation stuff
... seeing that they are doing the output correctly
... a validator would be a necessary component
... if other people want to use the Web Annotation model
timcole: So, single page won't be enough: shouldn't we
identify a set of annotation tests
... and seeing whether a client can generate a file which can be
validated and checked for errors
... I would like to see whether we can identify all test cases that are
in the model
shepazu: each test would have his own page, and that test would also contain the passing criteria
timcole: is it feasible, given that we have different kinds of objects and different kinds of implements, and some implementations would only pass a part of the annotation tests
shepazu: that's about different conformance classes
... if your annotation client would only work with, e.g., image objects,
we test that client only for the relevant test cases
... W3C doesn't actually test implementations, it tests the
implementability of the specifications
... i.e., this feature of this spec was implemented interoperably by two
or more user agents
... if that feature does not have two passing implementations, it is
marked at risk and possibly removed
... until we have two passing implementations, or we move the spec
forward without that feature
timcole: I expect not to find two implementations that implement all features
shepazu: you don't need that, it could be some kind of combination of different clients
timcole: good, my first question was: can we find all test cases
ivan: [about CSV working group]: each test had a scenario,
data file
... implementation had to produce something (JSON, metadata, etc.)
... each of these tests were run separately
... each implementation had to validate itself and return the results in
some accepted format
... about 350 different use cases
... to cover the various features
... if we have a structure like that, we need a certain number of
scenarios
<Zakim> azaroth, you wanted to +1 tasks on sample resources
azaroth: I also like the idea of setting up a series of
defined tasks
... and possibly downloaded, tested, and uploaded again, or done online
if possible
... question is that we would a group of people to implement the testing
framework
nickstenn: what would the tests for the model look like?
... actually, this means processing annotation data like clients would
do it in the real world?
... but the model is about the semantics of the model, not about
implementations
<azaroth> +1 to not testing UA behavior on consumption of an annotation
nickstenn: how could we possibly create a testing framework
that would work for all user agents?
... I think about giving a set of annotations, and asking to
implementers: can you client interpret it correctly?
timcole: the thing is that someone somewhere could
implement a tool that could generate, e.g., a selector
... secondly, could our testing framework distinguish between correct
and incorrect usage of a feature
... can a tool recognize the difference between a correct and an
incorrect annotation?
ivan: let's say we set up an HTML page
... and describe in human terms: this is the annotation the user is
supposed to do: select and comment
... the implementer would have to perform this task, and internally, you
would have to build up the annotation via the annotation model
... and the implementation needs to dump the model in, e.g., a JSON file
... then, the implementation shows that the model can describe that
action
... the other direction is that we provide annotation structures, and
see whether these annotation structures can be understood by
implementations
... it would be interesting to understand how current annotation clients
are tested
nickstenn: hypothesis tests on a granular level
ivan: this is a question for other implementers as well
... we need a feeling of what is realistic
azaroth: we only test syntax, not semantics
... we don't test the interaction
... if there are any comments about testing: put it on the mailing list
... and continue next week
<nickstenn__> sounds good to me
azaroth: adjourn
<azaroth> Thanks to bjdmeest for scribing! :)
<ivan> trackbot, end telcon