See also: IRC log
<markel> I didn't give any input
suggest we look at giorgio's comments (inline)
<vivienne> markel will be in sunny Australia next time
SH: can move this agenda item to next time when Giorgio and Yeliz are here
<vivienne> I'm happy to wait
SH: want to move to next week or try today?
MV: think it is excellent, just have a minor
comment
... maybe better to wait for Giorgio
[Giorgio joins]
MV: excellent job, Giorgio
... Yeliz and Shadi raised the issue of impact
... talked about what the research community in the past
... may not be considered as "novel"
... seems contradictory
... want new stuff but also about past work
GB: up to us how to define "novelty"
... but also happy to drop it
MV: novel paper also welcome
... might need to prioritize
... and balance between types of papers
... as long as they are sound and valid
GB: agree with you but uneasy to write in vey
specific terms what the criteria are and which are more important
... that said, novelty may be less important than soundness
... happy to decide that now if wanted
<markel> thanks Giorgio
<markel> +1 shadi
SAZ: agree with Giorgio
... for me relevance to the symposium topic would be most useful
SH: impact may be independent of novelty
... need impact but not necessarily novelty
... can have something very sound and very correct but won't have very much
impact
+1
<markel> +1
<giorgio> agree with simon: impact more important than novelty
<markel> yes
<giorgio> perfectly good.
<christos> ok
SH: can take a pass if that's ok?
[agreement]
<vivienne> +1
SH: may need to stay flexible
http://lists.w3.org/Archives/Public/public-wai-rd/2011Sep/0043.html
[SH reads comment #1 from email]
[[If the paper is mainly theoretical, it should be judged on the insightfulness of its ideas and the rigor of its arguments. If it is mainly empirical, it should be judged on the validity of its results and the appropriateness of its techniques. If it is mainly a methodology paper, it should be judged on the usefulness of its proposed methodology. If it is a design paper, it should be judged on the quality, novelty, and appropriateness of its solution to a proble
m; the rationale for the design decisions; the implementation; and the assessment of its success and weaknesses.]]
<markel> i'm reading
<giorgio> I'm trying to find shadi's comments
<giorgio> ok. got it now
<sharper> # section "Generic instructions for reviewers"
<sharper> - the first bullet uses a mix of terms defined as criteria and other
<sharper> that are not defined -- we need to review how we are using the terms and
<sharper> what ramifications this may have (for instance it may indicate
<sharper> prioritizing certain criteria for certain types of papers)
GB: don't you think that trying to achieve a high-level description is useful?
<markel> it seems that Shadi's comments go in line with the previous issue about prioritazing
GB: maybe not necessary for these short contributions
SAZ: think overall guidance is important
... just editorially may be confusing to use some terms that are and are not
defined later on
... need to be consistent
GB: happy for anybody to adjust the wording if
editorial
... want to avoid spending too much time in advance
SAZ: need to know how the approach was intended
to be, to clean-up editorially
... but also need to balance between quick&dirty and spending too much
time on theory
<vivienne> will the reviewers have a form based on this criteria to make it easier for them?
SAZ: this page sets the fundaments for the review process
GB: agree but don't think we should over-shoot
... should use reviews from the process to improve it
<giorgio> ok
SAZ: do not disagree that need to avoid
over-shooting
... but need to be responsible
... we also picked a very sensitive and delicate topic
MV: don't know a review process that is much more
regorous
... need to be careful not to spend too much time
SH: agree with all
... happy for SAZ to improve the section
<giorgio> yes
<vivienne> okay
SH: will also take a pass
<christos> yes
SH: sound ok?
<markel> i'm fine with that
[SH reads comment #2 from SAZ email]
SH: think we mean editorially
<giorgio> fine
SH: ok for SAZ change this or do we need discussion?
<vivienne> fine with Shadi changing
<sharper> +1
<markel> ok, shadi changing
SH: want to be open and flexible
... can make choices on the fly as needed
SH: initiate discussion
... are we going to allow more discussion
... how was that done in the past?
SAZ: can do it differently for each topic, as needed
<markel> no more than 2 hours I would say
SAZ: talked about having 5-10 minutes for each
presenter
... then a panel session chaired by the symposium chairs
... then an open Q&A
<vivienne> Does it work like a conference call?
SH: how much time do we spend?
<giorgio> we could set up a wiki page outlining our decisions on logistics
SAZ: typically 2-3 hours with break but will depend on what comes in
<markel> yes Giorgio +1. We can collect all the ideas about the weminar an put in a wiki page
SH: we are in kind of a looping mode right now
... what do want to do?
<markel> we've talked about this over the past meetings
SH: in the next meetings
GB: can start thinking of some of the topics
... work out some ideas
<sharper> GB: Thinking and formulating some of the questions and topics the chairs could raise on the panel
SAZ: also the wiki needs lots of love
... needs to think of new topics and also populate them
... even if outside our comfort zone and direct field of work
SH: will populate the wiki with topics on touch
interfaces
... should we also start initiating the next topic?
<vivienne> yes, it would be good to start looking at the next one
SH: stack them
<Zakim> shadi, you wanted to talk about Eval TF
http://www.w3.org/WAI/ER/2011/eval/eval-tf
SAZ: Eval TF developing methodology for evaluating conformance of websites to WCAG2
SAZ: very related to this first topic
... they are developing requirements
... also looking at existing resources
... need exchange between both groups
MV: agree with GB to prepare generic questions for the panel
+1
<giorgio> me too
MV: agree with SH not to get locked down into
evaluation too much
... and look at other topics too
+1
SH: can see that we may get locked down
... many interesting topics
... such as digital TV and more