Research and Development Working Group Teleconference

29 Sep 2011

See also: IRC log


Christos, Shadi, Simon, Markel, Klaus, Vivenne, Giorgio
Mario, Joshue, Yeliz, Peter


Finalise Paper Selection Criteria - http://www.w3.org/WAI/RD/wiki/Paper_Selection_Criteria

<markel> I didn't give any input

suggest we look at giorgio's comments (inline)

<vivienne> markel will be in sunny Australia next time

SH: can move this agenda item to next time when Giorgio and Yeliz are here

<vivienne> I'm happy to wait

SH: want to move to next week or try today?

MV: think it is excellent, just have a minor comment
... maybe better to wait for Giorgio

[Giorgio joins]

MV: excellent job, Giorgio
... Yeliz and Shadi raised the issue of impact
... talked about what the research community in the past
... may not be considered as "novel"
... seems contradictory
... want new stuff but also about past work

GB: up to us how to define "novelty"
... but also happy to drop it

MV: novel paper also welcome
... might need to prioritize
... and balance between types of papers
... as long as they are sound and valid

GB: agree with you but uneasy to write in vey specific terms what the criteria are and which are more important
... that said, novelty may be less important than soundness
... happy to decide that now if wanted

<markel> thanks Giorgio

<markel> +1 shadi

SAZ: agree with Giorgio
... for me relevance to the symposium topic would be most useful

SH: impact may be independent of novelty
... need impact but not necessarily novelty
... can have something very sound and very correct but won't have very much impact


<markel> +1

<giorgio> agree with simon: impact more important than novelty

<markel> yes

<giorgio> perfectly good.

<christos> ok

SH: can take a pass if that's ok?


<vivienne> +1

SH: may need to stay flexible


[SH reads comment #1 from email]

[[If the paper is mainly theoretical, it should be judged on the insightfulness of its ideas and the rigor of its arguments. If it is mainly empirical, it should be judged on the validity of its results and the appropriateness of its techniques. If it is mainly a methodology paper, it should be judged on the usefulness of its proposed methodology. If it is a design paper, it should be judged on the quality, novelty, and appropriateness of its solution to a proble

m; the rationale for the design decisions; the implementation; and the assessment of its success and weaknesses.]]

<markel> i'm reading

<giorgio> I'm trying to find shadi's comments

<giorgio> ok. got it now

<sharper> # section "Generic instructions for reviewers"

<sharper> - the first bullet uses a mix of terms defined as criteria and other

<sharper> that are not defined -- we need to review how we are using the terms and

<sharper> what ramifications this may have (for instance it may indicate

<sharper> prioritizing certain criteria for certain types of papers)

GB: don't you think that trying to achieve a high-level description is useful?

<markel> it seems that Shadi's comments go in line with the previous issue about prioritazing

GB: maybe not necessary for these short contributions

SAZ: think overall guidance is important
... just editorially may be confusing to use some terms that are and are not defined later on
... need to be consistent

GB: happy for anybody to adjust the wording if editorial
... want to avoid spending too much time in advance

SAZ: need to know how the approach was intended to be, to clean-up editorially
... but also need to balance between quick&dirty and spending too much time on theory

<vivienne> will the reviewers have a form based on this criteria to make it easier for them?

SAZ: this page sets the fundaments for the review process

GB: agree but don't think we should over-shoot
... should use reviews from the process to improve it

<giorgio> ok

SAZ: do not disagree that need to avoid over-shooting
... but need to be responsible
... we also picked a very sensitive and delicate topic

MV: don't know a review process that is much more regorous
... need to be careful not to spend too much time

SH: agree with all
... happy for SAZ to improve the section

<giorgio> yes

<vivienne> okay

SH: will also take a pass

<christos> yes

SH: sound ok?

<markel> i'm fine with that

[SH reads comment #2 from SAZ email]

SH: think we mean editorially

<giorgio> fine

SH: ok for SAZ change this or do we need discussion?

<vivienne> fine with Shadi changing

<sharper> +1

<markel> ok, shadi changing

SH: want to be open and flexible
... can make choices on the fly as needed

Initiate Webinar Logistics Discussion

SH: initiate discussion
... are we going to allow more discussion
... how was that done in the past?

SAZ: can do it differently for each topic, as needed

<markel> no more than 2 hours I would say

SAZ: talked about having 5-10 minutes for each presenter
... then a panel session chaired by the symposium chairs
... then an open Q&A

<vivienne> Does it work like a conference call?

SH: how much time do we spend?

<giorgio> we could set up a wiki page outlining our decisions on logistics

SAZ: typically 2-3 hours with break but will depend on what comes in

Any Other Business

<markel> yes Giorgio +1. We can collect all the ideas about the weminar an put in a wiki page

SH: we are in kind of a looping mode right now
... what do want to do?

<markel> we've talked about this over the past meetings

SH: in the next meetings

GB: can start thinking of some of the topics
... work out some ideas

<sharper> GB: Thinking and formulating some of the questions and topics the chairs could raise on the panel

SAZ: also the wiki needs lots of love
... needs to think of new topics and also populate them
... even if outside our comfort zone and direct field of work

SH: will populate the wiki with topics on touch interfaces
... should we also start initiating the next topic?

<vivienne> yes, it would be good to start looking at the next one

SH: stack them

<Zakim> shadi, you wanted to talk about Eval TF


SAZ: Eval TF developing methodology for evaluating conformance of websites to WCAG2

SAZ: very related to this first topic
... they are developing requirements
... also looking at existing resources
... need exchange between both groups

MV: agree with GB to prepare generic questions for the panel


<giorgio> me too

MV: agree with SH not to get locked down into evaluation too much
... and look at other topics too


SH: can see that we may get locked down
... many interesting topics
... such as digital TV and more

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.136 (CVS log)
$Date: 2011/09/30 08:11:24 $