Research and Development Working Group Teleconference

22 Sep 2011

See also: IRC log


Mario, Joshue, Shadi, Vivienne, Peter, Mate, Markel, Giorgio, Yeliz
Joshue, Markel


Welcome & Logistics (Regrets, Agenda Requests, Comments)

SAZ: Are there any agenda requests?

CFP Distribution (update and collation of lists already sent to and other perspective venues)

SAZ: The CFP was out, it has been sent to several groups and others have forwarded - which is great.
... What more can we do to promote it? I have gotten requests for clarification
... What promotion has been going on?
... Has anyone been circulating it?

<vivienne> I sent it through our university

Markel: I did send it to people I have been following who are involved in a11y metrics.
... I gave them the link, sent to 8 different research groups.

SAZ: Great

Giorgio: I sent it to three or four other people.

<giorgio> one list that comes to my mind i webaim

JOC: I also send it to about 6 academic colleagues.

SAZ: Are there lists etc that we could send it to?

<yeliz> I wonder if it could also be published in the newsletters?

<yeliz> for example, SIGACCESS or SIGWEB newsletters?

<yeliz> or usability newsletters

<Peter> I was planning to forward the CFP to OS mailing groups like mozilla accessibility, screen reader groups etc.

<yeliz> or may be we don't have enough time?

<yeliz> yes, some of them

SAZ: +1 to usablity newsletter

<markel> Did we get anything at the WAI-IG list?

SAZ: Do they have CFPs?

<vivienne> what about groups in LinkedIn?

SAZ: We should avoid double contacting.

Yeliz: I get regular newsletter from usability news .com
... The editor can be contacted about the call?

SAZ: Great

Yeliz: Or HCI Int

<yeliz> http://lists.ics.forth.gr/mailman/listinfo/hciinternational

Yeliz: The editor can be contacted about the call also and they upcoming events, announcements etc.

<yeliz> http://usabilitynews.bcs.org/

SAZ: For everyone in the group, if you do have a mailing list etc that you think interested then please check with me first.

<yeliz> sigaccess - http://www.sigaccess.org/community/newsletter/

<giorgio> I would suggst the mailing list of webaim.org

<Peter> Would it be easier to put the groups on a wiki? (avoid extra work for Shadi)

SAZ: I just want to check we don't crosspost.

<Peter> +1 webaim

<markel> +1 Peter

<yeliz> sigweb - http://www.sigweb.org/community/newsletters

SAZ: I was thinking about that Peter, yes. Why not?
... So do insert list into wiki, so we don't crosspost etc

<markel> DBWorld - http://www.cs.wisc.edu/dbworld/

Yeliz: If you are ok I can send a mail to a couple of groups SIGWEB, HCI Int etc

SAZ: Great

<markel> Interaction design - http://www.interaction-design.org/

SAZ: I'll put up that wiki page, so Yeliz please add to wiki.

<markel> yes, I'll do that

<Mario> I have to check with Klaus to send it also to ICCHP authors.

SAZ: So please check wiki if you are going to send.

<yeliz> I can also post it on the W4A Linkedin group and also Facebook group?

SAZ: Yes, this message went to the WAI-IG list
... Thats the announcement you should use.
... Try to avoid writing your own.

<Peter> (mozilla.dev.accessibility@googlegroups.com)

Yeliz: I can also post to W4A, on their FB page etc

<markel> I think SIGCHI does also have a list for the CFP

SAZ: I don't know if relevant but happy to

Yeliz: WebAIM newsletter?

<Peter> Hmm perhaps your right Shadi - will focus more on research groups.

SAZ: Ok, I'll check that myself.

<yeliz> http://ec.europa.eu/information_society/activities/einclusion/newsletter/index_en.htm

SAZ: I have sent it to the Commission.

Yeliz: It would be good to inform the eInclusion.

SAZ: I'll do that.

<Peter> (Jutta at http://idrc.ocad.ca/ Inclusive Design Research Centre)

<Peter> (oh ok :)

SAZ: I have sent it there also.
... Lets move on.

Fixing the Judging and Selection Criteria - http://lists.w3.org/Archives/Public/public-wai-rd/2011Sep/0030.html

<shadi> http://lists.w3.org/Archives/Public/public-wai-rd/2011Sep/0030.html

SAZ: regarding Yeliz's email on the criteria...we don't expect many contributions..we need high quality submissions and need to justify rejections...some of you are knowledgeable in organising conferences

YY: the CFP says that each submission will be reviewed by three reviewers
... it is necessary to establish the criteria even if we don't expect many submission

<Joshue> +1 to Yeliz. I liked your ideas that you outlined in your mail. Feedback is important either way. Whether successful or not.

YY: the SC should have the same criteria
... novelty is difficult one because we said that already submited papers are welcomed

SAZ: any thoughts?

<shadi> [[criteria including relevance, clarity, soundness and power of the arguments, generality of results/claims, novelty]]

YY: what would be the impact? soundness is an important one but we don't one to scare people

GB: the criteria in the CFP are general enough to judge the papers-

impact could also be included

<Peter> +impact perhaps easiest to measure

GB: it is difficult to measure clarity or novelty without seen the submissions...maybe these decisions have to be taken after reviews

<Joshue> Yes, totally agree with Giorgio.

GB: let's discuss about forms and tools for reviewers

<Joshue> Have we a scoring mechanism/

GB: we need to say which criteria are important and what we mean by that criteria
... I was pointing to the latter aspect

SAZ: both are necessary

<Joshue> One is wieghting and one is the core value of the paper.

GB: I agree but the second one is more difficult

<yeliz> appropriatness?

<yeliz> Appropriateness

<yeliz> ?

<yeliz> new one?

SAZ: impact, significance, relevance will get more clear once we have the definitions

<giorgio> why don't we try to formulate 1-sentence descriptions of these criteria?

<vivienne> I don't see anything on the academic authenticity?

Mario: GB's idea is a good one, I've reviewed for ICCHP for years but I've never seen such thing
... we need for the reviewer to put their
... rationale?

SAZ: should we start with a wiki page or should we discuss it now?

SAZ: we have to emphasize that other groups could benefit from this

Mario: not now in the call

GB: I'd go for a wiki page

<vivienne> I agree with a wiki page, would allow us all to add things as we go through it

GB: based on my experience reviewing papers

<yeliz> I agree with Giorgio

GB: the rationale is to improve the reliability and consistency of reviewers when evaluating a paper

SAZ: if we do it publicly we can get more input

<giorgio> if shadi could create an empty page for that I'll keep working on it.

<giorgio> yes

<giorgio> good.

SAZ: let's list the CFP criteria and look at definitions; can it be done by wednesday morning?
... YY, is that ok?

YY: I can also contribute adding definitions or some question for each criteria
... defining is difficult but useful for authors and reviewers
... is good for authors to get feedback so that they can improve their submission
... my only doubt is again regarding to last week discussion with Shawn
... about academia vs industry

<giorgio> so we need people of this group that are notacademians to review our proposals.

SAZ: there are two aspects (1) criteria and (2) how we define them
... GB made a good point

<yeliz> I agree

<giorgio> nono

<yeliz> to get a different perspective, its important

we need academic reviewers but knowledgeable on the topic

<yeliz> too academic

<yeliz> :-D

GB: I meant that in the wiki page should be reviewed by people who don't call themselves from academia

<yeliz> I agree with Giorgio

<yeliz> +

<yeliz> +1

GB: YY, SH, GB and MV are from the same family
... but SAZ, SLH's point of view in necessary

SAZ: let's have a balance SC

<yeliz> yes, that's why I think it was good to have Shawn

<yeliz> :)

<Mario> very good idea.

GB: to improve the CFP and the symposium we need to know how we will get feedback from participants

<yeliz> +1

GB: where participants are presenters and attendants

<Peter> +1 feedback form - consistency could be improved

SAZ: we have lots of things: define criteria, review from non-academic people, review form, etc.great ideas
... other thoughs?

YY: one last thing about rebuttal process...in some conferences...SH really likes it....after the review authors are allowed to submit his comments back to the SC

GB: we don't have enough time

I agrre


GB: we are asking 1000 word abstracts and the most typical answer we can get from authors is that there was no enough room for explanations

YY: I agree with GB, maybe it'd be useful for subsequent seminars

<yeliz> for the first one, I agree

<yeliz> definitelye

<yeliz> we can have in mind for the following ones

<yeliz> I agree Shadi

<giorgio> ok

SAZ: any other thoughT?
... another question related to the process...is participation..agreement was that accepted papers authors will have a bit of time

not more than 10min

SAZ: should have read the papers in advance
... there will be a panel : weminar chairs and some open questions
... all depending on the number of papers and questions...
... there will be physical limitations

<giorgio> wha kind of capcity can we handle?

SAZ: of how many people we can accept
... I have to check that
... I remember 150
... members of the RDWG should have a seat
... if we wanna do some other criteria such as first come first serve basis

<Mario> ok

GB: I agree on RDWG members, accepted papers + authors of rejected papers

SAZ: good point

<vivienne> sounds good

SAZ: do we need invitees?

<giorgio> why not?

<Peter> sure

SAZ: relevant people that don't have time to submit

<vivienne> good

I agree with that

SAZ: once we have that we can open to first come served

YY: we can give priority from...research groups

what about people from industry?

<yeliz> for example, yahoo accessibility group, Google accessibility group

SAZ: somebody from WCAG...back to the SC

GB: I'd be happy if Google, Yahoo or Amazon join but perhaps we have to be clear on the criteria for invitations.
... Inviting sb who is highly cited is a good idea
... excluding or missing groups might be a problem

<Peter> put criteria on the wiki? :)

<giorgio> who and why

SAZ: we need a criteria for this too: who do we invite, who has priority?

<Mario> Let's also think about these criteria in the Wiki.

SAZ: with 150 people we can accomodate most people
... people have to think who do we want on the phone

<giorgio> the real question is how many reservd seat are there, and how are they assigned.

SAZ: one criteria could be sb how could add value
... anything else to raise?
... thank you very much, I'll set the wiki pages
... we have an agenda for next week

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.136 (CVS log)
$Date: 2011/09/23 07:40:46 $