Re: Call for consensus - Localization Quality Précis (related to [ISSUE-34])

Hi Des,

I think the point is not whether you have a range +/- 100 or only +100, but
the difference between a range and voting like user evaluation which you
had described before.

With the range approach, if the value of the locQualityPrecisScore is 80,
you can say: the system scored 80% of a maxium.

With the user voting you described with the comparison to youtube, the
value 80 doesn't give you the information "80 is 80% of maxium 100". It
just tells you: the total sum of user votings is 80.

If we say that a value like 80 can have both the voting and the range
interpretation, applications which want to process locQualityPrecisScore
probably won't know what to do. E.g. if in a workflow there is a step
"everything below 80% quality needs a check", that makes only sense for the
range interpretation, but not for the voting approach.

Best,

Felix

2012/8/24 Des Oates <doates@adobe.com>

> Yves, yes in theory it is open ended but a range of +/- 100 would satisfy
> most use cases. (The vast majority of candidate translations vote counts I
> have seen to date on are in the 10s range or less, not the 100s).  If not
> then implementers could put in some form of logarithmic mapping with limits
> at +/- 100.  I don't think it is necessary to add another attribute
> specifically for this at this point. I think it is sufficient just to
> overload the locQualityPrecisScore as mentioned before, at least for now.
>
> On saying that, a standalone voting attribute has merits, since it could
> capture other information along with the aggregate score such as  total '+'
> counts, total '-' counts which are also useful metrics.  However I think
> that is out of scope for ITS2.0.
>
> Cheers
> Des
>
>
> -----Original Message-----
> From: Yves Savourel [mailto:ysavourel@enlaso.com]
> Sent: 24 August 2012 14:15
> To: public-multilingualweb-lt@w3.org
> Subject: RE: Call for consensus - Localization Quality Précis (related to
> [ISSUE-34])
>
> Hi Des,
>
> > This particular segment has 3 translation candidates that users can
> > vote on. Users can click either the checkbox (+ve vote) or the ‘x’
> > (-ve vote).  The aggregate of all users’ votes for each candidate
> > translation represents the ‘quality score’ for that candidate.
>
> So, just to see if I get this right: if, for a given translation, you have
> 5 'plus' and 12 'minus', your aggregated score will be -7?
>
> If that's the case, your system seems to be indeed open-ended and cannot
> be truly mapped to a range.
>
> Therefore I don't think having locQualityPrecisScore as value between -100
> and 100 will be more useful than if it's a value between 0 and 100. In both
> case it wouldn't be interoperable with any of the range-based scores used
> by other systems.
>
> Basically, I think a range-based value can only represent a rating, not a
> voting system.
>
> Maybe we need another attribute altogether that would represent voting?
>
> -yves
>
>
>
>
>


-- 
Felix Sasaki
DFKI / W3C Fellow