Re: Requirements for units of measure, accuracy and precision

I think we need to step back and decide if this is about requirements or
BP, and target the elements of conversation better to which deliverable is
affected.

IMHO there is a _requirement_ for a canonical mechanism to state aspects of
spatial properties.
I also think its a requirement for that mechanism to be common with broader
requirements for publishing metadata.  Its not a uniquely spatial concern -
but spatial is typified by this being relevant at dataset, service, feature
and property levels of granularity and the need for users to be able to
discover and interpret this information regardless of where it is best
placed.

If there is no consensus on an identifiable Best Practice - IMHO we should
note this as an unmet requirement in the BP document , and refer to the
catch-all (reuse something!) BP

Rob




On Thu, 1 Sep 2016 at 02:27 Joshua Lieberman <jlieberman@tumblingwalls.com>
wrote:

> Jeremy,
>
> These may not be the terms we want to make a best practice of. Precision
> (closeness of repeated measurements) and accuracy (closeness to truth) are
> really concepts that are impossible to compute directly, since one can’t
> either make all possible repeated measurements, nor actually make true
> measurements. What one can estimate and report is uncertainty (the range
> within which the true value has some likelihood of occurring if we
> accounted for all sources of error) and resolution (predicted precision or
> fineness of measurement).
>
> The spatial question in this is whether one should report multidimensional
> uncertainty and resolution, using such things as error ellipses or fuzzy
> geometric boundaries. It could be a real mess to make a map with fuzzy
> topographic contours, but the variation in hypsometric uncertainty between
> different published maps would probably shock people if they were made
> aware of it.
>
> Josh
>
> On Aug 31, 2016, at 11:13 AM, Jeremy Tandy <jeremy.tandy@gmail.com> wrote:
>
> Ignoring the UoM thing for now (although I can see @josh's point about it
> being needed to define CRS), can folks tell me whether BP 5 "Describe the
> positional accuracy of spatial data" [1] is on the right track to meet
> requirements about precision and accuracy? If not, please can one of you
> fine folks make some suggests content changes?
>
> Thanks in advance. Jeremy
>
> [1]: http://w3c.github.io/sdw/bp/#desc-accuracy
>
> On Wed, 31 Aug 2016 at 12:32 Joshua Lieberman <
> jlieberman@tumblingwalls.com> wrote:
>
>> Given that there are specific spatial aspects to both precision and uom,
>> for example a precision ellipsoid is a geometry and a unit of measure such
>> as an altitude is relatively to a datum such as a geoid, it seems
>> reasonable to me that these can be included SDW BP's without worrying
>> overmuch whether DWBP will ever get to these sorts of multi-dimensional and
>> affine space issues.
>>
>> Josh
>>
>> Joshua Lieberman, Ph.D.
>> Principal, Tumbling Walls Consultancy
>> Tel/Direct: +1 617-431-6431
>> jlieberman@tumblingwalls.com
>>
>> On Aug 31, 2016, at 07:19, Little, Chris <chris.little@metoffice.gov.uk>
>> wrote:
>>
>> Simon, Rob,
>>
>>
>>
>> I support submitting a formal public comment to DWBP, AND doing something
>> specific to spatial.
>>
>>
>>
>> Chris
>>
>>
>>
>> *From:* Rob Atkinson [mailto:rob@metalinkage.com.au
>> <rob@metalinkage.com.au>]
>> *Sent:* Friday, August 26, 2016 1:42 AM
>> *To:* Simon.Cox@csiro.au; frans.knibbe@geodan.nl; public-sdw-wg@w3.org
>> *Subject:* Re: Requirements for units of measure, accuracy and precision
>>
>>
>>
>> When this was raised and we asked Phil for guidance, we were basically
>> told DWBP is pretty much fixed, if we care it behooves us to address it
>> even ifs its a more general issue. I think we just need to note its a more
>> general issue, and provide what we feel is a BP otherwise there is really
>> no useful path to implementation.
>>
>>
>>
>> Given precision is a first class concern of spatial data - as spatial
>> resolution is going to be ever critical in the Internet of Things
>> especially, we need to handle this.  Precision may be common to a dataset
>> due to a common methodology, specific to a sensor, or time-varying - such
>> as GPS coordinates. The BP needs to address how to attach this at this
>> different levels of granularity. We need to cover both measured position
>> and gridded coverages as well, and preferably not with completely different
>> approaches. Precision is not generally an attribute of a CRS, unless we are
>> going to define CRS for every possible grid - and that would be a fairly
>> major departure from existing practice i suspect.
>>
>>
>>
>> UoM is a _similar_ case, and not inherently spatial, so i agree it can be
>> localised to CRS , TRS,  but I see no harm in tying it in to the same
>> pattern as spatial precision - whats needed is a BP to provide metadata
>> about values, to meet the hard requirement of spatial CRS and precision,
>> but its useful to suggest this would be applicable to UoM. Certainly having
>> a completely different set of patterns for attribution of uom and CRS feels
>> like sub-optimal practice.
>>
>>
>>
>> Rob
>>
>>
>>
>>
>>
>> On Fri, 26 Aug 2016 at 10:09 <Simon.Cox@csiro.au> wrote:
>>
>> Ø  The lack of information on units of measure and the apparent lack of
>> concern for proper indication of uncertainty in numbers are widespread in
>> spatial data and something should definitely be done about that. However, I
>> maintain that both problems are more general than spatial data and are
>> therefore out of scope for the UCR document. We have tried hard to limit
>> the UCR requirements to only spatial data on the web.
>>
>>
>>
>> Frans – I agree with this assessment in principle. Should we try to pass
>> it back up to the DWBP group?
>>
>>
>>
>> Best Practice 7 https://www.w3.org/TR/dwbp/#quality refers off to the
>> Data Quality Vocabulary, which may mean they think it is dealt with, but
>> I’m not convinced. It’s one more step away from the readers, at least, and
>> none of the examples show units of measure, or suggest where they would be
>> found. ‘units’ or ‘uom’ does not appear in their document (in the sense
>> that we mean, anyway).
>>
>>
>>
>> However, this document is still in ‘Working Draft’ status, although it is
>> certainly a pretty mature document by now. We could start by making a
>> comment on their public list
>> https://lists.w3.org/Archives/Public/public-dwbp-comments/ to the effect
>> that
>>
>>
>>
>> ‘The requirement to know the units of measure for quantitative data (or
>> another kind of ‘reference system’ for other kinds of data values) is not
>> mentioned in https://www.w3.org/TR/dwbp/. Units are required in order to
>> use quantitative data, so the Data on the Web Best Practices should include
>> a recommendation on the topic. This affects the following benefit
>> categories: Comprehension (C), Processability (P), Reuse (R)’
>>
>>
>>
>> But perhaps this is a place where some more formal coordination between
>> the groups is warranted?
>>
>> @phila – has this issue ever been discussed in the DWBP group? A quick
>> search of the email archive doesn’t turn anything up.
>>
>>
>>
>> --
>>
>> As a side comment – it is kinda remarkable how this gets overlooked. I
>> raised it in the Open Knowledge forum on their Data Packaging/Frictionless
>> Data initiative. As soon as I mentioned it there was acceptance that it is
>> in scope, but somehow no-one had brought it up until then.
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* Frans Knibbe [mailto:frans.knibbe@geodan.nl]
>> *Sent:* Thursday, 25 August 2016 10:50 PM
>> *To:* SDW WG Public List <public-sdw-wg@w3.org>
>>
>>
>> *Subject:* Requirements for units of measure, accuracy and precision
>>
>>
>>
>> Hello,
>>
>>
>>
>> In messages
>> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0172.html and
>>
>> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0109.html the
>> possibility of adding new requirements to the UC&R doc was brought forward.
>> Those should be requirements that
>>
>>
>>
>> A) the units of measure (UoM) in spatial data should be made clear
>>
>> B) the precision of spatial data should be made clear
>>
>>
>>
>> From the looks of it, those requirements would at least be requirements
>> for the BP deliverable.
>>
>>
>>
>> The topic was also discussed in the last SSN teleconference
>> <https://www.w3.org/2016/08/23-sdwssn-minutes>. I thought it would be
>> good to create a separate thread for these related issues.
>>
>>
>>
>> I will first repeat my initial response: The requirements are very good
>> requirements. The lack of information on units of measure and the apparent
>> lack of concern for proper indication of uncertainty in numbers are
>> widespread in spatial data and something should definitely be done about
>> that. However, I maintain that both problems are more general than spatial
>> data and are therefore out of scope for the UCR document. We have tried
>> hard to limit the UCR requirements to only spatial data on the web. This
>> constraint is specifically mentioned in the section on methodology
>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#Methodology>.
>> If we were to neglect this constraint, then the amount of requirements
>> could run out of hand quickly, the decisions on which requirements to
>> include or not would become very arbitrary and the deliverable teams that
>> are tasked with meeting requirements would inevitably find out that they
>> are not in a position to meet the requirements because they are not in
>> scope for their work. Of course the deliverable teams will work with
>> additional requirements next to the ones mentioned in the UCR document.
>> Those additional requirements will be based on general best practices. I
>> think the UoM and precision requirements fall in that class.
>>
>>
>>
>> That said, perhaps there is a way to shape the requirements in such a way
>> that they can be included in the UCR document, without violating the
>> spatial scope constraint too harshly. After all, it has been done for other
>> requirements too, to be honest.
>>
>>
>>
>> Let's start with the UoM requirement (A). I assume that is about the UoM
>> of coordinate data only. I think this is already implicitly covered by the
>> CRS requirements Linking geometry to CRS
>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#LinkingCRS>,
>> Determinable CRS
>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#DeterminableCRS>
>> and CRS definition
>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#CRSDefinition>:
>> If those requirements are met, it should always be possible to know the UoM
>> of coordinates, because the UoM will be part of the CRS definition. Perhaps
>> we should be explicit in mentioning that a CRS definition should include an
>> indication of UoM?
>>
>>
>>
>> As for the requirement B, if we change the wording a bit the requirement
>> could be made applicable to spatial data only and therefore be in scope:
>>
>>
>>
>> B2) The use of precision that matches uncertainty in coordinate data
>> should be facilitated and encouraged
>>
>>
>>
>> With this kind of wording I think the BP editors have a fighting chance
>> of meeting the requirement.
>>
>>
>>
>> Please share your thoughts...
>>
>>
>>
>> Regards,
>>
>> Frans
>>
>>
>