Final Report Use Cases

From Decision XG
Jump to: navigation, search

Decision Use Cases

One of the tasks set out by the incubator has been to collect use cases and requirements for a standard decision representation format. The intention has not been to be comprehensive, i.e. to cover all possible usages of such a format, merely to find enough use cases and requirements to motivate the existence and usefulness of a potential format. In addition, the use cases have been used as a basis for developing the small example formats that are described later in this report.

UC1 - Measuring Information Flow (decision states)

Proposed by: Jeff Waters.


Decision Aspects

  • Primary: Time, Process
  • Secondary: Overhead


eXtreme Design Components

There is a method proposed specifically for reusing Content Ontology Design Patterns, called eXtreme Design (XD). In summary, the idea is to use an agile and iterative approach for developing ontologies through the reuse of components (patterns). Also see the informal introduction provided by the Ontology Design Pattern Tour. The Context, Story and Competency Questions below are designed to support this XD process.

Context

An organization wants to set up a resource for its members to easily track the time spent in the various states of their decision-making. One of the goals is to provide situational awareness of the current/past/future decisions and time spent in the decision-making process. This may lead to improved tools for tracking/measuring information flow and streamlining the decision process.

Story: decision states

Decisions in this organization follow a typical information flow. First, a question is raised which starts the decision process. Second, the decision-maker spends time gathering information. Third, the decision-maker analyzes the information. Fourth, a decision is made. Fifth, the decision is described in some information format, perhaps in a text form or a slide presentation. Sixth, the decision is communicated to the intended audience, perhaps through a meeting, telecon, or e-mail. Seventh, feedback is gathered. And the process repeats. Each of these steps takes a certain amount of time and may involve a "waiting" subcycle. The decisions may serve as information input to other decisions and the latter must wait for the former to complete.


Competency questions (CQs) and contextual statements of decision information flow

  1.  When did a certain decision begin and end?
  2.  How much time was spent information gathering?
  3.  How much time was spent preparing the decision for presentation?
  4.  How much time expired between the decision being made and its communication to the intended audience?
  5.  How much time was spent information gathering/preparing/communicating versus analyzing/deciding?
  6.  How much time was spent "waiting" versus time spent processing the decision?
  7.  What's the average amount of time spent making decisions?

(Note: There are many other competency questions such as what was the subject matter of the decision, who made the decision, etc. but the ones mentioned above are particularly relevant for this use case focused on measuring decision/information flow via time spent in decision states.)

Background and Current Practice

This use case is derived from the need to measure the speed of information flow within an organization or enterprise. The objective is to determine a metric to measure whether the right information is getting to the right person at the right time. The decision process can be modeled as a sequence of states, some of which involve information gathering or information packaging and distribution. The amount of time spent in these states should be reduced if information is flowing effectively. The current practice is to look at indirect measures as opposed to having tools that are instrumented to produce a standardized decision representation to include time spent in decision states.

Goal

The goal is to ensure the states of decision making are represented and are usable for measuring the amount of time spent information gathering, packaging and distributing as opposed to states directly related to decision-making, such as assessing options, making decision, and assessing feedback.

In this use case, the goal is to capture, store and record time spent in various information processing states of the decision process to enable:

(1) users to assess the efficiency of the decision process from an information flow perspective;

(2) users to assess improvement from changes in organizational structure or tools designed to improve information flow;

(3) users to track individual information flow in the form of decision flow and time to process decisions across an organization to measure capability and efficiency of innovation.

Use Case Scenario

Employees of an organization can access a website tool showcasing a measure of information flow throughout an organization by showcasing the amount of time spent across the organization in key decision states.

Problems and Limitations

Cultural change implied by challenges to typical unscalable methods of communicating and packaging decisions, such as the use of slides, meetings, telecons, as opposed to lightweight, computer-oriented, flexible and immediate decision outputs.

Existing Work

[1] Information Velocity Metric for the Flow of Information Through an Organization: Application to Decision Support

[2] A Dynamic Process Model for the Design and Assessment of Network Centric Systems


UC2 - Open Linked Data

Proposed by: Jeff Waters


Decision Aspects

  • Primary: Subject Matter
  • Secondary: Context

eXtreme Design Components

The Context, Story and Competency Questions below are again designed to support the XD process.

Context

An organization or individual wants to use the open government data for making decisions.


Story: Open Linked Data Supports Decision-Making

The user would like to select applicable data sets in the form of RDF data stores (so that they can be easily combined), then select certain data components as options (e.g. cities) and select other data components as criteria for filtering and ordering of the options (e.g. number of earthquakes, amount of federal disaster relief, etc.). The user would then like to form a SPARQL query which could perform the query, filtering and ordering across the data sets to essentially assess and rank order the options based on the criteria. The rank ordering represents the assessment based on the weighted criteria. The user would then make the selection from the rank ordered list representing the final decision. In short, the user would like to use open linked data for decision-making.


Competency questions (CQs) and contextual statements of decision information flow

  1.  What Open Linked Data sets are supporting this decision?
  2.  Which Open Linked Data components are being used as options?
  3.  Which Open Linked Data components are being used as criteria?
  4.  What is the SPARQL query which supports the assessment of the options using the criteria?
  5.  What is the weighting of the criteria?
  

Background and Current Practice

This use case is derived from the need to make good use of open linked data and the need to use as the subject matter for decisions information which is already represented in a machine-usable semantic format. Currently, decisions are either not represented in a computer format, or are represented in a proprietary format, or are represented in a format consisting largely of free text (such as e-mails, reports, or slide presentations). Most decisions and decision processes are poorly recorded with little attempt for machine understandability. The information which goes into a decision is usually not readily available even if from government sources. Although significant efforts are now underway to make government data available in machine formats, even semantic formats, the information is generally utilized for queries or for human visualizations. There is a significant opportunity to utilize this data in further productive ways such as for the subject matter of decision-making.

Goal

The goal is to ensure that any proposed decision standard format is closely aligned with and can effectively utilize and leverage open linked data.


Use Case Scenarios

An individual would like to rate cities based on climate, location, population, quality of hospitals, and responsiveness of emergency services in order to decide where to retire. A city would like to rate its record of disaster relief funding received over the last 10 years in comparison with cities of similar size with similar levels of natural disasters so it can decide whether to employ additional resources on a specific grant application.

Problems and Limitations

Knowing what data is available, knowing the vocabulary to use for items of interest.

Existing Work

[1] For more on linked data, see http://linkeddata.org/.

[2] For more on RDF-enabling the linked data, see RPI Tetherless World Constellation at The Data-gov Wiki.

[3] Excellent Intro to SPARQL and Open Linked Data, see http://www.slideshare.net/fulvio.corno/sparql-and-the-open-linked-data-initiative.


UC3 - Metrics for Automatic Assessment

Proposed by:Jeff Waters


Decision Aspects

  • Primary: Measurement
  • Secondary: Assessment

eXtreme Design Components

The Context, Story and Competency Questions below are again designed to support the XD process.

Context

An organization or individual wants to specify the metrics used in making a decision and have those metrics support an initial automated assessment of the options.


Story: Metrics Support an Initial Automated Assessment

A user needs to make a decision, such as a computer purchase. There are several options which must be evaluated. The user wishes to specify the metrics (such as cost, hard disk space, warranty length, etc.) to be used in the assessment. The user wishes to specify these metrics in a manner which allows the user to specify the thresholds for what is allowed as an option (e.g. cost < $1500, hard disk space > 500GB, warranty > 1 year) and a weighting of the metrics based on user preference or other criteria (e.g. cost may be twice as important as hard disk space). The user wants to combine these metrics in a manner which allows the options to be rated on one overall scale (e.g. 1 (bad) to 100 (great)). In essence, the user wants to combine multiple measures (such as cost, quality, warranty) into one overall measure for purposes of rating the options and making a decision. The organization or individual would like the metrics to support an automated assessment of the options. In other words, each metric should be normalized in such a fashion that an automated assessment (scoring) of the options is possible resulting in a filtered and ordered set of options representing the best options for answering the decision question. The metrics should be easily configurable, reusable and sharable.


Competency questions (CQs) and contextual statements of decision information flow

  1.  What will be (are) (were) the metrics for this decision?
  2.  What are the weights given to the metrics?
  3.  What are the thresholds for determining which options to consider?
  4.  Is this metric to be considered better if higher (e.g hard disk space) or if lower (e.g. cost)?
  5.  How will these metrics be combined to generate an overall score for each option?
  

Background and Current Practice

This use case is derived from the need to make and document reasoned, rational decisions. Often a decision is made and there is no record of what metrics were used in the process and how they were weighted and what thresholds were considered. The result is that the decisions may appear arbitrary or ill-considered. Currently, decisions are either not represented in a computer format, or are represented in a proprietary format, or are represented in a format consisting largely of free text (such as e-mails, reports, or slide presentations). Most decisions and decision processes are poorly recorded with little attempt for machine understandability. The information which goes into a decision is usually not well documented. The ability to easily access and reuse decision components, such as metrics, is not well supported. The rationale behind a decision is often not apparent to those not involved in the decision process. Although complex decision-making cannot be boiled down into a simple formula, the attempt to define and quantify metrics, at least as a first cut approximation, is well worth the effort. In addition, more automated processes are being developed and a format for representing decision processes by computers are important to document, manage, archive and reuse in a standardized manner.

Goal

The goal is to ensure that any proposed decision standard format can effectively represent the metrics used to evaluate the options, including the weighting and thresholds given to those metrics by the decision-makers.


Use Case Scenarios

An individual would like to rate computers based on purchase price, speed, hard disk space, ram and warranty. A user would like to rate cities based on climate, location, population, quality of hospitals, and responsiveness of emergency services in order to decide where to retire. A user would like to rate earthquakes in terms of their significance for a weekly news report. A city would like to rate its record of disaster relief funding received over the last 10 years in comparison with cities of similar size with similar levels of natural disasters so it can decide whether to employ additional resources on a specific grant application.

Problems and Limitations

Objectifying subjective metrics, normalizing metrics to a common scale.

Existing Work

[1] A basic overview of metrics but in the context of product development, http://www.humiliationstudies.org/documents/Beaumontmetricsid.pdf.


UC4 - Interoperability

Proposed by: Don McGarry.

Decision Aspects

  • Primary: Status
  • Secondary: Context

Background and Current Practice

This use case is derived from the need to be able to exchange information in a standards-based way to properly inform decision makers of all available possible outcomes.

Goal

To represent the current operational picture along with anticipated actions of operational individuals driven from operational data.

Use Case Scenario

Taking a traditional Command and Control (C2) scenario we can specify the common elements:

  • Unified Command Structure
  • Forward Operating Command Substructure
  • Field / Operational Units

Problems and Limitations

Representing subject matter of decisions effectively in machine-understandable format.

Existing Work

[1] Naturalistic Decision-Making


UC5 - Situational Awareness

Proposed by: Don McGarry

Decision Aspects

  • Primary: Status
  • Secondary: Context

Background and Current Practice

This use case is derived from the need to understand the status of the organization. Every employee is a decision-maker at some level. An effective decision representation standard could be implemented in tools to enable capturing the real-time status related to decision-making, for example what decisions are being considered, where are we in the decision process, and what options are being considered. The combined representations could provide a real-time snapshot of the applied knowledge of an organization.

Goal

To represent the status of the organization through the capture and analysis of its decisions.

Use Case Scenario

Employees of an organization perform their normal duties utilizing tools which automatically capture decisions, status, criteria considered, options and assessments. Through the use of linked data, the subject matter of the decisions can be utilized to sort, analyze and present real-time and historical analysis of the thinking of the organization as represented by its decisions (planned, in process, and completed).

Problems and Limitations

Representing subject matter of decisions effectively in machine-understandable format.

Existing Work

[1] Naturalistic Decision-Making




Internal drafts used by the incubator during its work: