Presenter's Name: Alfred S. (Al) Gilman

Affiliation:

• Al is an independent consultant
• This presentation is supported in part by the U.S. Department of Education and National Science Foundation through research grants to the Trace Center, University of Wisconsin.
• http://trace.wisc.edu/about/

List your research goal(s) or the research question(s) you are trying to answer:

• There is a general backdrop of research and development goals at Trace, in the Protocols and Formats Working Group. and elsewhere. That will be discussed below. But for this session my goal is to shift the agenda.
• From visualization as display to visualized display as a strategy to draw a user into a spatially distributed virtual world.
• From visualization as a display mode dependent on the user's visual sensory mode to visualization as a common strategy used in conveying a spatial world as the context for data review and problem solving.
• From separate consideration of display modes and input modes to a consideration of interactive cycles of display and action that sustain the user's orientation in the virtual world that they understand themselves to be operating in. In particular, in mouse mode the user correlates hand action with eye tracking of the cursor subliminally. In the AUDIODOOM game, it is not quite so subliminal but it is clearly the spatial connotations of user commands that combine with the spatial connotations of audio feedback that combine to sustain the user's orientation to the virtual world in which the plot unfolds.
• From repaiiring access to a world that has been visualized in ignorance of the realities of the delivery context to using a model/view/controller architecture to synthesize an optimal interaction mode for the task at hand that employs spatial metaphors in building a virtual world to an appropriate degree; no more, no less.
• The broader research goal is to make contributions that will help establish in widespread practice a robust pattern of practice for binding problem worlds to interaction worlds which readily generates a space of interaction modes which contains diverse enough modes to optimize user interaction with these spatializable problem worlds under conditions of partial, single, multiple and severe disabilities.
• problem world: a carrier manifold for all aspects of a problem-solving task. This will contain representations of all pertinent information, together with choices for user decision and action, and any reference models supporting prediction of the consequences of these actions.
• Interaction world: a virtual world containing a user's perception of displays, conceptions of command inputs, and conception of a larger world on which this directly perceivable and actionable world is a window. For example, as a user browses a DAISY book the tree form of the surrounding table of navigation is part of the interaction world even 'though it is not continually presented in the audio stream.
• Reference:

Describe or list the complex information that you are concerned about making accessible:

• Any spatializable domain of knowledge. That is to say any relation or interaction where the user's continuing understanding of context is helped by the application of spatial metaphors in the virtual world model reflected in presentation of information and the acceptance and response to user action.

Which user task are you studying? Provide a scenario.

• We are studying models and the application of model mediation. The following use cases are presented only to illustrate the range of use cases to which this set of strategies can be applied.
• Use case that demonstrates "virtual reality doesn't have to be visual reality" is the AUDIODOOM game created by Jaime Sanchez et_al.
• use case: physician / patient / guardian consultations. Physician needs to explain diagnosis and treatment options arrived at with the aid of medical imagery.
• Patient or other medical decision maker may have visual disability
• Haptic presentation of tumor evidence may be more comprehensible to decision maker than visual presentation whether the decision maker can see or not.
• Use case: Pedestrian-scale GIS for route planning. Anyone with visual, mobility, or cognitive issues in wayfinding tasks. Shifting from map-data visualization as problem statement to route-planning as task statement makes it much easier to construct a dialog that works.

Which modalities (haptic, aural, visual) and input or output devices are you using to address accessibility issues? List any thoughts you have about using multiple modalities to create accessible interfaces:

• These remarks are derived from a basis heavily weighted toward a) blind interaction experience with haptic I/O devices, spatialized audio, sonification and Braille (which creates a spatial virtual world the way pure audio doesn't). It is also heavily influenced by the methods and researches of high-end visualization for life sciences from genetics to surgery.
• My thoughts arbout multiple modalities are these:
• We are always involving multiple modalities in any interaction there is at least a display modality and an input modality.
• Screen reader users are generally operating in a more multimodal delivery context than the content that they process anticipated. Definitely when using audio and Braille both.
• The key is to understand the geometry of the virtual world that persists in the user's cognition of what is going on. That is what one targets in rendering data and in refreshment stimuli to sustain orientation. One can't afford to over-remind the user as to their view orientation and world context, but by the same token one can't be letting them lose lock on this context often, either.

In 5 sentences, how are you attempting to address your research goal(s) or question(s)? Either list the specific technologies you are using or provide a general description of how you are using the technologies.

• Positive and negative results are collected from a broad variety of interaction worlds and user conditions.
• The mathematical models of signal and communication theory and processing from Electrical Engineering are used to cast user experience, visualization code, and problem solving environments in a common theoretical framework, such as the notion of interaction in a grok-locked loop.
• Disability success cases are compared with the capabilities of emerging and popular techniques for near-term consumer progress possibilities, such as in the area of GIS or map data services.
• Theory and Visualization research are searched for broad-spectrum reusable modules to nominate for disability-access research and development, such as the tools from the SLICER toolkit.

Include any visual or aural illustrations that you would like to use during your presentation:

• Three Domains diagram
• http://trace.wisc.edu/docs/ud4grid/images/image004.gif
• Description: http://trace.wisc.edu/docs/ud4grid/descriptions/three_domains_desc.htm

List resources that you will reference during your presentation:

• AUDIODOOM game demonstrates a) the power of the mind to create a geometrical virtual world with paltry spatialization in the stimuli and b) the interactive nature of maintenance of spatial orientation (input and output interact in contributing to this).
• http://www.google.com/search?q=jaime+sanchez+AUDIODOOM
• SANGIS, a public-information map site with a lot of view controls.
• http://www.sangis.org/
• SLICER, an open source medical imagery library with a lot of view-control capability.
• http://www.slicer.org/
• Three-domains model is the basis for considering alternative and adaptive bindings of problem worlds to interaction worlds.
• http://trace.wisc.edu/docs/ud4grid/#_Toc495220373
• Extreme model-fusion techniques let us throw silicon at the problem
• http://www.sdsc.edu/~ludaesch/Paper/icde01.html