د/ايمان زغلول قاسم

استاذ تكنولوجيا التعليم المساعد بكلية التربية بالزلفي

واجهة تفاعل للكفيف

Developing Dual User Interfaces for Integrating Blind and Sighted Users : the HOMER UIMS

Anthony Savidis and Constantine Stephanidis

Institute of Computer Science,
Foundation for Research and Technology-Hellas (FORTH),
Science and Technology Park of Crete,
P.O. Box 1385, GR 711 10, Heraklion, Crete, Greece,
Tel: +30-81-391741, Fax: +30-81-391740
Email: as@csi.forth.gr, cs@csi.forth.gr

© ACM

Abstract

Existing systems which enable the accessibility of Graphical User Interfaces to blind people follow an "adaptation strategy"; each system adopts its own fixed policy for reproducing visual dialogues to a non-visual form, without knowledge about the application domain or particular dialogue characteristics. It is argued that non-visual User Interfaces should be more than automatically generated adaptations of visual dialogues. Tools are required to facilitate non-visual interface construction, which should allow iterative design and implementation (not supported by adaptation methods). There is a need for "integrated" User Interfaces which are concurrently accessible by both sighted and blind users in order to prevent segregation of blind people in their working environment. The concept of Dual User Interfaces is introduced as the most appropriate basis to address this issue. A User Interface Management System has been developed, called HOMER, which facilitates the development of Dual User Interfaces. HOMER supports the integration of visual and non-visual lexical technologies. In this context, a simple toolkit has been also implemented for building non-visual User Interfaces and has been incorporated in the HOMER system.

Keywords:

UIMS; Aids for the impaired; Programming environments.


Introduction

Currently, accessibility to Graphical User Interfaces by blind users is enabled through systems which reproduce the lexical structure of User Interfaces (i.e. interaction objects and their relationships) to a non-visual form. These are characterized as adaptation oriented methodologies and the relevant systems, which employ filtering techniques to extract lexical information, are commonly referred to as screen readers. One of the main weaknesses of many such screen reader programs is that they explicitly introduce visually oriented concepts to the non-visual interaction. Such concepts come from the spatial metaphor, which was developed as a result of intensive research efforts based on the human visual information processing capability. Recent approaches aimed at excluding specific types of objects from reproduction and employed 3D audio output techniques for non-visual layout construction [5]. However, such lexical filtering and subsequent lexical transformation of the User Interface suffers from many significant theoretical drawbacks: (i) the original dialogue design has addressed the specific needs and abilities of sighted users, and consequently reproduction to a non-visual form requires syntactic re-structuring according to the needs and abilities of blind users; the translation of visual physical entities to corresponding non-visual structures is like trying to translate text from one language to another by simple substitution of words; also, the role of designers for non-visual dialogues is practically neglected, (ii) no knowledge on the application domain can be extracted and consequently the idiosyncrasy of the application is not taken into consideration; for instance, additional error prevention methods would be required for a nuclear reactor control system in comparison to a conventional computer game, (iii) considering that the trend of User Interface software technology is directed towards visual methods such as virtual reality and 3D representation (i.e. visual reality, for instance the perspective wall and cone trees [1]) it is likely that adaptation oriented techniques will become unrealistic or meaningless, and (iv) there are no facilities for developing non-visual User Interfaces which introduces the serious problem of lack of tools; with adaptation oriented methods, design, prototyping, testing and experimentation cycles are completely "vanished" since only some already existing visual applications can be provided to blind users in a fixed non-visual form.

It is therefore evident that there is a need for better quality User Interfaces accessible by blind users and for the corresponding development support. In this context, the Dual User Interface concept has been defined as an appropriate basis on which the previous problems can be efficiently addressed. A Dual User Interface is characterized by the following properties: (i) it is concurrently accessible by blind and sighted users, (ii) the visual and non-visual metaphors of interaction meet the specific needs of sighted and blind users respectively (iii) the visual and non-visual syntactic and lexical structure meet the specific needs of sighted and blind users respectively (iv) at any point in time, the same internal (semantic) functionality should be made accessible to both user groups through the visual and non-visual "faces" of the Dual User Interface (v) at any point in time, the same semantic information should be made accessible through the visual and non-visual "faces" of the Dual User Interface. The first requirement is important for enabling cooperation between a blind and a sighted user and thus avoiding further segregation of blind people in their working environment. The second requirement explicitly poses the need of a metaphor specifically designed for blind users in the non- visual environment. The third requirement expresses the need for separate non-visual interface design. Finally, the fourth and fifth requirements indicate that the underlying functionality and information structures should be made equivalently accessible, though via different interactive channels.

A tool has been developed, called HOMER, which facilitates the development of Dual User Interfaces. It falls in the domain of language-based User Interface Management Systems (UIMS) and, apart from Dual User Interfaces, it supports the development of separate visual and non-visual User Interfaces as well. The HOMER system can integrate visual and non-visual lexical technologies and it supports concurrently communication with a visual and non-visual lexical technology at run-time. The dialogue specification is facilitated through the HOMER language. This language enables abstraction of the physical constructs, which are introduced through lexical technology integration in the HOMER system, to metaphor independent interaction objects. Consequently, sub-dialogues which can be described with such abstract objects are specified with shared descriptions for both the visual and non-visual environments. The Athena and the OSF/Motif widget sets have been integrated as visual technologies, while a simple toolkit for non-visual interface development, called COMONKIT, has been developed and integrated in HOMER. The Dual User Interfaces generated by HOMER can function in two modes: (i) local collaboration, where the visual and non-visual versions run on the same host (support to interface designers for handling keyboard sharing for this case is provided), and (ii) remote collaboration, in which case the visual and non- visual versions run on remote hosts.

RELATED WORK

Solutions which appeared in the past for providing accessibility to Graphical User Interfaces by blind people were based on filtering techniques aiming to reproduce in a non-visual form an internally stored image of the visual display (off screen model). Examples of well known commercially available systems are: OUTSPOKENTM by Berkeley Systems Inc. and SYSTEM 3TM by the TRACE Research & Development Centre at the University of Winsconsin for the Macintosh; SLIMWARE WINDOWS BRIDGETM by Syntha-Voice Computers Inc., for MS- WINDOWSTM. Research has been also devoted for providing access to the X WINDOWING SYSTEM; two different approaches are discussed in [5]: (i) The one followed by the MERCATOR project at Georgia Institute of Technology, which transforms the User Interface of X-clients to an appropriate auditory representation. (ii) The other followed by the GUIB project, supported by the TIDE Programme of the Commission of the European Union (DG XIII), which addresses both the MS-WINDOWSTM environment and the X WINDOWING SYSTEM. The GUIB approach is based on a proper transformation of the desk-top metaphor to a non-visual version combining Braille, speech and non-speech audio. Additionally, other issues are being investigated by GUIB, including: different input methods which can be used instead of the mouse, the problem of how blind users can locate efficiently the cursor on the screen, issues related to combining spatially localized sounds (both speech and non- speech) with tactile information and design of appropriate non-visual interaction metaphors. Also, the GUIB project has supported two different activity lines: the adaptation oriented line, under which the above efforts fall, and the future development tools line, in the context of which the work reported in this paper was partially funded.

DUAL USER INTERFACE SPECIFICATION

Interface specification in HOMER is facilitated through the HOMER language which has been designed for supporting formal description of Dual User Interfaces, as well as of separate visual and non-visual User Interfaces. Regarding the visual and non-visual "faces" of a Dual User Interface, the language allows: (i) metaphor differentiation, by enabling employment of different metaphors for the visual and non- visual environment, (ii) structural differentiation, by facilitating different object hierarchies to be constructed for the visual and non-visual environment, (iii) syntactic differentiation, by providing methods for defining separate behaviour for objects and different dialogue sequencing regarding the visual and non-visual environment, and (iv) shared description of sub-dialogues, by enabling shared specification of particular syntactic commonalities which can be identified by abstraction methods. Also, the principle of equivalence on represented semantic structures can be preserved through notational facilities for constraining visual and non-visual presentation structures to internal semantic objects.

Virtual interaction objects revisited

The so called virtual toolkits aim at enabling platform independent User Interface implementation by providing a common programming interface to various lexical technologies. They deliver collections of interaction objects, usually called virtual interaction objects, for interface construction. Since such virtual objects have attributes explicitly related to the desk-top metaphor (e.g. spatial attributes, colours, etc) which forms a specific physical platform model, they are considered mostly generalisations of the physical entities rather than abstractions. In HOMER, virtual objects can be constructed as pure abstractions, without having lexical attributes; only the abstract behaviour (e.g. selector, toggle, valuator, etc) and the physically independent state attributes (e.g. state of a toggle object, content of valuator, etc) need to be notationally visualized. Apart from abstraction, generalisation is also supported. For instance, virtual interaction objects can be specified which form generalisations of various platforms implementing the desk-top metaphor. In this case, such desk-top metaphor specific objects are appropriate only for the visual environment; however, they are platform independent and consequently the visual dialogue specification will not need modification for other platforms. In HOMER, the following categories of virtual objects are provided: (i) Dual virtual interaction objects, which are pure abstractions of visual and non-visual interaction objects. This type of objects can be used for describing the dialogue for both blind and sighted users in a unified fashion, (ii) Visual virtual interaction objects, which are generalisations of visual physical interaction objects. This type of objects can be engaged only on describing the visual dialogue, and (iii) Non-visual virtual interaction objects, which are generalisations of non-visual physical interaction objects. This type of objects can be engaged only on describing the non-visual dialogue.

Specification of virtual object genesis and lexical instantiation schemes

There are two levels of interaction objects in HOMER: (i) the virtual level, which concerns the classes of virtual interaction objects, and (ii) the physical level, which consists of the classes of physical interaction objects. Only virtual object classes can be utilised for dialogue specification, while the physical level plays a two-fold role: firstly, it incorporates the description of the interaction objects supported by the lexical technologies which is necessary for technology integration; and secondly, it is used for specification of mapping schemes with which virtual object classes are associated with various physical object classes. The basic relationship between virtual interaction objects in the dialogue is the parenthood relationship. Dual objects have two parents, one for the visual and one for the non- visual environment, which need not be the same. Moreover, a Dual object may be parentless for one environment while not for the other. An arbitrary number of attributes and methods can be specified for either virtual or physical object classes; additionally, physical object classes support inherited attributes and methods which are automatically inherited to children objects. During dialogue specification, full notational access is facilitated for virtual interaction objects to the various physical attributes and methods of the associated physical visual or non-visual object class.

A virtual interaction object can be constructed through the genesis mechanism. Construction of objects at the virtual level does not imply construction at the physical level, since no facilities are provided for extending the set of interaction objects supplied by a target platform. The HOMER system provides methods for integrating and utilising the physical constructs which are available. Engagement in dialogue specification of a virtual interaction object is enabled only after a physical instantiation scheme has been defined for the target environment(-s); for instance, Dual objects require both a visual and a non-visual instantiation scheme to be specified. In Example 1, the genesis of a Selector object is demonstrated at the virtual level, supporting one method called Selected and two virtual attributes, totalOptions and userOption. Following the genesis of the virtual class Selector, a visual physical object class is introduced, named VIMenu; physical attributes like foreground colour, background colour and font are defined.

     
dual genesis     Selector [     
        int totalOptions;     
        int userOption;     
        method Selected;     
        constructor (int N) [     
                 totalOptions=N;     
                 userOption=-1;     
        ]     
        destructor []     
]     
visual object VIMenu [     
        int N, userOption;     
        string fg, bg, font;     
        string* optionsList;     
        method VISelected;     
        ...     
]     
     
Example1: Genesis of virtual objects and specification     
of physical objects.     
     
     
visual instantiation Selector [     
        simpleMenu : VIMenu [     
->      {me}.userOption:={me}visual.userOption;     
        method Selected : VISelected;     
        constructor [     
                 {me}visual.N={me}.TotalOptions;     
        ]     
->      ]     
        default=simpleMenu     
]     
1: {x}.attr {x}visual.attr {x}nonvisual.attr     
2: method {x} Selected [ stmts ]     
3: method {x}visual VIselected [ stmts ]     
4: method {x}nonvisual NVselected [ stmts ]     
     
Example 2: Specification of instantiation, access     
to virtual / physical attributes and virtual / physical     
method specification for virtual interaction objects.     

In Example 2, the specification of the visual instantiation for the virtual Selector class is demonstrated. Multiple physical instantiation schemes can be defined for each environment. The example scheme is named simpleMenu and the physical class in which the object will map is the VIMenu. The block indicated between arrows is the description of the instantiation scheme named simpleMenu. Attribute associations can be specified either through constraints or monitors. In this example, a constraint enforces the virtual variable userOption to always have the value of the physical attribute userOption; the qualifier visual is used to distinguish physical visual attributes. Following, is the correspondence of virtual to physical methods. The virtual method Selected is associated to the physical method VISelected; when physical VISelected is called, the virtual Selected will be called as well. The last four lines show the syntax for accessing virtual or physical attributes and methods. The notation {X} provides access to the virtual structure of object X while the notations {X}visual and {X}nonvisual provide access to the visual and non-visual physical structures respectively. Since for each virtual object class it is allowed to specify multiple physical instantiation schemes, it is possible to have objects of the same physical class which are instantiated differently at the physical level. This is the first level of polymorphism (i.e. from the virtual to the physical level) enabled in HOMER. Moreover, the physical level can be modelled in such a way that it forms a generalisation over platforms instead of exactly "visualizing" the conventions used in the particular lexical technology being integrated.

This would practically serve as a second level of polymorphism. Hence, virtual objects are potentially characterized by a meta-polymorphic lexical nature. In Figure 1, the intermediate levels for physically instantiating a Dual virtual object are indicated.

Figure 1:The various intermediate levels for physical realisation of virtual interaction objects in HOMER.

Dialogue agents

The controlling model in the HOMER language is realized via agents; the basic PAC [2] model is adopted, however, it has been enhanced with the introduction of a dual presentation component. Agents in HOMER manage collections of virtual interaction objects, event handlers and owned agents; all managed entities live as long as the owner agent lives. Agents fall in two categories regarding the way in which they are instantiated: (i) agents instantiated by precondition, which can not be directly re-used, since they access either global variables or variables belonging to hierarchically higher agents; however, simple preprocessor facilities are usually applied for creating class patterns and for customizing to specific declarations when required, thus enabling a type of generality to agent classes, (ii) agents instantiated by parameter passing, which are explicitly instantiated by statements and they can have an arbitrary number of parameters (either conventional variables or virtual interaction objects); parameterized agents are re- usable and can be applied to easily construct general interface components like dialogue boxes and even domain specific dialogue substructures.

     
     
1: agent Dbox create if (precondition)     
2: agent Dbox (call parameters)                  
   destroy if (precondition) [     
                 interaction objects, event     
                 handlers, methods, constraints,     
                 monitors, functions, data     
                 structure definitions, variables,     
                 agents     
        constructor [ stmts ]     
        destructor [ stmts ]     
]     
     
Example 3: Specification of dialogue agents enabled     
by precondition or by parameter passing, and constructs     
internally to agents.     

The skeleton of an agent specification in HOMER is indicated in Example 3. The first line shows the header of an agent instantiated by precondition, while the second a parameterized agent. Agents may optionally have a destruction precondition. Apart from objects, event handlers and children agents, the specification of arbitrary method implementations, constraints, monitors, functions, data structures and variables may take place within the definition body of agents. Also, communication events related to the application interface (e.g. creation of shared objects, receipt of messages in a channel, etc) can be also used as instantiation / destruction preconditions for agents. One way of achieving syntactic differentiation is via dialogue agents and virtual objects. Assume that task-structure differentiation is required for a particular simple task of deleting a file as shown in Figure 2. This scenario is implemented in HOMER via a single agent which owns all the objects indicated in Figure 2. Objects with a prefix VI, represent visual objects and those with prefix NV represent non-visual objects; the rest are dual objects. The designer defines the hierarchical structure in the context of the owner agent for both environments - notice that dual objects have two parents - and subsequently assigns the desired behaviour for each environments via corresponding method implementation. Figure 2:A simple scenario for task structure differentiation.

Event handlers

The model of event handlers is based on Event Response Systems (ERS) [4] with some modifications and extensions: (i) flags have been substituted by arbitrary preconditions, thus adding more expressive flexibility, (ii) an event may be processed in parallel by many event blocks, which allows orthogonally expansion of the number of event blocks without introducing processing priority problems, and (iii) an event handler in HOMER consists of collections of event blocks (similar to ERL handlers). Event handlers are attached either to the visual or non-visual instances of virtual interaction objects, hence, syntactic differentiation is facilitated through event handlers as well. An arbitrary number of event handlers can be defined per object. Since event handlers live in the context of agents, dynamic change of the number of event handlers attached to an object can be handled by the designer through proper management of instantiation / destruction for the owner agents.

     
visual eventhandler DrawArea [     
        (pointDragged==true) PointerMoved [     
                 // arbitrary code here which     
                 // accesses PointerMoved.x,y     
        ]     
        (true) PointerMoved [     
                 !! ButtonPressed (PointerMoved.x,     
                 PointerMoved.y,     
                 1);     
        ]     
]     
     
Example 4: Specification of a simple event handler.     

The specification of a simple visual event handler is illustrated in Example 4. Two event blocks are shown for the same event class, thus demonstrating the parallelism for event processing. Also, the first event block has a simple precondition while the second event block indicates the way in which unconditional handling of events can be specified (i.e. setting (true) as the precondition). The statement of the second event block (!! ButtonPressed) concerns the generation of artificial events (i.e. local event broadcasting).

الوقت من ذهب

اذكر الله


المصحف الالكتروني