Peter Brusilovsky
Carnegie Technical Education and
School of Computer Science, Carnegie Mellon University,
Pittsburgh, PA 15213 USA
plb@cs.cmu.edu
David W. Cooper
Antech Systems Inc.
1214 Progressive Dr., Suite 101
Chesapeake, VA 23320 USA
dcooper@antechsystems.com
Abstract: Electronic Performance Support Systems is a new and challenging area for application of adaptive hypermedia techniques. Some possible scenarios for using adaptive hypermedia for adaptive performance support were explored in the context of the Adaptive Diagnostics and Personalized Technical Support (ADAPTS) project. ADAPTS provides an intelligent, adaptive electronic performance support system for maintaining complex equipment. ADAPTS maintains a dynamic characterization of a technician's knowledge, experience, and preferences in the form of a user model. This model influences the diagnostic strategy, technical information content, and navigation support offered to a technician.
Keywords: adaptive performance support system, adaptive hypermedia, user model, adaptive presentation
Advanced diagnostic systems can offer a technician a wealth of information about what is wrong or about to go wrong in a system--even where the problem lies. Regardless of the sophistication of these diagnostic systems, however, the technician must ultimately decide how to resolve the reported problems. Typically, the technician uses the output of the diagnostics system as a starting point for troubleshooting, consulting a fault code index or symptom tree to interpret the diagnostics output. The technician determines what to do by locating procedural information in a technical manual to supplement his or her knowledge and experience with the equipment. The technician must be able to comprehend the information presented in the technical manual, but without being overwhelmed by too much information. In some cases the first information located is sufficient, but in other cases different, related information is needed or supplementary information may be required to bolster the technician's understanding. For example, other information may be required to explain how to set up for and conduct a series of tests until the fault is isolated and the correct remove and replace procedure executed. While the conversion of paper documentation to computer format expedites this search for information, the technician is still required to locate information that matches the current work context. Frequently, multiple searches and queries are required in a variety of products to locate all the information that applies in a given context.
The focus of the ADAPTS project was to offer an integrated adaptive electronic performance support system (EPSS) for maintenance technicians (Cooper et al., 1999). ADAPTS integrates adaptive guidance from diagnostics systems with adaptive access to technical information - thus supporting both sides of the process: what-to-do and how-to-do-it. ADAPTS adjusts the diagnostic strategy to who the technician is and what the technician is doing, dynamically adapting the sequence of setups, tests, and repair/replace procedures based on the technician's responses. New activities are planned depending on the technician's responses to current recommended activities. ADAPTS assembles information content on the fly in response to the steps of that diagnostic process. The technician receives dynamically selected technical support information appropriate for the contexts of the setup, test, and remove/replace procedure being performed. Key to the ability to adapt diagnostic strategy and provide contextual information is a dynamically maintained user model, which characterizes each technician. The characteristics may include knowledge about the equipment being maintained, experience with the equipment or specific procedures, equipment-related education, and preferences (e.g., prefers schematics to text descriptions). The user model determines what task to do, what technical information to select to describe the task, and how to best display that information for a given technician. (Figure 1).
The cycle of work with ADAPTS consists of two main sub-processes (Figure 1): adaptive diagnostics and adaptive interaction with a technician performing a task. The adaptive diagnostics is performed by a modified version of the TEAMS toolset (Deb, Pattipati & Shrsetha, 1997). The adaptive diagnostics engine accepts multiple sources characterizing equipment condition and uses a system model to recommend a diagnostic strategy. The diagnostic strategy, presented as a series of recommended tests, is adjusted as constraints are introduced by the technician or through other system inputs. Those constraints may include such diverse considerations as available tools and materials, technician experience and education (provided by a user model), combat environment, equipment configuration, maintenance history, and cost. On each step of the diagnostic process the diagnostic engine selects the most relevant task for the user to perform. This task is usually a big chunk of work, which consists of a sequence of subtasks such as various operations on equipment (including removing and installation of components) and checking measurable and observable parameters.
The identifier of the selected task is passed to the adaptive hypermedia engine, which provides adaptive support in the process of performing the task. Adaptive support includes adaptive guidance though the sequence of subtasks and adaptive presentation of relevant material for each performed subtask. The user interacts with the adaptive hypermedia engine through a standard Web browser. The results of the user's work with the task (confirmation that all subtasks are completed, results of the observations, or failure to perform the tasks) are passed to the diagnostic engine though a Java applet. Depending on the results, the diagnostic engine dynamically selects the next task to perform and starts the next cycle of work.
The adaptive hypermedia interface consists of two main windows - the outline frame (left frame on figure 2) and the content presentation frame (right frame on figure 2). Each frame can present several types of information. The user can select the desired type of information using named tabs on the top of each frame.
The main function of the outline frame is to provide an adaptive checklist of the task being performed. The adaptive task checklist helps each technician navigate through computer-presented maintenance information by suggesting an optimal path and indicating the current state of performing the task. It applies several adaptive navigation support techniques.
What differentiates adaptive navigation support from traditional hypermedia-based performance support is the customization to each technician's knowledge, experience, and preferences. Typical hypermedia systems identify a predefined course through technical information. ADAPTS, on the other hand, dynamically defines a unique course each time it presents technical information. The approach is comparable to a person giving directions to a driver after first asking where the driver is coming from and how familiar the driver is with the terrain. These directions differ if the driver is coming from the north versus south or is new to the area. In a similar manner, ADAPTS considers what the technician is doing and who the technician is before charting a course through technical information.
Before presenting information, ADAPTS consults the technician's user model and dynamically assembles custom-tailored, step-by-step directions for a troubleshooting task. ADAPTS uses a collapsible checklist of steps to guide the technician through a troubleshooting procedure (Figure 3). ADAPTS determines how to present this checklist based on a dynamic assessment of the user's expertise with that procedure. For example, ADAPTS collapses a subtask outline if the technician is experienced with the subtask. Inexperienced technicians automatically receive an expanded outline of subtasks that reveals details. Experienced technicians may expand the outline if they choose and are given greater flexibility to navigate within the checklist. Inexperienced technicians are given more assistance in step-by-step navigation. As a technician completes a step within the checklist, color-coding and icons identify completed, current, and remaining steps.
The duty of the content presentation frame is to display the relevant support information for the subtask selected in the outline frame. The problem here is that the amount of potentially relevant information could be very big and it's a serious challenge for a technician to find the information that is most suitable to his experience and context of work. While several navigation "tabs" are provided to classify the support information into several types and present each type in a separate window, the amount of information in each of these windows is still potentially too big. To provide further support ADAPTS uses an adaptive presentation technique called stretchtext (Brusilovsky, 1996b) to present a sequence of paragraphs of support information.
Stretchtext expands and collapses procedural paragraphs to reveal or hide details, similar to the expanding and contracting outline used for the procedural checklist (Figure 4). The use of stretchtext in ADAPTS is similar to its use in such systems as MetaDoc (Boyle & Encarnacion, 1994) or PUSH (Höök et al., 1996). For example, a particular paragraph could be collapsed in the default presentation if a technician is familiar with the information presented in this paragraph or if this information is not very relevant to the current context. It could also be collapsed if a technician is experienced with current procedural subtasks. The technician is free to expand and contract the stretchtext at will. To support the use of procedural information, the navigation component also custom-selects links to supporting information that will be offered to each technician. Technicians who are inexperienced with a step will be offered links to fundamental concepts, background information, and training segments (such as video clips or simulations). Experienced technicians will be offered links to more concise information that omits fundamentals that have already been mastered. Because the user model is continuously updated, the navigation path continuously adapts to the technician's changing level of expertise.
ADAPTS not only custom-selects links for a technician, it also cues the technician to the relevance of the links that are offered. Cues may be visual, such as different icons or different colors; textual, such as annotations or comments; or sorting, which places the most relevant links at the top of a list. Regardless of the technique used, the goal remains the same--guiding the technician to custom-selected support information that is not only applicable to the current context, but also appropriate given the technician's expertise.
The key to the intelligent performance of ADAPTS is the domain model. ADAPTS uses a popular concept network approach to domain modeling: the domain model is represented as a network of concepts of several types connected by various relationships. Two main types of domain concepts in ADAPTS are a component and a task. Both components and tasks form two separate hierarchies of concepts. One hierarchy is a tree of components: from the whole aircraft on the top, to subsystems, to sub-subsystems, down to elementary components called addressable units. Another hierarchy is a tree of tasks: from big diagnostic tasks that are handled by the diagnostic engine, to subtasks, to elementary steps. The two hierarchies are tightly interconnected because each task is connected with all components involved in performing the task.
The user model is the guide for personalizing the content and navigation in an ADAPTS application. It can also be used to influence the diagnostic strategy. ADAPTS uses a multi-aspect overlay user model. A technician's experience with a concept can be judged on many aspects, each weighted to indicate its relative influence on the decision. The user model independently accumulates several aspects (roles) of the experience and knowledge of each technician for each concept defined in the domain model. From this record, ADAPTS uses a weighted polynomial to estimate the proficiency of a user in locating, operating, and repairing equipment or performing each step of a recommended procedure. The weighting of aspects can be changed for different individuals, for example to account for learning styles. Factors measured in the ADAPTS prototype include whether and how often a technician has reviewed, observed, simulated, expressed understanding (self- evaluation), previously worked on, or received certification on specific equipment or procedures (Figure 5).
The user model continues to evolve as a technician uses the ADAPTS system, beginning with a stereotype that seeds the model for technicians with certain backgrounds. No formal test is used to initialize the model. The model grows in size as it records the technician's experience. It will follow a pattern of rapid expansion initially as new material is accessed for the first time, followed by decreasing growth rate up to a limit imposed by the extent of the domain model. Elements of the user model carry a time-stamp to be used in making judgements regarding when and how often certain information is accessed.
To support the user in performing a diagnostic task ADAPTS uses a variety of information types stored in its database (see figure 1) which are collectively called rich content . In addition to textual documents and diagrams, which are typical components of electronic technical manuals (IETMs), the rich content could include various pieces of multimedia: color photos, training videos, animations, and simulations. Moreover, the rich content could include variations of the same information fragment oriented to the users with different levels of experience. One of the functions of ADAPTS is to find pieces of the rich content that are relevant to the selected subtask, and to adaptively present it to the user. This functionality depends on the links between the domain model and the rich content. Establishing a connection between documents (i.e., IETM pieces) and the domain model is usually called indexing. Indexing is a key to both user modeling and adaptation. Various kinds of indexing applied in adaptive hypermedia systems are reviewed in (Brusilovsky, 1996a).
ADAPTS applies two kinds of indexing of the rich content. The first type is role-based indexing with components. Conceptually, it means that a piece of the rich content is linked by typed links with all components involved in it, while the type of links indicate the type of involvement (i.e., its role). For example, a piece of video that shows how to remove a component is indexed with a pair (component ID, role=removal). Similarly, a figure that shows the location of a component is indexed with a pair (component ID, role=location). Domain experts developed the set of possible roles.
The second kind of indexing refers to tasks and subtasks. The reason for indexing a piece of the rich content with a task or a subtask is that the rich content explains how to perform the task. For each piece of rich content, both the type of explanation and the level of explanation differ according to several factors. The type of explanation differs according to the material available for the content, such as text, figure, animation, or video, and its purpose. The level of explanation differs according to the estimated ability of the user to comprehend that material (for example, a reminder oriented to an expert who has done this task many times, or a complete description for a technician who has never performed the task). Tasks are usually indexed as a one-to-one relationship with a set of rich content dealing with a specific concept or topic. The specific rich content that is accessed from the set (to support each step in the task) depends on the user model. In contrast, a one-to-many indexing scheme is used with components so the technician sees many links as optional navigation paths.
ADAPTS is an electronic performance support system that integrates an adaptive diagnostics engine with adaptive access to supporting information. We think that integrated EPSS is a very attractive area for using adaptive hypermedia. Integrated performance support systems bring together an expert system like problem solving engine and an on-line information system. There are a number of examples of using adaptive hypermedia in both problem solving support systems and on-line information systems (Boyle & Encarnacion, 1994; de Rosis, De Carolis & Pizzutilo, 1994; Höök et al., 1996). However, an integration of these two types of systems in a single EPSS provides more possibilities for applying adaptive hypermedia. The problem support aspects of the system can provide reliable information about a user's current goal, and also serve as an additional source of feedback to the system about the user.
We consider as an important outcome of the ADAPTS project the fact that our system shows an example of applying adaptive hypermedia technologies in the area of integrated electronic performance support systems. We do not know any other electronic performance support system equipped with adaptive hypermedia, but we hope that more systems like ADAPTS will appear in the near future.
The ADAPTS project was developed for the US Navy at the Information Technology Branch of the Naval Air Warfare Center-Aircraft Division (NAWCAD) in St. Inigoes, MD.
Testability Engineering And Maintenance System (TEAMS) toolset was originally developed by Qualtech Systems Inc.
The ADAPTS software design was the result of collaboration among Carnegie-Mellon University, Antech Systems, Inc., http://www.antechsystems.com, and Qualtech Systems, Inc., http://www.teamqsi.com.
Deb, S., Pattipati, K., and Shrsetha, R. (1997) QSI's integrated toolset. Proceedings of IEEE autotest conference.
Boyle, C. and Encarnacion, A. O. (1994) MetaDoc: an adaptive hypertext reading system. User Modeling and User-Adapted Interaction 4 (1), 1-19.
Brusilovsky, P. (1996a) Adaptive hypermedia, an attempt to analyze and generalize. In: P. Brusilovsky, P. Kommers and N. Streitz (eds.): Multimedia, Hypermedia, and Virtual Reality. Lecture Notes in Computer Science, Vol. 1077, Berlin: Springer-Verlag, pp. 288-304.
Brusilovsky, P. (1996b) Methods and techniques of adaptive hypermedia. In P. Brusilovsky and J. Vassileva (eds.), Spec. Iss. on Adaptive Hypertext and Hypermedia, User Modeling and User-Adapted Interaction 6 (2-3), 87-129.
de Rosis, F., De Carolis, B., and Pizzutilo, S. (1994) User tailored hypermedia explanations. In: P. Brusilovsky and I. Beaumont (eds.) Proceedings of Workshop Adaptive Hypertext and Hypermedia at Fourth International Conference on User Modeling, Hyannis, MA, Aug 17, 1994, http://www.cs.bgsu.edu/hypertext/adaptive/deRosis.html.
Höök, K., Karlgren, J., Wærn, A., Dahlbäck, N., Jansson, C. G., Karlgren, K., and Lemaire, B. (1996) A glass box approach to adaptive hypermedia. User Modeling and User-Adapted Interaction 6 (2-3), 157-184.