Abstract

The integration of modern computer-based inferencing, networking, computer-mediated training and multimedia technology hold the promise of improved training and performance support for personnel in industry and government. This paper reports an effort to develop a network-enabled knowledge editing and modeling environment that offers just-in-time training and performance support, anytime, anywhere. The instructional component of the system is based upon a knowledge model elicited from a domain expert and represented in a pedagogically-motivated multimedia browser. The expert's diagnostic processes are mapped into an integrated rule-based advisory system that assists the user in task performance. The advisory system is tightly coupled with the knowledge model in a fashion that provides context-dependent explanations of the diagnostic process. The distributed, network-based design allows the system to be accessed any time, on demand, running seamlessly over the Internet or similar networks. El-Tech (Electronic Technician) is a first prototype of this approach to next-generation, just-in-time performance support systems with embedded training. El-Tech is being developed as a joint project with the Navy's Chief of Naval Education and Training (CNET).

1. Introduction

The technologies and methods that underlie current approaches to computer-mediated learning, distance learning, expert systems, performance support systems, world-wide-web browsers, and just-in-time training systems are rapidly converging. Training and performance support that were once presented in hardcopy are increasingly being delivered in computerized formats. However, simply automating or computerizing current training materials and processes is not enough - qualitative changes to enhance instruction and job performance are both needed and feasible.

Systems that embody the knowledge and reasoning capabilities of experts in the performance of skilled tasks hold the promise of providing much greater utility than current training programs provide. The goal, as Wehrenberg (1989, p 38) aptly expressed it is "to put the right person in the right place at the right time with the right skills ..."

Bradshaw (1993) discusses integrated performance support systems (IPSS) that are designed to improve the performance of field-deployed technicians by making training and other critical information available immediately as needed. Bradshaw states that pressures for efficiency will require more training to be delivered only as needed on the job, in some degree supplanting the training that is currently offered in dedicated programs. As an example, it is possible that computer-based performance support systems might provide video demonstrations of equipment checkout or maintenance procedures in a fashion that can provide just-in-time training for technicians.

This paper reports an effort to create tools to support distributed knowledge modeling and inferencing capability. This tool set allows domain experts and knowledge engineers to collaborate in the development of performance support systems to assist the end user of the system with performance of a task. The deployed system will have the ability to explain the conceptual logic behind its diagnostic reasoning strategy. This research effort extends work that started with NUCES, Nuclear Cardiology Expert System (Ford et al, 1996) by extending the knowledge modeling and inferencing components to operate seamlessly over networks, and by adding additional support capabilities as appropriate to the application domain.

The system holds the promise of supporting selected ancillary services in a more cost-effective way via network technology. Since it is anticipated that such systems can facilitate just-in-time training in addition to performance support, the deployed system will have the capability to track users' use of instructional materials, and to provide other services. The system has been implemented in Java to achieve platform-independence. The first working prototype of this system, El-Tech, demonstrates the feasibility of this type of cost-effective just-in-time training.

El-Tech is being constructed in a joint research effort with the Chief of Naval Education and Training, that is aimed at advancing and integrating the aforementioned technologies into a remotely accessible, just-in-time performance support system for electronics technicians. The Navy, at any given time, has approximately 45,000 students in training, mainly through courses taught by approximately 6,500 instructors. It is typically the case that Navy electronics technicians find themselves in-fleet and responsible for mission-critical equipment months after receiving rapid-fire, short duration training on the equipment, and after a long-period of time without any contact with the equipment. Likewise, regular equipment upgrades and evolving safety procedures and warnings regarding maintenance and repair of equipment require continuous refresher courses for the technicians. With budgets regularly being reduced, the Navy is looking to distance learning and new technologies to provide a more cost effective delivery of continuous education and training.

The target audience for El-Tech is the enlisted electronics technician. Typically, this technician has had 16 weeks of basic electronics training followed by as much as five months of advanced training on specialized equipment. The advanced training is especially fast-paced and covers a lot of equipment. As stated above, very often it is months after the courses end before technicians finds themselves responsible for the maintenance of the equipment for which they were trained. The role of El-Tech will be to assist these electronics technicians in their job performance.

2. Computer-based Training and Performance Support Systems

Collis and Verwijs (1995) present a survey of software that originally fell outside the educational process, but that are now being applied increasingly to educational situations, and that are characterized as hybrid electronic performance support systems. These products are grouped into three categories: support for group functioning, information retrieval and handling assistance, and guidance and information support that are integrated into the regular workplace. Our work encompasses elements of the latter two categories.

Computerized training and performance support has evolved significantly from the early computer-based training programs. Early work in computerized training and performance support featured lockstep, programmed instruction that grew out of the work of B. F. Skinner and the Behavioral Psychologists (Shrock, 1995). These early systems presented a standard curriculum and sequence of instruction for all, with testing and feedback to assess the attainment of pre-specified goals. Such systems mirrored the approach (in computer-mediated form) of the traditional classroom, and lacked explicit performance support capability. The approach of such early systems have been criticized both for being decontextualized, and for not being geared to the needs of the individual student, and hence, ineffectual in bringing about learning.

The current work seeks to avoid deficiencies attributed to such systems by presenting the student a constructivist model of a knowledge domain based upon knowledge that has been elicited from an expert in the domain. The knowledge model is well situated in the context of a specific problem domain, rather than in generic first principles. The performance support component of the system assists with the solution of contextualized, real-world problems.

3. An Architecture for Performance Support Systems with Embedded Training

The increasing maturity and integration of network communications, multimedia, performance support and training systems makes possible a new architectural approach to network-based performance support and training systems. Figure #1 presents a graphic of our system architecture that includes:

(1)    a pedagogically-motivated multimedia browser

(2)    a consultation component for performance support

An integrated development environment for the creation of browser and consultation component is part of the system.

Figure 1: System Archiecture

3.1. The Pedagogically-Motivated Multimedia Browser

Concept maps, developed by Novak (1984), have been designed to elicit and represent part of a person's cognitive structure by externalizing concepts and propositions. A concept map is a planar representation of a set of concepts that makes interrelationships among them evident. Concept maps have proven to be useful for knowledge elicitation (Ford et al, 1991), and as an interface to a model of a knowledge domain (Ford, Cañas, and Coffey, 1993).

The pedagogically motivated browser is based upon a hierarchically structured representation of a knowledge domain, organized by concept maps. This pedagogically-motivated browser provides a unique way of organizing a model of expert domain knowledge. The knowledge model is stored on network servers that contain concept maps and other mediating representations (see Ford, et al, 1991) such as textual media, audio, video, still and animated graphics. The architecture of the knowledge model permits its components to reside in a central repository or to be distributed over a network.

The knowledge model is accessible from anywhere on the network. The pedagogically motivated browser can switch seemlessly from a knowledge model of one domain to a knowledge model of a related domain. Navigation links, based upon the occurrence of a given concept in various concept maps within a given knowledge domain, can be established to other such models. The knowledge model itself adopts a specifically Constructivist approach by affording the user a freely browsable knowledge space which the user may peruse until satisfied with the explanation.[2]

3.2. The Consultation Component

Knowledge-based systems (KBS) in general, and expert systems in particular, have traditionally been used for classification or diagnostic tasks. Our approach extends traditional knowledge-based systems capabilities in order to provide training as needed, and reference information to support job performance. The consultation component acts both as a consultant providing domain knowledge and as a diagnostic front-end in support of just-in-time training.

After the initial knowledge model has been elicited from the domain expert, the consultation component is created as a system of rules that maps to the expert’s diagnostic process. The consultation part can present a full complement of media in the form of examples to assist in the diagnostic interaction. The consultation component offers great promise to mitigate the need for access to human domain experts. By integrating pedagogically-motivated browsers with knowledge-based systems, just-in-time training is provided for the learner/practitioner who is not in a traditional setting - somebody, for example, who needs to learn how to fix a piece of equipment while on the job actually conducting the repair.

3.3. The Editing Component

The editing environment contains two parts. The knowledge model editor fosters collaboration between the expert and the knowledge engineer to create the multimedia knowledge model that forms the basis of the system. The knowledge model is based upon a hierarchy of concept maps. The knowledge engineer has the use of a model editor that allows for creation and linking of concept maps, and for the population of the maps with audio, video, text and graphics. The model editor is an outgrowth of work on ICONKAT (Integrated Constructionist Knowledge Acquisition Tool) (Ford et al, 1991) and an initial version has been implemented.

The consultation editor provides basic editing capability to create the rule bases. An object oriented approach has been adopted in which the rule object includes attributes such the various media that might be accessed by the user during an interaction involving that rule, as well as an entry point into the knowledge model, should the user seek greater elaboration of the question at hand in the consultation. The consultation editor is currently under development.

4. El-Tech

The first prototype developed with this system, El-Tech, models the knowledge of an expert electronics technician on the RD-379A(V)/UNH, a fault-tolerant magnetic recorder/reproducer manufactured by Magnasync/Moviola corporation.

4.1. Knowledge Elicitation for El-Tech

Knowledge elicitation proceeded along several dimensions simultaneously. First, it was deemed desirable to create a basic description of the RD-379A(V)/UNH, that documents its components, their locations, and how they individually and collectively operate. Second, in order to support the performance of deployed electronics technicians, the routine maintenance and checkout procedures had to be identified and described in the system. The third area of knowledge elicitation involved identifying the symptoms of problems that might be noted by the technician, and the diagnostic strategy that might be employed to repair the problem.

Knowledge elicitation started with an informal discussion during which the three areas were identified, and then proceeded with the creation of concept maps to elicit the important concerns of the expert in each area. An iterative process of refining the concept maps and creating and locating support materials such as schematics, block diagrams, video of the expert, etc, then ensued.

4.2. The El-Tech System Interface

Figure 2 presents a graphic of the system interface. The leftmost window in Figure 2 depicts an interaction with the advisory component of the system, in which the system is asking the user about the machine’s performance in “direct mode.” The system explanation of the question has been invoked and the “direct mode”, as it pertains to a malfunction has been located in the “Malfunctions” concept map. The user has opened a window that gives a textual description of the importance of direct mode in the diagnosis of the fault, and the user is currently viewing a digital video of the expert elaborating on the point.

The malfunction model is closely coupled with a model that describes the nominal operation of the system so that the user can easily navigate through a description of a component’s normal behaviors and failure modes. The knowledge model contains two hierarchies of concept maps, the first presenting the detailed description of the equipment, and the other the detailed account of how to diagnose and repair faults.

The knowledge model includes block diagrams of the equipment, electronics schematics, photos that show components and their locations, textual passages transcribed from interviews with the expert, digital video that illustrates checkout and fault isolation procedures, and references to supplementary information contained in external sources.

(click on image for a larger view)

Figure 2: El-Tech System

The inference component of El-Tech is a rule-based expert system, with rules culled from the knowledge model elicited from the expert. The inference component is implemented using JESS (Java Expert System Shell) (Friedman-Hill, 1997), which allows creation and maintenance of centralized knowledge bases on a network. The Fault Isolation knowledge bases were first developed in CLIPS (Riley, 1997). Our system includes a graphical interface for the consultation session that runs a Java Applet, to replace the command line interface of CLIPS. A distributed collection of servers house these components (the knowledge model and the inference component) together with all the other media, making the entire system available anywhere in-fleet via a network connection.

Users of the system can seek a simple advisory consultation in which the system asks the user a series of questions concerning symptoms to be found in the equipment. The inference component converges on probable failure points at the component level çç- individual transistors, relays, switches, motors, amplifiers, or even a wiring fault, typically after the user has answered 3 to 6 questions. At any time the user can ask for an explanation of the line of reasoning the system is pursuing. Upon such a request, the inference component will transport the user into the knowledge model where the user can view graphics of the schematics and literal pictures that show test points of the components under consideration, oscilloscope settings, wave forms, and values for which the user should look.

5. Summary and Future Work

This project links navigational and knowledge-based tools into an integrated, network-enabled system suitable for distance, just-in-time learning and performance support. This research has led to the production of a working prototype system (El-Tech) that assists electronics technicians in their job performance. This approach to computer-mediated learning decouples training from the scheduling and location constraints of traditional instruction. The development and deployment of systems like El-Tech will make it possible for electronics technicians to have access to knowledge when and where it is needed. This approach is generalizable and is currently being used in another project with the National Imagery and Mapping Agency (NIMA).

Future work will explore the feasibility of incorporating other forms of reasoning into the system, such as case-based reasoning. System problems in the Navy trigger an OpNav47902K trouble report that details the nature of the problem, how and when it was discovered, length of time to repair, etc. The Navy has an extremely comprehensive record of these reports. A retrieval system for previous failures could prove a useful adjunct to the current system. It is anticipated that application of this technology to other knowledge domains, might well reveal other domain-specific ancillary services that can be integrated into the system.

6. References

Bradshaw, J. M., D. Madigan, D., Richards, T. and G. Boy. (1993). Emerging technology and Concepts for Computer-Based Training, Proceedings of the 6th Florida Artificial Intelligence Research Symposium, Ft. Lauderdale, FL.

Collis, B. A., and Carla Verwijs. (1995). A Human approach to Electronic Performance and Learning Support Systems: Hybrid EPSSs. Educational Technology. January.

Ford, K. M., Stahl, H., Adams-Webber, J. R., Cañas, A.J., Novak, J.D., & Jones, J. C. (1991). ICONKAT: An integrated constructivist knowledge acquisition tool. Knowledge Acquisition Journal, vol 3, pp. 215-236.

Ford, K. M., Cañas, A. J., Coffey, J. (1993). Participatory Explanation. Proceedings of 6th Annual Florida AI Research Symposium (pp. 85-90). Ft. Lauderdale, FL.

Ford, K. M., Coffey, J., Cañas, A.J., Andrews, E.J., Turner, C. W. (1996). Diagnosis and Explanation by a Nuclear Cardiology Expert System, International Journal of Expert Systems, vol 9, number 4, pp. 499-506.

Friedman-Hill, E. (1997). Jess, the Java Expert System Shell. [online reference] http://herzberg.ca.sandia.gov/jess/

Novak, J. D. & Gowin, D .B. (1984). Learning How to Learn. Ithaca, NY: Cornell University Press.

Riley, G. (1997). CLIPS, A Tool for Building Expert Systems. [online reference] http://www.ghg.net/clips/CLIPS.html.

Shrock, S. A. (1995). A Brief History of Instructional Development. Instructional Technology, Past, Present and Future. 2nd edition. Gary Anglin ed. Libraries Unlimited, Englewood CO.

Wehrenberg, S. B. (1989). The Future Just-In-Time Work Force, Personnel Journal,. pp. 36 - 44. February.

[1]Presented at the of the Eleventh Florida Artificial Intelligence Research Symposium, Sanibel Island, Florida, (May 1997). Copyright 1998, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.