Peter Gram
Pensacola Junior College

Jason Muldoon
Department of Psychology
University of West Florida

Abstract

Finding relevant information on the Internet can be a daunting task that would be enhanced if the material were organized and could be accessed in an efficient manner. Browsers based on a concept map-based interface and on a World Wide Web page-based interface were compared for ease in finding information necessary to answer a series of search questions based on the same domain material (developmental psychology). Users differed in the amount of concept map training they received and the type of learner they tended to be (meaningful vs. rote learners). The results indicated that the concept map-based interface resulted in better search performance for all learners although this difference was most pronounced for meaningful learners. Training in concept map construction appeared to have no more effect on search performance using the concept map-based interface, than control conditions. Taken together, the results suggest that organizing information via a concept map-based interface leads to more accurate search performance than the typically used web page-based browser.

Introduction

World Wide Web (from here on referred to as web) pages are the major technique used to organize and browse for information on the Internet. However, the web page is based on the now quite old paradigm of a printed page. Although the page is a very effective way of communicating printed information, there is no particular reason (except the comfort and convenience of using a known paradigm) that information on the Internet needs to be organized and browsed as web pages that mimic the printed page. Furthermore, we have taken the page paradigm further: web sites, like books, tend to be organized hierarchically, including a linear table of contents. Links between web pages are usually based on the words or phrases on the text itself that serve as clickable spots, leaving any semantics regarding the nature of the link and the target resource to the text itself, or to the entries in the table of contents. As a result, disorientation is common while navigating through the World Wide Web, since one never knows where a link will lead to until one clicks and follows the link (e.g., Otter & Johnson, 2000). Thus, obtaining information may require a protracted search because several links or web pages may have to be traversed, often with backtracking to previous pages, before the desired information is found.

Concept maps provide a new method for organizing and browsing through information and may be an effective navigational tool for hypermedia environments (Cañas, Ford and Coffey, 1994; Ford, Cañas and Coffey, 1993). Because of their graphical nature, concept maps may make desired information more accessible. A concept map (Novak, 1984) is a hierarchically organized, two-dimensional representation of a set of concepts and the labeled relationships among them with the most superordinate concept being placed at the top of the hierarchy. Concept maps can also contain crosslinks, which represent relationships among sub branches in the hierarchy.

The concept map interface used in this experiment was constructed using the concept mapping tools software package (CmapTools) developed by the Institute for Human and Machine Cognition. This software extends the use of concept maps beyond knowledge representation, to serve as the browsing interface to a domain of knowledge. Figure 1 shows the concept map-based browser as the interface to a multimedia on plants. The icons below some of the concepts provide access to complementary or auxiliary information to explain the concept in the form of pictures, images, audio-video clips, text, web pages, or to other concept maps that expand on the topic or in which the particular concept appears.

Figure 1: A Concept Map about Plants with Associated Resources

These linked media resources and concept maps can be located anywhere on the Internet. In Figure 1, the icon underneath the Minerals concept in the Plants concept map has been selected, and the names of a related concept maps are displayed, indicating that there is a concept map about Minerals (within the same context of plants) available, and that there is also a concept map about minerals resources in mars linked to this map. The Honeybees map in the back was displayed as a result of selecting the concept map icon under the Bees concept. The Flowers concept has icons for images and video (a blooming iris video is displayed to the left). Displayed also in Figure 1 are a text about leaves, a web page about photosynthesis and images of leaves and fruits, all linked to the Plants concept map.

Although concept maps have mainly been used to assess an individual's specific knowledge or cognitive structure, they have also been employed to present information to a student or group of students as an advance organizer (Ausubel, 1968; Wandersee, Mintzes & Novak, 1994; Willerman & MacHarg, 1991). The concept map-based browser evaluated goes further, through not only presenting the concept map-based knowledge model constructed by an expert to the user, but also exploiting it as the navigational tool by which the user browses and searches through a multimedia of the domain.

Little research has been conducted to determine if a concept map-based browser (or interface) is better for browsing and searching for information than a web page-based browser. The only data we are aware of is from pilot research conducted in our Institute that showed a nonsignificant trend (p=.057) favoring concept maps over both web pages and list interfaces for finding relevant information (Carnot, Dunn, Cañas, Baker, & Bense, 1999). The list interface used in this pilot study was a semi-hierarchical list of the major concepts in the domain with links to related information. It was roughly equivalent to a limited outline of the concepts that were emphasized in the concept map-based and web page-based browsers.

The only related literature concerning browsers similar to concept maps was based on knowledge maps or knowledge hypermaps. Rather than comparing knowledge maps with equivalent web page interfaces, all comparisons were made using either a list interface or hotwords within hypertext to link to related material (e.g., Reynolds & Dansereau, 1990; Hall, 1997). Traditionally, knowledge maps or node-link maps have been considered to be similar to concept maps in that they also contain concept-like nodes and labeled links between nodes (Chmeilewski, Dansereau & Moreland, 1998; Lambiotte, Dansereau, Cross & Reynolds, 1989). However, there are some significant differences between concept maps and knowledge maps. Knowledge maps restrict linking terms to a limited set, and often present them in abbreviated format. Information in nodes can be much larger than a single concept (e.g. a paragraph, Newbern, Dansereau & Patterson, 1997). Another difference is that concept maps are conceived to be primarily hierarchical in nature, while knowledge maps are not restricted to a hierarchical format (Lambiotte et al., 1989). There is evidence that when students learn to construct their own knowledge maps, it can help in the recognition and recall of the material studied (e.g., McCagg & Dansereau, 1991) and new material (Chmeilewski & Dansereau, 1998).

However, there is little evidence that when knowledge maps are used as browsers, they lead to better search performance or recall of material than list interfaces. For example, Reynolds and Dansereau (1990) found no difference in accuracy and time spent answering reference (search) questions, between participants using knowledge map-based interfaces and hypertext-based interfaces. Like our pilot study (Carnot et. al., 1999), there was a nonsignificant trend favoring those using knowledge maps over hypertext or list interfaces.

In fact, most significant positive results reported using knowledge maps as browsers are related not to the participants' performance but to their subjective feelings. Users reported less frustration and confusion when using a knowledge map-based interface rather than a hypertext-based interface (Reynolds & Dansereau, 1990). In related research by Hall and colleagues (Hall, 1997; Hall & Stocks, 1998), users were asked to browse or study rather than search for specific information in a given amount of time. Generally, these results showed that participants felt as if they had covered more of the underlying material, and that they had a better overview of the material. However, they did not necessarily recall more details of the material than users of a standard hypertext (list) interface (Hall & Stocks, 1998).

The novelty of the concept map-based interface for first-time users raises the question of whether training is required to effectively use concept maps and related representations. The proponents of knowledge maps have suggested that effective processing of knowledge maps requires training in map reading and construction (e.g., McCagg & Dansereau, 1991). If true, concept map training might be necessary when searching through large or complex concept maps. The current experiment addresses whether training and practice in concept map building can teach strategies that lead to improved search performance when using a concept map-based interface.

Use of concept map-based browsers and concept maps in general also may be related to a person's tendency toward meaningful learning. Meaningful learning requires that a learner put new information into an existing cognitive structure in a nonverbatim or non-rote manner, thereby allowing better use and retention of the knowledge. According to Novak (1998), meaningful learning requires: relevant prior knowledge, meaningful material and the choice of the learner to engage in meaningful learning. Previous research has suggested that people can be divided into meaningful and non-meaningful learners. Meaningful learners construct concept maps containing more concepts, more relationships and more branching than rote learners (Pearsall, Skipper & Mintzes, 1997). The question then becomes: which interface (web page or concept map) provides the most efficient search space for high and low meaningful learners?

The present experiment compares a concept map-based interface to a web page-based interface in terms of accurately finding information using search questions derived from the same instructional information. Both interfaces were developed using input from two domain experts, and were based on an overview chapter on developmental psychology. It was assumed that the concept map-based interface would serve as an advance organizer allowing subjects to readily search information. Even though our pilot work (Carnot et. al., 1999) showed a nonsignificant trend favoring concept map-based over list-based and web page-based interfaces, it should be noted that the n's per cell were extremely small. With a larger sample, carefully designed concept map-based browsers should provide better access to relevant information than web page-based interfaces in which the same information is covered. In the present study, these interfaces were systematically compared.

Further, the necessity of training was addressed by teaching one group of participants how to construct concept maps prior to the experimental session. (Control subjects were given other learning tasks). Finally, differences in the search performance of high and low meaningful learners were examined across the two interface conditions using a questionnaire developed by Donn (1990).

It was hypothesized that most people will correctly answer more search questions using the concept map-based interface (primarily because of its graphical nature) than the web page-based interface. Training in concept mapping will have a small but measurable effect on search performance. Finally, meaningful learners' best search performance will occur when using the concept map-based interface.

Method

Participants and Class Assignment

Sixty-five introductory psychology students were recruited from three introductory psychology classes, and received extra credit for their participation. In order to investigate the effects of prior experience with concept mapping, different existent classes were used. Since students could not be randomly assigned to a given class due to personal and working schedules, the type of instruction to a given class was randomly assigned. The three class types used, concept map, learning objective, and quiz classes (described below), yielded 20, 19, and 29 volunteer participants, respectively. In order to determine if the students in the three classes were reasonably equivalent in learning ability, motivation, etc., the students' overall GPAs were compared using a one-way between-subjects ANOVA. No significant differences in the classes' GPAs were obtained, (F (2, 61) = 1.80, p >.17). The classes were taught by the same instructor, had the same textbook and course materials, but had course assignments that differed across classes to test the effectiveness of different study methods. These study methods were the only differences in instruction across the classes. Evaluation of student activities related to each method accounted for 12% of a student's final grade.

Class Type

Concept Map Class

At the beginning of the semester, students enrolled in the class that used concept maps were given a one-hour, in-class training session on how to create a concept map. The professor introduced the important ideas needed to build a concept map, showed them how to distinguish among concepts, activities and linking words and how to hierarchically arrange concepts. The professor then made a sample concept map after which students broke up into groups of 3-4 people, and developed a "group" concept map based on a section of the book. It was also suggested that they do additional work at home. During the next class period, students spent 30 minutes reviewing the ideas involved in building concept maps, and practiced concept map construction.

During the term, participants were required to construct concept maps for each assigned chapter. Typically, two or three chapters were covered before the students were tested on that unit's material. Each unit test was composed of 40 multiple-choice questions and one short answer question. Students were required to turn in their concept maps before taking the test. If they failed to do so, they had to take a makeup exam after completing and turning in the concept maps. There were six unit tests requiring a total of 13 concept maps to be generated by each student independently during the semester (which again, accounted for 12% of their final grade).

Learning Objective Class

The students in the learning objective class were required to turn in answers to learning objectives assigned for each chapter in order to earn the 12% for their final grade. The learning objective questions were taken from the textbook's commercially available student study guide. For each assigned chapter, there were approximately 20 relatively global (as contrasted to detail) questions to be answered. Like students in the concept mapping class, students were required to turn in their answers to the learning objectives before taking each unit exam.

Quiz Class.

Students in the quiz class had chapter quizzes (also taken from the class text's student study guide) before each chapter was covered in class. Students were encouraged to study before each class. Students in these classes took a total of ten quizzes, but were allowed to drop the lowest four scores

Throughout the semester, students in the concept map and learning objective classes were given approximately 15 minutes per week to work on their concept maps or learning objectives in class in order to equate for the time taken by quizzes in the quiz class. Concept map, quizzes and answers to learning objectives were scored on a ten-point scale. Scoring for concept maps and learning objectives was lenient and primarily intended to encourage out of class work; scoring for quizzes was standard in terms of rewarding correct answers only. For the first two (of six) sets of concept maps and learning objectives, feedback by the professor was provided concerning concept maps and learning objectives and how to improve them. After the first two sets, little feedback was given, except for those students requesting help.

The Interface Conditions

Two developmental psychology experts modified the information contained in a general psychology text to aid in the construction of the concept map-based and web page-based interfaces. The chapter on human development came from the text used by those members of the three classes who volunteered to participate in the experiment. The development chapter was not required reading for the course. The concept maps and web pages were built from the definition lists at the end of the chapter, from the chapter outline and headings, and from the chapter itself.

Concept Map-based Browser

As mentioned earlier, the concept map-based browser was constructed using the software package CMapTools developed at the Institute for Human and Machine Cognition. In the current experiment, the concept map-based organization of the information consisted of three hierarchical levels of concept maps. Figure 2 shows the first level or "Overview" map, which provides links to the maps on developmental time periods, including prenatal, infancy and childhood, adolescence and adulthood.

Figure 2: Top Level Concept Map: Overview of Development

Level Two contained the developmental time periods per se (Figure 3). The individual concept maps covering physical, social-emotional, cognitive, and moral development in infancy and childhood were the Level Three concept maps (see Figure 4). Moral development was also covered in a Level Three concept map for Adolescence and Adulthood. Within the concept maps, information was arranged in a hierarchical manner, with higher level or more inclusive concepts occurring higher on the screen than lower level or more specific concepts.

Figure 3: Level Two Concept Map: One of the Developmental Time Periods

Figure 4: Level Three Concept Map: Cognitive Development in Infancy and Childhood

Web page-based Browser

Web pages were created using Adobe Pagemill ™ to cover the same material in the concept map-based interface, both of which paralleled the information contained in the textbook chapter. Figure 5 shows the top level web page that corresponds to the concept map presented in Figure 2. Wording in both the concept map-based and web page-based interfaces was made as similar as possible without violating normal sentence structure and related content of the web pages. Accessing resources in the concept map-based interface required more time than loading pages in the web page interface (due to the early version of the CMapTools software used). Loading time was equated by using software that added a delay to web pages.

Figure 5: Top Level Web Page

Search Questions

A set of 76 search questions were developed based on the concept maps and web pages. Most of these questions had several parts, which were scored independently, for a possible total of 189 points. Questions were designed to address material at different levels of meaning in the concept map (Levels 1 - 3, above). Consequently, the search questions reflected the hierarchical nature of the concept map-based and web page-based interfaces by sampling information at each of the three different meaning levels: high level or gist (Level One), medium level or supportive (Level Two) and detail (lower level) information. For example, a question that asked about the four developmental time periods would be a gist level question requiring a four-part answer (prenatal, infancy and childhood, adolescence and adulthood). A question that asked for the names and times of phases within the prenatal time period would be a supporting level questions requiring six parts, three for naming the periods (germinal, embryonic and fetal) and three for responding with the time periods associated with the phases. A question that asked for information about a specific type of change within a time period (i.e. childhood) was considered to be a Level 3 question if it appeared in a separate web page or concept map. Thus a question asking for names and descriptions of Kohlberg's three levels of moral development was a Level 3 question with six parts, again three for the names of each of the levels and three for the descriptions of these levels. Search questions matched the order of information presented in the web pages and concept maps.

Procedure

Prior to running, participants read and signed a consent form if they opted to volunteer. They then completed the Learning Approach Questionnaire (LAQ) developed by Donn (1990). The LAQ was used to determine whether a student was a meaningful learner (defined as one who constructs their own knowledge base) or a rote learner (defined as one who typically memorizes and retrieves information in the order in which it is presented).

Participants were run in groups ranging from 5-20 people in a student computer lab. In most cases, all people in the group were run under the same interface condition so that group instructions could be given. For the last session, approximately half of the group was run in each interface condition to even out the distribution of class types across conditions. Consequently, instructions were given separately to the two interface groups.

Instructions were given using a projector to demonstrate the key elements of a web page and a concept map. For both interfaces, the instructions made use of a simple example, and emphasized the use of resources, and the mechanics of following links. In all sessions, the experimenters walked around the room so that they were available for any problems or questions that participants might have. Participants were given 50 minutes to answer as many search questions as they could. They were instructed to answer questions in order, and to avoid skipping questions, but that if they did skip a question, they should not go back to it.

Results

Due to the hierarchical nature of the concept map-based and web page-based interfaces, the data were analyzed as a function of the level of semantic importance a specific search question required, i.e., high level (or gist), medium level (supportive) and lower level (detail) information. Most participants were unable to answer all of the search questions because of time limitations. Thus, the dependent variable was the number of search questions correctly answered at each meaning level divided by the number of questions actually covered at that level by a given participant. For example, if a subject attempted 10 of the gist questions, but only answered 4 correctly, that percentage would be 40. If they attempted 12 of the supporting level questions and answered 4 correctly, that percentage would be 33, etc. Examination of the range and the variability of the percentage scores at each meaning level indicated that they were basically equivalent (Range: gist 47.4 - 100, supporting, 52 - 100, detail, 49.0 - 100; SD: gist, 13.36, supporting, 12.57, detail, 12.24).

Furthermore, since research suggests that meaningful learners construct concept maps that are more complex than non-meaningful learners (Pearsall, Skipper & Mintzes, 1997), we assumed that they might be able to search browsers based on experts' concept maps more efficiently than non-meaningful learners. Consequently, participants were divided into high and low meaningful learners based on a median split of on their Learning Approach Questionnaire scores (Donn, 1990). Their correct percentage search question data were then analyzed using a four-way, learning style (high vs. low) by interface (concept map-based vs. web page-based) by class (concept map, learning objective, and quiz) by information meaning level (gist, supporting and detail) mixed ANOVA with meaning level being the only within subject variable.

The ANOVA yielded a significant main effect of meaning level [F (2, 94) = 7.14, p = .001], with a greater mean percentage of high-level questions answered than lower level questions (gist, M = 90.81; supporting, M = 85.23; and detail level M = 79.89). Several two-way interactions were found to be significant, including the interface by class interaction [F (2, 47) = 7.23, p = .002], the learning style by interface interaction [F (1, 47) = 5.22, p = .027], the interface by meaning level interaction [F (2, 94) = 7.02, p = .001], and the learning style by meaning level interaction [F (2, 94) = 3.154, p = .047]. The only other interaction to reach significance was the three-way learning style by interface by meaning level interaction [F (2, 94) = 3.12, p = .049].

Figure 6 shows the interface by class interaction, which suggests that participants who had quizzes and concept maps as their class assignments answered a greater mean percentage of search questions when using the concept map-based interface than when using the web page-based interface. No difference appeared to occur as a function of interface for the participants who completed learning objectives as their class assignments.

Figure 6: Mean percent correct as a function of interface and class

The interface by learning style interaction is shown in Figure 7. Although slight, it indicates that high meaningful learners' mean search performance was affected more negatively when using the web page-based interface than low meaningful learners' search performance.

Figure 7: Mean percent correct as a function of interface and learning style

Figure 8 shows the significant interaction between interface and meaning level, and suggests that at the supporting and detail level, those participants who used the concept map-based interface had higher mean search performance than participants who used the web page-based interface.

Figure 8: Mean Percent Correct as a Function of Interface and Meaning Level

The learning style by meaning level interaction is shown in Figure 9. There was a slight tendency for high-meaningful learners to have worse mean search performance at the supporting level than low-meaningful learners. At the gist level, however, high meaningful learners' performance was somewhat better.

Figure 9: Mean percent correct as a function of learning style and meaning level

For ease of explanation, the important three-way learning style by interface by meaning level interaction is shown in Figure 10, as two 2-way components as a function of meaning level and interface for low and high meaningful learners. This interaction suggests that high-meaningful learners searched most efficiently when using the concept map-based interface rather than the web page-based interface at each of the meaning levels. With the exception of the gist level, low meaningful learners also had higher mean search scores when using the concept map-based interface than when using the web page-based interface.

Figure 10: Mean percent correct as a function of meaning level and interface for low and high meaningful learners

Dicussion

This experiment addressed three major questions: a) Can a concept map-based browser, developed by a domain expert, help people search information more accurately than searching the same information using a web page-based browser? b) Does previous training in the construction of concept maps increase subsequent search efficiency when using a concept map-based browser? c) Are individual differences in meaningful learning related to search efficiency?

The interface by meaning level interaction provides an affirmative answer to the first question. Regardless of class training and individual differences in meaningful learning, most participants searched out more information and searched it more accurately when using a concept map-based browser than when using a web page-based browser.

In answer to the second question, the class training by interface interaction indicated that training in concept map construction had little or no effect on search efficiency. This is somewhat surprising given research which showed that the effective processing of knowledge maps depended on previous training (McCagg & Dansereau, 1991), and that training in knowledge map construction provided implicit transfer to other reading tasks (Chmeilewski & Dansereau, 1998).

Both the concept mapping and quiz groups' search performance was better when using the concept map-based interface than when using the web page-based browser. However, those trained in answering learning objectives, appeared to do equally well when using either the concept-map-based or web-page-based browsers. More specifically, the learning objective group's performance did not differ from those receiving concept map training or quizzes. Yet, they had equally high performance when using the web page-based interface. The reason why the learning objectives group did equally well using both interfaces is unclear. A post-hoc examination of the learning objective questions suggested they were somewhat similar to the search questions developed for the experiment. Whether this similarity in questions could account for these results requires further research.

The results of the three-way, learning style by interface by meaning level interaction provide an answer to the final question concerning individual differences in meaningful learning. It suggests that meaningful learners when using the concept map-based interface successfully searched out more information at all meaning levels than when they used the web page-based interface. Interestingly, with the exception of the gist level, so did low meaningful learners.

Taken together, the results of this experiment suggest that most individuals find it easier to access material that is relevant to them when information is browsed using a concept map-based interface than when it is organized using web page-based interface. The use of a concept map-based interface appears to require little or no training, and appears to benefit both high and low meaningful learners, although there is some evidence that meaningful learners can more easily find gist information than non-meaningful learners. Future research will be required to determine whether these findings are robust using different domains (e.g., statistics and mathematics) and different types of knowledge (e.g., procedural knowledge). At a minimum, our results justify further work in the area.

References

Ausubel, D.P. (1968) Educational psychology: A cognitive view. New York: Holt, Rinehart and Winston. (5)

Cañas, A. J., Ford, K.M., Coffey, J. C. (1994) Concept Maps as a Hypermedia Navigational Tool, Seventh Florida Artificial Intelligence Research Symposium, Pensacola, FL (2)

Carnot, M.J., Dunn, B., Cañas, A.J., Baker, G. and Bense, J. (1999) The Effectiveness of Computer Interfaces in Information Search. Paper presented at the Southeastern Psychological Association, Savannah, GA.(8)

Chmeilewski, T., Dansereau, D., & Moreland, J. (1998). Using common region in node-link displays: the role of field dependence/independence. Journal of Experimental Education, 66(3), 197-207.(11)

Chmeilewski, T.C. and Dansereau, D.F. (1998) Enhancing the recall of text: Knowledge mapping training promotes implicit transfer. Journal of Educational Psychology, 90(3), 407-413. (15)

Donn, J.S. (1990) The relationship between student learning approach and student understanding and use of Gowin's Vee in a college level biology course following computer tutorial instruction. Unpublished doctoral dissertation, Cornell University, New York. (19)

Ford, K. M., Cañas, A. J., & Coffey, J. C. (1993) Participatory Explanation, Proceedings of FLAIRS 93: Sixth Florida Artificial Intelligence Research Symposium, Ft. Lauderdale, FL, pp. 111-115. (3)

Hall, R.H. (1997, October). Guided Surfing: Development and Assessment of a World Wide Web Interface for an Undergraduate Psychology Class. Presentation at the third international North American Web Conference, Fredericton, New Brunswick. http://www.umr.edu/~rhall/naweb97/titlemap.html (10)

Hall, R.H. & Stocks, E.L. (1998, November) Guided surfing: A multimethod assessment of a layered hypermap WWW interface. Presentation at WebNet98:World Conference on the WWW and Internet, Orlando, FL http://www.umr.edu/~rhall/webnet98/ (16)

Lambiotte, J.G., Dansereau, D.F., Cross, D.R. and Reynolds, S.B. (1989) Multirelational semantic maps. Educational Psychology Review, 1(4), 331-367.
McCagg, E.C. and Dansereau, D.F. (1991 A convergent paradigm for examining knowledge mapping as a learning strategy. Journal of Educational Research, 84(6), 317-324. (12)

Newbern, D., Dansereau, D.F. and Patterson, M.E. (1997) Spatial-semantic display processing: The role of spatial structure on recall. Contemporary Educational Psychology 22(3), 319-337. (13)

Novak, J.D. and Gowin, D.B. (1984) Learning how to learn. New York: Cambridge University Press. (4)

Novak, J.D. (1998) Learning, creating and using knowledge: Concept maps as facilitative tools in schools and corporations. Mahwah, NJ: Lawrence Erlbaum Associates. (17)

Otter, M. and Johnson, H. (2000) Lost in hyperspace: metrics and mental models. Interacting with Computers, 13, 1-40. (1)

Pearsall, N.R., Skipper, J. and Mintzes, J. (1997) Knowledge restructuring in the life sciences: A longitudinal study of conceptual change in biology. Science Education, 81(2), p 193-215. (18)

Reynolds, S. and Dansereau, D. (1990) The knowledge hypermap: An alternative to hypertext. Computers in Education, 14(5), 409-416. (9)

Wandersee, J.H., Mintzes, J.J. and Novak, J.D. (1994) Learning: alternative conceptions. In D.L. Gabel (Ed.) Handbook on research in science teaching. (pp 177-210) ew York: Macmillan. (6).

Willerman M. and MacHarg, R. (1991) The concept map as an advance organizer. Journal of Research in Science Teaching, 28(8), 705-711. (7)

Also affiliated with the Department of Psychology, University of West Florida.