Abstract: Hypertext and hypermedia applications allow users to navigate through large sets of information in many different ways. It is impossible for an author to foresee all possible paths a user may follow. Adaptive hypermedia is a fairly new research field on the crossroad of hypertext (or hypermedia) and user modeling. Its goal is to improve usability of hypermedia through the automatic adaptation of hypermedia applications to individual users. Adaptive hypermedia systems (AHS) offer methods and techniques for adapting the content of information pages and the links between pages. A number of experiments have been conducted to demonstrate the benefits of adaptive hypermedia, mostly of adaptation of link structures. In this paper we argue that the benefits of using an AHS are a result of careful authoring, more than of the adaptive techniques themselves.
In a linear document (article or book) a single desired reading order is predefined. Readers always know where they are. When authors are writing a book and are adding pages they always know what they may expect the reader to have read when that reader reaches the page being written. In hyperdocuments this assumption is no longer valid. Given a rich link structure there are so many ways to navigate through a hyperdocument that it is impossible for an author to foresee which pages a user will have read when jumping to a certain page. Encyclopedia are a prime example of a type of hyperdocument that is written in such a way that the user can jump to any page, understand the information on that page and see links to other related pages that can also be understood. Most educational documents, especially textbooks, cannot be read in an arbitrary order because knowledge is built up gradually and (parts from) the first chapters contain prerequisite knowledge for the later, more advanced chapters. When the typical hierarchical structure of a textbook (with chapters, sections, subsections, etc.) is translated into a link structure it creates the illusion that the user can jump directly to any chapter or section she desires. However, prerequisite relationships between sections or chapters may result in users jumping to pages they cannot (yet) understand.
Adaptive hypermedia is a direction of research on the crossroads of hypertext (hypermedia) and user modeling. The goal of this research is to improve the usability of hypermedia. Adaptive hypermedia systems build a model of the goals, preferences and knowledge of the individual user and use and update this model throughout the interaction in order to perform adaptation of the hypertext to the needs of that user. A hyperdocument is adapted in two ways:
When hyperdocuments are adapted while the user is browsing and reading a new kind of confusion may be generated: when a user returns to a page she saw before the page may look different (even beyond recognition). Both the content and the embedded links to other pages may have changed. Authors need to balance this potential problem against the potential benefits of using adaptive techniques.
Adaptive hypermedia systems (AHS) have been used in many application areas, including on-line information systems [BE94,HBG96], help systems [FMRR90,FGQ93] and information retrieval systems [AFJM95,G97]. But the primary application of AHS has always been in education [BSW96, BES98, DC97]. (There are many more possible references, we have simply given a few of each type. In [B96] you can find more references.) When Vannevar Bush developed the concept of Memex [B45], he immediately extended the concept of hypertext links to include that of guided tours, which he called "trails". He saw the creation of trails as a potential new profession, the "trailblazer". Later, others have also promoted the use of guided tours as a way to use hypertext for communication [T88]. By combining hypermedia with information retrieval it has become possible to generate guided tours automatically. Museum information systems are a prime example of the application of generated guided tours. Based on artist names, styles, time periods, etc. these systems generate a linear presentation of artwork or artists. Adaptive hypermedia goes one step further: it brings non-linearity back to guided tours.
Several authors claim that adaptive hypermedia leads to better student performance [BE94,BP98,BES98]. However, this increase is not uniform: some students choose to ignore guidance, or even to always jump to pages the system suggests to avoid [BES98]. Also, whether following the guidance offered by an AHS actually helps depends on the "quality" of that guidance. Following bad advice may result in worse performance than trying your best without getting any advice. (We have no empirical data on this because all experiments have tried to show the benefits of adaptive hypermedia through the use of "good" guidance.)
Web-based education involves many different applications. Adaptive hypermedia is used mostly for interactive textbooks. Such applications offer information and may incorporate some tests. They do not include larger assignments, group work, message boards and other types of personal or collaborative support. As such, an AHS alone is certainly not enough to move a learning process from the classroom to the Web. The fact that this paper only discusses AHS is not meant to suggest that adaptive hypermedia is more than that component in a bigger environment. At the Eindhoven University of Technology we combine the use of our AHA adaptive system [DC98] with a HyperNews message board and a simple shared workspace platform [ABT98].
This paper first recalls the methods and techniques used in AHS, in Section 2, and illustrates some (possible) use in education. In Section 3 we describe how AHS are evaluated. We argue that AHS should not be evaluated by comparing them to non-adaptive hypermedia applications.
An excellent overview of methods and techniques can be found in Brusilovsky's survey paper [B96]. We briefly recall some of these definitions, add our own concepts, and illustrate them with educational motivations and examples.
Brusilovsky [B96] uses this term to indicate the adaptation of what is shown on a single screen or page. In reality there are two different aspects to this topic:
In current AHS the adaptation is always dynamic. Prerequisite explanations disappear when they are no longer needed. Comparative explanations appear not only on the page about the second concept but also on the first one, etc. This results in inconsistencies in the presentation which may confuse the reader. In [HH98] a different approach is taken: pieces of content (mostly additional explanations) are grayed out when the user is not advised to read them. Preliminary experiments have shown that users like this more stable form of presentation.
In order to achieve adaptive presentation several techniques exist. The two most common ones are:
The manipulation of links that are presented within nodes (pages) is typically done in one or more of the following ways:
The series of destinations indicated by the "next" button used with direct guidance is a linear guided tour. Depending on what a user (who does not always follow the "next" button) has read before the page to which the "next" button leads may be different for every user. The techniques of link annotation and link hiding offer some kind of non-linear guided tour. On each page the user sees a set of links that are recommended by the AHS, based on the user model. The links that are not recommended are either shown with a different annotation or are hidden, but they can be used if the user wishes to navigate against the advice of the AHS.
Some applications of AHS have been evaluated empirically. Such evaluations often consist of the comparison of user bahavior, satisfaction and/or performance using an adaptive versus a non-adaptive version of the same application. We briefly recall some such evaluations.
The most comprehensive evaluation of adaptive presentation in hypermedia was performed by Boyle and Encarnacion [BE94] with their adaptive stretchtext system MetaDoc. Three versions were compared: the original MetaDoc with all functionality and two "disabled" versions of MetaDoc: a stretchtext version which had all stretchtext functionality, but no user modeling and adaptation and a hypertext-only version which had no stretchtext functionality at all. (All the fragments were always shown, or "stretched".) Two kinds of tasks were used to compare these kinds of hypertext: eight reading comprehension tasks and five search and navigation tasks.
The experiment showed that the users of the adaptive stretchtext version found answers to comprehension questions significantly faster than users of the traditional hypertext version while showing significantly better comprehension. No significant difference was found regarding the performance in solving search and navigation questions (search correctness, number of visited nodes, and number of operations).
Brusilovsky and Pesin conducted one of the earliest studies of adaptive link annotation and link removal mechanisms using their system ISIS-Tutor [BP98]. Adaptive link annotation (color fonts and special symbols) was used to show classes of pages behind links. The study compared three versions of ISIS-Tutor: a non-adaptive version, a version with adaptive annotation, and a version with both adaptive annotation and adaptive link removal (links to non-recommended pages were removed.) The study (performed with 26 computer science freshmen) showed that the overall number of navigation steps, as well as the number of unforced repetitions of concept and task pages were significantly lower with both adaptive versions of the system. No difference was found between the link annotation with or without the additional link removal technique. No difference was also found between all three groups for the quality of mental maps of the hyperspace developed by students.
A similar study involving 25 undergraduate "teacher education" students was performed with the InterBook system [BES98], a Web-based descendant of ISIS-Tutor. The study compared two versions of the system: with and without adaptive link annotation. The study showed that link annotation encourages the novices to use non-sequential links more often. The study also showed that the students who follow the system's guidance are able to achieve better post-test scores.
An informal evaluation of the AHA system [DC98] in a hypermedia course showed that among the link adaptation techniques students disliked link removal and link disabling, which force users to follow guidance, and offer no way to ignore that advice. This shows that while the experiment with ISIS-Tutor shows that enforcing guidance may improve user performance it decreases user satisfaction.
Most (if not all) evaluations of adaptive systems are intended to justify the use of adaptive hypermedia in a given application. The adaptive application is usually built first. Then a second version is generated from the adaptive one by simply disabling all adaptive features. All content fragments are shown, and all links are shown without annotations. The two versions are then compared through user testing. We argue that this is not a fair comparison because the non-adaptive version of the application is not well designed and thus put at a disadvantage right from the start. We give some examples of design issues that are different in adaptive hypermedia applications versus non-adaptive applications.
From these examples we hope it is clear that in order to make a fair comparison between an adaptive and a non-adaptive application the non-adaptive version cannot simply be generated from the adaptive one but must be carefully redesigned, using the same information content. Such redesign is often not done because of lack of resources. The experiments described in sections 3.1 and 3.2 thus do not compare static and adaptive hypermedia applications in a fair way.
Adaptive hypermedia applications are very different from non-adaptive ones. Adaptive hypermedia techniques have many potential benefits and drawbacks:
AHS offer a new set of techniques for automating some of the implementation issues a hypermedia author is confronted with, such as the decision when some information becomes relevant, and when a link to some page is interesting for the user. Comparing adaptive hypermedia to non-adaptive hypermedia is like comparing a bicycle to a motorcycle. Of course there are differences you can find through empirical evaluations. You will find that a motorcycle runs faster, that it requires less physical effort from the user, and that it does not offer the same level of feedback on whether the road goes uphill or downhill, whether there are small bumps, etc. Of course it makes much more sense to compare different motorcycles. In the same way it makes sense to compare different adaptive methods and techniques, rather than comparing the use of "any" adaptive technique versus not using it.
Adaptive hypermedia has the potential to offer learners some freedom in the order in which they wish to study course material while ensuring that the presented material is always relevant and can be understood. However, whether this potential is reached depends on careful authoring. Adaptive hypermedia is not a magic wand but a powerful set of methods and techniques that can be used or abused.
More research is needed, especially at the conceptual level, to find out how authoring adaptive hypermedia can be made easier. Also, more evaluations need to be done to find out which adaptive techniques offer the best way to improve on both user performance and user satisfaction.