Incorporating learning styles in hypermedia environment: Empirical evaluation

Namira Bajraktarevic¹, Wendy Hall¹ and Patrick Fullick²

¹Intelligence, Agents, Multimedia Group
Dept of Electronics and Computer Science
University of Southampton
{nb99r, wh} @ecs.soton.ac.uk

²Research and Graduate School of Education, Southampton
plf@soton.ac.uk

Abstract

In this paper a hypermedia system that incorporates global and sequential learning styles is described. Elements of learning styles such as the sequencing of pages and structure have been incorporated in the courseware. Two different presentation style user interface templates were employed. The system provides an alternative to the “one size fits all” approach to development of web-based educational courseware, by creating learning materials to cater for individual learner preferences. The Felder-Solomon Learning Style Questionnaire was used to measure the learning style preferences of students. Information recall tests were created and scores that were obtained after matched^[1] and mismatched^[2] sessions have been compared. Significantly higher results were obtained for the matched session compared with the mismatched session. The findings are discussed with respect to their implication for further development of adaptive web based courseware.

1. INTRODUCTION

This document presents the results of the empirical-analysis of the experimental study relating to the application of learning styles in educational hypermedia and their effects on learning outcomes. The analysis intends to compare the post-test achievement score differences between matched and mismatched sessions of courseware and evaluate the correlation between the increase in post-test achievement scores and the session browsing times.

2. LITERATURE REVIEW

When designing instructional material, it is important to accommodate elements that reflect individual differences in learning. One such element addressed in this study is a learning style. There are a variety of definitions and interpretations of learning styles available in literature but in this paper the term learning style refers to “adopting a habitual and distinct mode of acquiring knowledge” as summarised by McLoughlin (1999). There are many models of learning style but this study focuses on the two learning styles that fall into the cognitive and information-processing model. Individual differences are observed by the way students interact with hypermedia based learning materials. It is important to study how the structure and organisation of information may contribute to a better learning in relation to individual differences.

2.1 Learning styles

Pask (1976) describes two particular learning styles: holist and serialist. He describes them in the following way: "holist learners learn in layers. They prefer an overview of where they are going first before learning a complex process. They like having a map, knowing where they are headed and what they are working toward. They enjoy having examples shown to them even if they are not capable of imitating the skill yet. Holist learners sometimes get confused by step-by-step instructions, especially if the steps are numerous and complex. They pick up bits and pieces within a broad framework. They may leave gaps, repeat themselves and over generalise. They may also be more comfortable with “topic” based learning". In addition, for serialist learners Pask said that such learners "find introductory overviews distracting and confusing. They expect to learn whatever they are shown immediately or they become frustrated because they don't have the ability of the global learner to see ‘the big picture’. They prefer to proceed step-by-step, in an orderly way, to the end result. They build their knowledge sequentially, they are impatient with ‘jumping around’ and they may lose sight of the broader picture. They may be more comfortable with inherently ‘linear’ subjects".

Serialist learners are in the majority, and most educational materials are laid out in a sequential rather than a global way. Pask has demonstrated that a mismatch between the teacher and learner’s styles seriously hinders the learning process. Riding and Cheema (1991) use wholist/analytics terminology to describe how individuals process information. Wholists, according to them, organise information into loosely clustered wholes, in order to construct an overall understanding of the given information. Analytics, in contrast, process information in clear-cut conceptual groupings. Felder and Silverman (1988) describe different individuals as "global" and "sequential". Global individuals are classified as holistic, systems thinkers, learning in large leaps. Sequential individuals are classified as linear, orderly and learning in small incremental steps. Clarke (1993) suggests that all these styles ‘differ more in name than nature’ and that they ‘can be classified into either a preference for a reasonable degree of structure and guidance [serialist] and a preference for considerable freedom to explore [holistic]’. This paper seeks to explore the relationship between matching and mismatching learning style preferences in hypermedia material based mainly on the theoretical work of the above-mentioned researchers.

2.2 Previous attempts at applying learning styles in hypermedia

A number of researchers have promoted constructivist approaches to the design of computer based learning materials, whereby courseware is designed with the priority on individual student requirements and styles, as opposed to the tutor-led factors inherent in an instructionist approach. Additionally, researchers have addressed the issue of tailoring the design of learning activities to match an individual’s learning preferences, in the context of computer based learning materials. This section presents some of the previous attempts at incorporating the wholist/analyst (global/sequential, holist/serialist) dimensions of learning styles into computer and hypermedia assisted instruction.

Kwok and Jones (1995) carried out an experimental study with a computerised ‘front-end’ study preference questionnaire (based on Ford, 1985) in order to suggest to the user a suitable navigation method through the system. They found that students at the far extremes of the learning style spectrum needed the navigational guidance, and it helped raise their interest in the material. Pillay and Willss (1993) reported the study where students with wholist/analytic cognitive style received instruction that matched and mismatched their cognitive style. The matched group performed better in the explanation and problem solving tasks. Riding and Sadler-Smith (1992) investigated an interaction between mode of presentation and style and their effect on learning performance. They believed that structure and organisation of the contents might interact with the wholist/analytical dimension of style. Their conclusion was that the mode of presentation has important effects on learning performance. Ford and Chen (2001) explored relationship between match and mismatch of instruction presentation style with student’s cognitive style (field dependent and field independent). They have found significant differences in performances on conceptual knowledge for students under two different conditions. Graff (1999) tested the relationship between three different hypertext structures (linear, hierarchical and relational) and the performance of the students with wholist-analyst cognitive styles. He suggested that providing different linking structure to individuals of different cognitive styles would make the learning from hypermedia more effective. No significant differences on recall of information were found. The experiment conducted as part of this study is most similar to Graff’s paper, in that an attempt has been made to incorporate two types of linking structure in the hypermedia material (liner and hierarchical), that will match sequential and global learning preferences. These linking structures were imposed on students by providing a different sequence for browsing the courseware, so that sequential students were provided with a linear sequence, while global students were allowed to ‘globetrot’ and had much more navigational freedom in the system.

3. ADAPTIVITY AND APPLICATION OF LEARNING STYLES

With application of individual differences in hypermedia such as the cognitive and learning style a few adaptive hypermedia systems were created recently, such as INSPIRE (Grigoriadou et al, 2001), CS388 (Carver, 1996), RAPITS (Woods and Warren, 1996) and AEC-ES (Triantafillou et al., 2002). In INSPIRE the authors adopted Kolb’s theory of experiential learning (activists, pragmatists, reflectors and theorist learning styles). Their user model consists of two parts: general information about user (age, sex) and current knowledge level unit. As the users progress through the system, they are monitored. Lessons are divided into layers and generated dynamically. There is also a presentation module responsible for modifying the appearance of knowledge modules. The authors used adaptive presentation techniques for different order of knowledge-modules. Learning style elements appear or do not appear inside a lesson. Adaptive navigation techniques such as annotation of the links in the ‘navigational’ area of the system are used. Students‘ progress is indicated in metaphoric form (filling glass). The links recommended by the system change from black and white to coloured. Carver (1996) created a system (CS388) that consisted of a range of learning style tools. The learning styles were assessed using the Felder-Silverman learning style model. Students were allowed to traverse the courseware according to their own learning style. Carver’s paper couples the Felder-Silverman learning styles with hypermedia to provide tailored lessons. In this approach the key was to determine what type of media are applicable and appropriate to different learning styles (such as graphs, movies, text, slideshows etc.). RAPITS (Woods and Warren, 1996) is an adaptive teaching system that compares student model to domain knowledge and automatically changes presentation style. The student can proceed through the lessons sequentially or switch to a non-linear hypermedia mode through the courseware. Student knowledge is assessed at the end of a topic. Triantafillou et al. (2002) created an adaptive educational system (AEC-ES) based on field dependent/field independent cognitive styles. Their system uses navigational support tools (concept map, graphic path, advance organiser) and adaptive presentation techniques. Students are provided with instructional strategies that suit their cognitive preferred style with an option to switch it to a non-preferred version.

3.1 Implications for the design of web-based learning materials in this study

Based on the learning style description by Pask, the following learning style representation in a hypermedia environment was compiled. The majority of these elements apply to the layout, sequencing and structure as well as the navigation of the user interface. The two principal considerations in designing hypermedia courseware to accommodate preferred learning styles are: the way in which the information is formatted and structured and how individuals process the given information. Hypermedia can be put to an advantage or disadvantage for the users depending on whether the material is matched or mismatched with the students’ preferences. The way that global and sequential students process information would appear to be directly relevant to effective learning from information presented as hypermedia. Figure 1 depicts an example of the global and sequential page layouts. The difference in presentation of the two types of formats is apparent. For the students with a global learning style preference, pages comprised elements such as a table of contents, summary, diagrams, overview of information etc. For sequential students, the pages contained small chunks of information, text-only pages with ‘forward’ and ‘back’ buttons.

Where L represents a link

Fig. 1. The user interface templates (sequential and global)

4. OVERVIEW OF THE EXPERIMENT

As learning styles are a significant contributing factor in terms of student progress, we can attempt to represent some of the characteristics of the learning preferences within hypermedia environment and help students in different parts of the learning cycle. To examine whether differently formatted hypermedia courseware has an impact on information acquisition during the browsing process, an empirical evaluation was conducted. Students were asked to complete a number of tasks. These tasks included the following: browsing, reading, searching for and memorising information displayed on the computer screen. At the end of each round of the tasks, the students were asked to answer a series of questions to test their recall of the supplied information. In this study, the achievement scores from two post-tests were compared. Test questions were tied to lesson objectives and levels of Bloom’s taxonomy (1956).

4.1 Procedure

Sample

The sample of students involved in this study comprised twenty-two, year-10 students (they are 14 year old students attending National Curriculum) in the first year of a two-year GCSE^[3] geography course. They browsed through two different versions of web-based courseware; one version matched their preferred learning style and the other did not. The learning styles incorporated into the hypermedia courseware were global and sequential learning styles.

Courseware choice

Both sets of courseware were part of the GCSE syllabus. First of the courseware sets ("Countries of the world") was developed and adapted from a GCSE geography book (Pallister, 2001). The instructional material for the second courseware set ("Ozone layer depletion") was a mixture of web resources. The courseware was new to the students, so interference from prior knowledge was not a concern.

Fig. 2. Global session layout

Learning style instrument

Before the start of the experiment the positions of the students on the learning styles dimensions were evaluated using a web-based, self-administered Felder-Soloman Index of Learning Styles Questionnaire (ILSQ) that assessed four learning style dimensions: introvert/extravert, verbal/visual, sensitive/intuitive and global/sequential dimension. This Index of Learning Styles (ILS) has been developed by Soloman and Felder (1988). Attention in this study was focused on the bi-polar nature of the LSQ scales, particularly the global-sequential dimension. Students can have from mild to moderate and strong learning preference on that scale.

Procedure

At the start of the study, students read a short explanation concerning the use of the system. They then logged onto the system and their learning styles were recorded. The students then completed pre-test on the first learning subject and then proceeded to browse and study the material that matched their learning preferences. Having completed that, the students were presented with a recall-type post-test. The questions were knowledge questions as they tested recalling of facts, terms and concepts as suggested in Bloom’s taxonomy. In the second part of the study, students logged in again, and completed pre-test about the second subject. They proceeded to browse and study material, which was in the adapted so that did not match their learning preference. Upon completion of the second browsing session the students completed the post-test. The two main dependent variables in this study were the achievement scores obtained in the two post–tests and the session-browsing times. For this study significance testing or hypothesis testing was used.

5. RESULTS

This section presents the performance results for the knowledge attainment and browsing times differences between matched and mismatched sessions. It also shows any dependency between the length of browsing times and the increase in scores achieved after browsing both sessions.

5.1 Data collection and analysis

To compare performance between students the times for both browsing tasks and the answers to all four sets of questions (pre-tests and post-tests for matched and mismatched sessions) were recorded. The population consisted of 21 students, where 9 were with sequential preferences (6 balanced, 2 moderate and 1 with strong preference) and 12 with global preferences (6 mild and 6 with moderate preferences). The results were analysed to determine whether significant differences between score means occurred for different session types. T tests were used to compare the mean scores between matched and mismatched conditions and a significance level of p<0.05 was adopted. Q₂ denotes the number of correctly answered questions after the 1^st post-test (after the session in preferred style) and Q₄ denotes the number of correctly answered questions after the 2^nd post-test (after the session in non-preferred style)

5.1.1 Difference between score means for matched and mismatched sessions

Fig. 3. Scores after the post-questionnaires with mean values for all students, with confidence interval of 95%

To determine if the differences between achievement scores were significant, the two-sample, paired, two-tailed t-test was used. The main results are presented in Table 1. The hypotheses for the test scores were that:

H₀: Post-test-score means for matched and mismatched sessions are the same

H₁: Post-test-score means for matched session are significantly higher than for mismatched session

Table 1: SPSS results for the t-test for score differences for all students

The mean values for the scores after the 1^st and 2^nd sessions are listed in Table 1 and it appears that the mean scores for Q₂ are much higher than for Q₄ (8.00>5.095). Analysis of student performance indicated that students achieved higher scores when studying matched session than mismatched session. Mean values and standard deviation are also listed in the Table 1. From the first post–test and second post-test achievement scores, it has been found that there was a very significant difference of correctly answered questions t(21)=6.84; p=0.000, 2-tailed. Figure 3 depicts the fact that the mean value of scores (Q₂) after matched session is much greater than the mean score value for mismatched session (Q₄).

Table2: Post test and pre test score gains differences

This table shows the total score gains between matched conditions was less than ⅓ (23) higher than in the total score for score gain in mismatched conditions. Summing the scores brought the total number of points for a matched session to 72 points, with an average score difference of 3.4 and 49 points for mismatched session, with the average score difference reduced to 2.33 points. This indicates that the higher score difference was achieved between pre and post-tests in a matched session.

Table3: Score gains between two pre and post-tests for matched and mismatched conditions

When the subscale of score increases or gains is scrutinised (Table 3), results reveal that more students (4) in mismatched session did not have any score increase (0%) than in a matched session (0). The biggest difference is in the number of students between a mismatched session and a matched session is in 30% and 50% score gains (5 students). The same number of students in both sessions achieved a score increase of 20% and 70%. On average the two groups are comparable in terms of pre-test to post-test gains.

5.1.3 Results of the pre-tests: Q1 (matched) – Q3 (mismatched)

Table4: Pre test score (Q1-Q3) differences

Table 4 indicates the results of the pre test scores for matched and mismatched conditions. It shows that pre-tests in matched conditions are more than ⅓ (38 points) higher than in their pre-test for mismatched conditions.

5.1.4 Difference between browsing times for matched and mismatched sessions for all students

The t-test tests were also performed for session browsing times. The completion times for browsing were measured from the start of each round until the students started answering post-tests. It was expected that the students would spend less time browsing their matched session than their mismatched session.

Table 5: SPSS results for t-test for browsing times for all students

The hypotheses for the browsing times for all students were:

H₀: Browsing times for matched and mismatched sessions are the same

H₁: Browsing time for a matched session is significantly shorter than for a mismatched session

The above table shows that the means between two browsing times do not differ hugely, where t2>t1 (20.68 >18.87mins). The results do not indicate high significance between browsing times for matched and mismatched sessions, i.e. t (21)=-0.495, p<0.626.

5.2 Relationship between the length of browsing times and the increase in scores for all students for matched and mismatched sessions

To check for any correlation between the length of browsing times between the sessions and the scores achieved in the post-tests, a Pearson-rho correlation coefficient was calculated. The times show that students on average spent different lengths of time for the two different sessions and obtained different score increases. 6 out of 21 students achieved the same scores after the matched and mismatched sessions, even though there is a high difference in the times they spent on average (~16 mins). In one third of cases, students spent a considerably longer time browsing the matched session (32.5mins on average), one half of them spent less time browsing the matched session (3.77 mins), and one sixth of them spent an almost identical amount of time on two sessions and in this instance attained the same score. On average there was an 18 % score increase for all the students between matched and mismatched sessions. The results from the analysis reveal that the Pearson’s correlation coefficient is r=0.086, r²=0.00739, or less than 1 % of browsing time difference accounts for score difference, with a p-value of 0.356, which is clearly demonstrating not a very significant correlation. It can be concluded that the post-test scores obtained in the study are a reflection of the lesson–tailoring, rather than the length of time that the students spent browsing the courseware.

5.3. Summary of results

Alternative hypothesis 1 was confirmed. All the students achieved significantly higher scores while browsing matched session for all students, t (21)=6.837, p<0.000 (two–tailed).

Alternative hypothesis 2 was not supported. Browsing times for the matched session was not significantly shorter than for mismatched session for all students, t (21)=-0.495, p<0.626.

Alternative hypothesis 3 was not supported. All students did not achieve significantly higher scores in a shorter time, i.e. there was not a strong correlation for the matched session r = 0.086, p<0.356, nor for the mismatched session r=-0.012, p<0.479.

6. DISCUSSION

The main goal of this empirical evaluation was to evaluate if there was a very significant difference in the means of scores achieved in the matched-learning-style-session versus the mismatched-learning-style-session. The relationship between score-increases achieved between the matched and mismatched hypermedia environment and browsing times were also tested. The results obtained suggest insightful data with respect to student’s learning preferences. The following discussion based on the experimental findings, concerning the identified factors provides an insight into possible reasons for these findings.

Knowledge scores from post-tests: difference between means of post- scores from matched and mismatched sessions

In analysing the responses to the knowledge questions, the scores for the two session types suggested that there was a very strong relationship between matching students’ learning style to the courseware as the findings suggest that all the students achieved significantly higher scores while browsing the session that matched their learning styles.

The browsing times difference between matched and mismatched sessions

With regards to the browsing times between the two sessions, the evidence suggests that there is no significant difference between the lengths of time students spent on the two sessions. On average, the time it took to browse the matched session was only slightly different than the time it took to browse the mismatched session for all students (18.87<20.68 mins.). The conclusion is that there was not a statistically significant difference between browsing times for the matched and mismatched sessions. The lack of any significant difference between browsing times for the matched and mismatched groups may be due to a number of factors. The speculation would be that since the first session was tailored to suit their preferred style of learning, it grabbed their attention for a longer period of time. The results indicate the browsing times are not affected by the session-type may seem surprising, but a closer examination shows that it is not unreasonable. It may be suspected that students who were not ‘burdened’ with the additional links and learning style elements (in the sequential session) would perform significantly faster on the browsing, but their scores indicate otherwise. In fact, the results do not indicate a trend towards lower times. The amount of information displayed on the screen within a global session might have an impact on the browsing times too, although that is not to say that the study tasks in sequential sessions were not less cognitively demanding.

The browsing times versus score difference increase

The data analysed indicates that there was not a correlation between student performances and browsing times. Browsing times have shown not to affect the increase in scores after each session. 24% of students achieved a higher score difference in a shorter time. Those students who spent longer (19%) browsing their matched session achieved higher mean scores, while those students who spent less time browsing their matched session (5%) achieved lower scores. The correlation coefficient calculated for all students for a matched session was r=0.086, r²=0.0074 (or only 0.74%) variance of score differences is explained by browsing times, with p<0.356 and r=-0.012 for mismatched session, r²=0.014 with p<0.493.

7. CONCLUSION

This was an initial study evaluating what impact the incorporation of learning-styles within educational hypermedia courseware has on learning outcomes. In this study, with its emphasis on GCSE students, the main hypotheses postulated, regarding the mean scores difference, were found to be particularly pertinent and well founded. The findings suggest that students benefit from the learning materials being adapted to suit their learning preferences. The results revealed that students have obvious different preferences for lesson presentation type. The results suggest that the learning outcomes can be improved if designers of hypermedia courseware provide a different sequence and presentation of materials to accommodate individual learning style differences. Hence possibilities for promoting more effective learning are realised. These solid results indicate that learning styles provide a good basis with which to adapt hypermedia to individual needs. Hypermedia design features, based on student’s learning styles, such as structural and linking mechanisms, have significant bearing for the future development of adaptive hypermedia systems. The next stage of this experiment is to develop a more adaptive version of the system that automatically tailors itself to users’ learning needs. It combines the learning styles described in this experiment with a variety of learning strategies. It provides adaptive navigational guidance and it supports cognitive learning strategies. The system prototype is currently being evaluated.

REFERENCES

1. Bloom, B.S, Englehart, M.B., Furst, E.J., Hill, W.H. and Kratwohl, O.R. (1956) Taxonomy of educational objectives: The classification of educational goals, Handbook 1: The cognitive domain. New York: Longman

2. Brusilovsky, P (1996) Methods and techniques of adaptive hypermedia. User Modelling and User adapted Interaction.6, pp. 87-129

3. Carver, C. A. (1996) Enhancing student learning by incorporating learning styles into adaptive hypermedia, [online], http://www.eecs.usma.edu/cs383/tools.htm, [Accessed November 2001]

4. Carver, C.A., Howard, R.A. and Lavelle, E. (1996) Enhancing student learning by incorporating learning styles into adaptive hypermedia. Proceedings of 1996 ED-MEDIA World Conf. on Educational Multimedia and Hypermedia, Boston, USA, pp.118-123

5. Clarke, J.A. (1993) Cognitive style and computer assisted learning: problems and a possible solution, Association for Learning Technology Journal (ALT-J), Vol.1, (1), pp.47-59

6. Felder R.M. and Silverman L.K. (1988) Learning and teaching styles in engineering education. Engineering education 78, pp. 674-681.

7. Felder R.M., and Solomon, B. Index of learning styles questionnaire. [online], http://www2.ncsu.edu/unity/lockers/users/f/felder/public/ILSdir/ilsweb.html, [Accessed November 2002]

8. Ford, N. (1995) Learning styles and strategies of postgraduate students. British Journal of Educational psychology, 16 (1), pp. 17-26

9. Ford, N. and Chen, S.Y. (2001) Matching/mismatching revisited: an empirical study of learning and teaching styles. British Journal of Educational Technology, Vol. 32. No. 1, pp. 5-22

10. Graff, M. (1999) Cognitive style and hypertext structures. Proceedings of the 4^th European Learning styles conference, Central Lancashire

11. Grigoriadou, M., Papanikolaou, K., Kornilakis, H. Magoulas, G. (2001) INSPIRE: An Intelligent system for personalized instruction in a remote environment. [online] Available at http://wwwis.win.tue.nl/ah2001/papers/papanikolaou.pdf, [Accessed March 2003]

12. Kwok, M. and Jones, C. (1985) Catering for different learning styles. Association for Learning Technology Journal (ALT-J), 3, 1, pp. 5-11

13. McLoughlin, C. (1999) The implications of the research literature on learning styles for the design of instructional material. Australian Journal of educational technology, 15(3), pp. 222-241

14. Pallister, J., Bowen, A., Clay, R., Dilandro, C., Phillipson, O., (2001) Longman Geography for GCSE. London: Longman, pp. 202-206

15. Pask, G. (1976) Styles and strategies of learning. British Journal of Educational Psychology, 46, pp. 128-48

16. Pask, G. (1988) Learning strategies, teaching strategies, and conceptual or learning styles. In R.R. Schmeck (Ed.), Learning strategies and learning styles. New York: Plenum Press

17. Pillay H., and Willss, L. (1996) Computer assisted instruction and individual cognitive style preferences in learning: Does it matter? Australian Educational Computing, Vol. 11, No.2, pp. 28-33

18. Riding, R. and Cheema, I. (1991) Cognitive styles: An overview and integration. Educational Psychology, 11(3 & 4), pp.193-215

19. Riding R.J. and Sadler-Smith, E. (1992) Type of instructional material. Cognitive style and learning performance. Educational studies 18, pp. 323-340

20. Triantafillou, E., Pomportsis. A, Georgiadpu E. (2002) AES-CS: Adaptive Educational System base on cognitive styles. AH2002 Workshop, Adaptive Systems for Web education, 2^nd International Conference, Adaptive Hypermedia and adaptive web based systems

21. Woods, P. and Warren, J. (1995) Rapid Prototyping of an intelligent tutorial system (RAPITS). Proc. ASCILITE'95, Melbourne, pp. 557-563

[1] A ‘matched’ session refers to one where the hypermedia material is adapted so that it matches a student’s learning style.

[2] A ’mismatched’ session is a session where the hypermedia material presentation and navigation do not match the student’s learning style.

[3] GCSE refers to General Certificate of Secondary Education