For the past three decades, my colleagues and I have been tracking down evidence-based principles for the design of multimedia instruction, that is, instruction involving words and graphics (Mayer, 2021, 2022).  Our work has produced over 200 experimental comparisons, yielded a set of 15 evidence-based principles for multimedia instructional design (along with boundary conditions for their applicability), and contributed the cognitive theory of multimedia learning, which is grounded in an understanding of how the human mind works.  As with any piece of active scientific inquiry, this effort has evolved and expanded over the years thanks to the efforts of a cohort of researchers from around the world as represented in the most recent edition of The Cambridge Handbook of Multimedia Learning (Mayer & Fiorella, 2022).

In his recent chapter, "The Case for Rethinking Multimedia," Hinderliter (2021, p. 57) first complements our work: "Mayer's Cognitive Theory of Multimedia Learning (CTML) has been a mainstay of instructional design for nearly two decades."  Then, he immediately diminishes our work: "but recent developments in theoretical frameworks relevant to multimedia learning point to a reversal of many previously held assumptions."  In his efforts to build a case against our work, Hinderliter (p. 57) seeks to "examine multiple issues regarding CTML that question its influential status within the instructional design community."  The goal of this essay is to systematically evaluate each of five areas of concern raised in Hinderliter's chapter.  I close with an examination of what I consider to be some areas for improvement in our field's quest for evidence-based principles of multimedia instructional design.

Criticism 1: The Research Base Is Limited to 60 Experiments

Criticism

Hinderliter's first unwarranted claim is that our proposed instructional design principles are based on "sixty previous tests conducted within 20 studies."  Hinderliter points out that some studies are limited in terms of the small number of participants, use of narration rather than printed text, or reliance on system-based timing of presentations. 

Response

Instead of the 60 experimental tests that Hinderliter attributes to us, our latest edition of Multimedia Learning (Mayer, 2021) includes over 200 experimental tests that we have conducted covering a variety of instructional venues.  In a field where claims are often based on a single study, we have maintained a commitment to replicating an effect multiple times before we offer it as a potential instructional design principle. In short, we agree that design principles should be backed up by a body of rigorous experimental tests, published in peer-reviewed journals.  With over 200 published experimental tests, our lab is contributing to that worthy goal. I conclude that Hinderliter's first criticism is unwarranted.

Criticism 2: The Design Principles Are Based Solely on Outdated Technologies

 Criticism

Hinterliter castigates CTML for being based on research conducted during "the last decade of the twentieth century" (p. 63).  He falsely claims that all of the studies "offered in support of CTML include paper-based treatments...as well as computer-based treatments involving HyperCard stacks on a monochrome Macintosh IIci computer" and that "few previous studies have been thoroughly replicated using adequate sample size within modern online instructional environments" (p. 63). Hinterliter correctly notes that today's students are "highly acclimated to the use of animation, video, and narration," but suggests that our work does not involve these media. 

Response 

Hinterliter appears to be basing his critique on a review published more than 20 years ago (Mayer, 2002), and he does not cite any of our research studies published since then.  He ignores current reviews of our work (e.g., Mayer, 2021) that are based on more than 200 experimental tests including educational technologies such as narrated animation, instructional video, interactive simulations and games, animated pedagogical agents, and immersive virtual reality.  Most studies have been replicated numerous times.  I conclude that Hinterliter's second claim is unwarranted.

Criticism 3: The Results Do Not Replicate

 Criticism

In perhaps his most destructive claim, Hinterliter states: "The landmark results of early research into cognitive learning principles have proven difficult to replicate" (p. 63).  In spite of making a serious claim against the findings of our research program, Hinterliter offers almost no evidence other than two one-sentence summaries that he alleges show that the modality principle cannot be replicated (i.e., Penney, 1975; Tabbers et al., 2004). 

Response

Even a cursory review of our research shows that our research team has been committed to replication.  As shown in Table 1, we have tested each of our multimedia design principles multiple times.  Certainly, we and others have not always obtained positive results, but these cases form the basis for identifying what we call boundary conditions--that is, situations in which a principle is more or less likely to apply.  Identification of boundary conditions represents a sign of the growing maturity of research on multimedia learning, and offers a basis for clarifying, refining, and adding to the cognitive theory of multimedia learning. 

Table 1

Multimedia Learning Principles Adapted from Mayer (2021)

Note.  ES = median effect size based on Cohen's d.  Tests = number of positive results out of number of experiment tests.

For example, the modality principle does not apply when there are words but no graphics, consistent with Penny's (1975) early work with word lists, or when the lesson is self-paced, consistent with Tabbers et al. (2004).  As chronicled by Fiorella and Mayer (2022), across 76 studies testing the modality principle by researchers around the globe, most failures to find positive effects were consistent with predictions that could be derived from the cognitive theory of multimedia learning. For example, the modality principle is predicted to be diminished or eliminated when the demands on working memory are low--such as when the lesson is slow-paced or self-paced, the material is simple, or the learners have sufficient prior knowledge. 

The detection of theory-grounded boundary conditions for the modality principle does not necessarily indicate a lack of empirical support for the cognitive theory of multimedia learning, but rather helps us gain better insights into how it should be best applied.  The principles listed in Table 1 should not be considered as rigid rules to be applied in all situations, but rather should be adapted to situations in line with what we know about how people learn as reflected in the cognitive theory of multimedia learning.  Overall, I agree that the modality principle has not been found in all studies, but when an effect is not found it is worthwhile to determine whether this can be explained by CTML.  In addition, overall, across the 76 studies reviewed by Fiorella and Mayer (2022), the median effect size favoring the modality principle was d = 0.65, which is in the medium-to-large range.  I conclude that Hinterliter's third criticism is unwarranted and misleading. 

Criticism 4: The Approach is Based on the Wrong Ism

Criticism

Hinderliter's fourth criticism is that our work is based on the wrong ideological approach.  In particular, he calls us "cognitivists" and suggests the true ism should be "constructivism".  For example, he misrepresents CTML as being based on the false idea that the human mind works like a computer:  "Constructivists may consider the conceptualization of students' minds as analogous to computers to be a gross oversimplification, but CTML is among many theories advanced by cognitivist thinkers that remain embedded within modern instructional practice" (p. 64). As another example, he misrepresents CTML as being opposed to fitting instruction to the needs of individual learners: "Constructivist pedagogy recommends the customization of lesson plans in order to suit the unique individuality of each learner, while cognitivism seeks to determine the 'true' manner in which students learn new information en route to global prescriptions that will benefit all" (p. 64).

Response

As I have repeated noted in my writings, I consider the cognitive theory of multimedia learning to have its roots in constructivist theories of learning, which assume that learning involves actively attending to relevant aspects of incoming information, mentally reorganizing presented material into a coherent representation, and integrating what is presented with what the learner already knows (Mayer, 2021, 2022). In short, CTML is based on the fundamental idea that we do not learn like computers, but rather meaningful learning is a constructive process of sense making. Similarly, our work is based on the idea that different learners come to the learning task with different kinds of prior knowledge and different information processing capabilities; therefore, it is useful to consider that different principles may apply for different types of learners. In recent years, as the field matures, we have been able to focus on what I call boundary conditions for when each multimedia design principle most strongly applies including specifying for whom a principle is most relevant.  Thus, Hinderliter grossly misrepresents CTML by accusing it of being based on equating the human mind with a computer and on advocating a one-size fits all approach to instructional design. 

Lastly, Criticism 4 is based on the idea that contributions to our field should be judged based on their ideological approach. I have long held that searching for the perfect ism is not a productive activity for educational psychology (Mayer, 1997). I do not think our field is served by taking an ideological approach rather than a scientific approach based on testing theories against research evidence.  Overall, I conclude that Hinterliter's fourth claim is not only unwarranted but also misguided. 

Criticism 5:  The Research Only Deals with Typical Students

 Criticism

Hinderliter's final claim is that "the broad applicability of CTML's principles is often in conflict with issues of accessibility" particularly for learners with challenges such as "vision impairment, hearing impairment, cognitive impairments, or low language proficiency" (p. 64).  He proposes focusing on "neurotypical native English speakers raises concerns regarding the generalizability of such findings" (p. 65). 

Response

I agree that most of our research has not targeted special populations, although we have examined the applicability of the redundancy principle and multimedia principle for students learning in their second language (Lee & Mayer, 2017; Mayer & Lee, 2015; Mayer, Lee, & Peebles, 2014).  In addition, some of our collaborative studies also involve learning in languages other than English (Li et al., 2019, 2022; Wang et al., 2018; Xie et al., 2019).  Certainly, an important next step in research on multimedia design principles is to examine how they apply to students who are not neurotypical, including students with specific learning challenges.  Overall, I conclude that Criticism 5 is warranted, but rather than suggesting a flaw in CTML it reflects a path for future research that can strengthen it. 

Looking to the Future of Multimedia Instructional Design

Don't get me wrong, I am not against criticism (although I prefer constructive criticism rather than unwarranted criticism).  In this section, I sketch out my own constructive suggestions concerning areas for improvement in research on multimedia instructional design that warrant attention in the future. 

• Broaden the context of learning:  Conduct studies in authentic learning venues such as classrooms and online courses, and with new media such as virtual reality, interactive simulations and games, animated pedagogical agents, instructional video, and narrated animation.
• Broaden the learner population: Include students that represent the diversity of the student population in terms of age, gender, race, ethnicity, language proficiency, nationality, and individual differences in terms of cognitive, physical, and sensory skills.
• Broaden the measures of learning process and outcomes: In addition to transfer and retention posttests, incorporate embedded tests of learning outcomes, as well as biometric, cognitive neuroscience, eye-tracking, and data mining measures of learning processes.
• Incorporate affective and social processes: Expand CTML to include the learner's affective and social processes during learning.
• Incorporate motivational processes: Expand CTML to include the learner's motivational processes including interest, self-efficacy, and beliefs.
• Incorporate metacognitive processes: Expand CTML to include the learner's metacognitive processes, including learning strategies and executive function skills. 

In the future, I expect to see an ever increasing empirical research base, a growing set of evidence-based design principles, more refined descriptions of boundary conditions, and a more inclusive theory of multimedia learning. 

Conclusion

In spite of the many unwarranted criticisms I have confronted in this chapter, I prefer to frame Hinderliter's chapter as a well-meaning call for improvement in research and theory in our field.  I appreciate Hinderliter's interest in our work and look forward to our field's continuing efforts to build principles of instructional design that are based on research evidence, grounded in theories of how people learning, and relevant to today's educational challenges. 

References

Fiorella, L. & Mayer, R. E. (2022). Principles for managing essential processing in multimedia learning. In R. E. Mayer & L. Fiorella (eds.), The Cambridge handbook of multimedia learning (3rd ed; pp. 243-260). Cambridge University Press.

Hinderliter, H. (2021). The case for rethinking multimedia. In B. Hokanson, M. Exter, A. Grincewicz, M. Schmidt, & A. A. Tawfik, A.A. (eds.), Learning: Design, engagement and definition (pp. 57-68). Springer.

Lee, H., & Mayer, R. E. (2018).  Fostering learning from instructional video in a second language. Applied Cognitive Psychology, 32, 648-654. https://doi.org/10.1002/acp.3436

Li, W., Wang, F., Mayer, R.E., & Liu, H. (2019). Getting the point: Which kinds of gestures by pedagogical agents improve multimedia learning? Journal of Educational Psychology, 111 (8), 1382-1395. https://doi.org/10.1037/edu0000352

Li, W., Wang, F., Mayer, R. E., & Liu, T. (2022). Animated pedagogical agents enhance learning outcomes and brain activity during learning. Journal of Computer Assisted Learning, 38 (3), 621-637. https://doi.org/10.1111/jcal.12634

Mayer, R. E. (1997). Searching for the perfect ism: An unproductive activity for educational research. Issues in Education, 3 (2), 225-228.

Mayer, R. E. (2002). Multimedia learning. Psychology of Learning and Motivation, 41, 85-139. https://doi.org/10.1016/S0079-7421(02)80005-6

Mayer, R. E. (2005). Cognitive theory of multimedia learning.  In R. E. Mayer (ed.) The Cambridge handbook of multimedia learning (pp. 202-212). Cambridge University Press. 

Mayer, R. E. (2009). Constructivism as a theory of learning versus as a prescription for instruction.  In S. Tobias & T. M. Duffy (Eds.). Constructivist theory applied to education: Success or failure? (pp. 184-200). Erlbaum. 

Mayer, R. E. (2014). Multimedia learning. In J. Michael Spector, M. David Merrill, E. Jan, & M. J. Bishop (eds.), Handbook of research on educational communications and technology (4^th ed., pp. 385-399). Springer.

Mayer, R E. (2021). Multimedia learning. Cambridge University Press.

Mayer, R. E. (2022). Cognitive theory of multimedia learning. In R. E. Mayer & L. Fiorella (eds.). The Cambridge handbook of multimedia learning (3rd ed; pp. 57-72). Cambridge University Press. 

Mayer, R. E. & Fiorella, L. (2022). (eds.). The Cambridge handbook of multimedia learning I3rd ed). Cambridge University Press.

Mayer, R. E., & Lee, H. (2015). Visual aids to learning in a second language: Adding video to an audio lecture.  Applied Cognitive Psychology, 29, 445-454. https://doi.org/10.1002/acp.3123

Mayer, R. E., Lee, H., & Peebles, A. (2014). Multimedia learning in a second language: A cognitive load perspective.  Applied Cognitive Psychology, 28 (5), 653-660. https://doi.org/10.1002/acp.3050

Penney, C. G. (1975). Modality effects in short-term verbal memory. Psychological Bulletin, 82 (1), 68–84. https://psycnet.apa.org/doi/10.1037/h0076166

Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74 (1), 71–81. https://doi.org/10.1348/000709904322848824

Wang, F., Li, W., Mayer, R. E., & Liu, H. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110 (2), 250-268. https://psycnet.apa.org/doi/10.1037/edu0000221

Xie, H., Wang, F., Mayer, R. E., & Zhou, Z. (2019).  Coordinating visual and auditory cueing in multimedia learning.  Journal of Educational Psychology, 111 (2), 235-255. https://psycnet.apa.org/doi/10.1037/edu0000285