Introduction

With the dramatic shift to online learning with the arrival of the COVID-19 pandemic, faculty, staff, and students within higher education worldwide have made the sudden but necessary initial steps to incorporate technology into the learning environment in ways never imagined. However, forward-thinking administrators are wondering, “what comes next?” Simply shifting lectures to web conferencing is not revolutionary. Declining freshmen US enrollment of 13% has causes major financial instability in higher education budgets (Smalley, 2020).  Administrators face the need to make brave and creative choices. Administrators also want to insulate their institutions from negative repercussions of the next major instructional interruption. Immersive learning answers this call and has already had a two-decade research base to pull from (Beck et al., 2020). Given that many XR experiences are sustainable (Bucea-Manea-Țoniş et al., 2020) and do not require the learner to be on campus, a major shift to XR-for-learning might be the greatest change in higher education since the invention of the university.

Nevertheless, XR is not going to settle for a rebottling of ‘the next big thing’ in education. Following a fad is not a good idea. Instructional designers are best situated to consult on this topic because these professionals are comfortable analyzing instructional tools looking past any purported hype. Especially with decreased technology prices and increased access to XR, campus administrators might want to buy the technology first and think about use second. Instructional designers are obligated to advise on the best use of the technology even if that advice is sought after the purchase. This chapter will focus on research-based recommendations for XR design decisions.

Definitions

Extended reality

Terms in XR represent the evolving and changing human and computer interface. The terms ‘extended reality’ or ‘cross reality’ refer to “technologies and applications that involve combinations of mixed reality (MR), augmented reality (AR), virtual reality (VR), and virtual worlds (VWs)” (Ziker, Truman, & Dodds, 2021, p. 56). Immersive learning definitions draw from Milgram and Kishino’s key taxonomy (1994) emphasizing the continuum of experiences that range from where a computer adds to a learner’s reality with overlays of information, or a computer experientially transports a learner to a different place and time by manipulating sight and sound. Moreover, the social and connected nature of virtual reality experiences signals an association with the word metaverse, first used in Stephenson’s 1992 fictional novel, Snow Crash, to describe a three-dimensional (3D) space where users, embodied as avatars, interact with others and the virtual space. In that fictional writing, the metaverse was designed to be the next version of the Internet; an Internet that one entered as a reality in 3D.  With XR, this is still possible; the future lies ahead. Díaz, Saldaña, and Avila (2020) observe that within higher education, the incorporation of XR has already provided a rich research base for experiences that include interactivity, corporeity (users represented as avatars), and persistence.

Virtual reality

The terms metaverse, virtual reality (VR), mixed reality (MR) and cross reality (XR) are used interchangeably in common parlance despite nuanced differences that are debated among experts. All terms imply instances of the user having an immersive experience facilitated by technology. Virtual reality has traditionally been more popular terminology than XR (see Figure 1).

Figure 1

Google Search Term Totals

Note. Scores are a Google popularity index with no values. Blue is XR, red is VR. (Data source: Google Trends, 2021 https://www.google.com/trends).

Beck, Morgado, and O’Shea (2020) point out that varied immersive learning environments (ILEs) have immersion as the key characteristic, it “is the locale where the technical, narrative, and challenging aspects occur” (2020, p.1045). VR tends to refer to independent immersive experiences facilitated by headsets. The interchangeable use of terms in this field is a characteristic of the early evolution of a branch of technology. In this chapter, XR is used to represent all immersive experiences. Note that historically, ID would refer to users as learners. Given the interconnections between instructional design, user experience (UX), and human-computer interaction (HCI), the terms users and learners are used interchangeably in this chapter.

Presence

When asked, users tend to mention the feeling of being there or presence as the key feature of XR. It can “unlock doors to social experiences and give people a sense of belonging and fulfillment in a world changed by a pandemic that keeps many physically apart” (Hackl, 2020, para. 2).  Lee (2004) defined presence as “a psychological state in which virtual objects are experienced as actual objects in either sensory or non-sensory ways” (p. 27). Presence has been studied in many facets. Users feeling presence is a best practice within XR.

Storytelling

Serrat (2008) defines storytelling as “the vivid description of ideas, beliefs, personal experiences, and life-lessons through stories or narratives that evoke powerful emotions and insights” (p.1). Stories bring the user through the experience and answer the critical question: Why are you making the user do this experience? Higher education users, often at adult ages, want that question answered.  Users will not proceed with an experience if they do not know why they are being asked to do it. Storytelling has a direct connection to XR via experiences.  XR users describe attending events or going to places. As such, becoming familiar with storytelling as a design feature is another best practice when considering XR.

Instructional Design Theory and Approaches

The foundational theory for most XR experiences is experiential learning theory. In cases where users create within XR, constructivist learning theory also applies. These theories recommend these elements for use in education:

• Be of high quality: XR experiences cannot be haphazard; the lesson must be pre-planned.
• Expose the learner to something different, a variation in the user environment. It is not enough to replicate reality; XR experience should be different from the non-XR (as in manipulatable/changeable).
• Include experimentation or manipulation of cause and effect. The user must be able to change something.
• Include reflective components. All theories stress the inclusion of guided pondering and contemplation.
• Have a direct tie to future action. XR experience should change or impact a future experience.

Criticism of these theoretical approaches suggests that learners do not always learn in the sequential nature that theories suggest (Lindsey & Berger, 2009). For example, learners can learn from third-person observations in XR. Nevertheless, Lindsey and Berger (2009) recommend that the experiential approach to instruction include three key features: the experiences must be framed, activated, and then reflected upon (see Figure 2).

Figure 2

Experiential Approach to Instruction (Lindsey & Berger, 2009)

VR Design Model

Instructional designers venturing into 3D immersive designs will recognize the same skill set use for 2D design. Díaz, Saldaña, and Avila state that “the creation of virtual spaces to host training activities must follow similar design criteria in terms of rigor and quality as the design criteria of training spaces for the real world” (2020, p. 105). This chapter combines three different design models (see Figure 3): the ADDIE Design Model (Branson, 1978), Design Thinking (Brown & Wyatt, 2010) from user experience (UX), and the 3D Learning Experience Design Model (Kapp & O'Driscoll, 2009).

Figure 3

A comparison of ADDIE, Design Thinking, and the 3D Learning Experience Design models.

Analysis

The first stage, also known as the empathy or participant-centered stage, asks the key question: Why is immersive learning the solution to the instructional problem? The experience must be instructionally grounded (Kapp, 2020). XR should not be selected for use in higher education just because it is perceived as ‘amazing’ or ‘cool’. Given that the brain often believes what the eye sees, the expansive effects of XR are too influential to be casually selected. Kapp (2020) recommends that in any case where declarative knowledge is the goal, XR is not the correct choice. In many current situations, XR might not be the best selection when measured against expense, environmental sensitivity, and socio-cultural awareness. However, there is some large-scale research indicating that XR does outperform the competition when considering user emotions. There is some positive early research on the use of VR for procedural skills, communication skills, and corporate culture (Bailenson, 2020).

IDs must know what technology is available to the users. If all users do not have VR headsets, IDs should recommend WebXR (web browser accessible 2D VR). Users should also have some connection to prior immersive experiences that make XR a logical choice (Kapp, 2020). Furthermore, XR is recommended where the real-world learning experience would be dangerous, expensive, or impossible.

In summary, XR may be cost-effective for (See Appendix A on how to engage in leadership discussions regarding XR costs):

• Non-declarative knowledge learning
• Environmentally sensitive or sustainable applications
• Socio-culturally respectful applications
• Procedural skill learning
• Applications where the emotional influence is are dominant (i.e., emergency services: medical/police/fire/military)
• Communication & cultural skills (workplace relationships)
• Where the XR technology already exists or is easily accessible
• Where the learning would otherwise be dangerous (going inside a nuclear reactor chamber), expensive (field trips to far off locations), or impossible (watching the landing of a Mars probe from the surface of Mars).

Design and Development

Design. No XR experience currently suits all needs in higher education. Therefore, priorities must be determined. This is the phase where the solution is contextually situated or defined and framed. Administrators must choose which characteristics of XR will be most important to their users. Choices can be between access, immersion, and function (Dodds & Peres, 2020). For example, if it is most important that as many users as possible engage in the learning, then accessibility is the most important characteristic. IDs will need to find platforms that offer the greatest amount of accessibility. Those same platforms might sacrifice immersion and functionality to strongly deliver on accessibility (see Appendix B for how to evaluate a XR platform).

IDs should note that mainstream XR platforms tend to replicate reality, instead of engaging the phantasmagorical. Instruction should be designed around the user, rather than having the user adapt to the platform. Personalization within XR is a compelling characteristic that gives the user control over the experience (see Appendix C for suggested resources to research design choices).

Development. XR experiences can include a story arc (See Appendix D), a tutorial of user affordances, intentional user actions, and place the user into first or third person experiences (Spillers, 2020). VR currently uses the HCI elements of gaze, voice, gestures, sound, and interactive menus.  IDs should note data collection abilities and privacy protections. For further XR development research, seek user interface (UI) style guides from companies such as Unity and Microsoft, mixed reality guides, and the W3 web standards.

Implementation

Research on the implementation of XR in higher education is in its nascent stages but there is promise if decisions are made wisely (Radianti, Majchrzak, Fromm, & Wohlgenannt, 2020). Because of the immersive nature of XR and drawing on other HCI field experience, users have expectations of how an XR experience should progress; users take their conceptions of reality into virtual reality. Every choice and affordance available within the experience should support the user. The interactions should be action-oriented to best take advantage of XR. For example, users should be able to flip switches and guide an airplane down for a landing, not simply select a multiple-choice answer to do so. User testing is critically important in all phases of design. IDs should test beyond the direct development team with diverse and inclusive cases and incorporate international collaboration to check for cultural or language bias.

Evaluation and Optimization

Traditional assessments used outside of XR are a common design choice. However, XR allows for a much wider selection for assessment and evaluation. Users can give audio or video feedback or modify objects. Users can express their knowledge, skills, and abilities directly within the platform. For example, users can move to indicate an answer to a question. Users can directly interact with some platforms as knowledge creators.

Conclusion

Immersive learning environments have the potential to save resources (i.e., fossil fuels, health, time) and increase user access. The COVID-19 pandemic has reminded us of lost shared experiences. XR is about building shared experiences. XR choices should focus on providing an experience to the user that they cannot experience via other media. IDs are reminded that there are users that cannot engage in XR because of vertigo, technical specifications, health concerns, or expense.  XR is on the cusp of mainstream but it still considered a status symbol. Furthermore, gender, identity, and privacy issues continue to plague many XR experiences.

More research is needed in areas of accessibility. XR platforms are changing at an incredible pace. Major technology companies like Facebook, Google, Apple, and Microsoft all have significant research interest in XR. When companies of these sizes invest over time scales of 10 or 20 years, higher education must pay attention. XR experiences made for the social or work realms will need to be taught in higher education as critical skills and behaviors.

Immersive learning and XR is clearly not a fad. IDs have the critical role of consulting on media choices for their campuses. IDs can lead the way by advocating, recommending, designing, assessing, and researching learning options. The key features of shared experiences, depth of personalization, and compelling story arcs support this media choice for future opportunities. It is time to step through the looking glass of immersive learning and into XR.

References

Bailenson, J. (2020). Is VR the future of corporate training? https://edtechbooks.org/-hXkG

Beck, D., Morgado, L., & O'Shea, P. (2020). Finding the gaps about uses of immersive learning environments: a survey of surveys. Journal of Universal Computer Science, 26, 1043-1073.

Branson, R. (1978). The interservice procedures for instructional systems development. Educational Technology, 18(3), 11-14. http://www.jstor.org/stable/44418942

Brown, T., & Wyatt, J. (2010). Design thinking for social innovation. Development Outreach, 12(1), 29-43. https://edtechbooks.org/-Lxs

Bucea-Manea-Țoniş, R., Bucea-Manea-Țoniş, R., Simion, V. E., Ilic, D., Braicu, C., & Manea, N. (2020). Sustainability in higher education: The relationship between work-life balance and XR e-learning facilities. Sustainability, 12(14), 5872. https://edtechbooks.org/-pfkw

Díaz, J., Saldaña, C., & Ávila, C. (2020). Virtual world as a resource for hybrid education. International Journal of Emerging Technologies in Learning (iJET), 15(15), 94-109. https://edtechbooks.org/-PLj

Dodds, H. & Peres, T. (2020). A taxonomy of VR platforms. https://edtechbooks.org/-nnxZ

Google. (2021). Google trends. [Data set: Worldwide, 2004-present, XR and VR search terms] https://www.google.com/trends

Hackl, C. (2020). Social VR, Facebook Horizon and the future of social media marketing.https://edtechbooks.org/-bSp Kapp, K. M. [Immersive Learning Research Network]. (2020). iLRN2020 featured plenary presentation: What works in 3D learning environments. [Video]. YouTube. https://youtu.be/gZpHg0G_cMw

Kapp, K. M., & O'Driscoll, T. (2009). Learning in 3D: Adding a new dimension to enterprise learning and collaboration. John Wiley & Sons.

Lee, K. M. (2004). Presence, explicated. Communication theory.14(1), 27-50. https://edtechbooks.org/-rdr

Lindsey, L., & Berger, N. (2009). Experiential approach to instruction. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models (Vol. 3, pp. 117-142). Routledge.

Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321-1329. https://edtechbooks.org/-YoVW

Ng, C. (2011). Plotting out plot. https://edtechbooks.org/-SoV

Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778, 1-29. https://edtechbooks.org/-nihN

Serrat, O. (2008). Storytelling. Knowledge Solutions - Asian Development Bank. https://edtechbooks.org/-VANQ

Smalley, A. (2020). Higher education responses to coronavirus (COVID-19). National Conference of State Legislatures [NCSL]. https://edtechbooks.org/-BZsW

Spillers, F.  (2020). How to design for virtual and augmented reality. Interaction Design Foundation. https://www.interaction-design.org/

Ziker, C., Truman, B. & Dodds, H. (2021) Cross reality (XR): Challenges and opportunities across the spectrum. In J. Ryoo & K. Winkelmann (Eds.), Innovative learning environments in STEM higher education: Opportunities, challenges, and looking forward (pp. 55-78). SpringerBriefs in Statistics.

Appendix A

How to engage in leadership discussions on VR costs

1. Determine user need.
2. Determine existing technology including access.
3. Develop personas based on user roles, including diverse users (Microsoft, 2016).
4. Establish instructional goals.
5. Research effective use of VR in similar environments, subjects, or user groups.
6. Prototype and test VR with users.
7. Research costs for purchase, maintenance, access, safety, and upgrades.
8. Present on efficiency:

• Does VR achieve equivalent learning outcomes?
• Does VR cost less per user experience than traditional instructional methods?
• Does VR add access to previously inaccessible user groups due to danger, cost, physical or instructional access?

Appendix B

VR platform analysis for accessibility, immersion, and functionality

For more description, see Dodds & Peres (2020).

Accessibility

1. Equitable use
2. Flexibility in use
3. Simple and Intuitive to Use
4. Perceptible information
5. Tolerance for error
6. Low physical effort
7. Size and space for approach and use
8. Ease of technology

Immersion

1. User's expectations match
2. User's actions have non-trivial impact
3. Consistent world conventions
4. Deep play ability
5. Presence
6. Human factors
7. Feedback
8. Technical factors
9. Latency
10. Avatar

Functionality

1. Input devices
2. Selection, manipulation, and 3D user interaction
3. Navigation
4. Menus and interfaces
5. Systems
6. Quest

Appendix C

Resources for XR Design Choices

Immersive Learning Research Network  https://immersivelrn.org/  This group focuses on research-based conclusions. It holds regular weekly, monthly, and annual events including guided adventures in XR.

Educators in VR https://educatorsinvr.com/  This group has an active Facebook group and Discord channel. It is a good place to meet practitioners in VR in education.

Microsoft’s Inclusive Design approach is worth considering: https://edtechbooks.org/-THfv

Microsoft Mixed Reality guidance focuses primarily on augmented reality devices like the Hololens, but much of the research-based advice applies to XR https://edtechbooks.org/-iQCp

APPENDIX D

Storytelling in VR