Abstract

Increasing interest in user experience design (UXD) in the field of learning design and technology (LDT) signals a growing recognition of the importance of the individual experience of using learning technologies to learning—the learner experience (LX). However, a need exists to better define and conceptualize the phenomenon of learning experience design (LXD). Imprecise, interchangeable, and reductive usage of terms and concepts related to LXD frustrates efforts to situate and connect the established traditions of our field with complimentary methods and processes external to LDT (e.g., UXD, human-computer interaction). To approach this need, we performed qualitative content analysis on a corpus of 15 chapters from a recently published edited volume focused specifically on LXD in the field of LDT. Our research questions focused on identifying key terms and concepts, exploring how chapter authors characterized LXD, and examining the perspectives that informed authors’ conceptions of LXD. We approached these questions using a rigorous, multi-phase inquiry process in which we conducted systematic, iterative open-coding. These coding efforts led to the emergence of a rich tapestry of terminology, methods, and concepts associated with LXD. Importantly, while book chapter authors drew from outside the field of LDT, the manner in which they intentionally located their work within established traditions of this field was particularly revealing. Grounded in the voices of these researchers and practitioners, we assert that LXD is a human-centric, theoretically-grounded, and socio-culturally sensitive approach to learning design, intended to propel learners towards identified learning goals, and informed by UXD methods. On the basis of this operational definition, directions for future research are proposed.

Introduction

User-centered design (UCD) and user experience (UX) methods increasingly are being applied in learning design contexts (Cheng, 2019; Dimitrijević & Devedžić, 2021; Jahnke et al., 2020; Matthews & Yanchar, 2018; Shernoff et al., 2020; Stefaniak et al., 2020). This signals a shift in the field of learning design and technology (LDT), moving the field towards more human-centered approaches to designing digital environments for learning (Matthews et al., 2017; McDonald et al., 2019; Quintana et al., 2000; Soloway et al., 1994). Human-centered approaches to learning design seek to provide learners pleasing and effective digital learning tools that are easy to use and that efficiently propel them towards their learning goals (Robinson et al., 2017; Roman et al., 2020). Arguably, the move towards more human-centered methods of design in LDT began with the field distancing itself from the term instructional design and its focus on creation and delivery of educational and training materials (Mor & Craft, 2012). In its place, the term learning design is preferred (Bower et al., 2010), with a focus on the design of learning activities (Beetham & Sharpe, 2007; Oliver et al., 2007). While these changes were happening in LDT, usability and UX methods and processes began to gain prominence in the field of software engineering (Hassenzahl, 2013). Learning design practitioners took heed and began adopting these approaches in their own design practice (Kilgore, 2016). Consequently, the title learning experience designer emerged (Korkmaz, 2018) to describe the job of someone engaged in learning experience design (LXD: Georgiou & Ioannou, 2021; Harrati et al., 2016; Minichiello et al., 2018). Interestingly, these terms and associated concepts have become common parlance in the field of learning design, but with surprisingly limited clarity around what LXD actually is or what it entails. We seek to address this issue in the current paper.

In learning design practice, LXD is thriving. A search on prominent job sites (e.g., indeed.com, monster.com) for the term “learning experience designer” yields thousands of results. In practitioner circles, the LXDCON learning experience design conference is currently in its 6th year (lxd.org/lxdcon/). Higher education certificates and degree programs in LXD are increasing (e.g., Oregon State University’s LXD certificate, Brandeis University’s MS in LXD). Technology tools have emerged that brand themselves as learning experience platforms (e.g., lemonadetraining.com, 360learning.com). Further, conversations around LXD abound on social media (e.g., Facebook, LinkedIn). The growing interest in LXD is not limited only to practice, but extends to academic circles as well.

Within the realm of academic scholarship, the use of terms and concepts associated with LXD is increasing (Figure 1). To explore the extent of this, we performed a simple bibliographic analysis. We ran a search using the Web of Science database on educational publications and citations over the past 20 years using the terms user-centered design, user experience design, learner experience design, and learning experience design (Figure 2). Our results showed that these terms began to gain prominence in the educational research literature starting between 2005-2008, with a substantial increase in publications using these terms starting in 2015. The term learning experience design is somewhat widely used in the literature, with more publications and citations than any of the other terms that were reviewed. Prevalence of this term is increasing substantially, with over a 22% increase between 2018-2019. The second-most prevalent term from our analysis is learner experience design. This term also shows a steadily increasing trend in citations and publications, with a 17% increase in 2019 over the previous year.

Figure 1. Total number of articles including search term published per year

Figure 2. Sum of times articles including search term were cited per year

Law and colleagues state, “It is an intriguing phenomenon that the notion of User Experience (UX) has been widely disseminated and speedily accepted in the Human-Computer Interaction (HCI) community, however, without it being clearly defined or well understood” (2009, p. 709). Arguably, this sentiment also extends to the LDT community. With increasing prevalence and interest in LXD, it is somewhat surprising that terms and concepts related to LXD have not been clearly defined. What is LXD? How does LXD differ from instructional design (ID)? What does a learning experience designer do? Is there a difference between learner experience and learning experience? Answers to these questions are elusive, due in part due to the recency of the LXD phenomenon. Rapid evolution is common in the field of LDT, but can present challenges in establishing common definitions and terminology (Lowenthal & Wilson, 2010; Moore et al., 2011; Volery & Lord, 2000). Furthermore, the field of LDT in some ways is defined by “imported” perspectives (McDonald & Yanchar, 2020) that link the field to outside disciplines. Such imported perspectives are often quite useful in learning design contexts, but understanding their main characteristics, how they compare and differ from one another, and potential deficiencies when applied to learning design is crucial, lest these perspectives be applied to ill effect. Although LXD draws influence from human-computer interaction (HCI), UCD, and UX (Jahnke et al., 2020; Schmidt, Earnshaw, et al., 2020), serious efforts have yet to emerge in the literature to map their characteristics, similarities and differences, and potential flaws in relation to canonical traditions of LDT. Confounding this, terms associated with learning design sometimes have “multiple, complex and sometimes competing roles and meanings” (Cross et al., 2008). For example, the term learning design can be used to describe the practice of design, the product of design, or a field of study (Conole, 2018; Koper & Olivier, 2004).

We argue that a common foundation of consistent terms and distinct concepts is essential to situate and connect LXD work in our own field with that of our sister disciplines (e.g., HCI, UCD, UX, etc.). As noted by Moore and colleagues (2011), ambiguous use of terms and lack of conceptual clarity can confuse efforts to “perform meaningful cross-study comparisons and build on the outcomes from the previous studies” (p. 129). In the vein of promoting consistency and clarity regarding the terms and concepts used in this article, we provide brief definitions in Table 1 below. The authors are well aware that these definitions are not exhaustive and that many of the terms’ meanings and definitions remain the subject of some debate. Further, because no definitions for learning experience (LX) or LXD have yet been agreed upon in the literature, the definitions for these terms are provisional. They are synthesized from the findings of the present research and should therefore not be interpreted as conclusive.

Table 1. Terms and concepts pertinent to the notion of LXD

Term	Definition
User- Centered Design (UCD)	Offshoot of human-centered design used to describe iterative design practice that actively seeks user validation across all phases of design. Recognizes that users’ needs, abilities, and desires should drive design at each stage of the process. Does not prescribe specific methods, but can accommodate a variety of investigative and generative methods (cf. Abras et al., 2004; Chandran et al., 2020; Norman & Draper, 1986; Robinson et al., 2018; Rose et al., 2018; Schmidt et al., 2020; Signoretti et al., 2019; Wang et al., 2018; Zhong & Schmiedel, 2021).
Learner- Centered Design (LCD)	Extension of user-centered design that reconceptualizes the role of the user as a learner. Emphasizes the importance of promoting understanding, performance, and expertise when designing learning technologies. Includes instructional scaffolding, supports, motivation, diversity, and addressing the gap in expertise between learners and experts (cf. Guzdial et al., 1995; Quintana et al., 2000; Soloway et al., 1994).
User Experience (LX)	Individual, perceptive quality that manifests through involvement, interaction, and observable/measurable experience with a technology or product. A consequence of internal factors related to the user, characteristics of the designed system, and the context of interaction. (cf. Hassenzahl, 2008; International Organization for Standardization, 2019; Law et al., 2008, 2009; Tullis & Albert, 2013).
Learner Experience (LX)	An emerging focus area of LIDT located at the crossroads of UX, learning design, and educational technology; concerned with the UX of learners during technology-mediated learning. Focuses on a specific class of user (the learner) engaged in a particular task (related to learning) while using a distinct type of technology (a technology tool designed for learning). Considers issues of how experiential elements might influence learning effectiveness and how perceptual factors might impact learner performance (cf. Jahnke et al., 2020; cf. Schmidt et al., 2020; Tawfik et al., 2021).
User Experience Design (UXD)	Design practice coined by Norman (2013) that seeks to consider every aspect of the relationship between the user-in-context and product. Modern UXD practice adopts a narrower focus on the user’s immediate experience of using a technology product’s user interface (UI). A variety of methods and processes are canonical to UXD, including design thinking, empathy mapping, usability, user stories, etc. Exact origins and precise definition remain the subject of ongoing debate (cf. Hanlon et al., 2021; Law et al., 2008, 2009; Marcus, 2002).
Learning Design (LD)	Can be used to describe design practice, design product or a discrete field of study. Learning design products are formal, reusable elements that can be meta-tagged, searched for, and shared widely via standardized digital methods. Learning design practice is application of a rule-set to describe the teaching-learning process from an instructor’s perspective (cf. Alonso et al., 2008; Conole, 2018; Hummel et al., 2004; Koper, 2005). Serious efforts to define can be found in Koper and Olivier (2004) and the Larnaca Declaration on Learning Design (Dalziel et al., 2016).
Learning Experience Design (LXD)	Situated at the crossroads of learner-centered design and UXD, the term LXD can be used to describe design practice, design product or a discrete field of study. Concerned with both the effectiveness of designed learning interventions and the interconnected and interdependent relationship between the learner-as-user, the designed intervention, and the learning context. Transdisciplinary and complex by nature, LXD practice requires an extensive repertoire of knowledge, skills, and abilities across a range of disciplines. (cf. Abbott, 2020; Chang & Kuwata, 2020; Gray, 2020; Jahnke et al., 2020; Schmidt et al., 2020; Stefaniak & Sentz, 2020; Vann & Tawfik, 2020).

The purpose of this paper is to highlight the need for greater semantic and conceptual clarity around the concepts and language of LXD that increasingly are gaining prominence in the field of LDT and to approach this need through a content analysis of a corpus of recently published literature by active LXD researchers. Given the need for greater semantic and conceptual clarity around the concept of LXD, we performed a qualitative content analysis to articulate the phenomenon of LXD as portrayed in a corpus of 15 book chapters that were recently published in the edited volume Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology (Schmidt et al., 2020). Our approach bears some resemblance to that of Kou and Gray’s (2019) analysis of UX practitioners’ discussions on StackExchange, an online affinity group where practitioners ask and respond to questions germane to UX design practice. A strength of their work is how they captured and characterized the vocabulary of UX design practice from the voices of actors embedded within a situated context, as opposed to seeking practitioners' opinions in response to predefined questions. We sought to do likewise in our study; however, as no comparable LXD community exists with whom we could engage, we therefore selected what we understand is the only collected body of LXD knowledge to-date, the Learner and User Experience Research edited volume (Schmidt et al., 2020). The purpose of our study, therefore, was to explore how authors conceived of LXD as evidenced by the definitions, characteristics, parameters, and contexts found in their book chapters. The research questions that guided our inquiry were:

RQ1: What key terms and concepts are used across the corpus of book chapters and with what prevalence?

RQ2: How do authors characterize LXD within their book chapters?

RQ3: What perspectives inform authors’ conceptions of LXD?

Methodology

The current research was performed by the first and second authors of this paper, the lead researcher (a university professor) and the assistant researcher (a trained doctoral student), respectively. Our methods borrow from the tradition of grounded theory in our application of open-coding techniques (Glaser & Strauss, 2017; Strauss & Corbin, 1998). All data were first reviewed to gain a sense of the whole, with impressions recorded in field notes. These notes were then reviewed and initial interpretations were made. This formed the basis of an emergent coding scheme. After this, all data were re-read and coded. This process unfolded across three phases, in which we performed preliminary open-coding (Phase 1), provisionally applied and refined the coding scheme (Phase 2), and finalized the coding scheme and completed all analyses (Phase 3). We provide a detailed description of our research processes in the following sections.

Phase 1 Procedures

In Phase 1, we performed a preliminary review of our data set and recorded our impressions in field notes. We began by systematically reviewing four chapters to orient our inquiry and identify characteristics such as definitions, operationalizations, and problem statements. Using an iterative process, preliminary categories emerged that first were recorded in a spreadsheet, then refined, and finally used to create a structure for systematically annotating all book chapters. The resulting spreadsheet sections included: definition, positionality, context, theoretical perspective, learning domain/subject matter, areas of convergence/divergence, type of chapter (e.g., conceptual, empirical, case-study), and key terms. Following this, the assistant researcher systematically annotated 12 chapters based on these spreadsheet categories, and the lead researcher annotated three. Over the course of this procedure, the lead and assistant researchers met regularly to discuss the coding process, make refinements to the structured spreadsheet and annotations, and begin developing a preliminary coding scheme for Phase 2 of the research. During these meetings, coding categories were further developed and refined, guidelines for analysis were established, and preliminary findings were recorded. Upon completion of Phase 1, all chapters had been annotated using a structured process, and preliminary coding procedures had been established. These served as the inputs for Phase 2 of our analysis.

Phase 2 Procedures

To facilitate our Phase 2 systematic open-coding process, we used the computer-aided qualitative data analysis software (CAQDAS) tool Dedoose (https://www.dedoose.com/), as illustrated in Figure 3 and Figure 4. The open-coding procedures and development of the coding scheme went through several iterations. First, the initial open-coding scheme from Phase 1 was imported into Dedoose and used to code excerpts of book chapters. The lead researcher first coded one book chapter and made refinements to the coding scheme while the assistant researcher observed. The lead researcher explained procedures and reasoning for coding decisions using a think-aloud process. Next, the assistant researcher applied the coding process from the first stage to one chapter while the lead researcher observed, provided guidance, and answered questions. Both researchers then collaboratively coded one book chapter as a dyad, after which the assistant researcher coded another chapter independently. During this process, the coding scheme was finalized using an iterative process (Table 2). Finally, the lead and assistant researchers independently coded book chapters using the finalized coding scheme. The researchers met regularly to discuss coding discrepancies or issues with the coding scheme and to resolve these issues. Upon completion of the third stage, inter-rater reliability estimates were calculated based on the lead and assistant researchers’ coding.

Figure 3. Example of a fully coded chapter in the qualitative analysis software, Dedoose

Figure 4. Code application chart from qualitative analysis software, Dedoose, showing codes applied across chapters

Coding category	Examples
Key aspects of LXD	socio-cultural perspectives, technological usability, pedagogical usability, empathy as key component, hedonic qualities, human-/user-centered, personal, cross-/multi-/inter-/trans-disciplinarity, learning outcomes-oriented.
Design perspectives of LXD	Color design, contextual design, emotional design, intuitive design, pedagogical design, design thinking, learner-centered design, user-centered design.
Disciplinary perspectives of LXD	HCI, instructional design, learning design, LDT, UX, UXD.
Theoretical perspectives of LXD	Cognitive load theory, community of practice, flow theory, connectivism, activity theory.
Methods/Methodology of LXD	A/B testing, analytics, card sorting, cognitive walkthroughs, ethnography, eye-tracking, EEG, focus groups, heuristic evaluation, expert review, needs assessment, contextual analysis, participatory design, co-design, personas, prototyping, scenarios, think-aloud, cognitive interviews.
Key terminology related to LXD	Design thinking, empathy, learner experience, learning experience, user-centered design, learner-centered design, think-aloud, co-design.

Table 2. Emergent coding categories and examples.

Coding Reliability

Mentoring and dyad coding approaches were employed to promote coding reliability as described above. We also performed inter-rater reliability calculations to contribute to the rigor of our coding results. The lead and assistant researcher coded and compared 37% of the entire corpus of excerpts, yielding a Cohen’s Kappa estimate of 0.765. These results fall in the category of good agreement according to Landis and Koch (1977) or excellent agreement by Cicchetti (1994).

Phase 3 Procedures

In phase 3 we sought to explore trends across terms coded as key terminology. To generate a precise list and frequency count of these terms, we stripped all references from chapters to avoid inflation of the terms’ frequency count and combined them into a single document. We then ran searches on all coded terms and established corresponding frequency counts. Alternate formulations of terms were also used, such as plural forms, alternative spellings, etc. In the third step, all terms were stratified along the dimensions of LXD components, design approaches, methods, methodological approaches, etc. Some terms that were initially coded as key terminology were deemed to be so general that they did not meaningfully contribute to our research question (e.g., fidelity, functionality, flow, etc.), and hence were removed. The remaining results were reviewed and pruned, with coding categories that only described one or two terms being collapsed into existing categories or removed. Category descriptors were revised accordingly. Frequency counts also were updated as the refinement process unfolded.

Findings

RQ1: What key terms and concepts are used across the corpus of book chapters and with what prevalence?

Research question 1 focused on the prevalence of key terms and concepts. These were identified in book chapters and coded as key terminology. A total of 44 terms were identified, which we categorized across four categories: (1) LXD attributes, (2) LXD research and evaluation methods, and (3) LXD approaches.

LXD Attributes

The coding category LXD Attributes represents the terms used by authors that contribute to the overarching gestalt of LXD. These are terms that serve to shape and form the general concept of LXD. Figure 5 presents terms we assigned to the LXD Attributes category and their corresponding frequencies in a descending order. Among all 11 terms in this category, two closely related terms user experience (31.3%) and usability (20.4%) have the highest frequencies. This could suggest that these two related concepts are perhaps predominant aspects of LXD. The third most-mentioned term in this category is learning design (12.6%), perhaps underscoring the critical importance of learning to the phenomenon. Two terms worth special notice are learning experience (8.3%) and learner experience (7.5%). While this could support the centrality of individual experience to LXD, the terms are sometimes used interchangeably, suggesting potential confusion as to how the terms are distinguished from one another (a point we address in the discussion). The remaining terms are used less frequently relative to those discussed above, and while they comprise less than 20% of term frequency in this category, they could imply that design thinking and related concepts of empathy and prototyping play a role in characterizing LXD.

Figure 5. Frequency of terms related to LXD attributes.

LXD Research and Evaluation Methods

The coding category LXD Research and Evaluation Methods represents the research and evaluation methods that authors used or discussed related to LXD practice. Figure 6 depicts the frequencies of terms categorized as LXD research and evaluation methods. Twenty terms were assigned to this category. Of these, the top five most frequent terms are personas (16%), analytics (14.8%), think-aloud (13.7), requirements (9.9%), and scenarios (5.3%). Many, if not all, of these terms are related to formative design and evaluation, suggesting that LXD methods could have a particular focus on these aspects of design practice. The next five most frequent terms are cognitive interview, focus group, contextual analysis, card sorting, and cognitive walkthroughs, all of which have similar frequencies (3.8%-4.6%) and, combined, account for 16.7% of the terms in this category. Nearly all of these are evaluation methods, perhaps highlighting the centrality of evaluation to LXD practice. In aggregate, the remaining 10 terms’ frequencies account for 19.4% of the total frequencies and provide insight into the multitude of research and evaluation methods that are actively being used in LXD.

Figure 6. Frequency of terms related to LXD research and evaluation methods.

LXD Approaches

The coding category LXD Approaches represents the specific design frameworks or approaches that were used or discussed by authors related to their LXD practice. Table 3 shows the frequency of terms related to LXD approaches, including co-design (58.6%), participatory design (20.7%), emotional design (13.5%), universal design for learning (4.5%), and contextual design (2.7%). The top two design approaches have some similarities and together count for 79.3% of the total frequency. Intentional collaboration with participants during the design process could represent a significant departure from tradition. Indeed, Gray (2020) asserts that “the use of participatory design and co-design approaches [...] is not present in traditional LDT scholarship” (p. 7).

Term	Description	Frequency
Co-design	Often contrasted with top-down approaches to design in which a user’s role is more passive, co-design conceives of users as active participants in the design process who are valued as equal contributors (Roschelle et al., 2006).	58.6%
Participatory design	Similar to co-design, but differentiates itself in that users are not seen as equal contributors. Although user input is solicited in participatory design, the design team is the arbiter of decision-making (Engelbertink et al., 2020).	20.7%
Emotional design	Approach to designing multimedia in an appealing and engaging manner so as to promote positive emotional responses in learners (Mayer & Estrella, 2014).	13.5%
Universal design for learning	An approach to learning design that seeks to account for learner differences by providing multiple means of representation, action and expression, and engagement (D. Rose & Meyer, 2002)	4.5%
Contextual design	A systems-focused design process that prioritizes user data collected in-situ as the base criterion to inform product development (Holtzblatt & Beyer, 2014)	2.7%

Table 3. Descriptions and frequencies of terms related to LXD approaches.

RQ2: How do authors characterize LXD within their book chapters?

Research question two focused on authors’ characterizations of LXD within their book chapters. Some authors succinctly defined LXD (Table 4), whereas others provided broader descriptions. Generally speaking, analysis of authors’ portrayals of LXD revealed substantial agreement. However, while descriptions and definitions of LXD tended to agree that LXD is the result of integrating design practice from other fields (e.g., HCI, architecture, product design, software design, etc.) into instructional and learning design, a point of divergence was how author’s described the integration of LXD with disciplinary perspectives from outside the LDT field. Authors described this integration using the terms cross-disciplinary, multi-disciplinary, interdisciplinary, and transdisciplinary. These terms suggest a spectrum of integration. On the cross- or multi-disciplinary end of the spectrum, LXD is viewed as the application of UXD methods in a learning design context, essentially replacing the term user in UXD with the term learner. On the transdisciplinary end of the spectrum, LXD is conceived of as having the potential to become a new and emergent paradigm in our field that derives from HCI, UX design, and LDT, but that ultimately transcends the currently established disciplinary boundaries of LDT. Some authors positioned LXD as interdisciplinary, for example, as a result of the combined inputs from both UX design and learning design, or as an integration of LXD and UX design.

Authors	Definition
Vann and Tawfik (2020)	“How the interface design aligns with principles of human-computer interaction and learning processes to support student knowledge construction” (p. 1).
Jahnke et al. (2020)	“Focus on improving the usability and LX of [...] learning technology from the perspective of [...] the learner” (p. 2).
Chang and Kuwata (2020)	“Practice of designing learning as a human-centered experience that leads to a desired goal” (p. 2).
Stefaniak and Sentz (2020)	“Process of designing products that are relevant to the everyday experiences of users or learners [that] encompasses the ability for a designer to address all the ways a learner [...] will interact with the product (intervention) being developed” (p. 1).
Abbott (2020)	“An approach that foregrounds learners and their desired outcomes in a goal-oriented way, acknowledging individual experience” (p. 1).

Table 4. Examples of LXD characterizations provided by chapter authors

Although the above points of divergence were noted, we also found that a number of key constructs are shared across authors. One example is using UXD processes and methods in the learning design process, for example, participatory design, co-design, and design thinking. In the following sections we detail how authors characterized LXD in their chapters using four broad themes: (1) human-centric, (2) theoretically-grounded, (3) informed by UX methods, and (4) socio-culturally sensitive. These themes are briefly summarized in Table 5. Interestingly, these themes share similarity with Floor’s (Floor, 2018) practitioner description LXD, namely, that LXD is human-centered, goal-oriented, theoretically-grounded, and interdisciplinary.

Theme	Description
Human-centric	LXD focuses centrally on human experience from the perspective of the learner, as well as other learning technology users (e.g., teacher, LMS administrator).
Theoretically-grounded	Theory is foundational to LXD, which is principally inspired and guided by theoretical perspectives that have found resonance in the field of LDT (but also draws from theories rooted in outside traditions such as HCI and UX).
Informed by UXD methods	LXD is informed by UXD methods, but these methods are adapted and extended in LXD so as to be more appropriate and effective within a learning design context.
Socio-culturally sensitive	LXD seeks to promote empathetic understanding of the learner, their socio-cultural context, as well as the context in which they engage in socially-mediated meaning making.

Table 5. Four broad themes that characterize LXD

Human-centric

Authors’ descriptions and definitions of LXD reveal that, much like UXD, human experience is the central focus in the LXD process. However, in a LXD context, key differences are evident. Raza and colleagues assert, “the field has started exploring and adopting human-centered or user experience design methods” (2020, p. 2). While Raza uses the term human-centered, terms such as learner and instructor were used with more prevalence. For example, Abbott emphasizes the learner: “within LXD the learner’s needs, experiences, desires, and emotions are crucial” (2020, p. 2). McCarthy, Watanabe, and McNamara emphasize the teacher: “teachers play an important role in the success of educational technology in the classroom, yet instructors are often ignored as both facilitators and end-users” (2020, p. 2). Many authors seem to conceive of users as being a general term and learners being a unique category of user, as illustrated by Jahnke and her colleagues: “The focus of UX is [...] quite broad, with applicability to any technology in any context for any user. [LXD], however, has a narrower focus on improving the usability and LX of only one type of technology—learning technology—from the perspective of only one type of user—the learner (2020, p. 1).” LXD is therefore characterized as being centrally focused on a human in the specific role of the learner, as well as the more general roles of other learning technology users (e.g., teacher, LMS administrator).

Theoretically-grounded

Every book chapter adopted a theoretical lens to describe LXD work, although with varying levels of detail. This suggests that theory is central to LXD practice. The theories that guide LXD draw not only from the tradition of LDT, but also from outside disciplines such as HCI and UCD. For example, Kimmons (2020) discusses a theory that receives little attention outside of the visual arts—color theory. He carefully connects color theory to learning theories of motivation and self regulation: “Though the connection between color and learning may not be obvious at first, by influencing learner emotion, attitude, and interest, color can influence learner behaviors and attitudes, which in turn will influence their learning” (p. 5). Multidisciplinary theoretical perspectives seem necessary in LXD because “designers that approach the interface from only a learning theory perspective may encounter unforeseen obstacles due to user experience (UX) challenges” (Vann & Tawfik, 2020, p. 1). Across chapters, learning theory was consistently privileged over UX design. Bowen and colleagues explain, “while important to user experience (UX) and eventual product viability, [engagement, likability, and usability] should not be the sole focus of early testing. When developing tools intended to foster learning, it is paramount to explicitly define and test the learning theories on which those tools depend through deliberate learning experience (LX) design” (p. 3). Theory guides LXD practice. Although LXD practice draws from theories rooted in outside traditions such as HCI and UX, it is principally inspired and guided by theoretical perspectives that have found resonance in the field of LDT.

Informed by UXD Methods

LXD practice readily adapts learning design processes to align with those of design thinking and adopts design thinking techniques, such as empathy-based approaches for assessing needs. Design thinking was a recurring topic across many of the book chapters. This is perhaps unsurprising, as UX design is heavily influenced by design thinking, as Gray (2020) notes: “‘Design thinking’ has been taken on perhaps most substantially by practitioners known as user experience (UX) designers” (p. 1). Authors applied design thinking to their own learning designs, such as the MOOC described in Cavignaux-Bros and Cristol (2020). Although design thinking is frequently represented as a process model in the literature, Stefaniak and Sentz’ (2020) consideration of this key concept was more nuanced, suggesting that design thinking should be “treated like more of a mindset rather than a specific method” (p. 3).

LXD practice applies various UX design techniques and methods, such as participatory design, co-design, think-aloud, cognitive walkthrough, etc. Across all book chapters, authors described application of UX design methods in learning design contexts (for an overview, see Figure 3). Of these, one method that was frequently discussed and applied was usability evaluation. However, conceptions of usability in learning design contexts deviated from more traditional views (e.g., International Organization for Standardization, 2019). Traditional usability perspectives focus on technological usability. While technological usability was acknowledged as important to LXD, it was conceived of primarily as a prerequisite to or a conduit for learning, but not as a central driver. For example, Oprean and Balakrishnan’ (2020) framework for immersive learning underscores the importance of usability in promoting learner engagement. The traditional foci of technological usability—ease-of-use, effectiveness, efficiency, and user satisfaction—were seen as insufficient to inform learning design. As Quintana and colleagues (2020) argue, not all technological usability heuristics “are directly relevant to an educational context [...] they require an integrated approach, one that does not artificially separate usability and learning considerations” (p.3). Gregg and colleagues (2020) state, “online learning design requires more than the implementation of technical usability strategies and techniques” (p. 3). Given the inadequacy of technological usability alone to inform learning design, some authors advocated for the application of alternative forms of usability that specifically target learning design, such as pedagogical usability. According to Gregg and colleagues (2020), pedagogical usability “refers to a category of usability strategies meant to operationalize learning-centered design principles in online learning environments” (p. 3). Extending this, Jahnke and colleagues (2020) argue that “usability evaluation of technology-enhanced learning should embrace a broader conceptualization of usability, considering (a) the social dimension, (b) the technological dimension, and (c) the pedagogical dimension” (p. 2), which they label sociotechnical-pedagogical usability. To summarize, LXD practice is informed by UX design methods, but these methods must be adapted and extended for more appropriate and effective application in learning design contexts .

Socio-culturally Sensitive

While socio-cultural theory is widely used to inform the design of learning and instruction in LDT (e.g., social constructivism, activity theory, distributed cognition), socio-cultural sensitivity is not necessarily intrinsic to the methods and processes of instructional and learning design. In contrast, socio-cultural sensitivity is central to LXD. Gray (2020) asserts: “Rather than assuming that learners have similar characteristics and experiences, which often advantages certain types of students in powerful structural ways, [learning designers should] identify mechanisms whereby learning experiences can value unique and subjective learner qualities” (p. 10). This can be a challenge in learning design contexts in that, as Schmidt and colleagues (2020) maintain, “Learning design teams tend to be small (2-3 members) or consist of an individual learning designer. Such teams can lack sufficient socio-cultural perspective to design for a culturally sensitive and diverse learner experience” (p. 6). Key to developing socio-cultural sensitivity is empathy. Chang and Kuwata (2020) state, “Human-centered LXD includes empathetic understanding of the learner, the socio-cultural and technical context in which they are embedded, and the individual and socially mediated meaning making process as driven by the learners” (p. 3). Development of such empathetic understanding is central to questions of equity. According to Raza and colleagues, such understanding can uncover students’ perceptions and help designers in their “noticing and understanding situations in which learners’ experiences differ based on their race and gender and in turn how these differences impact overall classroom culture” (p. 5). A variety of methods to promote socio-cultural sensitivity were employed by authors. For example, development of personas, which “can provide context for designers to consider [...] socio-cultural perspectives more intentionally in their learning designs” (Schmidt, Earnshaw, et al., 2020, p. 6). Other methods such as as participatory design and co-design were employed by a other authors, which is notable in that such approaches are “discussed infrequently in an LDT context, and [...] almost completely lacking in explicit support through design processes and methods” (Gray, 2020, p. 9).

RQ3: What perspectives inform authors’ conceptions of LXD?

Research question 3 focused on the perspectives that informed authors’ conceptions of LXD. Our open coding procedures revealed two distinct categories that contribute to how authors conceive of LXD. Firstly, we found that author’s conceptions of LXD are, perhaps unsurprisingly, influenced by their conceptions of learning. Authors’ conceptions of learning were stratified into five categories: learning as (1) process, (2) personal endeavor, (3) contextually-situated, (4) experience, and (5) goal oriented. These categories are summarized in Table 6. Secondly, we found that authors’ conceptions of LXD were informed by a variety of perspectives, including (1) design, (2) disciplinary, (3) methodological and procedural, and (4) theoretical. These perspectives are summarized in Table 7.

Learning Perspective	Description
Learning as process	Knowledge construction A transaction between internal and external factors or an individual and the environment Transfer of knowledge from the learning space to a real-world environment A process that occurs within nebulous environments of shifting core elements—not entirely under the control of the individual Progressive
Learning as personal endeavor	Effective cognitive processing Individual meaning-making Through personal inquiry and mental models
Learning as contextually-situated	Well-situated within a relevant context Mediated by intentional interaction and communication with learning technologies and a broader socio-cultural context A social effort Occurs one-on-one or in a group/team
Learning as experience	Emotions influence learning Aesthetics influence learning The quality of the experience influences learning, such as cognitive engagement and affective responses
Learning as goal-oriented	Accomplishing results Bridge gaps between current and desired knowledge, skills, and abilities Meet the needs of equity in classroom settings Effectiveness

Table 6. Perspectives on learning that influence authors’ conceptions of LXD

Category	Description	Examples from book chapters
Design perspectives	Design approaches applied or discussed by authors to inform and/or advance LXD practice.	Color design Contextual design Emotional design Intuitive design Pedagogical design Design thinking Learner-centered design User-centered design User experience design
Disciplinary perspectives	Disciplines explicitly mentioned by authors that contribute to and/or influence LXD.	Human-computer interaction Instructional design Learning design Learning, design, and technology User experience
Methodological and Procedural perspectives	Methods and procedures applied or discussed by authors in LXD practice.	A/B testing Analytics Card sorting Cognitive walkthroughs Ethnography Eye-tracking / EEG Focus groups Heuristic evaluation / expert review Needs assessment Contextual analysis Participatory design and co-design Personas Prototyping Scenarios Think-aloud / cognitive interviews
Theoretical perspectives:	Specific theories explicitly applied or discussed by authors to guide LXD practice.	Cognitive load theory Color theory Community of practice Flow theory Connectivism Distributed cognition Activity theory Sociotechnical theory Social constructivism Actor-network theory

Table 7. Perspectives informing authors’ conceptions of LXD

Discussion

According to Chang and Kuwata (2020), “There is a need to provide a concrete definition of LXD to guide the conceptualization and practice of learning design” (p. 2). Adapting UX to the field of learning design has led to adoption of associated terminology, but there has been little work to-date in the way of systematically defining LXD in a broad sense, operationalizing LXD in a way that could prove useful from the perspectives of research and practice, or aligning this concept with the theoretical foundations of our field. To approach the need for greater semantic and conceptual clarity around the phenomenon of LXD in the field of LDT, we performed content analysis on a corpus of recently published literature by active LXD researchers. On the basis of this research, we define learning experience design as a human-centric, theoretically-grounded, and socio-culturally sensitive approach to learning design, intended to propel learners towards identified learning goals, and informed by UXD methods.

Our first research question attempted to identify key LXD terminology used in those book chapters and to establish how frequently those terms were used. Findings related to this research question provide insight into key terminology used by authors and the frequency with which various terms were used. Three thematic categories emerged from our coding process related to key terminology: (1) LXD attributes, (2) LXD research and evaluation methods, and (3) LXD approaches. A variety of related terms is found within each of these categories. Analysis of the frequency with which these terms were used provides insight into the prevalence of terms across the book chapters. Our intent with presenting frequencies of terminology usage is not to suggest that certain terms are more or less important to LXD, but instead to present a lexicon of prominent nomenclature used by a segment of the LXD discourse community. However, this lexical repository has limitations. Firstly, it was drawn from a narrow sample of only 15 book chapters. Secondly, and because of this, it is incomplete and biased. For example, some authors might have repeated a term multiple times, whereas others might have only mentioned a term once or twice. We are therefore cautious in our interpretation, lest we conflate the signifiers (the terminology) with what they might signify. Establishing the conceptual relationships between these terms remains a direction for future research.

Our second research question sought to explore how authors characterized LXD in their book chapters. A range of perspectives from both within and outside the field of LDT informed authors’ conceptions of LXD, including design perspectives, disciplinary perspectives, methodological and procedural perspectives, and theoretical perspectives. However, eclipsing these were authors’ perspectives on learning. These perspectives were explicated in our approach to the third research question, which sought to identify and stratify perspectives that informed authors’ conceptions of LXD. Learning takes primacy in LXD as a contextually-situated, personal endeavor towards a learning goal, the process of which is interpreted through the lens of individual experience. Taken together, the two categories of perspectives that emerged from our analysis (i.e., perspectives of learning and perspectives of LXD) represent both central and peripheral influences. That is, peripheral aspects of LXD include design, disciplinary, etc. perspectives, whereas most central to how authors conceived of LXD were perspectives related to learning. This suggests that further inspection of the underlying assumptions of perspectives related to UX could be needed—a direction for future research. Further, future research is warranted that seeks to intentionally combine imported theoretical perspectives (e.g., from UX or HCI) with the canonical perspectives of LDT, similar to Gibbons’ (2013) masterful combination of perspectives from fields such as architecture and interaction design with LDT.

Implications for Practice

From a practice perspective, the work presented here contributes substantially not only towards defining the work of LX designers, but also towards more formally endorsing the work that LX designers have been doing for some time as legitimate and relevant manifestations of learning design practice. Applying external methods and processes to learning design practice (i.e., usability testing, cognitive walkthroughs, etc.) can draw criticism in that these approaches do not specifically address learning or performance improvement. Foregrounding the centrality of learning when drawing from external disciplines such as UX and HCI can underscore the relevance of these external methods and processes to learning design, which can further serve as powerful justification for exploring, adopting, and applying such procedures within one’s own design practice. Further, this work establishes precedent for LX designers related to terminology, specific methods, design approaches, theories, and perspectives on learning. It also provides a foundation to build upon, perhaps towards development of practitioner guides and textbooks such as the approachable work of Steve Krug (2010, 2014) in the realm of usability evaluation or Tullis and Albert’s widely used textbook on user experience evaluation (2013). With a scarcity of resources on LXD in the field of LDT, there are many possibilities for contributing to practice.

Implications for Research

Although the work in this article has sought to provide clarity around terms and concepts related to LXD, further work is needed to define the phenomenon, to identify its component parts, and to articulate its conceptual boundaries. Currently, the primary theoretical inputs for LXD are drawn largely from HCI and UX and then interpreted from the perspective of learning. However, as illustrated by the many examples in the Learner and User Experience Research (Schmidt et al., 2020) edited volume, it is clear that many researchers are operating on what Honebein and Reigeluth (2021) would characterize as their own personal instructional design theories, which are “a set of ideas focused on how to ‘create’ instruction rather than ‘describe’ instruction” (p. 3). Extending this is Bowen and colleagues’ (2020) notion of “theories of change” in LXD, which serve to frame designers’ “early investigations into a specific learning problem, their successive iterations in learning design, and their repeated testing with targeted learners” (p. 1). Explicating these tacit, individual theories and their underlying assumptions and influences presents a potential direction for future research that could productively move the field towards establishing a theory of LXD that is born of the discipline of LDT. This could represent a step towards the “originary theory” for which McDonald and Yanchar (2020) advocate. Originary theory is “(a) a set of models, frameworks, principles, or other products of inquiry that (b) describe, conceptualize, or otherwise structure knowledge from the unique perspective of the field in which the theory is generated, and that (c) offers a contribution to knowledge that cannot be fully reduced to, or explained by, the theoretical contributions of other fields” (p. 638). Not only can such theoretical guidance unveil novel methods and processes to inform practice, it also can serve to shape the field and influence professional identities. In this light, we present in the following section a preliminary conceptual model of LXD.

Towards a Conceptual Model of LXD

This article has provided an overview of the terminology used by authors, the manner in which they characterized LXD, and the perspectives that influenced their conceptions. As a whole, this research could provide a signpost for future researchers seeking further clarity in terms and concepts related to the emerging focus area of LXD. Synthesizing the findings presented here, LXD draws from multiple external disciplines. It is a confluence of disciplinary, design, methodological, and theoretical perspectives that are both internal and external to LDT (Figure 7). As evidenced by the current research, external influences can be found in abundance in LXD practice and are reflected in the terms and concepts used to communicate and characterize the phenomenon. However, LXD is greater than the sum of its parts. Taken together, LXD emerges not as a patchwork of borrowed influences but instead as a distinct, cohesive expression of learning design. Indeed, it is the influence of learning design from which LXD derives its unique character. That is, all external methods, theories, and design approaches are encountered, interpreted, adapted, and applied in LXD through the lens of learning design. We illustrate this point in Figure 7 and discuss the implications in the following paragraphs.

Figure 7. Conceptual model of LXD as a confluence of disciplinary, design, methodological, and theoretical perspectives, both internal and external to LDT, and with a central focus on learning.

Figure 7 provides a conceptual model of LXD. Within this model, the four perspectives detailed previously in Table 7 (i.e., design perspectives, disciplinary perspectives, methodological and procedural perspectives, and theoretical perspectives) are illustrated as four intersecting circles. The overlapping sections do not represent the influence or weighting of perspectives, but instead are intended to visually represent their interconnected nature. Superimposed atop these four intersecting circles is a white circle, intended to portray the boundary of LXD much like the plasma membrane of a cell defines the cell boundary. That is, this boundary separates and delineates that which is internal from that which is external to LXD. By external, we refer to the outside perspectives alluded to by McDonald and Yanchar (2020) that are often incorporated into LDT. Although these authors point to theoretical perspectives specifically, LDT also readily imports disciplinary, design, and methodological perspectives; for example, design thinking is a prominent perspective that recently has captured the attention of the LDT community (Hokanson & Gibbons, 2013; Hokanson & Kenny, 2020; McDonald et al., 2019). In order for external perspectives to permeate the boundary of LXD, they pass through what we characterize as a “learning filter.” This takes place as external perspectives are considered from the perspective of learning, as detailed in Table 5 (i.e., learning as process, personal endeavor, contextually-situated, experiential, and/or goal oriented), and adapted to the extant methods, processes, and theories that have found resonance in LDT. This is an indirect process that occurs over time as learning designers encounter tensions in adapting external perspectives to their own design practice and make consequent adjustments.

An example of “filtering” an external perspective through a learning lens can be found in Gray’s (2020) alignment of HCI and UX methods and concepts to the ADDIE model, in which the need to recognize and reconcile divergent views of design is highlighted. This is achieved through “further engagement in the research-practice divide and the differing definitions and conceptual vocabulary that describes design activity” (p. 6). Another example is found in the application of usability evaluation methods to digital learning environments. The broad focus of usability evaluation on any user in any context has been recognized as dissonant with the needs of learning designers. Usability tends to focus on users and how they generally interact with and experience digital products, systems or services. However, using products to accomplish a range of goals is at odds with the very specific ways that learning technologies are designed to support learners in attaining learning objectives. In LXD, usability does not focus on any user performing any task with any technology, but instead focuses on a specific class of user (the learner) who is engaged in a particular task—related to learning—while using a distinct type of technology (a technology tool designed for learning). In recognition of this tension, some researchers advocate for a form of usability evaluation that focuses more intentionally on learning called pedagogical usability (Lim & Lee, 2007; Moore et al., 2014; Nokelainen, 2006). Others suggest an even broader conceptualization that considers “(a) the social dimension, (b) the technological dimension, and (c) the pedagogical dimension” (Jahnke et al., 2020, p. 2) of learning, or “sociotechnical-pedagogical” usability. Both of these examples serve to illustrate the tensions and complexities associated with filtering external perspectives through a lens of learning. Future research is needed that explicitly considers the various components of the model related to empirical and theoretical support.

Conclusion

Although the work presented here serves as an initial step towards providing greater clarity around terms and concepts related to LXD, a number of questions remain. For example, who is the learner in LXD? Is it only the individual engaged in learning (Tsay et al., 2018; Wood & Shirazi, 2020)? If we accept that UX as an emergent quality predicated by all aspects of the user's interaction with a given technology system (International Organization for Standardization, 2019; Madariaga et al., 2021; Norman & Draper, 1986) and that key to UX is a central focus on the user and the user’s needs (Hassenzahl, 2004), then when UX methods are applied in LDT, its focus on the user of a technology system necessarily shifts to a focus on the user of the learning technology. This conceptualization is in accord with Quintana and colleagues’ (2020) depiction of learners as a distinct category of users who differ in terms of domain expertise, heterogeneity, motivation, etc., but distinguishes itself in that learning technology users may include not only learners but also teachers, designers, system administrators, etc. (Chew et al., 2018). On this basis, it follows that experience design in the context of learning technologies is not limited to considerations of the end user only in the role of learner, but also to end users in other roles (e.g., teacher, instructor, administrator). This suggests an important conceptual distinction between the practice of learning experience design and the notion of learner experience. We define learner experience as a quality that is uniquely perceived by an individual actor that, to paraphrase Hassenzahl and Tractinsky (2006), can be conceived of as a “subjective, situated, complex, and dynamic” (p. 95) quality that emerges based on the learner’s internal state, the characteristics of the learning technology, and the context within which the learner interacts with the learning technology. As a manifestation of human-centered learning design practice, it would seem LXD practice should account for the roles of all end users of a learning technology related to how those roles might influence the individual learner experience. In accord with the definition we provide above, however, we argue the concept of learner experience should remain circumscribed to the role of the individual learner.
Findings presented here suggest the field of LDT is shifting toward a more human-centered approach, but the concept of LXD is still emerging. This paper firstly established the increasing prominence of terms associated with LXD and secondly summarized and synthesized current and emerging views of LXD in the field of LDT based on a corpus of 15 book chapters published in a recent Open Access book. The diverse views demonstrate the multiplicity of LXD, which may contribute to a better scoping of this concept. The key constructs presented in this research may serve as a reference for future studies. However, our proposed definition and conceptual model is but the first attempt toward systematically defining and operationalizing LXD, as well better aligning LXD with the theoretical foundations of our field. Further research is warranted.

References

Abbott, D. (2020). Intentional learning design for educational games: A workflow supporting novices and experts. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/11_intentional_learn

Abras, C., Maloney-krichmar, D., & Preece, J. (2004). User-centered design. In Bainbridge, W. (Ed.). Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications.

Alonso, F., López, G., Manrique, D., & Viñes, J. M. (2008). Learning objects, learning objectives and learning design. Innovations in Education and Teaching International, 45(4), 389–400. https://doi.org/10/fnsbb4

Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: Designing and delivering e-learning. Routledge.

Bowen, K., Forssell, K. S., & Rosier, S. (2020). Theories of change in learning experience (LX) design. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/lx_theories_of_change

Bower, M., Hedberg, J. G., & Kuswara, A. (2010). A framework for Web 2.0 learning design. Educational Media International, 47(3), 177–198. https://doi.org/10/d38ghw

Cavignaux-Bros, D., & Cristol, D. (2020). Participatory Design and Co-Design—The Case of a MOOC on Public Innovation. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/participatory_and_co_design

Chandran, S., Al-Sa’di, A., & Ahmad, E. (2020). Exploring user centered design in healthcare: A literature review. 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 1–8. https://doi.org/10.1109/ISMSIT50672.2020.9255313

Chang, Y. K., & Kuwata, J. (2020). Learning experience design: Challenges for novice designers. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/LXD_challenges

Cheng, K.-H. (2019). Parents’ user experiences of augmented reality book reading: Perceptions, expectations, and intentions. Educational Technology Research and Development, 67(2), 303–315. https://doi.org/10/gjh33b

Chew, S. W., Cheng, I.-L., Kinshuk, & Chen, N.-S. (2018). Exploring challenges faced by different stakeholders while implementing educational technology in classrooms through expert interviews. Journal of Computers in Education, 5(2), 175–197. https://doi.org/10/gjksz2

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284. https://doi.org/10/ds968s

Conole, G. (2018). Learning design and open education. International Journal Of Open Educational Resources, 1(1). https://doi.org/10/ggxvq4

Cross, S., Conole, G., Clark, P., Brasher, A., & Weller, M. (2008). Mapping a landscape of learning design: Identifying key trends in current practice at the Open University. In: LAMS conference. 25th July, 2008, Cadiz, Spain.

Dalziel, J., Conole, G., Wills, S., Walker, S., Bennett, S., Dobozy, E., Cameron, L., Badilescu-Buga, E., & Bower, M. (2016). The Larnaca Declaration on Learning Design. Journal of Interactive Media in Education, 2016(1), 7. https://doi.org/10/ggxvpb

Dimitrijević, S., & Devedžić, V. (2021). Utilitarian and experiential aspects in acceptance models for learning technology. Educational Technology Research and Development. https://doi.org/10/gjh33f

Engelbertink, M. M., Kelders, S. M., Woudt-Mittendorff, K. M., & Westerhof, G. J. (2020). Participatory design of persuasive technology in a blended learning course: A qualitative study. Education and Information Technologies, 25, 4115–4138. https://doi.org/10.1007/s10639-020-10147-x

Floor, N. (2018). Fundamentals of learning experience design. Retrieved from: https://lxd.org/fundamentals-of-learning-experience-design/

Georgiou, Y., & Ioannou, A. (2021). Developing, enacting and evaluating a learning experience design for technology-enhanced embodied learning in math classrooms. TechTrends, 65(1), 38–50. https://doi.org/10/ghpfk8

Gibbons, A. S. (2013). An architectural approach to instructional design. Routledge.

Glaser, B. G., & Strauss, A. L. (2017). Discovery of Grounded Theory: Strategies for Qualitative Research. Taylor & Francis.

Gray, C. M. (2020). Paradigms of knowledge production in human-computer interaction: Towards a framing for learner experience (LX) design. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/paradigms_in_hci

Gregg, A., Reid, R., Aldemir, T., Gray, J., Frederick, M., & Garbrick, A. (2020). Think-aloud observations to improve online course design: A case example and “how-to” guide. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/15_think_aloud_obser

Guzdial, M., Kafai, Y. B., Carroll, J. M., Fischer, G., Schank, R., & Soloway, E. (1995). Learner-centered system design: HCI perspective for the future. DIS ’95, 143–147. https://doi.org/10/dfwn4p

Hanlon, R., Barry, M., Marrinan, F., & O’Sullivan, D. (2021). Towards an effective user interface for data exploration, data quality assessment and data integration. 2021 IEEE 15th International Conference on Semantic Computing (ICSC), 431–436. https://doi.org/10/gjs38m

Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior, 61, 463–471. https://doi.org/10/f8rwz5

Hassenzahl, M., & Tractinsky, N. (2006). User experience—A research agenda. Behaviour & Information Technology, 25(2), 91–97. https://doi.org/10/djx9nq

Hassenzahl, Marc. (2004). The thing and I: Understanding the relationship between user and product. In M. A. Blythe, K. Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From Usability to Enjoyment (pp. 31–42). Springer Netherlands. https://doi.org/10.1007/1-4020-2967-5_4

Hassenzahl, Marc. (2013). User experience and experience design. The Encyclopedia of Human-Computer Interaction (2nd Ed.). Copenhagen: Interaction design foundation.

Hassenzahl, Marc. (2008). User experience (UX): Towards an experiential perspective on product quality. Proceedings of the 20th International Conference of the Association Francophone d’Interaction Homme-Machine on - IHM ’08, 11. https://doi.org/10/fbmn7c

Hokanson, B., & Gibbons, A. (2013). Design in educational technology: Design thinking, design process, and the design studio. Springer Science & Business Media.

Hokanson, B., & Kenny, R. (2020). Creativity and critique as formative processes in design thinking. Journal of Formative Design in Learning, 4(1), 2–4. https://doi.org/10/gj6ng6

Holtzblatt, K., & Beyer, H. (2014). Contextual design: Evolved. Synthesis Lectures on Human-Centered Informatics, 7(4), 1–91. https://doi.org/10/ghbvmh

Honebein, P. C., & Reigeluth, C. M. (2021). Making good design judgments via the Instructional Theory Framework. In J. K. McDonald & R. E. West (Eds.). Design for Learning. EdTech Books. https://edtechbooks.org/id/making_good_design

Hummel, H., Manderveld, J., Tattersall, C., & Koper, R. (2004). Educational modelling language and learning design: New opportunities for instructional reusability and personalised learning. International Journal of Learning Technology, 1(1), 111–126. https://doi.org/10/bs7cjs

International Organization for Standardization. (2019). Human-centred design for interactive systems (ISO/DIS Standard No. 9241-210). https://www.iso.org/standard/77520.html

Jahnke, I., Schmidt, M., Pham, M., & Singh, K. (2020). Sociotechnical-pedagogical usability for designing and evaluating learner experience in technology-enhanced environments. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/sociotechnical_pedagogical_usability

Kilgore, W. (2016, June 20). UX to LX: The Rise of Learner Experience Design - EdSurge News. EdSurge. https://www.edsurge.com/news/2016-06-20-ux-to-lx-the-rise-of-learner-experience-design

Kimmons, R. (2020). Color Theory in Experience Design. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/color_theory

Koper, R. (2005). An introduction to learning design. In R. Koper & C. Tattersall (Eds.), Learning Design: A Handbook on Modelling and Delivering Networked Education and Training (pp. 3–20). Springer. https://doi.org/10.1007/3-540-27360-3_1

Koper, R., & Olivier, B. (2004). Representing the learning design of units of learning. Educational Technology & Society, 7(3), 97–111.

Korkmaz, C. (2018). “ID 2 LXD” from instructional design to learning experience design: The rise of design thinking. In A.-P. Correia (Ed.), Driving Educational Change: Innovations in Action. https://ohiostate.pressbooks.pub/drivechange/chapter/id-2-lxd-from-instructional-design-to-learning-experience-design-the-rise-of-design-thinking/

Kou, Y., & Gray, C. M. (2019). A practice-led account of the conceptual evolution of UX knowledge. CHI ’19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13.

Krug, S. (2010). Rocket science made easy: The do-it-yourself guide to finding and fixing usability problems. New Riders.

Krug, S. (2014). Don’t make me think, revisited: A common sense approach to web usability (3rd ed.). New Riders

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. https://doi.org/10/dtzfj3

Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P. O. S., & Kort, J. (2009). Understanding, scoping and defining user experience: A survey approach. CHI ’09, 719–728. https://doi.org/10/dxtf84

Law, E., Roto, V., Vermeeren, A. P. O. S., Kort, J., & Hassenzahl, M. (2008). Towards a shared definition of user experience. CHI EA ’08, 2395–2398. https://doi.org/10/fh9qbk

Lim, C. J., & Lee, S. (2007). Pedagogical usability checklist for ESL/EFL e-learning websites. Journal of Convergence Information Technology, 2(3), 67–76.

Lowenthal, P., & Wilson, B. G. (2010). Labels DO matter! A critique of AECT’s redefinition of the field. TechTrends, 54(1), 38–46. https://doi.org/10/b839j7

Madariaga, L., Nussbaum, M., Gutiérrez, I., Barahona, C., & Meneses, A. (2021). Assessment of user experience in video-based learning environments: From design guidelines to final product. Computers & Education, 167, 104176. https://doi.org/10.1016/j.compedu.2021.104176

Marcus, A. (2002). Dare we define user-interface design? Interactions, 9(5), 19–24. https://doi.org/10/br9cz4

Matthews, M. T., Williams, G. S., Yanchar, S. C., & McDonald, J. K. (2017). Empathy in distance learning design practice. TechTrends, 61(5), 486–493. https://doi.org/10/gf8rk8

Matthews, M. T., & Yanchar, S. C. (2018). Instructional design as manipulation of, or cooperation with, learners? TechTrends, 62(2), 152–157. https://doi.org/10/gjh384

Mayer, R. E., & Estrella, G. (2014). Benefits of emotional design in multimedia instruction. Learning and Instruction, 33, 12–18. https://doi.org/10/gf4zw9

McCarthy, K. S., Watanabe, M., & McNamara, D. S. (2020). The design implementation framework: Guiding principles for the redesign of a reading comprehension intelligent tutoring system. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/9_the_design_impleme

McDonald, J. K., West, R. E., Rich, P. J., & Pfleger, I. (2019). “It’s so wonderful having different majors working together”: The development of an interdisciplinary design thinking minor. TechTrends, 63(4), 440–450. https://doi.org/10/gjh36m

McDonald, J. K., & Yanchar, S. C. (2020). Towards a view of originary theory in instructional design. Educational Technology Research and Development, 68(2), 633–651. https://doi.org/10/ggwwx7

Minichiello, A., Hood, J. R., & Harkness, D. S. (2018). Bringing user experience design to bear on STEM education: A narrative literature review. Journal for STEM Education Research, 1(1), 7–33. https://doi.org/10/gjh39q

Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same? The Internet and Higher Education, 14(2), 129–135. https://doi.org/10/fnvdjr

Moore, J. L., Dickson-Deane, C., & Liu, M. Z. (2014). Designing CMS courses from a pedagogical usability perspective. Perspectives in Instructional Technology and Distance Education: Research on Course Management Systems in Higher Education, 143–169.

Mor, Y., & Craft, B. (2012). Learning design: Reflections on a snapshot of the current landscape. Research in Learning Technology, 20, 85–94. https://doi.org/10/gh2g3r

Nokelainen, P. (2006). An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students. Educational Technology & Society, 9(2), 178–197.

Norman, D. (2013). The design of everyday things: Revised and expanded edition. Basic books.

Norman, D. A., & Draper, S. W. (1986). User centered system design: New perspectives on human-computer interaction. CRC Press.

Oliver, R., Harper, B., Wills, S., Agostinho, S., & Hedberg, J. (2007). Describing ICT-based learning designs that promote quality learning outcomes. In H. Beetham & R. Sharpe (Eds.), Rethinking pedagogy for a digital age: Designing and delivering e-learning. Routledge.

Oprean, D., & Balakrishnan, B. (2020). From engagement to user experience: A theoretical perspective towards immersive learning. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/10_from_engagement_t

Quintana, C., Krajcik, J., & Soloway, E. (2000). Exploring a structured definition for learner-centered design. In B. Fishman & S. O’Connor-Divelbiss (Eds.), Fourth International Conference of the Learning Sciences (pp. 256–263). Erlbaum.

Quintana, R. M., Haley, S. R., Magyar, N., & Tan, Y. (2020). Integrating learner and user experience design: A bidirectional approach. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/integrating_lxd_and_uxd

Raza, A., Penuel, W. R., Jacobs, J., & Sumner, T. (2020). Supporting equity in schools: Using visual learning analytics to understand learners’ classroom experiences. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/supporting_school_equity

Robinson, H. A., Sheffield, A., Phillips, A. S., & Moore, M. (2017). “Introduction to teaching online”: Usability evaluation of interactivity in an online social constructivist course. TechTrends, 61(6), 533–540. https://doi.org/10/gjh374

Robinson, J., Lanius, C., & Weber, R. (2018). The past, present, and future of UX empirical research. Communication Design Quarterly, 5(3), 10–23. https://doi.org/10.1145/3188173.3188175

Roman, T. A., Callison, M., Myers, R. D., & Berry, A. H. (2020). Facilitating authentic learning experiences in distance education: Embedding research-based practices into an online peer feedback tool. TechTrends, 64(4), 591–605. https://doi.org/10/gjh36j

Roschelle, J., Penuel, W., & Shechtman, N. (2006). Co-design of innovations with teachers: Definition and dynamics. In Proceedings of the international conference of the learning sciences (pp. 606–612). New Brunswick: International Society of the Learning Sciences.

Rose, D., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Association for Supervision and Curriculum Development.

Rose, E. J., Björling, E. A., Kim, A., & Alvarez, N. Y. (2018). Usability testing with teens: Adapting human-centered design and UX methods. Proceedings of the 36th ACM International Conference on the Design of Communication, 1–6. https://doi.org/10.1145/3233756.3233955

Schmidt, M., Earnshaw, Y., Tawfik, A. A., & Jahnke, I. (2020). Methods of user centered design and evaluation for learning designers. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/ucd_methods_for_lx

Schmidt, M., Tawfik, A. A., Jahnke, I., Earnshaw, Y., & Huang, R. (2020). Introduction to the Edited Volume. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/introduction_to_ux_lx_in_lidt

Schmidt, M., Tawfik, A., Jahnke, I., & Earnshaw, Y. (Eds.). (2020). Learner and user experience research: An introduction for the field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux

Shernoff, E. S., Von Schalscha, K., Gabbard, J. L., Delmarre, A., Frazier, S. L., Buche, C., & Lisetti, C. (2020). Evaluating the usability and instructional design quality of Interactive Virtual Training for Teachers (IVT-T). Educational Technology Research and Development, 68(6), 3235–3262. https://doi.org/10/gjh33h

Signoretti, I., Marczak, S., Salerno, L., Lara, A. d, & Bastos, R. (2019). Boosting agile by using user-centered design and lean startup: A case study of the adoption of the combined approach in software development. 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 1–6. https://doi.org/10.1109/ESEM.2019.8870154

Soloway, E., Guzdial, M., & Hay, K. E. (1994). Learner-centered Design: The Challenge for HCI in the 21st Century. Interactions, 1(2), 36–48. https://doi.org/10/d2kw7v

Stefaniak, J., & Sentz, J. (2020). The Role of Needs Assessment to Validate Contextual Factors Related to User Experience Design Practices. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/role_of_needs_assessment

Stefaniak, J., Shah, S., Mills, E., & Luo, T. (2020). Keeping the learner at the focal point: The use of needs assessment and persona construction to develop an instructional resource center for instructional designers. International Journal of Designs for Learning, 11(2), 142–155. https://doi.org/10/gjpw9f

Strauss, A., & Corbin, J. . (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Sage.

Tawfik, A. A., Gatewood, J., Gish-Lieberman, J. J., & Hampton, A. J. (2021). Toward a Definition of Learning Experience Design. Technology, Knowledge and Learning. https://doi.org/10/gh27hp

Tsay, C. H.-H., Kofinas, A., & Luo, J. (2018). Enhancing student learning experience with technology-mediated gamification: An empirical study. Computers & Education, 121, 1–17. https://doi.org/10.1016/j.compedu.2018.01.009

Tullis, T., & Albert, W. (2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics. Elsevier.

Vann, S. W., & Tawfik, A. A. (2020). Flow theory and learning experience design in gamified learning environments. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.). Learner and User Experience Research: An Introduction for the Field of Learning Design & Technology. EdTech Books. https://edtechbooks.org/ux/flow_theory_and_lxd

Volery, T., & Lord, D. (2000). Critical success factors in online education. International Journal of Educational Management, 14(5), 216–223. https://doi.org/10/c7p4bt

Wang, S., Zhang, Q., Huang, W., Tian, H., Hu, J., Cheng, Y., & Peng, Y. (2018). A new smart mobile system for chronic wound care management. IEEE Access, 6, 52355–52365. https://doi.org/10.1109/ACCESS.2018.2864264

Wood, R., & Shirazi, S. (2020). A systematic review of audience response systems for teaching and learning in higher education: The student experience. Computers & Education, 153, 103896. https://doi.org/10.1016/j.compedu.2020.103896

Zhong, V. J., & Schmiedel, T. (2021). A user-centered agile approach to the development of a real-world social robot application for reception areas. Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 76–80. https://doi.org/10.1145/3434074.3447132