Introduction

Words like disruptive and revolutionary have often been used to describe the potential impact of massive, open, online courses (MOOCs) on access to elite universities and their faculty (Carver & Harrison, 2013; Toven et al., 2014). Yet the promise of equitable and inclusive access for a global audience of learners has not always been realized, and some scholars have argued that MOOCs may even perpetuate educational disparities as they do not spread benefits equitably. For example, learners from developing countries may be particularly disadvantaged and even experience social identity threat when courses do not adequately attend to their learning traditions, contexts, and needs (Kizilcec et al., 2017). Inequitable course designs include a narrow focus on Western epistemological perspectives and the prevalence of unidirectional, or pedagogies of transmission, utilized in courses (Rhoads et al., 2013). Designing for diversity, equity, and inclusion (DEI) in MOOCs, therefore, presents a variety of challenges, but because of the potential reach of MOOCs opportunities exist for advancing innovative pedagogies and content that take into account learners from a variety of global contexts (Ebben & Murphy, 2014). Knowing this, MOOC faculty may have good intentions for advancing DEI goals in their course designs. Yet more research is needed to understand their specific aspirations and the extent to which they are successful in realizing these goals with the help of design teams.

Our research explores the coherence between DEI goals that faculty articulate in advance of the design process and their enactment within the resulting MOOCs. Our study details work within an instructional services unit specializing in the design, development, and production of open online courses at a Research I university in the midwest of the United States. The purpose of the study is to gain an understanding of the types of DEI goals identified by faculty and how those goals are instantiated in corresponding course designs. Additionally, we intend to use the results to inform the development of processes (e.g., pre-proposal consultations assisting faculty in the development of actionable DEI design goals, embedding DEI reflection points during each phase of design, seeking qualitative feedback from colleagues or potential learners pre-launch, and sourcing qualitative learner feedback related equity and inclusion post-launch) to engage in a systematic and purposeful approach for the realization of equitable and inclusive design goals in MOOCs.

We use the following research questions to guide our study:

1. What goals for DEI do faculty identify in their MOOC project proposals?
2. In what ways are stated faculty DEI goals manifest in the final design of their MOOC?

Conceptual Framework

As designers of learning experiences, we see design as a potential avenue to mitigate issues related to equity and inclusion in open education. We draw on theories of intercultural and transformative learning and a framework for diversity scholarship as a conceptual framework to guide data analysis and the synthesis of results.

Intercultural Education and Transformative Learning Theory

When designing online learning experiences, faculty and design teams should consider the social identities of learners, the situatedness of learners and the curriculum, and pedagogical strategies that promote respectful, intercultural dialogue between learners. Theories of intercultural education and transformative learning have the potential to serve as frameworks for faculty and design teams to create more equitable and inclusive online courses that center the needs of diverse learners and advance the democratization of online education in a global society.

Intercultural education emphasizes “dialogue, social inclusion, interaction, and exchange through...empathy, flexibility, and curiosity” (Portera, 2008, p. 399). Intercultural education creates the possibility for personal and social growth through interacting with “individual[s]of different cultural origin[s]” (Portera, 2008, p. 485). Intercultural education lies between two epistemological poles in which “differences and similarities are taken into consideration, brought into contact, and bring about interaction” (Camilleri, 1985 as cited in Portera, 2008, p. 486 ). Technology has increased opportunities for intercultural communication within online learning spaces and as such, intercultural education may “represent the most appropriate response to the challenges of globalization and complexity,” as educators and institutions continue to strive to demonstrate their commitment to diversity, equity, and inclusion within online learning experiences (Portera, 2008, p. 488). Online learning experiences that draw on existing knowledge of learning in different socio-cultural contexts may foster intercultural competence and communication and create space for curricular and pedagogical flexibility. In turn, these designs could increase equity and inclusion within learning environments and augment learning for those across the globe (Gunawardena, 2014). Online intercultural education requires learners to engage in discursive communication with others, which requires critical introspection and the capacity to simultaneously embrace the similarities and differences of others.

The tenets of transformative learning theory offer a framework for carrying out online intercultural education as the central premise of the theory is the transformation of frames of reference through “critical reflection on the assumptions upon which our interpretations, beliefs, and...points of view are based” (Mezirow, 1997, p. 7). Learning experiences that foster transformative learning require:

• Autonomous thinking
• Imaginative problem-solving
• Self-direction
• Group deliberation
• Learner-centered discourse
• Content that reflects the lives and experiences of learners (Mezirow, 1997, p. 10)

We use intercultural education and transformative learning theories as lenses in the subsequent literature review focusing on the design and implementation of MOOCs. We explore three main themes in the review: 1) the democratization of online education, 2) the motivations of faculty engaging in equitable and inclusive design, and 3) the impact of equitable and inclusive design on learning.

Democratizing Online Education

The democratization of online education extends beyond open enrollment in MOOCs. According to Rhoads et al. (2013), democratizing open online education requires thinking beyond expanded access to educational resources and must include divergent epistemological perspectives and engage non-dominant “social actors, institutions, and nations” (p. 106). Relatedly, Chen et al. (2020) investigated what they refer to as a “third way” of course design that reimagines the epistemological perspectives of a large Western-centered MOOC for a local, non-Western context through a process the authors refer to as an “ecological circle for MOOC development” (p. 20). More specifically, the Learning How to Learn MOOC was collaboratively redeveloped by a cross-national team for a Chinese context and renamed The Tao of Learning. The authors indicated that during the redesign of the course, culturally-specific content and analogies within the course were updated to be relevant to learners in China.

Rhoads et al. (2013) call for the use of liberating and democratic pedagogies in open online education, and may also include critical pedagogical perspectives drawing on Freire’s (2014) work. Critical pedagogies are liberating in the sense that learners are encouraged to critically reflect on content and the context in which they are learning as “there is no such thing as a neutral education” (Shaull, 2014, p. 34). Morris et al. (2017) expand the notion of critical pedagogy to the digital space—specifically to MOOCs—explaining that critical digital pedagogy revolves around collaborative communities that are open to multiple voices and international perspectives that communicate beyond social, cultural, and political boundaries outside of traditional higher education institutions.

In their study, Dennen and Bong (2018) found that courses encouraging cross-cultural dialogue between learners with divergent national cultures (e.g., individualistic and collectivist) tend to make those in the non-dominant culture susceptible to feelings of otherness. To reduce feelings of otherness, flexible and dynamic course design and facilitation recognize and respect the cultures of learners, support learners so they feel safe to experiment with those who are different from them, and promote dialogue as a way to help learners understand each others’ lived experiences (Dennen & Bong, 2018; Shahini et al., 2019). Relatedly, Mittlemier et al. (2018) reported that learners’ tended to engage in collaboration when content was directly related to their personal backgrounds. Mittlemier et al. suggest that when incorporating collaboration in online courses, educators should consider flexible grouping strategies and encourage learners to share their experiences as sources of content for other learners.

Faculty Considerations When Designing for DEI in Online Courses

It is important to consider faculty motivations for creating MOOCs in general to understand the reasons they may choose (or not choose) to pursue DEI goals. Scholars also present a view of MOOC instructors who desire to reach a wide audience of global learners. Freitas and Paredes (2018) explored faculty motivations driving MOOC development and found that faculty valued the opportunity to widely share specific knowledge to a general audience. Kolowich (2013) asserted that MOOC instructors most frequently cite the desire to reach a worldwide audience of learners and a goal of increasing access to instructional materials. Similarly, Hew and Chung (2014) noted that faculty may be interested in engaging in MOOC design and production processes because they are motivated by a sense of altruism, with the goal of reaching learners who might otherwise not have access to educational experiences. Kleinman (2018) described faculty who were interested in utilizing MOOCs as a means of sharing widely on a topic they are passionate about and feel is of great societal importance, thus increasing learners’ awareness on critical topics. Although these scholars do not focus on faculty intentions concerning DEI, these studies do shed light on faculty motivations for expanding reach and access, which is a related goal.

Beyond reaching global audiences of learners, one MOOC instructor demonstrated an ambition to incorporate the voice of the learner into the instructional materials through the creation of locationally-specific data science problems (Quintana et al., 2018).  In another study, a group of instructors indicated a desire to meet the unique needs of MOOC learners during course design, and some even described efforts towards personalization (Bonk et al., 2018). 

Impact of Designing for Equity and Inclusion on Learning

Learners who enroll in MOOCs enter with a variety of expectations and prior experiences. Intercultural differences in the way learners perceive expectations and communicative norms could lead to tensions during interactions and discussions, which may inhibit learner engagement and motivation reinforcing feelings of difference (Andersen et al., 2018; Lawrence, 2013). Intercultural competence can be defined as the “cognitive, affective, and behavioral skills and characteristics that support appropriate and effective interaction in a variety of cultural contexts” (Bennett, 2014, p. 157). Croft & Brown (2020) suggested that a lack of racial and cultural diversity amongst higher education faculty could perpetuate implicit biases within their online courses, because they base their assumptions about online learners on personal experience, believing them to hold the same kinds of privileges that they have themselves. Within the MOOC environment, faculty are designing for undefined audiences and may view them as a homogenous group (Macleod et al., 2016).

Some of these tensions can be addressed through the design of the learning environment itself. In terms of bringing learners into an experience in the first place, Kizilcec et al. (2019) theorized that diversity statements within MOOCs could influence enrollment patterns, but suggested that more work is needed to develop diversity statements that are effectively reliable in advancing these goals. Kizilcec et al. (2019) also investigated the impact of psychological cues (e.g., written content, visual design, and interaction design) on enrollment and participation in a statistics MOOC and found that changes to a course image and description appeared to lead to increased enrollment for women (Kizilcec et al., 2017). Thus, the learning environment should promote a sense of safety and trust for sharing ideas and critical reflections related to content and tasks that welcome diverse perspectives and multiple literacies (Blayone et al., 2017; Loizzo & Ertmer, 2016; Marshall, 2014; Stewart, 2013). In order to foster intercultural competence and respect learners’ prior experiences and beliefs regarding communicative norms, Loizzo and Ertmer (2016) suggest acknowledging the notion of “lurking as learning” by removing discussion forum posting requirements and encouraging collaboration beyond platforms (p. 1022). Moreover, the peer-review process for assignments can be utilized as a feature to build intercultural competencies for learners to have space to reinforce content and expand their worldviews through collaborative interactions (Bali, 2014; Loizzo & Ertmer, 2016).

Methods

Research Team

As researchers, our interpretations are filtered through our social identities, and “all research is “positioned” within a stance” (Creswell, 2013, p.  215). We provide a description of our research setting and our positionality as researchers for the sake of transparency (Malterud, 2001). Our research team consists of four learning experience designers who work within a provost-funded instructional services unit to support faculty interested in advancing online learning experiences. As a team, we shared responsibilities for research design, data collection, and analysis. We examined course content for every course and distributed the data analysis work as evenly as possible. Additionally, we took care to analyze courses for which we were not the assigned learning experience designers. During our investigation, we worked together in person and also asynchronously using collaborative word processing tools. We frequently engaged in reflexive dialogue during our research meetings when we discussed our values, beliefs, and interpretations of data to develop a shared understanding.

Context

In concert with a university-wide initiative, the instructional services unit developed a DEI strategic plan, which focused on their specific context—open online courses.  At this time, the MOOC proposal form was updated to reflect the unit’s commitment to DEI asking faculty to outline goals and describe how their course design would contribute to the university becoming more diverse, equitable, and inclusive. The question is open-ended, broad, and yielded a variety of faculty responses and goals. Faculty were not explicitly asked to relate their DEI goals with the university or instructional unit’s broader strategic DEI plan.

When we began the investigation, this updated DEI question within the proposal had been in use for three years, allowing for the development of a sufficient number of courses to provide insight into the manifestation of faculty course design goals. We were specifically interested in understanding the range of goals that were articulated in proposals and the extent to which these goals were instantiated in various MOOCs.

Study Design

Data collected for this study included faculty responses to the DEI question in the MOOC proposal and textual elements from corresponding courses. We conducted the study in three phases (see Figure 1):

• Phase 1: Select MOOC proposals. We established a set of inclusion criteria to select course proposals and corresponding course designs:
  • the proposal must respond to the DEI question about how the proposed MOOC addresses DEI goals set forth by the university;
  • the proposed course must fall into the open-content category (i.e., with no restrictions on enrollment);
  • the proposed course must be live (i.e., not archived).

Based on these criteria, we identified 11 MOOC proposals in total. Six of the selected proposals were for single MOOCs, or stand-alone courses that are typically estimated to take four to six weeks to complete. The other five proposals were for a MOOC series or a set of three to six related courses meant to be taken together.

• Phase 2: Code MOOC proposals. We modified the National Center for Institutional Diversity’s Framework for Diversity Scholarship (NCID, 2021) (see Appendix A) for use in the MOOC context and used it to deductively code 11 MOOC proposals (see Appendix B).
• Phase 3: Review live MOOCs. We examined the MOOCs (n=32) that were developed from the 11 proposals identified through our selection process, looking for evidence of ways in which DEI goals were realized in the design of each course through deductive and inductive coding.

Figure 1

Overview of Three Phases of the Study

Analysis

We engaged in qualitative content analysis (Elo & Kyngäs, 2008), using an iterative and flexible approach that included inductive and deductive coding to identify themes (Deterding & Waters, 2018). Using a deductive approach, we analyzed stated course design goals in response to the DEI question from the MOOC proposal. To begin our analysis, we explored potential thematic categories and discovered NCID’s Diversity Scholarship framework. NCID’s mission is to bring together interdisciplinary scholars to pursue research to create a more equitable and just society. The framework presents categories of diversity scholarship, which we modified for use in the online course design context and used as a coding scheme for the first phase of analysis (see Appendix A).

While coding the equity goals, we first identified discrete excerpts from MOOC proposals related to one or more dimensions of the modified NCID framework. Second, we coded these excerpts, applying dimensions from the modified NCID framework. Third, we reviewed the coded goals taking into consideration the affordances of the MOOC environment and what would reasonably be instantiated in a course for global learners. For example, some proposals included additional DEI goals that went beyond the course design and were related to the impact on faculty’s pedagogy in general or the dissemination of course materials to university personnel for professional development. Goals such as these were excluded from our analysis.

After the initial analysis of the proposed DEI goals, we performed an artifact analysis of course elements found within each MOOC. We examined a variety of textual elements found within the design of each individual MOOC including course syllabi, video transcripts, discussion prompts, course readings, and assessments. Visual and graphic elements of the course were not included in this stage of the analysis. Each researcher was assigned two single MOOCs and 5-7 individual courses within a MOOC series. Our individual analyses of assigned courses consisted of an initial close reading (line-by-line) of textual course elements in which relevant excerpts or descriptions were recorded, coded, and memoed in relation to the associated NCID dimensions identified in the initial proposal analysis.

Following the initial close reading, the research team convened to discuss and come to a consensus about the extent to which the coded artifacts related to the DEI goals from the proposal would be considered in the analysis. Coded instances were counted in the final tally if they met the following criteria (see Tables 2-3):

1. Elements were deemed by the team to be related to the stated DEI goals from the proposal (e.g., stated goal of featuring rural library examples to increase diversity in the field of librarianship coupled with rural library case studies throughout the course)
2. Elements were sustained across the course OR where they were deemed to be impactful in relation to learner experience and other course elements (e.g., sustained assessment techniques drawing on learner lived experiences/contexts or a single culminating assessment drawing on learner lived experiences/contexts).
3. Elements that appeared more than once throughout the MOOC series (e.g., introductory videos) were counted as one instance.
4. Textbooks used throughout a single course were counted one time.

Our analysis was limited in the sense that faculty and design team intentionality was not addressed and the analysis was from the perspective of outside designers attempting to draw connections between proposed goals and course elements after the course had already been designed and launched.

Findings

Proposed DEI Goals

Faculty proposed a range of DEI goal types in their MOOC proposals. An overview of the findings for the initial DEI goal analysis in relation to the NCID Framework themes for both single MOOCs and MOOC series is presented in Table 1. Our findings show that on average, single MOOC proposals tended to outline more goals related to DEI than MOOC series. Both single MOOCs and MOOC series proposed goals most commonly aligned with the Addressing Social Inequality theme.

The Climate Change Action single MOOC was an outlier in that our team coded seven DEI-related goals in the proposal across four of the five themes: Addressing Social Inequality, Recognizing the Impact of Power and Privilege, Including Multiple Perspectives, and Valuing Individuals. (see Table 1).

Table 1

Prevalence and Description of DEI Goals by NCID Framework Theme

Manifestations of DEI Goals Within Courses

In the following section, Tables 2 and 3 delineate tallies of coded instances in each course or series. Notable findings and outlier cases are organized by the themes used to code DEI goals in proposals, which will allow us to represent relationships between goals stated and their instantiations (or lack thereof in some cases).

Table 2

Instantiations of DEI Goals by NCID Framework Theme: Single MOOCs

Table 3

Instantiations of DEI Goals by NCID Framework Theme: MOOC Series

The following section is organized thematically by goal type. We describe findings from courses and series in which we identified the highest number of instantiations of proposed DEI goals. We also describe notable or unique instances as well as outliers. A summary of types of DEI goal instantiations for MOOCs and series organized by theme is included in Table 4.

Table 4

Summary and Descriptions of Common Instantiations of DEI Goals in MOOCs

Addressing Social Inequality

Addressing Social Inequality was the most common theme applied to goals in single MOOC proposals with five out of six proposing this goal type. Of the proposals coded with this theme, all but one, Implications for Decision Making in Accounting, included manifestations of the goals proposed. Introduction to Dentistry exhibited the highest number of manifestations of the theme (n=9). Within the introductory module, five lecture videos, three readings, and one quiz focused on addressing the lack of access to dentistry education and resulting lack of diversity in the profession in the US. While not the highest number of instances, Climate Change Action includes specific examples of actions to take each week related to inequality and climate injustice at the individual, community, political, and adaptation levels (e.g., individual sustainability, community organizing, and writing to politicians). Each week, learners engage in a self-reflection quiz in which they are prompted to check off the actions they took toward climate justice. Additionally, the culminating assessment within the course is an individual climate action plan related to each of the levels of action outlined each week.

Of those three series, Basics of Web Applications (n=59), a four-course series, exhibited the highest number of manifestations of proposed goals. Goals were made visible through tutorial videos the instructor referred to as “code walkthroughs” in which coding mistakes by experts were normalized.

Although How to Manage Public Libraries on a Budget, an eight-course series, did not exhibit the highest number of instantiations of this goal (n=46), an interesting pattern emerged in the data. The series was developed by a large instructional team of outside experts led by one faculty member. Courses taught by the lead faculty member had the highest number of goal manifestations.

Including Multiple Perspectives

Four out of the six single MOOC proposals included goals related to the theme of Including Multiple Perspectives and manifestations of goals were found in each of the four courses that proposed this type of goal (see Table 4). Climate Change Action included the highest number of instantiations of the theme (n=23). The proposed goals related to this theme included perspectives that would not typically be found in a MOOC and seeking guests representing diverse experiences and social identities. This course was unique in that it was co-developed by students working with the faculty and design team, which was outlined in the syllabus. Each of the seven weeks featured seven students interviewing seven guest experts in the field and additional readings outlining the stories of community organizers outside of academia.

Using Python for Statistical Calculations, a three-course series, was the only MOOC series proposal to include a goal related to the theme of Including Multiple Perspectives, which focused on including examples and data from outside the US and offering explanations and where possible, alternatives to US-centric jargon. In total, seven manifestations of the stated goal were identified. Within the series, two videos spent time explicitly breaking down some of the jargon used that was US-centric (e.g., describing the US census process and various acronyms related to datasets) and three videos touched on topics applicable to settings outside of the US such as population health and people who have taken swim lessons. Furthermore, extension activities within the resources section provided exercises using global current issues datasets.

Valuing Individuals

Three of the six single MOOC proposals included goals related to the theme of Valuing Individuals. Of those three, only two included manifestations of the goals (Strategies for Searching in Health Sciences and Climate Change Action). Implications for Decision Making in Accounting stated the goal of creating course content and assessments that “reflect the diversity of the country and the globe”. While there were many case studies outlined detailing both fictional and real companies, the diversity was mainly reflected in the function of the company. Within examples provided, one reference to a Canadian company was identified and company types included restaurants, vineyards, bakeries, and jewelers. Many examples included specific references to the city in which the university is located.

Two out of five MOOC series, Utilizing Python for Statistical Calculations and Advanced Applications of Python, included goals related to the theme of Valuing Individuals. Within Utilizing Python for Statistical Calculations, the proposed goals of meeting learners where they are and providing opportunities for learner choice were made visible through the use of a variety of additional resources such as YouTube Python tutorials, a supplemental resource with frequently used notations and definitions, and a learner-sourced frequently asked questions page to provide additional support for those who may not have as much experience with Python.

Advanced Applications of Python proposed the goal of providing a personalized learning experience through the use of automated feedback on low-stakes exercises. All five courses in the series utilized an interactive OER textbook with auto-graded exercises and elaborative feedback. The textbook was specifically developed for a residential course related to this series. Additionally, this course linked out to two tools geared toward personalized learning. One tool allowed learners to set and track learning goals for the course and the other was used to allow learners to create their own drawings via code to share with others.

Recognizing the Impact of Power and Privilege

Two out of the six single MOOC proposals included goals related to the theme of Recognizing the Impact of Power and Privilege—Climate Change Action and Higher Education Leadership. Both courses exhibited instantiations of the proposed goals. Climate Change Action included a question and answer session with a climate activist outlining the disproportionate impacts of climate change on under-resourced communities around the world. Additionally, a link to a website exploring this topic in relation to Ecuador was included as a specific example.

The instantiations identified in Higher Education Leadership involved an activity in which learners were prompted to examine campus maps and label physical spaces on campus that represent power and importance. Additionally, a discussion prompt asking learners to reflect on the ways in which structures on campuses perpetuate inequality.

Finding Common Ground

Higher Education Leadership was the only single MOOC proposal to relate to the theme of finding common ground. One module focusing on the topic of contested discourse (discussed above in the Including Multiple Perspectives section) ended with a discussion prompt asking learners to reflect on the multiple student perspectives with which they just engaged and outline how they would work toward resolving similar disputes on their campuses. 

Advanced Applications of Python was the only MOOC series proposal to outline a goal related to the theme of Finding Common Ground. The proposal emphasized the importance of learners engaging in social learning with others with different lived experiences. Two assessments with social learning components were identified. One instance in the first course involved a linked tool that allowed learners to post drawings created with code to a gallery for peer comments and feedback (similar to a digital gallery walk activity). The second instance was identified in the fourth course and was a culminating peer-review assignment. Both instances involved the social aspect of learners communicating with one another about their work; however, no specific communication guidelines or reminders for students to practice providing constructive feedback were included.

Discussion

What Types of DEI Goals Were Proposed?

A range of goals were proposed with respect to the NCID themes and also their level of specificity. A clear pattern emerged in relation to the high frequency with which broad, less specific goals focused on expanding educational access were proposed. Notably, the goal for Climate Change Action and Enacting Social Change Through Narrative Experience moved beyond expanding access and incorporated more specificity and cited the need for learner agency and activism (via climate justice and local activism). It is perhaps reasonable to posit that the subject matter of these courses and their relation to social justice issues necessitated the invitation to action on the part of the learner.

The proposal of broad goals related to expanding access to education aligns with current research on faculty motivations for designing MOOCs (Kolowich, 2013; Hew & Cheung, 2014; Freitas & Paredes, 2018). Numerous studies critique some MOOC advocates for being overly ambitious about their potential for expanding educational access and educational equity across the globe (Ebben & Murphy, 2014; Toven et al., 2014; Portmess, 2013). As learning experience designers, it may be helpful to provide faculty with research that critically examines such claims and provides further insights into the complexity of designing MOOCs for global audiences (Portmess, 2013; Rhoads et al., 2013; Kizilcec et al., 2017). Furthermore, our findings suggest that learning experience designers may be able to assist faculty to develop shared, specific DEI goals at the onset of the project in order to promote their enactment in the final course design (Cvitanovic et al. 2020).

How Were Goals Enacted in Course Designs?

Our findings revealed a relationship between goal type and the way in which goals were enacted in courses. Didactic video and text content within the MOOCs and series was the most prominent way in which proposed DEI goals were realized in relation to the most common goal type, Addressing Social Inequality (see Table 1). Conversely, Valuing Individuals and Finding Common Ground related goals were more frequently enacted through activities in which learners share and produce content via learning activities and peer-review type assessments. Bali et al. (2020) delineate typologies of open education practices on a continuum from content-centric to process-centric. Within content-centric practices, the content is the main focus whereas process-centric practices focus on the process of learners sharing and producing their own content contributions. In Figure 2, we use the content and process-centric typologies described by Bali et al. to represent the ways in which DEI goals were realized in courses.

Figure 2

Instantiations of DEI Goals within Courses 

Based on our findings, we draw two conclusions related to the content and process typologies described by Bali et al. First, video and text content—especially when it does not have to be created by faculty or design teams—may be an efficient way to address DEI goals when barriers to other means may be present (i.e., time and resources). Second, MOOC platform affordances do not always allow for the realization of process-centric goals. For example, some platforms provide little in the way of dialogue and sharing and tend to rely on asynchronous threaded discussions and peer-feedback communication tools. Studies have shown, however, that learner participation in MOOC discussion forums involves a relatively low number of learners and those that do participate are generally a homogenous group of professionals from the Western world who are well-educated (Gillani & Enyon, 2014; Ayer et al., 2018). Alternatively, Ito et al. (2020) suggest connected learning environments provide opportunities for learners to share “work, skills, and knowledge across networks, groups, and communities…[through] [b]logging, publishing work, or streaming” (p. 61). These forms of engagement and communication are typically not available MOOC learning platforms but may be possible by integrating or linking out to other tools as was the case in Climate Change Action and Advance Applications of Python.

Our findings also suggest differences between DEI goal development and course design for single MOOCs and MOOC series. Single MOOCs tend to exhibit a greater number of enacted DEI goals whereas MOOC series were less consistent in terms of enacting DEI goals. We postulate this difference may be attributable to three potential features of designing MOOC series. First, the scope of work involved in designing a series of MOOCs is substantially more than that of designing a single MOOC. Second, MOOC series require coordinating design efforts across multiple courses increasing the complexity of the design work. Third, MOOC series are often led by teams of faculty members, with individual faculty responsible for parts of the design. The individual motivations of faculty on teams may present challenges for cohesively enacting DEI goals across courses, which may have been the case for the How to Manage Public Libraries on a Budget series team. MOOC series design projects will likely require higher levels of support and attention in order to realize DEI goals. Learning experience designers and faculty leads should attempt to ensure DEI goals are collaboratively developed, that plans for implementation are put into place for each course, and that time is taken to holistically examine the series for the enactment of goals.

Limitations and Future Work

Our study is limited in the sense that it is situated within a single instructional services unit at a single university. Our inclusion criteria resulted in a fairly small number of courses and series to analyze, namely courses that were included within the unit’s new proposal template and were “live” at the time of analysis. Since the time of writing, several more courses and series have been completed that meet our inclusion criteria. Furthermore, our chosen methodology, document analysis, may be insufficient to adequately address the nuances of faculty stated and realized goals (Bowen, 2009). Future work could include interviews with faculty to flesh out course goals with respect to DEI more specifically. Future data sources could include interviews with MOOC learners to better understand their experience of taking a course or series, to provide further data concerning whether or not course DEI goals were realized.

Our team is committed to examining our design processes and conducting further design-based research to explore opportunities to enact equitable and inclusive design principles. Since the time of writing, we have created resources to support faculty and design teams. First, we developed a guide (with feedback from our colleagues) for using the modified NCID framework as a set of lenses for critical reflection and a catalyst for action at various points during the design process. Second, a number of our team members, along with colleagues from the instructional services unit formed a group tasked with developing a repository of equitable and inclusive design examples to share with faculty during course design. Third, the instructional services unit is developing processes to assist faculty in the development of actionable DEI goals within the proposal that will be communicated to design teams. Beyond our institution, resources such as design guides and repositories will be important as universities are forced by COVID-19 to enter online spaces with a renewed attention to issues of racial injustice and inequity worldwide. Providing additional research-informed guidance for faculty and course design teams will be essential for continued efforts to design equitable and inclusive online learning experiences open to global audiences.

References

Ayer, N., Sukhathankar, H., Deshmukh, U., & Sahasrabudhe, S. (2018, December). Impact of learner-centric discussion forums on learner engagement in skill development MOOC. In Proceedings of the 2018 IEEE Tenth International Conference on Technology for Education. 69-72. https://edtechbooks.org/-eUKo

Andersen, B.L., Na-songkhla, J., Hasse, C., Nordin, N., & Norman, H. (2018). Perceptions of authority in a massive open online course: An intercultural study. International Review Education, 64, 221–239. https://edtechbooks.org/-oPps

Maha B. (2014). Why doesn't this feel empowering? The challenges of web-based intercultural dialogue. Teaching in Higher Education, 19(2), 208-215. https://edtechbooks.org/-ANGm

Bali, M., Cronin, C., Jhangiani, R.S. (2020). Framing Open Educational Practices from a Social Justice Perspective. Journal of Interactive Media in Education, 10, 1–12. https://edtechbooks.org/-juPQ

Bennett, J.M. (2014). Intercultural competence: Vital perspectives for diversity and inclusion. In B.M. Ferdman & B. Deane (Eds.), Diversity at Work: The Practice of Inclusion (155-176). John Wiley & Sons. https://edtechbooks.org/-PoiX10.1002/9781118764282.ch5

Blayone, T. J., Barber, W., DiGiuseppe, M., & Childs, E. (2017). Democratizing digital learning: theorizing the fully online learning community model. International Journal of Educational Technology in Higher Education, 14(1), 1-16. https://edtechbooks.org/-rRsa

Bonk, C. J., Zhu, M., Kim, M., Xu, S., Sabir, N., & Sari, A. R. (2018). Pushing toward a more personalized MOOC: Exploring instructor selected activities, resources, and technologies for MOOC design and implementation. International Review of Research in Open and Distributed Learning, 19(4). 92-115. https://edtechbooks.org/-uUXJ

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative research journal, 9(2), 27. https://edtechbooks.org/-EKon  

Carver, L. & Harrison, L. (2013). MOOCs and democratic education. Liberal Education, 99(4), 20-25. https://edtechbooks.org/-oMa

Creswell, J. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.) [eBook edition]. Sage. https://edtechbooks.org/-snD

Croft, B., & Brown, M. (2020). Inclusive open education: presumptions, principles, and practices. Distance Education, 41(2) 1-15. https://edtechbooks.org/-imzC

Cvitanovic, C., Colvin, R.M., Reynolds, K.J., & Platow, M.J. (2020). Applying an organizational psychology model for developing shared goals in interdisciplinary research teams. One Earth, 2(1), 75-83, https://edtechbooks.org/-kBKf

Deterding, N. M., & Waters, M. C. (2018). Flexible coding of in-depth interviews: A twenty-first-century approach. Sociological methods & research, 50(2), 1-32. https://edtechbooks.org/-AuJg  

Ebben, M. & Murphy, J.S. (2014). Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328-345. https://edtechbooks.org/-IAWM10.1080/17439884.2013.878352

Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of advanced nursing, 62(1), 107-115. https://edtechbooks.org/-hRL 

Freire, P. (2014). Pedagogy of the oppressed: 30th anniversary edition [eBook edition]. Bloomsbury Academic.

Freitas, A. & Paredes, J. (2018). Understanding the faculty perspectives influencing their innovative practices in MOOCs/SPOCs: a case study. International Journal of Educational Technology in Higher Education, 15(5), (1-13). https://edtechbooks.org/-ZKJA10.1186/s41239-017-0086-6

Gillani, N. & Enyon, R. (2014). Communication patterns in massively open online courses. The Internet and Higher Education, 23, (18-26). https://edtechbooks.org/-UADc

Gunawardena, C. N. (2014). Globalization, culture, and online distance. In O. Zawacki-Richter & T. Anderson (Eds.), Online distance education: Towards a research agenda, pp.75-107. https://edtechbooks.org/-uknZ          

Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58. https://edtechbooks.org/-keh

Ito, M., Arum, R., Conley, D., Gutiérrrez, K., Kirshner, B., Livingstone, S., Michalchik, V., Penuel, W., Peppler, K., Pinkard, N., Rhodes, J., Tekinbaş, K., Schor, J., Sefton-Green, J., & Watkins, S. (2020). The connected learning research network: Reflections on a decade of engaged scholarship. Connected Learning Alliance: Irvine, CA. https://edtechbooks.org/-SwJT

Kizilcec, R. F., Saltarelli, A. J., Reich, J., & Cohen, G. L. (2017). Closing global achievement gaps in MOOCs. Science, 355(6322), 251-252. https://edtechbooks.org/-uuMf

Kizilcec, R. F., & Saltarelli, A. J. (2019). Psychologically Inclusive Design. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM (Vol. 10, No. 3290605.3300704). https://edtechbooks.org/-owo

Kleinman, M. A. (2018). Teaching at scale: Instructor experiences with massive open online courses (MOOCs) (Publication No. 11006724) [Doctoral dissertation, University of Michigan]. ProQuest Dissertations Publishing. https://edtechbooks.org/-eogJ  

Kolowich, S. (2013). The professors behind the MOOC hype. The Chronicle of Higher Education, 18. https://edtechbooks.org/-ThxDt

Lawrence, G. (2013). A working model for intercultural learning and engagement in collaborative online language learning environments. Intercultural Education, 24(4), 303-314. https://edtechbooks.org/-mCXQ10.1080/14675986.2013.809247

Loizzo, J., & Ertmer, P. A. (2016). MOOCocracy: the learning culture of massive open online courses. Educational Technology Research and Development, 64(6), 1013-1032. https://edtechbooks.org/-fIee

Macleod, H., Sinclair, C., Haywood, J., & Woodgate, A. (2016). Massive Open Online Courses: Designing for the unknown learner. Teaching in Higher Education, 21(1), 13-24. https://edtechbooks.org/-eee

Malterud, K. (2001). Qualitative research: Standards, challenges and guidelines. The Lancet, 358(9280), 483-488. https://edtechbooks.org/-UZds

Marshall, S. (2014). Exploring the ethical implications of MOOCs. Distance Education, 35(2), 250-262. https://edtechbooks.org/-Xnj

Mezirow, J. (1997). Transformative learning: Theory to practice. New directions for adult and continuing education, 1997(74), 5-12. https://edtechbooks.org/-uIQ

Mittelmeier, J., Rienties, B., Tempelaar, D., & Whitelock, D. (2018). Overcoming cross-cultural group work tensions: mixed student perspectives on the role of social relationships. Higher Education, 75, 149–166. https://edtechbooks.org/-gXR

Morris, S. & Stommel, J. (2017). 11. Open Education as Resistance: MOOCs and Critical Digital Pedagogy. In E. Losh (Ed.), MOOCs and Their Afterlives: Experiments in Scale and Access in Higher Education (pp. 177-197). Chicago: University of Chicago Press. https://edtechbooks.org/-bFED

National Center for Institutional Diversity. (2021, October 26). NCID’s statement on diversity research & scholarship. Framework for diversity scholarship. https://edtechbooks.org/-KdI

Portera, A. (2008). Intercultural education in Europe: Epistemological and semantic aspects. Intercultural Education, 19(6), 481-491. https://edtechbooks.org/-pyhd  

Portmess, L. (2013). Mobile Knowledge, Karma Points and Digital Peers: The Tacit Epistemology and Linguistic Representation of MOOCs/Savoir mobile, points de karma et pairs numériques: l’épistémologie tacite et la représentation linguistique des MOOC. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 39(2), 2-8. https://edtechbooks.org/-zfuL

Quintana, R. M., Brooks, C., Smothers, C., Tan, Y., Yao, Z., & Kulkarni, C.  (2018, June). Mentor Academy: Engaging Learners in the Creation of Data Science Problems for MOOCs. Proceedings of the 13^th International Conference of the Learning Sciences (ICLS). (pp. 1415-1416). June 24-27. London, England. https://edtechbooks.org/-scSS

Rhoads, R.A., Berdan, J., & Toven-Lindsay, B. (2013). The open courseware movement in higher education: Unmasking power and raising questions about the movement’s democratic potential. Educational Theory, 63(1), 87-109. https://edtechbooks.org/-kuB10.1111/edth.12011

Shaull, R. (2014). Forward in P. Freire, Pedagogy of the oppressed: 30th anniversary edition (pp. 29-34)  [eBook edition]. Bloomsbury Academic.

Shahini, R., Davis, H. C., & Borthwick, K. (2019). Design recommendations to address cultural issues in multicultural MOOCs: a systematic literature review in A. Plutino, K. Borthwick, & E. Corradini (Eds.), New educational landscapes: Innovative perspectives in language learning and technology, (pp. 55-62). Research-publishing.net. https://edtechbooks.org/-kNM 

Stewart, B. (2013). Massiveness + openness = new literacies of participation? MERLOT Journal of Online Learning and Teaching, 9(2), 228-238. https://edtechbooks.org/-tEmI

Stommel, J., Friend, C., Morris, S.M. (2020). Introduction. In J. Stommel, C. Friend, S.M. Morris (Eds.), Critical digital pedagogy: A collection (p. 1). Hybrid Pedagogy. https://edtechbooks.org/-FXn

Toven-Lindsey, B., Rhoads, R., & Berdan, J. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. Internet and Higher Education, 24, 1-12. https://edtechbooks.org/-hWk 

Appendix A

Modified NCID Framework

Appendix B

Below are two sample MOOC proposals that were de-identified and coded using the modified NCID framework outlined in Appendix A.

How does your Initiative help the University of Michigan to become more diverse, equitable, and inclusive?

Appendix C