As previously discussed, a learning management system (LMS) is designed to organize and manage course creation; provide appropriate access to certain users; create and organize learning activities; and provide reports for all users (Mabed & Kohler, 2012). For any LMS implementation there are a number of people or stakeholders that will be impacted by the LMS. While some individuals may have a variety of roles within the LMS, each role has a unique perspective on what is expected from the LMS and what would make the LMS usable. If you want your LMS to be used, you need to look at usability from the perspectives of your various stakeholders.

Stakeholders

Your stakeholders will be anyone affected by the LMS (Wagner, 2008). As you begin listing the various individuals, it is important to consider their roles within the system, as well as the purpose, goals, and intentions of your institution. In addition to learners and instructors, there are instructional designers, system administrators, and technology support to consider. Each of these groups of stakeholders will approach the LMS based on their specific needs and goals.

Usability

According to the International Organization for Standardization, usability is “the extent to which a product can be used by specified users to achieve specified goals, with effectiveness, efficiency and satisfaction in a specified context of use” (n.d.). In order to look at the usability of an LMS, you will need to identify your users (stakeholders), and their goals; and then determine how effective and efficient the system is in meeting those goals. In addition, you need to look at your stakeholders’ satisfaction throughout the process; even if two system have equal ratings on effectiveness and efficiency, they may have very different satisfaction ratings (Kauffman, 2015).

Usability is related to the two strongest predictors of technology use - perceived ease of use and perceived usefulness (Davids, Halperin, & Chikte, 2015). Where “perceived ease of use” is how the technology appears to not require much effort, and “perceived usefulness” is how beneficial the technology appears to support the user in accomplishing their work (Davis, 1989). We can look at these two predictors through the lenses of user interface design (UI) and user experience (UX). The UI will have the strongest impact on how effective and efficient the system is, whereas the UX will impact overall satisfaction.

The User Interface (UI)

The user interface (UI) of a system encompasses the visual elements displayed to the user and define how a user interacts with the system (Ming, n.d.). The UI will determine “the degree to which a person believes that using a particular system would be free of effort” or “perceived ease of use” (Davis, 1989, p. 320). In addition, it is the UI that dictates the common quantitative measures of usability related to effectiveness and efficiency such as, the number of clicks, the time on task, error rates, and deviation from the optimal path (Hornbæk & Law, 2007).

While there continue to be many opportunities for further research on usability guidelines for an LMS, a commonly used heuristic for interface design is Nielsen’s 10 Usability Heuristics for User Interface Design (Nielsen, 1995).

Nielsen’s 10 Usability Heuristics for User Interface Design

1. Visibility of system status

Is the system available? If not, when will it be back up?

2. Match between system and the real world

Does the language used match the real world?

3. User control and freedom

Can users recover from mistakes?

4. Consistency and standards

Are the pages, words, menus, etc. consistent across the system?

5. Error prevention

Does the system support the user in optimal practices and prevent mistakes such as incomplete information?

6. Recognition rather than recall

Does the user have access to all the required information without having to remember?

7. Flexibility and efficiency of use

Do the background functions support efficient use?

8. Aesthetic and minimalist design

Are the system menus and dialogues direct and to the point, not adding additional cognitive overload and distracting users from their goals?

9. Help users recognize, diagnose, and recover from errors

Do the error message use plain, non-techical language to enable quick and efficient recovery from errors?

10. Help and documentation

Are the help resources easy to locate when needed?

These 10 broad guidelines can apply to a wide range of interfaces including an LMS. In terms of achieving specific goals, obstacles and issues in these areas make a system ineffective and inefficient.

In contrast to Nielsen’s 10 items, Krug (2006) uses a common-sense approach to interface design with his first “law of usability” being “Don’t make me think.” The word usage, visual display of clickable items, organization, and consistency across pages should support the individual in using the system, as opposed to trying to figure out how to use it. Thus, cognitive effort is reserved for goal attainment.

All of these broad guidelines would apply to an LMS. However, the purpose of a company’s website is generally to invite user’s to become customers, while the purpose of an LMS is to provide a platform for learning. Additionally, as each of your stakeholders has a different role in the learning process so they will also have varying goals. For example, the learner needs to learn specific content; the instructor needs to facilitate and provide appropriate feedback; the LMS / IT support needs to troubleshoot a wide range of issues. This difference means that not only do you and your stakeholders need to look at the UI, but to also identify explicit and implicit connections to educational theories.

The User Experience (UX)

The user experience (UX) includes the interaction with the interface, but looks more at the individual’s overall satisfaction with the experience. While the goals of the individuals are important, an individual’s overall experience with a system includes the look and feel of the interface as well as the emotions and motivations provided by the system. The satisfaction that contributes to a system’s usability is tied to the more qualitative measures such as preference, talk alouds, observations, and interviews. Also tied to satisfaction is the technology acceptance model predictor of “perceived usefulness” which is “the degree to which a person believes that using a particular system would enhance his or her job performance” (Davis, 1989, p. 320). You can have an effective and efficient system, but if the overall experience is not seen as helpful, or high in satisfaction, individuals will be reluctant to use the system.

In a review of education theory and strategies, Peters (2014) focused on developing heuristics for the design of learning interfaces (see figure 1). These guidelines consider education theory and research as well as the UI, so that they better address the UX of a learning system.

Peters (2014) 11 Heuristics for the Design of Learning Interfaces

1. Relevance of media and reduction of extraneous load
2. Learner control and freedom
3. Support for learning objective(s)
4. Alignment with specific learner needs
5. Appropriateness of look and feel
6. Support for the cognitive aspects of learning
7. Support for the affective aspects of learning
8. Appropriateness of media and tools
9. Accessibility
10. Usability
11. Feedback and responsiveness

For each of the heuristics, some of the design aspects will fall under the role of the instructional design and as well as the underlying architecture of an LMS. An instructional designer will have the most control over the following: “relevance of media and reduction of extraneous load”; “support for learning objective(s)”; and “alignment with specific learner needs.” All three of these areas are dependent on information outside of the LMS. The architecture of the LMS will dictate the following: “learner control and freedom”; “appropriateness of look and feel”; and “usability.” It is interesting to note that the one heuristic “usability” speaks specifically to the interface design, while all the others incorporate educational theory.

Both the LMS architecture and the instructional designer will have impacts in relation to the following: “support for the cognitive aspects of learning;” support for the affective aspects of learning;” “appropriateness of media and tools;” “accessibility;” and “feedback and responsiveness.” The LMS needs to have the functionality, and ability to meet these needs, but it is up to the instructional designer to implement them. For example, the instructional designer will select the media and tools to use in a course, while the LMS provides the functionality to add media and has a list of tools which can be used. Another example is with accessibility, the LMS needs to provide the option to add alt-text to an image, but it is up to the instructional designer to determine what text to add.

The effectiveness, and efficiency from an LMS is dependent upon the structure and functionality of the UI; whereas satisfaction is dependent upon the UX. Due to the nature of an LMS the role of the instructional designer can either enhance the positives of the underlying structure of the system, or detract from the overall experience when designing learning. Thus, usability is not solely dependent upon the system. An understanding of all stakeholders and their goals will help the instructional designer build upon the usability of the LMS.

Making it Real

Whether you are looking to implement an LMS in an educational institution, such as a college, or in a corporation, the roles of your stakeholders will be similar. In some cases, you will have one individual or group that takes on several roles. For example, often the instructor role and the instructional designer role are performed by one person, or the LMS administrator and LMS / IT support are handled by one person or group of people. In this section we look at each of the main roles for any LMS implementation. For each role there is a short description; at least one persona (see figure 2); a close-up look at the actions and goals for that group in relation to one specific activity - a quiz; and a summary of the questions this stakeholder group would ask in relation to an LMS adoption and implementation.

Learners

Who are your learners? Your learners will be the largest group using the LMS in terms of headcount. The goals of your learners will be to efficiently access their learning. This includes the ability to see all their courses, track their progress within each course, and know what they need to do next. Your learners should always be front and center, however, they are the group that will see the least in terms of the overall functionality of the LMS, and their access is controlled by the course design.

As learners will be the most widely varying stakeholder group, four personas were created to highlight key differences across various LMS implementations. All the names are fictitious.

What is a Persona?

A persona is based on real statistics about your stakeholders; it is a fictional character that is created to help identify specific needs and wants, as well as what motivates an individual and what they value (Dam & Siang, 2018). While this strategy is often used in the creation of a system, it can be helpful in the broad evaluation of systems and in the selection of your stakeholder representatives. A persona brings a real-world layer to your initial evaluation (Usability.gov, n.d.). To consider the goals and needs of each of your stakeholders, one or more personas were developed for each of the main roles in an LMS. While the personas used here may not match your specific stakeholders, they should give you an idea of who you should consider when gathering a group of your stakeholders together for adoption and implementation of an LMS.

Learner Personas

Matt Smith is a traditional undergraduate student who has not yet decided on a major. He is excited to explore a wide variety of subjects. He expects to see a wide range of courses and finds the variety motivating. He is comfortable with technology and accesses his courses on his phone, tablet, and computer depending on what he needs to accomplish. He prides himself in being self-sufficient and does not feel the need to bother his instructors or the IT department for much help.

Emily Barnes is a graduate student who has finally saved up enough money to complete her M.B.A. She has been working in a small family-owned company but would like to branch out on her own, or perhaps take her family’s company into a new area. She feels the M.B.A will build upon the skills and knowledge she has gained through real world experience. She was happy to find the nearest university is finally offering their M.B.A. program online so she can still continue to work and go to school. At the same time, the university is close enough that she can easily meet with her instructors or classmates on campus when she has questions.

Samuel King is a brand-new employee who needs to better understand the expectations of the institution. He brings with him a wealth of industry experience and is looking forward to applying his skills and knowledge in his new position. He was hoping to hit the ground running, but has quickly realized that there are some unique aspects of the company that he needs to better understand. He is concerned that training will take away from his work.

Amanda Williams has been working at the company for over ten years. She struggles with keeping track of all the required annual training. Additionally, she looks forward to continued learning opportunities to expand her skills.

Learner Taking a Quiz

Select one of the personas, and consider the following actions related to taking a quiz from that individual’s perspective. Where would obstacles occur? What specific tasks would hinder that learner from having a positive experience?

In taking a quiz, here are the needs and actions of your learner stakeholder group:

• Identify the need to take the quiz.
• Navigate to the quiz.
• Read instructions.
• Ask questions, if needed.
• Start the quiz.
• Answer each question.
• Track the time left, if there is a time limit.
• Submit responses.
• Receive a score.
• Review feedback.
• Identify how the quiz contributes to determining the final course pass/fail status.
• Identify what to do next.

Learner Perspective Summary

In looking at the actions for taking a quiz, the overall needs of the learner stakeholder group can be extrapolated into the following goals. They are presented as questions to help guide you in the evaluation and implementation of an LMS from the learner perspective.

• Where are my courses?
• What is required of me and what is optional?
• How do I know exactly what needs to be done and when?
• What do I need to do next?
• What is my grade? or How do I know if I passed?
• How can I access help with the content?
• How can I access help with technology?

Instructors [Delivery Aspects]

The role of the instructor will vary and depend upon the design of the course. This stakeholder group is looking specifically at the delivery aspects of a course, in other words, aspects in play when there are actual students in the course. The design aspects are addressed in the instructional designer stakeholder group.

Your instructors will have the highest interaction rate with the LMS, of all your stakeholders. For example, a learner will submit an assignment thus accessing the tool just enough to understand the instructions and upload a file; while the instructor will have to grade each learner’s submission, thus accessing the tool multiple times. This workload can be reduced by tools that automatically grade and provide feedback. However, your instructors while smaller in number than your learners, will be accessing your LMS the most.

Katelin Chow is a tenured professor who appreciates the opportunity to teach online as it affords her the opportunity to travel for her research. She has appreciated working with the university’s instructional designers to create online courses she enjoys. She misses the interaction of a traditional campus course, but feels the benefits for her research outweigh her concerns.

Arthur Richardson is responsible for conducting the majority of the continuous training required by his company. He has been pressured to move many of the traditionally lead courses to an online format. He has moved a couple of the trainings to the online format but does not feel that this new format will work for all the courses he currently teaches.

A Quiz - Instructors [Delivery Aspects]

Generally speaking, most question types in a quiz are automatically graded by the LMS. However, depending on the objectives of the course, the instructor may need to manually grade essay-type question. Additionally, the instructor may need to override automatically graded questions. Here are the needs and actions of the instructor stakeholder group in relation to a quiz:

• Identify the open and close dates.
• Identify a time limit, if set.
• Determine if students can access the quiz.
• Update instructions, if needed.
• Identify when quizzes have been submitted
• Provide feedback, if needed.
• Review questions and responses.
• Override question scores, if needed.
• Determine grade for each student.
• Ensure that the grades are determining the final course pass/fail status appropriately.

Instructor Perspective Summary [Delivery Aspects]

In looking at the delivery aspects for a quiz, the overall needs of the instructor stakeholder group can be extrapolated into the following goals. They are presented as questions to help guide you in the evaluation and implementation of an LMS from the instructor perspective.

• Can I see what is available to students?
• What items need grading and feedback?
• How do I communicate with my students - individually, in groups, and as a whole class?
• Can I add or update information or materials?
• How can I override automatically graded items?
• How do I access, update, and/or modify the gradebook?
• How do I provide feedback?

Instructional Designers [Design Aspects]

Your instructional designers will play a pivotal role in your LMS in how they maximize the tools and underlying structure. Even with the best tools available, the design of the course will determine the experience of the learner and the instructor. The instructional designer has the responsibility to use the strengths of the LMS. In some cases the instructor and instructional design roles may be performed by the same individual. As noted in the UX section, the instructional designer plays an important role in the design of the user experience in the context of learning.

Michael Rodriguez did not intend to become an instructional designer, but through a series of opportunities he discovered that he loved not only participating in, but developing the training for the employees of the bank. What began as a banking career took a few turns and he now works for an international company providing training around the world. He prefers to use a mix of options in developing training, but the well-developed self-paced modules are the easiest to modify for the many languages used in the company.

Jacqueline Stewart gained her first experience with instructional design when she was hired as a student worker in the center for teaching and learning. She easily picked up on the technology side of things and quickly found herself helping more and more faculty. When she looked for graduate programs she sought a program with a strong foundation in educational theory. She enjoys working with faculty on strategies for integrating technology, but looks forward to the opportunities where she can design online courses based on sound educational theory.

A Quiz - Instructional Designers [Design Aspects]

In selecting the use of the quiz tool, the overall design aspects will need to meet the course objective(s) and provide logical checkpoints for learning. If the instructional designer is working with a subject matter expert, some of the aspects of designing and creating a quiz will be performed collaboratively with the subject matter expert. Here are the needs and actions of the instructional designer stakeholder group in relation to a quiz:

• Identify the types of questions available.
• Select appropriate question types based upon the course objective(s).
• Create questions within the LMS question bank.
• Designate the quiz settings such as open and close dates/times, time limits, review options, etc.
• Create instructions for the learner.
• Designate grading for individual questions.
• Connect the quiz to the gradebook.
• Designate completion tracking, if available.

Instructional Designer Perspective Summary [Design Aspects]

In looking at the design aspects for a quiz, the overall goals and needs of the instructional designer stakeholder group can be extrapolated into the following goals. They are presented as questions to help guide you in the evaluation and implementation of an LMS from the instructional designer perspective.

• What tools and options are available?
• How can I organize a course?
• How can I create, edit, and modify information, instructions, and materials?
• How are items connected to the gradebook?
• Can I create dependencies?
• Can I designate completion tracking for the activities?

LMS Administrator

In addition to the design and delivery of courses to your learners, there are a wide range of functions that need to be maintained behind the scenes. Your LMS Administrator may be someone in your organization if you have a self-hosted solution, or a designated contact person if your LMS is managed by a vendor. This is the person that ensures that your LMS is running, secure, and updated.

Melissa Lee has worked in both higher education and in corporations in managing a variety of technologies. She has a solid background in information security and has been involved in several large-scale technology implementations. She discovered her love for LMS implementations while working for a human resources department. Having experience with both hosted and self-managed systems, she understands the pros and cons of each type of management. While she appreciates the access to additional support personnel with a hosted system, she likes the ability to control updates and minor changes with a self-managed system.

A Quiz - LMS Administrator

Your LMS Administrator will not necessarily be involved with a specific quiz. However, this role is responsible for ensuring that the system as a whole is maintained and supports the instructional designers, instructors, and learners in their goals in relation to a quiz. Here are the needs and actions of the LMS Administrator stakeholder group in relation to a quiz:

• Assign appropriate users (learners, instructors, and instructional designers) to their designated courses.
• Identify issues in the efficiency of the system.
• Archive older materials.
• Manage the system backup and recovery plan.
• Perform updates and maintenance.
• Maintain the security of the system, and user data.

LMS Administrator Perspective Summary

In looking at the responsibilities of the LMS Administrator in relation to a quiz, the overall needs of the LMS Administrator stakeholder group can be extrapolated into the following goals. They are presented as questions to help guide you in the evaluation and implementation of an LMS from the LMS Administrator perspective.

• How are students enrolled?
• How are instructors assigned to courses?
• How efficient is the system?
• How are old courses archived?
• What is the backup and recovery plan?
• What kind of reports can I run to tell how well the system is running?
• When and how are updates released?
• How does the system maintain security?

LMS / IT Support

All your users will need access to technology help. This role may be provided by other stakeholder groups such as instructors for learners, provided by the institution's IT department, or by the LMS vendor. This stakeholder group will actually see the LMS through all the other stakeholders’ perspectives.

Casey Jordan has worked in IT for over 20 years. He has a talent for asking the right questions to help users overcome technology hurdles. He manages a team that supports the entire institution with any and all technology questions. With the conversations of implementing an LMS he has concerns about adding yet another system to the list that his team supports.

A Quiz - LMS / IT Support

Your LMS / IT Support will need to be available to all users involved with a specific quiz. They are responsible for ensuring that all the other stakeholders - the institution, LMS Administrator, instructional designers, instructors, and learners are able to meet their goals in relation to a quiz. Here are the needs and actions of the LMS / IT Support stakeholder group in relation to a quiz:

• Ask questions about the issue.
• Identify if the system is having an issue.
• Determine if the user is accessing the LMS with a recommend device and browser.
• Identify whether the user has the correct access.
• View the system from the user’s perspective.
• Distinguish between user and system caused issues.

LMS / IT Support Perspective Summary

In looking at the responsibilities of the LMS / IT Support role in relation to a quiz, the overall needs of the LMS / IT Support stakeholder group can be extrapolated into the following goals. They are presented as questions to help guide you in the evaluation and implementation of an LMS from the LMS / IT Support perspective.

• Can I see what a specific user sees?
• Can I access system information?
• Does the LMS work on a variety of devices and browsers?
• What issues can be fixed without consulting the instructor, instructional designer, LMS Administrator, or the institution?

The Organization

This role may not seem a true stakeholder role; however, there will be reasons why an organization would invest in an LMS. In this context “organization” can be a college, a company, a department, human resources, etc. In exploring potential individuals in your organization, there will be someone who could represent the interests of the organization.

Anne Martin is the human resources director at her company. While she vast experience with all the aspects of human resources, the favorite aspect of her job is creating learning opportunities for employees. She is looking forward to working with the new instructional designers recently hired as part of the LMS implementation. She is hoping that the new LMS will be able to generate the reports she needs, as well as providing adequate information for employees to track their own learning.

Andrew O’Connor works in the Registrar’s office. His current responsibilities include coordinating with the LMS Administrator to ensure that both students and instructors are appropriately enrolled in the correct courses. He has also been tasked to work with the LMS Administrator to create reports related to retention efforts. He understands the need for these reports, but has concerns about the abilities of the LMS.

A Quiz - The Organization

Your organization will not necessarily be involved with a specific quiz. However, the organization has an invested role in the results of the quizzes in relation to determining successful completion. Here are the needs and actions of the organization as a stakeholder group in relation to a quiz:

• Track successful completion of quizzes.
• Identifying successful completion.
• Extracting reports for institution-wide requirements.

The Organization Perspective Summary

In looking at the role of the organization in relation to a quiz, the overall needs of the organization stakeholder group can be extrapolated into the following goals. They are presented as questions to help guide you in the evaluation and implementation of an LMS from the organization perspective.

• How to determine if the right learners are completing the needed courses?
• How to run reports on system usage?
• How to pull reports on specific institution requirements?
• What data is available to help determine if the learning is successful?

Conclusion

The usability of an LMS will partially be determined by the effective and efficient attainment of the goals of each of your stakeholder groups. It is important to carefully consider how each role will be affected by your LMS in order to identify those goals. While an LMS is designed to organize and provide learning opportunities, not every stakeholder group is focused only on the learning. Understanding the goals of each group will help you identify obstacles to your users’ satisfaction with the LMS.

In addition to the effective and efficient attainment of the goals of your stakeholders, the design of the interface will affect the overall user experience. General guidelines for interface design and heuristics specific to learning environments were provided to guide you in examining an LMS.

The descriptions and questions provide in the second half of this chapter will help you identify individuals in your organization who will best represent your stakeholder groups. In terms of an LMS evaluation you will want to make sure that each of your stakeholder groups is represented.

References