In this chapter, we will explore the evaluation procedures used when exploring new technologies such as a Learning Management System (LMS). Evaluation of a technology change is critical. Not only do you need stakeholder buy-in, but you also need data that the stakeholders have bought into the idea of change. When conducting an evaluation, you want to engage both faculty, staff, and students on their feelings and attitudes towards the new platform. When making a large-scale technology change, it is essential to have data to support the change. For example, in higher education organizations, faculty conduct research; therefore, the design of your survey tool and how you communicate the data is essential. In a corporate atmosphere, you need to consider the attitudes and feelings of the employees; therefore, the survey instrument should include those as well. Finally, the development of the evaluation plan needs to occur during the planning stage.

The evaluation plan consists of:

• Survey instruments
• Communication plan for faculty to students
• A timeline of when data collection will take place and type of data that will be collected
• Duration of survey availability
• Information on when and how data will be analyzed
• Details on how the data will be used and communicated.

Instructional Design Evaluation

You may have learned, are learning, or will learn about evaluation from the Instructional Design vantage. In many cases, it is a training needs analysis where formative and summative evaluation is employed. Formative evaluation provides data for revision and improvement. One-to-one or small group format is typically used to collect data to ensure the designed training meets the training objectives. Summative evaluation happens after training has been launched to determine whether the training produced the intended outcomes. The instructional design process is a systematic method of approaching an instructional problem. In this case, it is not an instructional problem; regardless, it is still essential to employ a systematic method to ensure a smooth transition from one system to another.

Evaluation of a technology product or system such as a Learning Management System (LMS) should not be an afterthought. You do not want to conduct the pilot and then think, "We should create a survey." It needs to be a well-planned, systematic procedure. The following sections will guide you through the thought processes and procedures to consider before making any significant decisions about changing technologies such as an LMS.

Timeline

When should you start thinking about your evaluation procedure? You should be planning the evaluation when you are considering a technology change. While reviewing new technology systems, begin the planning process. If your organization is considering new technologies, then change is likely to occur, and that is the time to start planning. Therefore, at the onset of this process, begin to design the evaluation. Consider the following questions:

• What is the culture of my organization? Is the culture accepting of change?
• Does leadership support this change?
• Who are the stakeholders?
• How will the results be disseminated transparently?

The evaluation design should depend on the culture of your organization. You may want to send a pre-survey to determine the feelings of the stakeholders before implementing the pilot. Cultural awareness will help guide the design of the evaluation plan and how you approach change in the organization. Again, begin designing the evaluation plan at the onset of change. It is crucial to have the feedback of the stakeholders and evaluation committee on the evaluation plan. Once you have approval on the overall project, begin the survey instrument design. The development takes time; therefore, the sooner you start the process, the better.

Stakeholders

Often when thinking of an LMS change, end-users are typically thought of as the first and foremost stakeholders. Stakeholders are those who have a share in the organization or have an interest in the organization. There can be primary stakeholders and secondary stakeholders; all have varying influence in the organization, and with the change. In a higher education context, faculty and students would be the primary stakeholders. Although students come and go, faculty do not, however, student or end-user input is still important. There should be end-user representatives on the LMS committee who serve as stakeholders in the evaluation process. Other secondary stakeholders include university staff, such as administrative and support staff. Often the LMS is used for advising, student orientation, and professional development, where administrative staff are the ones administering many of these entities. Also, administrative staff are the eyes and ears of faculty. Keeping the administrative staff informed of changes, allows for another communication channel for the impending technology change. Information Technology Systems (ITS) should also be involved as they are the entity that supports enterprise or system-wide technologies. ITS also typically governs and vets technologies for privacy statements, data storage, and security as well as ADA statement. All new technologies must be ADA compliant (Americans with Disabilities Act) for consideration.

ITS will also determine if the technology meets the privacy and security standards. For instance, some companies will have access to student data and sell it to other companies. Storage is another component, and some organizations will require all data to be stored in the country of residence. After product vetting, ITS will determine whether they can support this product. For example, if a faculty, student, or staff, is having technical issues, they typically call a Help desk that is associated with this group.

Finally, organizational administrators are key stakeholders since they often hold the purse to purchase the product. There may be a financial cap on the product (see Table 1). Therefore, develop a communication plan to keep the administration informed on critical decisions continually for budget planning and sustainability.

Table 1

Stakeholder Impact

Design of evaluation instrument

The evaluation instrument needs to be developed before the pilot, especially if you plan on having pre- and post-surveys (Appendix A). Reach out to other organizations or to the LMS company to determine if there is a survey instrument available that can be modified for your organization. For example, we reached out to a similar organization that had recently gone through an LMS transition. This organization did a pre-and post-survey. Our organization only did a post-survey. Therefore, the survey was modified based on how our organization was conducting the pilot.

Designing with the End User in Mind

When designing the survey, you want to keep the end-users in mind. You will need a survey for the pilot faculty, students, and potentially for others who may not have participated in the pilot but were able to have some hands-on experience. For example, in our pilot, we were limited by the number of students who could access the LMS, but all faculty could have access. Therefore, we had training sessions that allowed non-pilot faculty to have hands-on experience with the tools and features of the new LMS. They could not teach/have student interaction in the LMS. Therefore, you may have three variations of the same survey but slightly different for each audience. You also want to keep the surveys separate as the pilot participants' input will be significantly valued over the opinion of a person who may have an hour of "hands-on" experience. Once you have developed your survey, have the LMS committee, particularly the faculty, approve the survey. In our case, some of the faculty wanted more student demographic information regarding the students who were using the new LMS. The LMS committee needs to give the final approval for the survey instruments as this also creates stakeholder buy-in.

During the instrument development stage, you may want to employ various techniques such as the think-aloud protocol to ensure usability. The think-aloud protocol allows you to fine-tune your instrument and confirms the end-user will read it the way you have designed it. Once you have developed your survey, find 3-5 faculty to read through the survey. While they are reading through the survey, they should state their thoughts aloud. When employing this protocol, take thorough notes and then make adjustments to the survey instrument. It is important to remember that if the end-users are unsure of what you are asking for, you will not get the data you want and need.

Criteria to Consider Before Designing

Depending on your organization, you may be able to deploy one survey to collect data. One organization conducted pre- and post-surveys. Another organization not only conducted surveys but also employed faculty interviews. It is essential to understand the culture of your system and gather the data that will be appropriate for your organization.

One factor when designing a survey is not to compare the old system with the new system in terms of features. As Kim and Lee (2008) stated, older systems will not have the same functionality or tools since technology is continually improving. As a result, it is imperative to consider the end function of an LMS: teaching and learning. Kim and Lee (2008) proposed seven aspects of criteria: instructional management, interaction, evaluation, information guidance, screen design, technology, and organizational demand. The first four aspects are directly related to instruction. Criteria related to instructional activities are screen design, technology, and organizational demand.

Other criteria for consideration are security and compliance. The LMS must be completely accessible with regards to ADA (Americans with Disability Act) laws as well as adhere to security and privacy policies. Some states or schools may have policies regarding where and how student data is stored. For example, in Canada, student data must be housed within the country. Finally, the cost is a consideration typically for upper administration. (Appendix A - Survey questions for pilot faculty)

If the evaluation committee intends to collect qualitative data, the semi-structured questions should be designed and vetted by the LMS team before the pilot. Often, qualitative feedback is semi-structured, which means that the questions are guiding points, but the interview or focus group should be guided by the conversation, not the list of questions.

To conclude, whatever method of evaluation, whether it be a survey or a conversation, they all should be developed early and be vetted by a group of stakeholders such as the LMS committee.

Duration of evaluation

There are two points to consider for the duration of the survey. First is the actual time it will take the participants to take the survey. Research has shown that participation declines if the survey takes longer than 20 minutes to complete. Also, completion rates drop if there are more than three open-ended questions. Consequently, the design of the survey should be carefully considered.

The second point is the duration of the survey and/or evaluation. Will the evaluation span the whole pilot, or will it be at the end of the pilot? The evaluation period may depend on how many pilots are occurring and the culture of your system. Also, if you are conducting interviews or focus group sessions, those will need to be strategically planned to optimize attendance. The timing of qualitative data collection will depend on the duration of the pilot. If the pilot is only one semester, then focus groups and interviews will be clustered near the end of the semester.

Response Rate

It is essential to have high response rates on your data. For example, you should have 90-100% of the pilot faculty. When deploying an electronic survey, the minimum/average response rate is approximately 30% to ensure validity. You want to use techniques to maximize the return rate. Many survey tools (e.g., Qualitrics, Survey Monkey) have contact lists where you can email the recipients from within the survey tool. Contact lists allow you to set up follow up emails that will only send out to those who have not completed the survey. Our LMS administrator exported class lists from the Banner system and imported them into Qualtrics. To assist with a high student response rate, you may want to set up the survey, so students have to choose the class they are enrolled in the demographics section of the survey, this way, if instructors want to offer extra credit points for a majority of the class taking the survey, you can provide them with the proper information. Typically, the surveys are anonymous; therefore, accessing student names are not possible. If you use a contact list, with some electronic surveys, there is a name attached to the data. If this is the case, be sure to indicate that you will protect the privacy of the participant and that the survey is not anonymous, but it is confidential.

If you choose to incorporate focus groups or interviews, you will need to either have a very detailed notetaker or record the sessions. If you decide to record the sessions, a consent form will need to be developed and signed by the participants. A lower percentage rate is acceptable for focus groups and interviews due to the amount of data and detail. Still, it will be important to have an equal representation of all areas such as colleges, students, and staff. There is no acceptable response rate for focus groups and interviews but reporting on the equity of representation will be necessary.

Student data

It is imperative to collect student data, but often there is a low response rate for students. One organization had between a 14-18% response rate. Another organization had a 61% response rate for students. The size of the pilots was much different from one university piloting 50+ classes and another piloting 15 classes. The project leaders continually emailed the faculty participating in the pilot asking them to encourage their students to take the survey. Many of the faculty gave extra credit to students who took the survey or stated if 80% of the students took the survey then the class would get extra credit. This assisted with the student response rates and validated the data.

Data Analysis

You have deployed your survey and/or conducted your interviews. Now you have a lot of data. What are your next steps? First, you do not want to download the results and email them out to the faculty. You need to organize your data in an easy to read format for people who may be unfamiliar with the technology. First, organize your data into graphs to visualize the results. Many survey tools will generate graphs for you. Next, organize any qualitative data (open-ended questions) by theme. For example, qualitative data could be organized under three themes: positive, negative, neutral. Thematic organization allows the readers to see that the positives outweigh the negatives (hopefully).

If you have incorporated qualitative methods such as interviews and focus groups, you need to analyze the notes or recordings. If you recorded the sessions, transcripts need to be created either manually or through a paid service. This should be a part of the evaluation plan and incorporated into the evaluation committee's time or budget. Once the transcripts have been completed, they need to be reviewed for themes. We will not go into qualitative analysis in this chapter, for more information, refer to Saldaña (2015). Keep it simple and again find the positive, negative, and neutral themes. You may choose to display them as a chart or a graph, but if you collect qualitative data, include quotes.

Crafting your Final Report

Communication is essential! You need to ensure that you document that you have communicated. For example, an instructor may complain that he was unaware of the impending LMS change. If your organization documents all the communication methods you employed, you can easily inform that stakeholder. Make stakeholders aware of the change and ensure you provide adequate data. You have to make your opportunities to use as many venues as possible. For example, you need to use both electronic and paper formats to ensure all stakeholders are aware of the change. Often, informative emails are overlooked; therefore, using a flyer or paper distribution can assist with delivering the message. If the faculty has a senate or group that meets regularly, try to get on the agenda. Often departments have regular department meetings. Be aware of the departments that use the LMS regularly or that have one or more online programs and request to be on their departmental agenda. Overall, take a grassroots approach for informing the faculty/staff that change is coming and how that change occurred.

The LMS evaluation report should consist of an executive summary, members of the committee, purpose, steps taken to choose a pilot company, any limitations of the pilot, and results of the pilot along with your next steps. The executive summary serves as an overview of the report in its entirety. It is similar to an abstract in a journal article.

The dissemination of the evaluation should be highly publicized to faculty, staff, students, and administration. For instance, another organization conducted three pilots for their LMS evaluation, all of which were located on their website. Again, communication is essential. As project managers, it is vital to inform the stakeholder of each stage in the process. For example, you may provide monthly updates on the pilot detailing how it is going and then conclude with a decision. Once a decision has been determined, it is essential to get the message out. You may want to get on the Faculty Senate agenda, the IT advisory committee agenda, post in the campus news, and/or have the provost/president send out a message directly to the faculty and staff.

What do you include in your report? Begin with the background and explain the historical context. How long have you been on the current system, who was involved with the decision to look for a new LMS, and the objectives for the new LMS? Next, you need to include a purpose statement. What is the purpose of finding a new LMS? For example, there is high dissatisfaction among faculty with the current LMS, or in other cases, a cloud-based solution with minimal downtime is needed. Next, you may want to provide the context of the pilot. If you were not involved with the pilot activities, you should meet with the project manager who was involved and ask them to write that section. If that is not feasible, you then want to move on to the design of the evaluation plan. Be sure to include the response rate for all surveys, when the survey was conducted, who was included, and the questions asked. Be as transparent as possible. Finally, based upon the data and the committee decision, write a section on conclusions and recommendations (Appendix C).

To conclude, the evaluation process is an essential process for the sustainability of the LMS. If the pilot and evaluation of the pilot are conducted in a haphazard method, the chances of faculty buy-in decrease as well as overall adoption. Technology change is hard on the end users; therefore, the more you can document and demonstrate the positives to the stakeholders, the more likely the technology will be adopted in a seamless manner.

References

Appendix A

Canvas Satisfaction survey

Q1 Which of these describe(s) your Canvas course? Check all that apply.

• Face-to-face course
• Fully online course
• Hybrid course
• Combined course
• Undergraduate
• Graduate
• Other

Q2 Please rate your experiences in Canvas: *Bb = Blackboard*

Q3 Which of the Help features have you used for Canvas?

Q4 Would you recommend Canvas as a replacement for Blackboard

• Yes
• Unsure
• No

Q7 Please enter your reason for your recommendation. (optional)

Q6 Which one of the following statements most closely resembles your beliefs about [new LMS] versus [old LMS].

• [new LMS] is unequivocally better than [old LMS]
• It will take some time to learn [new LMS] more thoroughly, but I think it is better than [old LMS], overall
• [new LMS] is an acceptable replacement
• [new LMS] and [old LMS] are about the same, as far as I can tell.
• [new LMS] is an unacceptable replacement.
• It will take me some time to learn [new LMS] more thoroughly, but I think it's worse than [old LMS], overall.
• [new LMS] is unequivocally worse than [old LMS].

Q5 Anything else you want the LMS Evaluation Committee to know?

Appendix B

Canvas Pilot Initial and Midterm Faculty Evaluation

Please rate your experiences:

Would you recommend [new LMS] as a replacement for [old LMS]?

• Yes
• No
• Unsure

Canvas Pilot Initial and Midterm Student Evaluation

Conducted [Date], 2016 Please rate your experiences:

Would you recommend [new LMS] as a replacement for [old LMS]?

• Yes
• No
• Unsure

Which one of the following statements most closely resembles your beliefs about [old LMS] vs. [new LMS]?

• [new LMS]?is unequivocally better than [old LMS]
• It will take me some time to learn [new LMS]?more thoroughly, but I think it's better than [old LMS], overall.
• [new LMS]?is an acceptable replacement.
• [new LMS]?and [old LMS] are about the same, as far as I can tell.
• [new LMS]?is an unacceptable replacement.
• It will take me some time to learn [new LMS]?more thoroughly, but I think it's worse than [old LMS], overall.
• [new LMS]?is unequivocally worse than [old LMS].

Would you recommend Canvas as a replacement for Moodle?**

• Yes
• No
• No preference

Appendix C – Example report

Table of Contents

• Learning Management System Transition Report
• Executive Summary
• Background and Context
• Membership of the Committee
• Committee Activities
• Canvas Pilot
• Facts and Figures of the Pilot
• Evaluation of Canvas
• Conclusions and Recommendations
• Adoption of Canvas for 2018-19
• Extension of Blackboard
• Timeline for Transition to Canvas
• 24/7 Technical Support Available
• Faculty Professional Development & Orientation to Canvas
• Caveats
• Appendix A: Rationale from committee to pilot Canvas
• Appendix B: Canvas Pilot – Faculty Evaluation
• Appendix C: Canvas Pilot – Student Evaluation

Learning Management System Transition Report

This report is a summary of the work to date by the Learning Management System (LMS) Evaluation Committee. This committee was formed as a subset of the IT Advisory Council.

Executive Summary

UNC Wilmington is reviewing the current learning management system, Blackboard Learn. The university moved to this system in 2010. UNCW hosts Blackboard Learn on site.

On average, universities evaluate LMSs approximately every 8 years. This current review gives UNCW an opportunity to re-assess its learning management needs and evaluate alternatives.

This evaluation is timely given the anticipated growth of online classes at UNCW. The objectives of the review are to:

• Identify UNCW’s current academic community needs for an LMS.
• Evaluate options to meet current needs and best position UNCW for a rapidly changing future.
• Recommend an LMS to pilot at UNCW.

With the growing online programs running on both traditional (15 week) and accelerated (7-week) schedules, the committee reviewed LMS’s that were cloud based to minimize down time for security patches and upgrades.

The Learning Management System Evaluation Committee engaged in a number of investigative and exploratory tasks over the course of this academic year. The committee’s deliberations indicated that the Canvas LMS potentially offers significant improvements over Blackboard, as recognized by faculty, students, and staff. These advantages include increased reliability, greater ease of use, user-friendliness for mobile devices, plus the flexibility and adaptability in meeting the growing needs of faculty and students. Given these considerations, the LMS Evaluation Committee recommends that we adopt Canvas as the campus LMS beginning in Summer 2018 and that The Office of eLearning and the Learning Management System team immediately begin assisting faculty with this migration. The committee also recommends that we continue to run Blackboard for a one-year overlap period, until May 2019, to give ample time to transition to the new system.

Background and Context

The 2017-2018 academic year marks our 8th year as a Blackboard campus. Our current version, Blackboard Learn, was released in April 2010 and has been showing signs of its age, such as a lack of responsive design and an outdated user interface.

Since the advent of Blackboard, newer learning management systems have been developed with usability and sustainability in mind, to meet the modern demands of the academy. They offer cloud-based solutions that can scale up system resources during peak usage. As well as support a broad range of plugins, product extensions; and up-to-date mobile apps for faculty and students. Some of these newer systems have a more contemporary look and feel with a user centered product development and support model, implementing small fixes and improvements continuously without extended downtimes.

Therefore, given our current situation and the new offerings available, a Learning Management System Evaluation Committee convened in September 2017 to examine Blackboard and the viable alternatives to determine whether a new system was needed.

Membership of the Committee

The Committee is co-Chaired by the director of the Office of ELearning and the IT LMS manager. The committee is composed of 17 members representing all campus units. Specifically, there are ten faculty and seven non-faculty on the committee. They are: two representatives from the Cameron School of Business, three representatives from the College of Arts and Sciences, three representatives from the College of Health and Human Services and two representatives from the Watson College of Education. Along with faculty, there are committee members representing Human Resources, Information Technology Systems, the Office of eLearning, and Randall Library. Students from SGA were solicited to be on the committee, but the committee did not receive a response.

Committee Activities

The committee meet eight times in Fall 2017 to discuss, analyze and undertake the following activities: In the initial meeting, the committee members were reminded of their charge: Choosing an LMS appropriate for UNCW now and to accommodate future growth. The co-chairs also presented background information about LMSs to educate the committee members.

Additionally, they also received input from the committee members of the criteria for choosing an LMS. Finally, the co-chairs presented the three LMSs for the committee to consider: Blackboard, Canvas, and Desire 2 Learn. These were chosen based on credibility, reliability, technology (specifically cloud-based), service, and longevity. The committee met six times throughout the Fall semester. Two meetings were conducted via WebEx with personnel from [organization] and [organization] to hear about their experience with Canvas, and one with the [organization] to hear about its experience with Desire 2 Learn. The discussion with representatives from the other institutions covered reasons for choosing an LMS, transitioning from one LMS to another, working with the vendor and the vendor’s quality of service, etc. Additionally, the committee viewed a demonstration of Canvas.

Inclusiveness and transparency were the driving principles for the committee. As such, in addition to the six in-person meetings, the co-chairs met with each committee member individually to clarify questions and encouraged them to seek input from the colleagues of their respective college. The committee also sent a survey via the Provost’s Perspectives newsletter, the SWOOP and a survey link within Blackboard to ask for faculty and staff assessment of Blackboard. The data collected were presented to the LMS committee members to determine the current level of satisfaction. [co-chair] also attended some individual unit faculty meetings and with [co-chair] attended the Faculty Senate Steering Committee and the IT/Library Committee to discuss the work of the LMS committee.

After multiple meetings of examining the various LMSs the committee arrived at two options to vote upon:

1. Pilot Canvas. This does not require a Request for Proposal since UNC-GA has a contract with Canvas.
2. Not pilot Canvas and conduct a RFP to look at other LMSs.

Overall, 88% of the committee voted to pilot Canvas. The breakdown of the votes was:

• Eight faculty for piloting Canvas
• One faculty for RFP to demo other vendors
• One faculty “No to both options”
• Seven staff for piloting Canvas

Canvas Pilot

Facts and Figures of the Pilot

During the Spring 2018 semester, a group of pilot faculty signed up to teach their courses using Canvas for the entire semester. Included in the pilot were:

• 14 faculty and courses
• 14 disciplines across the four colleges
• 367 students
• 6 fully online courses
• 6 hybrid courses
• 2 face-to-face courses

There was a mix of undergraduate and graduate courses.

The Learning Management Systems staff worked with the Canvas team and others in ITS to complete technical buildout, such as Banner integration, so that the pilot experience would have high fidelity with how the system would behave in full implementation, rather than a diminished or trial version. Faculty had the opportunity to attend training with Canvas personnel at the end of the Fall 2017 semester. The pilot faculty were encouraged to utilize the Canvas Help features which included 24-hour support via toll-free telephone line, web chat, and online help forms as well as having a team of GA’s to assist faculty with questions. Four open labs sessions were also scheduled throughout the semester that allowed non-pilot faculty to receive assistance with evaluating Canvas and non-pilot faculty were encouraged to log into the system for review.

Evaluation of Canvas

During the middle of March, faculty and students were surveyed about their impressions of Canvas and how it performed throughout the semester (see Appendices B and C). Results indicated that Canvas was a suitable replacement for Blackboard. Faculty support to replace Blackboard with Canvas was 71% in favor, 28% unsure, and none against. Students replied 53% in favor, 19% against, and 30% unsure.

Conclusions and Recommendations

Adoption of Canvas for 2018-19

On April 3, 2018, members of the committee voted unanimously to recommend the adoption of Canvas as the campus LMS, due to the number of potential benefits, particularly with respect to the reliability, adaptability, ease of use, and relevance to UNCW’s instructional mission. Therefore, the adoption of Canvas should be initiated immediately, in preparation for full implementation for the 2018-19 academic year.

Extension of Blackboard

In conjunction with the formal transition to Canvas, the committee also recommends that the university keep Blackboard available for use throughout the migration, until the end of the Spring 2019 semester. System updates and patches will be applied to Blackboard during this timeframe. All new tool integration requests will be applied to Canvas. This will provide ample transition time for faculty to move their course materials to Canvas. Blackboard will also need to be available for an additional year after the last course has completed in the event of a grade dispute. This availability will be limited to the Learning Management team only.

Timeline for Transition to Canvas

The transition to Canvas can begin immediately for all non-accelerated programs. The timeline would reflect the following:

• Summer 2018

Canvas is available to the entire university community and is available to any faculty that are not teaching as part of an accelerated program.

• Fall 2018

Faculty and staff continue to migrate course materials.

Courses associated with CRN numbers will take priority over non-banner/professional development courses

• Spring 2019

Online accelerated programs migrated to Canvas

Training from ITS and OeL and online resources for faculty migration would begin with Canvas availability to all faculty. The last courses that can be taught in Blackboard will be Spring 2019 courses with an end of life on May 17th, 2019

24/7 Technical Support Available

The committee recommends that the university purchases the Tier 1 level of 24hour technical support (e.g. toll-free telephone line, web chat, and online help forms), all available 24 hours per day 7 days per week from Canvas. This should help with the transition for both faculty and students. The committee recommends an analysis at the end of the year to determine whether this service should be continued into another academic year.

Faculty Professional Development & Orientation to Canvas

Professional development will be offered through a joint effort between the Learning Management Systems team and Office of e-Learning through a variety of instructional events: group training and one-on-one support.

Caveats

There are two products currently integrated into the Blackboard environment that are not integrated with Canvas. The first is Skillport which utilized the Blackboard building block and will have to be redesigned for integration.

The second product which is a part of Blackboard is SafeAssign. SafeAssign is a Blackboard product that cannot be integrated into Canvas. The Canvas also included piloting Turnitin which is currently integrated into Canvas and will be available to faculty as an alternate solution to SafeAssign.

Appendix A: Rationale from committee to pilot Canvas

Committee members were asked to provide a rationale for their vote. The reasons for piloting Canvas were:

• Student-centered design
• Currently being used by NC public schools (K-12); 83% of our 2017 freshman class are North Carolina residents.
• Seven of the 17 UNC system campuses already using Canvas
• While I appreciate the concerns of a few committee members that we have not explored all options, I feel as if we have explored those options that are viable for a campus of our size (and growing).
• Canvas has a stable mobile application both for grading and student interaction. Grading through iPad is important to faculty.
• Stable mobile application for student use.
• The need to have a reliable, supportive, responsive, and Cloud based platform.
• Need for a flexible, adaptable, and convenience for instructors and students.
• After meeting with School using Desire to Learn, I am very wary of going in that direction due to lack of support and lack of high performing mobile application.
• I am wary of Blackboard for similar reasons of Desire to Learn, lack of support and a poor mobile application.
• The Canvas demonstration eased the concerns I was given by my faculty with regards to ease of grading, mobile application, more creative use of apps for delivery of content, and the numerous clicks within Bb.
• Canvas seems to have a culture of support and collegiality that seems to be unmatched in this industry.
• I watched as the energetic representative demonstrated the flexibility, adaptability, and convenience of this tool for instructors and students. They also reinforced their responsiveness and ability to work with institutions to solve problems. The conversation with another university demonstrated that while no solution is perfect, Canvas provides much that can be used and adapted to fit our needs.