Introduction

Gamification has been applied to instruction in both didactic and experiential training to enhance learner engagement and motivation (Sera & Wheeler, 2017). As noted by Sardi et al. (2017) in their systematic review, there are a number of related terms being used to refer to the application of game design concepts, including game-based learning, gamification, educational gaming, and serious gaming. As education and the integration of technology continues to evolve, so do the definitions of such terms. Certainly, more terms will be coined in the coming years. Consensus on the use and definition of terms to guide both researchers and practitioners as they share their work publicly would be helpful; however, it is not the intent of this article. Instead, we use a widely accepted, broad definition that gamification is the “use of game design elements in non-game contexts” (Deterding et al., 2011, p. 10).

Gamification in Healthcare Education

Across disciplines within healthcare education, students report positive perceptions of gamification in some form or another. Nursing programs have reported the incorporation of gamification in education through digital badges (White & Shellenbarger, 2018), and nursing orientation gamification has been used to improve both productivity and retention of knowledge (Brull et al., 2017). Medical education has incorporated gamification within residency training to support credentialing, examination scores, surgical technique, and clinical decision-making skills. Gamification of online study tools that supplement traditional classroom education for exam preparation had a positive impact on otolaryngology training examination scores for otolaryngology residents (Alexander et al., 2018). Time spent gaming resulted in significant improvement in performance on a laparoscopic simulator compared to residents who practiced with the simulator alone (Adams et al., 2012). In addition to surgical technique, surgeons must have excellent decision-making skills. Lin et al. (2015) found evidence to support the validity of a web-based gaming platform for training and assessment of surgical decision-making. These studies and many others over the past decade have shown a positive impact of gamification on learning experiences and assessments across many programs in healthcare education.

Gamification in Pharmacy Education

Similarly, pharmacy education has noted positive outcomes from the use of gamification within the pharmacy curriculum. The American Association of Colleges of Pharmacy (AACP) introduced charges to its 2013-2014 Academic Affairs Committee in order to explore the role of gamification in pharmacy education, explore how colleges can support implementation, and identify areas where the impact would be most promising. Based on their research, the Committee recommended that colleges of pharmacy integrate serious games into their core curriculum for learning and professional development. Given the importance of student learning, the Committee also recommended that AACP, as an organization, develop serious games for institutions to utilize despite the cost and complexity. The Committee recommended that faculty collaborate with experts in the field of instructional design to facilitate quality game development (Cain et al., 2014). Since AACP’s Academic Affairs Committee’s report in 2014, there has been effort underway by various colleges of pharmacy to leverage the benefits of gamification within both the didactic and experiential curriculum. Some benefits include increased student knowledge in topics covered, enhanced empathy towards patients, and increased student engagement (Aburahma & Mohamed, 2015). For example, the University of Pittsburgh School of Pharmacy developed a mock investment game to educate students on important aspects of the pharmacy industry. Students showed significant improvement in 19 domains assessed in the study (Wolf et al., 2018). After a faculty development workshop at one university, pharmacy faculty reported eagerness to implement a variety of active learning strategies featuring gamification (Barone et al., 2018).  In response to dwindling applicant pools, several colleges collaborated to design and implement new components (e.g., educational games, guest speakers, team-based learning) into the admissions process. This resulted in an increase in applications (Salazar et al., 2018). In this article we explore a slightly different scenario—gamification of a graduate level pharmacy course using a gameshow format with guidance from the college’s instructional designer (ID).

Gamification Integration into College of Pharmacy

Faculty profiles collected by the Office of Faculty Affairs have identified that faculty in the Taneja College of Pharmacy (TCOP) have doctoral degrees in pharmacy and related fields and not in education; thus, the college chose to incorporate a Technology, Instruction, Evaluation, Design (TIED) team within the Office of Academic Affairs (OAA). The TIED team’s role is to provide faculty with guidance and support in developing pedagogical skills, implementing educational innovations, and evaluating implementation of innovations. Additionally, the TIED team partners with faculty in scholarly endeavors associated with these collaborations. The TIED team is made up of one ID and one Learning and Development Manager. The ID on the team has experience and training in design and development of online educational games. The positive results the ID has seen from effective game design within reading software motivated the ID to incorporate such game elements into other instructional areas. Faculty development and educational sessions by the ID have allowed faculty to successfully integrate gaming elements including escape rooms, Kahoot, rap battles, competitions, and more into their courses. When preparing to attend the 41^st annual meeting for Association for Educational Communications & Technology (AECT) 2018, the ID enrolled in a workshop on gamifying courses. As part of the pre-workshop, the ID presented an innovative idea to the pharmacy faculty to solicit volunteers to embark on the endeavor together. Two eager faculty members agreed to pilot the gamification of a course; a strategy which is not commonly used in pharmacy education. The faculty members shared their syllabi and current course outlines with the ID. During the workshop, “Redesign Courses into a Competition-Based Game-Show Format” by Kiran Budhrani (2018), the ID gathered resources and planning tools in preparations for gamifying the PHA6603C Internal Medicine Elective to be offered for the first time in the following academic year. The workshop facilitator used Deterding’s (2011) definition of gamification in describing the ID’s efforts at gamifying a course in multimedia instructional design. In this article, we describe the systematic process used to analyze the course, and to design, develop, implement, and evaluate gamification using a gameshow format. We include the selection of game elements and how we applied them to the non-game context of a graduate level pharmacy course.

Methods

Although designers in various fields (e.g., engineering, software development, education) set about their assigned tasks along different pathways, these pathways share several common elements. Some kind of analysis of the problem, need, or situation takes place. The analysis reveals what needs should be addressed. The needs drive the design plan which is developed and implemented. Evaluating the product, of course, must also occur to ensure the outcome meets the need. When appropriate, the analysis of the evaluation results may drive another iteration of the design process to improve the product. These design components make up the ADDIE model: Analysis, Design, Development, Implementation, and Evaluation (Branch & Kopcha, 2014; Clark, 2015; Kurt, 2017).

The ADDIE instructional design model was developed in the 1970s for designing military training. In its purist sense, the ID moves through the phases linearly with no modifications to the product until after the evaluation phase is completed. While the seemingly strict parameters served their purpose for the military’s needs, the ADDIE model without modification is not a perfect fit for all training development (Clark, 2015; Kurt, 2017). An alternative Instructional Systems Design (ISD) is the agile model. Agile models focus on speed, flexibility, collaboration, and efficiency. It typically implements minimally viable products that are developed in a perpetual beta, or a constant state of design and redesign. Both ISD models have a range of modified approaches. Variations on the agile model include Successive Approximation Model (SAM) and more (Instructional Design Central, n.d.). Instructional designers typically modify ADDIE by incorporating mini-iterations in various stages of the process (Clark, 2015; Kurt, 2017).

Modified ADDIE: ADDIE in the Real World

Effective educators in a classroom scan the students’ body language, ask questions to informally assess understanding, and engage students in learning experiences. Then they adjust their teaching to meet the students’ needs. After a final evaluation, they may reflect on their results and make changes for the next semester. They have executed all phases of ADDIE even without awareness of any process model. They repeat these phases in each class, at the module level, and again at the end of the course for an overall analysis driving modifications for the next term. Action research is a more formal method than simply following one’s intuition as an educator. It involves the same tasks just described with deliberate data collection and analyses. Action researchers typically conduct their study within their own classrooms and may not share results beyond the colleagues in their own institutions. Design-based research, on the other hand, elicits additional stakeholders in the instructional design process. A noted benefit of design-based research, or formative experiments, is the collaboration in authentic settings (Bradley et al., 2012; Reinking & Bradley, 2008). Of course, design cases can be conducted using an array of ISD models and their modifications. For this design case, the team chose a modified ADDIE model.

Rationale for ADDIE

In reviewing the history of instructional design, Reiser (2001) points out that “most of the models include design, development, implementation and evaluation of instructional procedures and materials intended to solve those problems” (p. 58). Branch and Kopcha (2014) also state “all instructional design processes consist of at least five major activities” (p. 80) which they list as analysis, design, development, implementation, and evaluation. As is evident in most, if not all instructional design models, the TIED team at TCOP uses the ADDIE model as a basic framework for instructional design process. In fact, the steps of ADDIE are part of the ID’s job description. The ADDIE model is often criticized as inflexible, linear, complex, and inefficient; these valid critiques revolve around the purist, original implementations of the process model (Clark, 2015; Kurt, 2017) .  Within our college, the model is used more as guidance to ensure all appropriate input is considered in designing an array of outcomes to include program curriculum, course design, assessments, and learning experiences within a course. In the case of designing learning experiences within an Internal Medicine Elective, there was no need for speed or rapid prototyping. The instructors were effective educators who intuitively reflect and seek to improve their course regularly. Adding the college’s ID and following a modified ADDIE model afforded the instructors the opportunity for more effective course changes. Collaboration in authentic settings was both a benefit and a challenge. Given that the ADDIE model was commonly used in the college already, using it as a guide in this design case provided both the instructors and ID with shared vocabulary to increase effective communication throughout the process. Below we describe the actions taken in each phase of ADDIE, recognizing that some portions of each phase may overlap other phases.

Analysis

In the analysis phase of ADDIE, the ID analyzes data to determine the problem to solve, gap to fill, or need to be met. For instructional design, this stage often involves analyzing available resources, the particular classroom setting, demographics of the students, learning objectives, and pedagogical goals (Instructional Design Central, n.d.; Kurt, 2017). In this design case, the team began with the pedagogical goal of increasing motivation within an elective course. Then they conducted an analysis of the context and needs associated with this case.

Context Analysis

Colleges of Pharmacy are accredited by the Accreditation Council for Pharmacy Education (ACPE), which provides standards and guidance documents (ACPE, 2015) to ensure pharmacy students are “practice ready” and can contribute as a valuable member of the healthcare team (Beatty et al., 2014). Successful matriculation throughout the didactic curriculum sets the stage for the foundational knowledge and skills necessary for clinical application. Courses are developed to support these standards and are mapped to competencies with topics that follow both horizontal and vertical alignment. Gaps in competencies and subject matter are assessed, then topics are placed accordingly into the didactic and experiential curriculum.

The PHA6603C Internal Medicine Elective course was created after the curriculum was built with the goal to further prepare third-year pharmacy students (PY3) for the rigors and expectations of their Advanced Pharmacy Practice Experience (APPE) clinical rotations. This face-to-face, three-hour lecture and lab course would aim to support enhanced student learning and emphasize critical thinking and clinical application. Students taking this elective course would have foundational knowledge of the topics with limited individualized application and assessment in a clinical setting. A maximum of twenty students could enroll in the three-credit elective course which ran during the PY3 Spring semester (the students’ last didactic semester) for a total of 16 weeks. The maximum number of students was set to allow for individualized assessment and opportunities for one-on-one clinical simulations. Students were randomized to select their elective courses based upon their last name on a first come first serve basis. Historically, students had selected electives based on the area of practice, degree of difficulty, and faculty participating in the course. Electives are two to three credit hours, and classes take place once per week. Student workload for their PY3 ranges between 18 to 20 credit hours per semester.

Needs Analysis

Content-Gaps

National and college decreasing trends in student performance on the North American Pharmacist Licensure Examination (NAPLEX) and the increase in student expectations for practice-readiness upon graduation created the need for an internal assessment to better prepare our students for success in these areas. The TCOP Curriculum Committee, along with the TIED team, reviewed the current curriculum including the progressive mapping of professional competencies. Discussions on self-assessment of the curriculum through enhanced simulations and course design occurred at the same time as course coordinators were petitioning to start an Internal Medicine Elective as part of the pharmacy curriculum. Creation of the course would need to focus on individualized application of knowledge and assessment of critical thinking and clinical reasoning. Establishing a means for evaluating critical thinking and providing scenarios where students felt comfortable being uncomfortable in both low and high stakes environments was essential to the success of the course.

Learning-Gaps

Core courses (not electives) occupy the majority of the students’ time, leaving coordinators of electives continually searching for ways to motivate and encourage student participation and attention during and after class. Incorporating gamification principles aimed to help elevate student engagement for efficient learning and build intrinsic motivation.

Faculty Workload

Integrating activities with individualized assessment and detailed feedback can be time intensive and overwhelming for faculty to incorporate. The necessity for immediate feedback adds to the stress and time intensive nature of such activities. Creating assessments that limit faculty workload while at the same time aligning with course and activity objectives with timely feedback is essential.

Design

The design phase of ADDIE involves brainstorming and making decisions regarding the delivery method, learning objectives, lesson planning, and determining the resources (Instructional Design Central LLC, n.d.; Kurt, 2017). The collaborators in this case determined the goal of gamifying the elective course, selected appropriate game elements, created the rulebook, selected the assignments to be gamified, and created the rubric to assess the assignments.

Goal Setting

Course coordinators enlisted the help of the TIED team once the course was pre-approved by the College’s Curriculum Committee. The coordinators and TIED team met for a total of 10 hours to design and develop the overall course platform. During their first meeting, coordinators described their vision and goals of the course to the ID with further discussion on course design and implementation. During this design phase, the vision and recommendations of the ID were shared with the pharmacy faculty to align with the gamification workshop the ID was attending. The coordinators quickly decided to take advantage of this opportunity as this method directly aligned with the needs addressed above (Budhrani, 2018).

Game Elements

After attending Budhrani’s workshop at 41st Annual Meeting of the Association for Educational (2018), TCOP’s ID met with the course coordinators to review guidance materials provided including a document on game elements. Course coordinators reviewed the proposed elements and the elements of the television (TV) show and discussed ideas for purposeful integration. Elements were chosen based upon the course needs and applicability in this setting. A consensus was made to mimic the course after the competitive cooking reality TV Show, Master Chef. Competitors in the show have various key elements that encourage intrinsic motivation and challenge the participants to be creative and innovative as they prepare their dish. Surprise ingredients introduced throughout the competition also allow for “on the spot” thinking and an opportunity for enhanced critical thinking.

Through applying a gameshow format, the Internal Medicine Elective course hoped to encourage friendly competition with activities related to several key internal medicine topics. When reviewing similarities between the show and the course, the course coordinators were able to link “cooking” and “healthcare” in that chefs (pharmacists) must use the correct ingredients (medications) to create the best meal (patient care) possible. Chef terminology (i.e., layered learning or scaffolded learning) offered a direct correlation with student expectations and accreditation standards. Introducing these concepts to the students provided them with clear expectations and encouraged life-long learning as required by “Standard 9” (ACPE, 2015). Course coordinators created a rulebook as part of their syllabus to set the stage for expectations and to tie in the Master Chef theme.

Assignment Selection

There were three main assignments in the course where the coordinators worked with the ID to implement gamification principles and ensure alignment with the Master Chef theme. Assignments were chosen based on complexity and workload encompassing 80% of the total course grade. Assignments included a topic presentation (individual assessment), journal club debate (student pairs), and case simulation (6 total individual assessments). Activities were created to evaluate higher levels of learning that correlated with the expectations of students as they prepare for clinical practice. Assignments were scaffolded and encouraged friendly competition and autonomy of the learner.

Assessment Instrument

Course coordinators also incorporated entrustable professional activities (EPAs) into course assessments to ensure professional education expectations to transition from learner to clinician were not overshadowed by game theme elements (see Table 1). The purposeful integration of both EPAs and various chef roles into course rubrics supported course gamification without minimizing the significance of patient care. An example of an assignment rubric is provided in Appendix A. “Secret ingredients” in the form of new lab values or diagnoses were incorporated into the case simulations to provide a quick assessment and utilize clinical reasoning as they adjusted their recommendations. Students were encouraged to also utilize gaming principles as they created their own active learning activities during their topic presentations. This allowed for autonomy and creativity for enhanced motivation and retention of information.

The team reviewed each of the assignments to make sure they aligned with the objectives, assessment followed the gamification principles, and also encouraged motivation, participation, and higher level of thinking. All activities allowed students to assess their progress and enhance reflection and goal setting. Assignments were created to limit faculty workload and provide an efficient way for assessment and feedback.

Table 1

Entrustable Professional Activity (EPA) Milestone Level Descriptions

Development

The development phase includes production and testing of the content planned in the previous stages. In this phase the ID puts the plan into action through three steps: drafting, production, and evaluation (Instructional Design Central LLC, n.d.; Kurt, 2017).

Coordinators met on a weekly basis to draft and produce course content including lecture slides and outlines, clinical cases, rubrics, knowledge checks, and active learning scenarios. Since the course was an elective, and most--if not all--material had been previously taught, emphasis was made on key topics of patient safety and complexity when creating assignments and cases. Assignments mimicked those expected during student experiences on clinical rotations to scaffold expectations and provide additional opportunities for didactic application before experiential learning. A prototype was created in the learning management system Canvas, and coordinators used this prototype to develop and update their course prior to implementation. Coordinators worked closely with the ID on a biweekly basis to apply best practices and purposeful integration of techniques into the course in preparation for the inaugural offering. Coordinators enlisted the help of five current fourth year pharmacy students who were already on rotations to ensure learners could identify the purpose and application of gamification. Rotation students also helped to provide areas of confusion and topics of interest based on current experiences. Coordinators used Canvas to post all assignment details, rubrics, and the leaderboard prior to the start of class. Additional gamification was provided real-time through cases, verbal defenses, and an escape room activity. 

Implementation

In an iterative design process, the implementation phase is where course materials are shared with students through a learning management system and delivery of instruction takes place. Course design elements are implemented along with assessment of student learning (Instructional Design Central LLC, n.d.; Kurt, 2017). The ID is actively working to ensure the course is running efficiently and gathers real time feedback for immediate redesign when necessary. 

The first iteration of this gamified Internal Medicine Elective course received overwhelming interest from the PY3 students. The class enrollment was maxed out quickly (N = 20), and students continued to ask if the coordinators were able to add students to the course. Students enrolled in the course varied as far as scholastic ranking with grade point averages ranging from the lowest 10% up to the top performers. Students were excited about the ability to prepare for their clinical rotations and to enhance their clinical skills.

The course itself ran smoothly, and the coordinators provided clear expectations for students regarding deadlines, assignments, and grading rubrics that aligned with gamification principles. On the first day of class, the coordinators explained the different performance levels used for the “Topic Presentation and Patient Case Simulation” activities regarding performance and expectations and how they aligned with the theme of the course:

• “Unsatisfactory: Line Cook” (i.e., first-year level);
• “Below Expectations: Fish/Vegetable/Meat Cook” (i.e., second-year level);
• “Meets Expectations: Station Chef” (i.e., third-year level/current level);
• and “Exceeds Expectations: Sous Chef” (i.e., fourth-year level).

Coordinators also emphasized that although these rubrics were built on a scale of 60 total possible points, students would be graded out of 45 or 50 points in order to align with their level of training in the program (third-year). The theme of the course was carried forward with food-themed badges (cupcake and corn necklaces), which were awarded to the winners of activities and assignments such as our journal club debate. Students were incredibly enthusiastic about the honor of wearing the necklaces as a point of pride in their work until the next competition took place. Finally, a Master Chef escape room was used to provide insight on how to interpret and apply patient information for appropriate therapy assessment and management.

Evaluation

Evaluation within the ADDIE process includes both formative (i.e., throughout the whole process) and summative evaluation after the implementation phase (Instructional Design Central LLC, n.d.; Kurt, 2017). For this particular design case, the focus during the evaluation phase was not on formal evaluation of the intervention itself. Instead, the ID worked with the coordinators to determine if the course and programmatic objectives were met and to further review if identified gaps were filled. Gaming elements that were added to the course hoped to increase motivation of the students with a main emphasis on retention of information given the complexity and importance of effective application.

In the first iteration of the course, coordinators relied on course evaluations to evaluate interventions. In such evaluations, student comments were also utilized to provide further details into overall critique of gamification elements of the course. Informal verbal student feedback was overwhelmingly positive regarding the gamification of the course and the content of information provided to their pharmaceutical education. For the formal course evaluations through the college, all 20 students reported a score of 5 out of 5 regarding the statements, “I understand how this course will benefit me as a pharmacist” and “The course helped increase my knowledge and competence.” High average scores were also received regarding clearly stated learning outcomes (4.9); content aligned with stated objectives (4.9); expectations were clear (4.8); and the required assignment and activities enhanced my learning experience (4.8). Free responses indicated that students felt this course would benefit all students and that it promoted their learning in a way that could translate into clinical practice during their fourth-year pharmacy rotations. Some representative comments from course evaluations included, “The structure of this course is great!,” “Every single assignment that is done in this course helps you refresh what you have learned so far in other pharmacotherapeutics courses,” and “All of the activities felt like they enhanced my learning, and I thought the overall course format was very conducive to reinforcing knowledge we have already come across.” Course coordinators and the ID met to review student evaluations of the course and to brainstorm ideas for future offerings. There were some comments about harsh grading, but this was not in relation to the gamification of the course and instead the expectations for patient care as compared with previous courses. Unfortunately, due to the fact that this was the inaugural class for the course, the coordinators did not have prior course grades to compare.

Performance and Future Plans

While there were no quantifiable comparison data to drive decision making, our team decided to include self-reflections of the faculty, the ID, and the students as performance indicators of the course. Discussion with the ID lead to ideas for future evaluation through a pre- and post-perception analysis of the EPAs (see Table 2), analysis of clinical rotation graded performance overall, and review of specific grading for the Internal Medicine APPE clinical rotation. This would allow both the coordinators and the ID to quantify immediate and longitudinal improvements in student knowledge and retention. Additionally, course coordinators will continue to work with the ID to add in opportunities for students to level up through advancing through the food pyramid and gaining access to additional assessments for added points towards their final grade.

Conclusion

The positive feedback from the learners who participated in the gamified Internal Medicine Elective course support the use of gamification for professional health education. It can be inferred that the use of gamification for educational purposes does not seem to reduce credibility of the information presented, but instead supports learner engagement, retention, and application of skills. Incorporating gamification into the classroom was time intensive and increased faculty workload at first. This was in opposition to what the overall goal of this structure was originally. Most of the time was spent creating the activities and rubrics to assess student performance while aligning with the overall theme of the game show. Several hours were spent discussing course design with the ID and meeting prior to each activity to ensure alignment with the objectives and appropriate execution. The coordinators are hopeful that this will decrease with each offering. In the future, the coordinators plan to enhance clinical case simulations to align with the course theme and encourage individualized assessment. With the feedback provided from students, aspects of the course design have been added to core skills-based courses to continue to build upon intrinsic motivation and provide avenues for individualized clinical reasoning and application for all PY3 students.

Using student feedback, coordinators have incorporated additional concepts including a written consult with game design elements into the elective to allow for individualized assessment of both verbal and written communication methods. With the second offering, students were provided examples of ways to incorporate gamification into their topic presentations and this has been added to the assignment description. As the TCOP develops an updated modified block curriculum, future and continuous involvement of the TIED team will be essential to follow pedagogical advances while at the same time providing efficient and effective assessment methods for faculty with respect to workload.

Implications

Student engagement and self-directed learning is vital for higher education, especially in medical education. Faculty continue to struggle with a balance of creating options for individualized student assessment and engagement and the workload associated with such endeavors. Gamification has proven to be an effective means to challenge students with autonomy and opportunities for mastery while at the same time limiting faculty workload and time spent on such assessments (Barone et al., 2018). While incorporating such elements into a course can be time consuming and overwhelming, teaming up with an ID can alleviate the stress and allow for well-implemented gamification.

References

Aburahma, M. H., & Mohamed, H. M. (2015). Educational games as a teaching tool in pharmacy curriculum. American Journal of Pharmaceutical Education, 79(4) 59; http://dx.doi.org/10.5688/ajpe79459

Accreditation Council for Pharmacy Education (ACPE). (2015). Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree. Accreditation Council for Pharmacy Education. https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf.

Adams, B. J., Margaron, F., & Kaplan, B. J. (2012). Comparing video games and laparoscopic simulators in the development of laparoscopic skills in surgical residents. Journal of Surgical Education, 69(6), 714-717. http://doi.org.ezproxy.hsc.usf.edu/10.1016/j.jsurg.2012.06.006

Alexander, D., Thrasher, M., Hughley, B., Woodworth, B. A., Carroll, W., Willig, J. H., & Cho, D. Y. (2018). Gamification as a tool for resident education in otolaryngology: A pilot study. Laryngoscope, 129(2), 358-361. https://doi.org.ezproxy.hsc.usf.edu/10.1002/lary.27286

Barone, J. A., Aleksunes L, & Hermes-DeSantis E. R. (2018, July). An interactive and integrated pharmacotherapy and skills-based curriculum: Bridging basic and clinical science. [School Poster]. 119^th Annual Meeting of the American Association of Colleges of Pharmacy, Boston, Massachusetts, United States.  

Beatty, S. J., Kelley, K. A., Ha, J., & Matsunami, M. (2014). Measuring pre-advanced practice experience outcomes as part of a PharmD capstone experience. http://dx.doi.org/10.5688/ajpe788152

Bradley, B. A., Reinking, D., Colwell, J., Hall, L. A., Fisher, D., Frey, N., & Baumann, J. F. (2012). Clarifying formative experiments in literacy research. In P. J. Dunston, S. K. Fullerton, C. C. Bates, K. Headley, & P. M. Stecker (Eds.), Sixty‐first yearbook of the Literacy Research Association (pp. 410– 420). National Reading Conference.

Branch, R. M., & Kopcha, T. J. (2014). Instructional design models. In J. Spector, M. Merrill, J. Elen, & M. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 77-87). Springer.

Brull, S., Finlayson, S., Kostelec, T., MacDonald, R., & Krenzischeck, D. (2017). Using gamification to improve Productivity and increase knowledge retention during orientation. Journal of Nursing Administration, 47(9), 448-453. https:/doi.org.ezproxy.hsc.usf.edu/10.1097/nna.0000000000000512

Budhrani, K. (2018, October). Redesign courses into a competition-based game-show format. [Conference workshop].41st Annual Meeting of the Association for Educational Communications & Technology, Kansas City, MO, United States.

Cain, J., Conway, J. M., DiVall, M. V., Erstad, B. L., Lockman, P. R., Ressler, J. C., Schwartz, A. H., Stolte, S., & Nemire, R. E. (2014). Report of the 2013-2014 Academic Affairs Committee. American Journal of Pharmaceutical Education, 78(10), S23. https://doi.org/10.5688/ajpe7810S23

Clark, D.R. (2015). ADDIE Model. The Performance Juxtaposition Site. https://edtechbooks.org/-XAT

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: Defining “gamification.”. MindTrek, 9-15.      https://edtechbooks.org/-Rmdf

Instructional Design Central, LLC (n.d.). Instructional Design Models. Instructional Design Central. https://www.instructionaldesigncentral.com/instructionaldesignmodels.

Kurt, S. (2017). ADDIE Model: Instructional Design. Educational Technology. https://edtechbooks.org/-vDu

Lin, D. T., Park, J., Liebert, C. A., & Lau, J. N. (2015). Validity evidence for surgical improvement of clinical knowledge ops: A novel gaming platform to assess surgical decision making. The American Journal of Surgery, 209(1), 79-85. https//doi.org.ezproxy.hsc.usf.edu/10.1016/j.amjsurg.2014.08.033

Reinking, D., & Bradley, B. A. (2008). On formative and design experiments: Approaches to language and literacy research. Teachers College Press.

Reiser, R.A. (2001). A history of instructional design and technology: Part II: A history of instructional design. Educational Technology Research and Development. 49, 57-67. https://edtechbooks.org/-nghy

Salazar, M., Pattipati, S. N., Havard, P., Ofstad, W., Duncan, W., Hussain, D., Fuentes, D. G., & Hughes, J. (2018, July). Collaboration across departments and university stakeholders reduces impact of the decreasing PharmD applicant pool. [School Poster]. 119^th Annual Meeting of the American Association of Colleges of Pharmacy, Boston, Massachusetts, United States. 

Sardi, L., Idri, A., & Fernandez-Aleman, J. L. (2017). A systematic review of gamification in e-health. Journal of Biomedical Informatics, 31. https://doi.org/10.1016/j.jbi.2017.05.011

Sera, L. & Wheeler, E. (2017). Game on: The gamification of the pharmacy classroom. Currents in Pharmacy Teaching and Learning, 9(1), 155-159. https://doi.org.ezproxy.hsc.usf.edu/10.1016/j.cptl.2016.08.046

White, M., & Shellenbarger, T. (2018). Gamification of nursing education with digital badges. Nurse Educator, 43(2), 78-82. https://doi.org.ezproxy.hsc.usf.edu/10.1097/nne.0000000000000434

Wolf, C., Bott, S., Hernandez, I., & Grieve, L. (2018, July). Teaching about the healthcare industry through gamification. [School Poster]. 119^th Annual Meeting of the American Association of Colleges of Pharmacy, Boston, Massachusetts, United States.