• Visuals in Learning Design
  • Cover
  • I. Introduction
  • II. Principles
  • III. Practices
  • IV. Projects
  • V. Appendices
  • Download
  • Translations
  • UX Evaluation

    • UX Evaluation

    A UX evaluation is intended to help the designer determine if a design effectively achieves its goals and what changes should be made for improvement. UX evaluations utilize a variety of constructs and methods specific to a particular project. To conduct a UX evaluation, you must (1) identify your target product, (2) determine your question(s), (3) plan your method(s), (4) collect and analyze data, and (5) provide suggestions for moving forward.

    Before proceeding with this project, please read the following chapter by Earnshaw et al. (2018) for further guidance on how to conduct a UX evaluation:

    Earnshaw, Y., Tawfik, A. A., & Schmidt, M. (2018). User Experience Design. In  R. E. West (Ed.), Foundations of Learning and Instructional Design Technology. EdTech Books. https://edtechbooks.org/-ENoi 

    Evaluation Report

    For most products, evaluation reports can be very short (1-2 pages) and should include the following sections: Product Description, Questions, Methods, Results, and Suggestions.

    1. Product Description - What is the product, and where can it be found? What are its purposes, goals, technical specifications, etc.? What are similar products or competitors?
    2. Questions - What is it you are trying to understand about the product (e.g., aesthetics, utility, usability, value)? Some questions might include the following:
      • Does the user find the product to be intuitive or easy to learn?
      • Does the user have positive emotional or aesthetic reactions to the product?
      • Does the product efficiently allow the user to achieve her goals?
      • Are aspects of the product confusing or distracting?
    1. Methods - How will you answer your question(s)? These can be qualitative or quantitative in nature and might include interviews, surveys, focus groups, observational notes, tracked data, etc. A good UX evaluation utilizes multiple data collection and analysis methods.
    2. (Collect data for 5-10 minutes during class.)
    3. Results - What have you found to answer your questions? What new questions arise, or what unexpected results did you encounter?
    4. Suggestions - So what? How can the product be improved?

    Conducting an Evaluation in Formal Coursework

    If you are conducting an evaluation in a class for a student project, each student should have the opportunity to conduct their own evaluation during class, using fellow students and the instructor as participants or research aides. Each in-class evaluation should have a limited timeframe. As such, you should arrive in class with sections 1, 2, and 3 of your evaluation report already completed so that you can conduct your evaluation efficiently and effectively. You may also supplement your evaluation by conducting elements of it with additional participants outside of class.

    Example Evaluation Scenarios

    You may use these evaluation scenarios to guide you in constructing your own.

    1. You want to understand how a website’s design appeals to users’ aesthetic sensibilities and intuitive expectations. You conduct a focus group with participants and ask them questions that try to understand how users feel as they navigate the website, whether they are experiencing frustrations with certain elements, etc.
    2. You want to understand which buttons in an app are clicked first and which are ignored or not seen. You organize participants into groups of two and ask one participant to be the user and one the observer. The observer records how many times each button is clicked in a 60-second session, and these data are then compiled by the evaluator.
    3. You want to understand what parts of an infographic draw a user’s attention. You organize participants into groups of two and ask one participant to be the user and one the recorder. As the user is presented with the infographic, the interviewer asks questions like “what are your eyes drawn to first,” “what is confusing,” etc. Answers are recorded by the observer and compiled by the evaluator.

    Example Guiding Questions

    You may use these guiding questions to help you decide on your own for focus groups, interviews, surveys, etc.

    1. How would you describe the product (in a few words)?
    2. Overall, how easy is the product to use?
    3. How does the product compare to competitors?
    4. Why would you use the product (or how might you begin using it)?
    5. What features are essential (could you not live without)?
    6. What features are not essential (could you live without)?
    7. What do you like best about the product?
    8. If you could change one thing, what would it be and why?
    9. What is most frustrating about the product?

    Evaluation Criteria

    These criteria may be used in a formal course for an instructor to evaluate student evaluation reports.

      Unsatisfactory Basic Proficient Distinguished
    Clear Writing is unclear or difficult to follow throughout. Writing is generally clear, but in some places may be difficult to follow. Writing is clear but is occasionally difficult to follow. Writing is clear and easy to follow.
    Conventional Writing does not follow APA 6. Writing generally follows APA 6 with some errors. Writing follows APA 6 but with a few minor errors. Writing follows APA 6 with no errors.
    Concise / Complete Writing exceeds the maximum or is below the minimum word length requirement. Writing meets word length requirements but may exclude key conceptual elements. Writing meets word length requirements and generally includes necessary conceptual elements. Writing meets word length requirements and does not leave out any necessary conceptual elements.

    This content is provided to you freely by EdTech Books.

    Access it online or download it at https://edtechbooks.org/design/ux_evaluation.