• Evaluation and Design
  • Evaluation Basics
  • Evaluation Standards and Meta-evaluation
  • Evaluation within a Design Context
  • Evaluation for Instructional Designers
  • Evaluation Planning
  • Research Principles Primer
  • Evaluation Approaches for Designers
  • Evaluation in the Analysis phase
  • Evaluation in the Design Phase
  • Evaluation in the Development Phase
  • Evaluation in the Implementation Phase
  • Personnel Evaluation
  • Data Collection Primer
  • Reporting Evaluation Findings
  • A Research Principles Primer
  • User Testing
  • Download
  • Translations
  • Reporting Evaluation Findings

    Wise men speak because they have something to say;

    Fools because they have to say something. (Plato)

    The following chapter presents ideas related to presenting evaluation findings. There is no one correct way to write a report, but depending on the purpose and intended audience, specific guidelines will help you present your evaluation finding accurately and effectively.

    It is not enough for evaluators to conduct a rigorous and insightful investigation; they must also present their findings, interpretations, and recommendations in a logical and compelling manner. Unfortunately, reporting is often a step that evaluators fail to address adequately. Evaluators need to consider how the result will be presented for maximum impact and usefulness. They need to consider the best ways to present information to stakeholders so they will read the report and consider the recommendations being presented.

    Basic structure of an evaluation report

    All reports will follow a basic outline. This includes an:

    If you have written a formal proposal, several of these sections will have already been written and might only need to be updated for accuracy. For example, you may have proposed specific methods, but things may not have worked out as intended, or you now have specific details of how the data was captured (e.g., response rates and data sources). You can repurpose the revised sections of the proposal for the final report. The rest of the report (i.e., findings, discussion, conclusions, and recommendations) can only be written once you have collected and analyzed data, then interpreted the findings. In some cases, certain sections can be omitted or collapsed into one section or may need to be separated into two specific sections. You need to have a good reason for altering the basic structure of a report but remember that you will not necessarily write the report sections in the order you eventually present them. For example, if an abstract or executive summary is required, these are often written last.

    Interim Reporting

    The evaluation report is often seen as a final step in the process. However, it is good practice to begin reporting results long before the evaluation is concluded when working for a client. Discussing results with stakeholders as they are obtained can be a learning opportunity. The evaluator can get the client’s reaction, and stakeholders can provide insights and explanations that help the evaluator understand the data they have collected. This includes getting the client’s interpretation of the results. Rather than acting as the expert, the evaluator and client might work collaboratively to develop conclusions and recommendations.

    Report Formats

    There are many ways a report can be formatted. The best way to format a report will depend on the purpose and the intended audience. A report may take the form of an executive summary, an in-person presentation, or a formal written report. Depending on the circumstances, you may need all three.

    For academic reporting of an evaluation project or publication in academic journals, you may need to follow specific formatting guidelines (i.e., APA). There are good reasons for following these guidelines; most notably, researchers are often required to search through hundreds of studies and evaluation reports each year. Having these reports structured in a particular way makes their reading more efficient. Using appropriate heading and formatting helps the reader productively navigate the information. The tone need not be extremely formal; however, the writing should not be too casual (i.e., idiomatic or colloquial). Traditionally reports have been written in third person, past tense; however, many journals now allow first-person reporting in some situations.

    Remember, good writing requires rewriting. It would be best to plan time to write your report so you have time to rework sections, complete copy editing tasks, and reconsider how best to present information.

    Logical and Emotional Arguments

    The data you collect provides answers to the evaluation questions you posed and supports the recommendations you make. However, reporting is a form of storytelling. An appealing story can present a logical or an emotional argument. Most academic evaluation reports are expected to present a logical argument, including a rational appraisal of the evidence. However, the most compelling reports also include an emotional appeal. The problem with presenting information logically is that it doesn’t always persuade our brains to pay attention. Even though it’s logical, it doesn’t get remembered, retained, or recalled because it doesn’t tap into an emotion. Still, if you base your recommendations solely on emotional arguments, you may have difficulty getting readers to take your evaluation results and recommendations seriously.

    An effective evaluation report presents information that is relevant to the specific problem and evaluation questions proposed; then, because evaluations rely on value-based criteria, the answers we arrive at, and the recommendation we make need to be persuasively argued from multiple perspectives. Your evidence must be presented in a way that speaks to both our logic and our emotion.

    Results Section

    It is important to remember that you have nothing to say until you have collected and analyzed the data. Because evaluations often acquire a vast amount of quantitative data, you should organize these data into charts and tables before writing the results section (see examples). This way, you can decide how to present the findings, which should be included in the results section, and which might best be located in an appendix or a supplemental resource. Qualitative data is presented differently than quantitative data but can be coded and quantified to represent the strength of trends and patterns.

    The quality of what you report will depend on the quality of your data and your ability to analyze these data. The purpose of the results section is to organize the evidence you obtained in a way that the reader can understand. You need to focus attention on significant aspects of the data. This will include interesting information as well as those results that do not support your hypothesis. Sometimes an evaluator may present the findings separately (i.e., the fact) and then have a separate discussion section. Other times, the results and discussion section can be combined.

    Negative results

    At times an evaluation will uncover information that may be difficult for the client to accept. No one likes to be the bearer of bad news. There are, however, ways to lessen the potential backlash. An evaluator might:

    Discussion Section

    In addition to focusing attention on significant aspects of the data, you should provide readers with an understanding of the issues and introduce new ways of seeing the information in the discussion section. You will need to promote an accurate, balanced, and fair portrayal of the results. This section should persuasively present the results so the logical argument you wish to make is clear. However, check your biases; look for alternative explanations and unintended consequences that occurred or may occur. All your interpretations need to be reasonable, and they need to be supported by the data; still, you need to present the result in context. Don’t push a perspective that matches what you hoped to find if the data does not support that interpretation. Include and discuss all results needed to answer the evaluation questions. If you have multiple evaluation questions or purposes, you may want to present the result organized by the evaluation questions.

    The American Evaluation Association’s Guiding Principles (GP) provide some guidance for writing reports and expectations of evaluators when they communicate findings. These state that evaluators should

    Clearly, there is no one way to represent the good of society or the best interests of the public. However, sometimes we inadvertently allow our biases to cloud our judgment. Care should be taken not to advocate for a specific recommendation that is not fully supported by the data and does not consider the views and values of various stakeholders.

    Conclusions Section

    Discuss your conclusions in order of importance or by the evaluation questions. You need not summarize all the methods and findings again, but you should remind the reader of the evaluation’s purpose and answer the evaluation questions. You may compare your results with those from other studies and provide possible explanations or reasons for any discrepancies should the results differ. Make sure to mention any inconclusive or inconsistent results and explain them as best you can (e.g., based on a negative case evaluation). You may suggest additional research that may be needed to clarify your results.

    Briefly describe the limitations of your evaluation to show reviewers and readers that you have considered your study’s weaknesses. At the end of the conclusion section, state or summarize your main conclusions once again before listing recommendations. Your conclusions and recommendations require a good understanding of the data. They also go beyond simply presenting the results as your judgments are based on the values of the client and the evaluator.

    Recommendations

    The recommendations are perhaps the most critical part of the report and are most often an aspect of the report that will be criticized. Sometimes the recommendation can be avoided if the client wishes to decide for themselves how to proceed. However, most often, the client wants the evaluator’s opinion.

    Any recommendations you make must be ethical, practical, and realistic. Obviously, it would be best not to suggest the client do anything immoral or unethical. However, the recommendations must also be practical and realistic. Often recommendations are rejected, not because they would not solve a problem or meet a need, but because they cannot be implemented. The recommendation may be too costly, extremely challenging to implement, or may have undesirable consequences. For example, implementing a specific solution may address one specific need but cause additional problems. When possible, recommendations should be made in consultation with the client and other stakeholders.

    Chapter Summary

    • A common mistake evaluators make is failing to plan time for writing the final report.
    • The main purpose of any report is to answer the evaluation questions and make recommendations.
    • The format of your report will depend on the purpose of the report and the intended audience. 
    • Before you write your final report, you need to collect and analyze your data so you have something to say.
    • The quality of your report will depend on the quality of the data your collect and your ability to analyze these data.
    • It is best practice to discuss interim results with your client. When possible, include the client in interpreting the results and when developing recommendations. This can facilitate the evaluator's understanding and makes it easier to deliver negative results. 
    • The purpose of the results section is to organize the evidence you obtained in a way that the reader can understand. 
    • The discussion section should provide a contextual interpretation of the findings. 
    • The conclusion section summarizes the main findings and makes recommendations.
    • The recommendations are often the most likely part of a report stakeholders will criticize.  
    • Recommendations need to be ethical, practical, and realistic. The recommendations must also take into account the possible ramifications of implementing the evaluator's suggestions. 

    Discussion Questions

    1. Consider some ways a report might be biased. Describe what you might do to alleviate these problems?
    2. Suppose that, based on the evidence you collect, you feel it is necessary to make extensive revisions to an instructional product you evaluated. Describe issues stakeholders might have implementing the suggestions.  

    This content is provided to you freely by EdTech Books.

    Access it online or download it at https://edtechbooks.org/eval_and_design/evaluation_reports.