Some typical evaluation approaches and designs used by instructional designers in the analysis phase are summarized here.

Gap Analysis (also called a Needs Analysis)

A search of this topic will likely produce different descriptions of what a needs analysis (or needs assessment) might entail. Some sources refer to a needs assessment as an evaluation whose purpose is to determine the degree to which an objective or goal was met. We refer to these types of evaluations as objectives-oriented or effectiveness evaluations.

What many call a needs analysis might more appropriately be referred to as a gap analysis. The terms need and gap are often used synonymously, but technically, an identified gap necessitates a solution; the gap defines the need. Said another way, we need a solution to fill a gap. These two activities (i.e., the gap and needs analysis) are usually performed together as one process. A gap analysis is used to identify and describe the difference (i.e., gap) between what is and what we want it to be. The needs analysis uses the information from the gap analysis to identify the cause of the problem and evaluate potential solutions. Thus, a gap analysis identifies problems (i.e., gaps), and a needs analysis recommends a potential (needed) solution.

You may detect various kinds of gaps. The gap might be a performance gap, a knowledge gap, a competency or ability gap. Many of these gaps lead to a need for training or practice, and other gaps are reduced by providing an educational product or tool. Sometimes the evaluator recognizes the gap cannot be filled. Before you can confidently recommend a solution, you will need to identify the cause of the gap. You will also need to consider how important it is to fill the gap and the likelihood that any solutions you propose will have the desired effect.

A gap can represent a perceived, present, unrecognized, or potential need. The gap may refer to the difference between intended goals (or objectives) and the actual (or desired) outcome (i.e., perceived need). It may represent the difference between a desired state and the current situation (i.e., a present need). Or, it may refer to something users don't have but could use (need) to accomplish some task (i.e., something a student or provider needs to accomplish a task). In addition, a gap may represent something an individual may need in the future (i.e., a potential need) even though the individual may not appreciate or be aware of the need (i.e., unrecognized need).

Those conducting a needs analysis are often susceptible to an availability heuristic cognitive bias, where decisions are based on the first viable option or solution that comes to mind. For example, it is not uncommon for people to quickly conclude that training must be developed or technology needs to be purchased before identifying the actual cause of the problem and whether that solution would be appropriate.

The basic steps required to accomplish a generic needs analysis are:

Step 1:	Describe the context and intended users/learners.
Step 2:	Identify the problem (need or gap).
Step 3:	Describe the various users experiencing the problem.
Step 4:	Identify the cause or reason for the problem.
Step 5:	Assess how important it is that the problem be solved.
Step 6:	Recommend solutions.

There are several ways a Needs Analysis might be applied

Performance gap analysis

In their 1997 book, Mager and Pipe present a model for analyzing performance problems. Various depictions of the models exist (flowchart 1, flowchart 2, flowchart 3).

In Mager and Pipe’s model, once the gap (problem) has been identified, the evaluator must identify the cause of the problem, then select an appropriate solution for the performance problem. This is done by asking a series of questions. Each question is intended to provide information so the evaluator can determine the actual cause of the problem and propose a targeted solution.

It is not uncommon for instructional designers to produce training whenever it's asked for, even when training isn't going to solve the performance problem or produce the desired results. Training is only the best intervention when the performance issue is the result of a lack of knowledge and skill. Some issues that would not require training as a solution include:

	Being out of practice	Skill is rarely used
	Performance is punishing	Its boring, difficult, unpleasant
	Performance is unrewarded	Poor compensation, lack of acknowledgment
	Nonperformance is rewarding	Conflicting activities are more enticing, lack motivation
	Obstacles exist prohibiting the performance	Policies, conditions, facilities, lack of resources, tools
	Need to perform well isn’t important	Intended outcomes are trivial, little impact

Training needs analysis

Training is needed when the performance deficit is caused by a lack of knowledge or skill. In these situations, a training needs assessment should be conducted. A training needs analysis identifies the knowledge, skill, and abilities that can be facilitated through instruction. When training is required, gaps may represent knowledge and skills an individual will need in the future or abilities a person needs now to accomplish a particular job or perform a specific task. While a training need may be identified in the analysis phase as part of a gap analysis, some form of task analysis will also be needed, which is often carried out in the design phase.

However, when training is needed, it is not uncommon for instructional designers to spend an exorbitant amount of time and resources developing training for basic knowledge and skills while devoting insufficient time creating instructions for important topics which may be difficult to teach and challenging to develop. A training needs and task analysis will not solve this problem. The designer must use the information provided by a training needs analysis to make design decisions that will facilitate learning in an appropriate, effective, and efficient manner.   

The goal of a training needs assessment is to create learning objectives for the training so assessments, instructional materials, and learning activities can be designed and developed. The basic decision-oriented question that needs to be answered is—what do we need to teach? Some ways we can answer this question include:

• Consulting with experts
• Referring to establish curriculum content standards
• Identifying prerequisites and prior knowledge needs
• Listing special qualifications and personal attributes needed to accomplish a particular job or task
• Completing a task analysis

Consumer Reviews

A consumer review is often utilized once a gap is revealed and one or more products (i.e., solutions) exist that could potentially fulfill the identified need. In an educational setting, solutions may include a training program, policy, service, or product. The evaluator's role in a consumer review is to provide an expert opinion.

Results from a gap analysis and consumer reviews inform an evaluator's recommendation to create a new product or utilize an existing solution. The purpose of a consumer review is to provide information to the decision-makers about the various solutions so they can decide which, if any, should be employed. If none of the existing solutions are viable, an evaluator might recommend developing a new product or revising an existing product.

The evaluator's judgment of the product might be determined through measurement or personal experience. For example, the value of a product may be measured and compared against specific performance benchmarks. Or, the evaluation may be based on the evaluator's professional opinion, other users' experience and preference, or both. The evaluator must capture and report findings in a way that helps individuals make an informed decision regarding a solution's merit, worth, and value.

Once a significant problem or need has been identified, the basic steps required to accomplish a consumer review are:

Step 1:	Identify existing products (potential solutions).
Step 2:	Establish criteria for judging each solution.
Step 3:	Test each product using the standards (criteria) you selected.
Step 4:	Present results using a comparative analysis of the products.
Step 5:	Make a recommendation.

References

Mager, R. F., & Pipe, P. (1997). Analyzing Performance Problems: Or, You Really Oughta Wanna (3 ed.): Center for Effective Performance.