Sampling Techniques and Procedures

As previously mentioned, there are many reasons why you would use a sample rather than a census when conducting research. And as mentioned, there are many things that could go wrong. One of the things that could go wrong is the selection of a sample. The primary goal of sampling is to create a representative sample, one in which the smaller group (sample) accurately represents the characteristics of the larger group (population). If the sample is well selected, the sample will be generalizable to the population.

There are many ways to obtain a sample. The techniques used will vary based on the circumstances under which the study is conducted as well as the aims of the research. The way in which samples are drawn will affect the quality of a study.

Prior to choosing a selection method, you should have defined the population and the purpose for the study. Clearly defining the target population is important, meaning you will define both the size of the population and the accessibility of the population. As noted previously, anticipated survey response refusal will affect the size of the sample needed. Accessibility of individuals within the population will also affect the sample selection procedures. There are two general approaches to sampling: random and non-random. However, additional consideration should be made based on whether the study will be a qualitative study.

Definitions

Quantitative sampling. Surveys are typically designed to produce descriptive numerical statistics (e.g., scores, ages, strength of opinions, frequencies) that can be used to describe various characteristics found within the population. Any qualitative data obtained is typical categorized and quantified. Sampling for these studies must produce representative samples because generalizability is important. A distinctive aim of these studies is to gain a general understanding of the characteristics found in the population.

Qualitative sampling. Qualitative studies are not interested as much with generalizability as they are with understanding a phenomenon. The sample must produce good informants for the study. The characteristics of the respondents are more important than the size of the sample. These samples will be smaller and less representative but should provide researchers access to a good representation of key informants. Often the aim of qualitative research is to get a deeper, fuller understanding of the topic or phenomenon.

Random sampling. A selection technique where every unit in the population has an equal chance of being selected. The unit of analysis often involves individuals but may be intact groups.

Non-random sampling. While random sampling may be preferred, there are many ways in which a planned random sample may become less random. In non-random sampling (or non-probability sampling), researchers are unable to select participants at random from the population. This includes situations where circumstances (e.g., survey refusal leading to low response rates or missing contact information) diminish the likelihood that the sample provides a good representation of the population. Follow-up contact or a post-survey examination of demographic characteristics are often needed to verify the degree to which survey results might be considered generalizable.

Random selection and random assignment. These two terms should not be confused. Random selection is used to establish a sample. If done properly, the results of the study are believed to be generalizable. Random assignment is use in experimental studies. Randomly assigning individuals to two different groups is done in an attempt to make the two groups comparable. Random selection affects claims of generalizability. Random assignment is the basis for experimental claims of causality.

Random Sampling Techniques

Simple Random

For this type of sampling, each individual (or unit) in the population has an equal and independent chance of being selected. In probability sampling, another name for random sampling, the researcher can select the level of chance. In order to produce a true random sample, the population must be known. A known, finite population is one where all members of the population can be identified and are accessible. This kind of sampling also assumes that all who are selected to be part of the sample will respond.

A random sample does not guarantee that the sample will properly represent those in the population. Sampling is not a precise science. There is still a chance that a randomly selected sample will be skewed in some way—by this I mean the sample under- or overrepresents some group or characteristic found within the population. The Central Limit Theory tells us that when an infinite number of samples are taken, the distribution of the sample means will be normally distributed, and the average of the sample means will be that of the population. However, our understanding of the normal curve likewise indicates that the mean of any one sample may be extremely different from the population. Still, while the result we obtain will not be perfect, care should be taken to attain the best result possible. Random sampling is used when we don’t have specific information about those in the target population and wish to remove human bias from the selection process. Random sampling is believed to be the best way to avoid selection bias.

Systematic Sampling

Systematic sampling is an adaptation of random sampling which does not give everyone an independent chance of being selected. For example, the selection process may choose every fifth person in a list. This is not completely random because an individual’s position in the list limits the chance they will be selected (the selection is dependent on the individual's position in the list); the randomness of the selection becomes even more problematic if the list is compiled in a way that introduces a systematic bias.

Stratified Sampling

A stratum is an identifiable, mutually exclusive subgroup within a population. Stratified sampling attempts to guarantee representation from each important strata within the population. Membership in a stratum must be homogeneous so the sampling would not allow selection of an individual who has membership in two distinct strata. Stratification is considered to be a random sampling technique because individuals are randomly selected from each stratum. Stratified sampling could be equal or proportional. The researcher could select an equal number of participants from each stratum, or they could select participants proportionally based on the estimated size of each stratum. Proportional sampling is preferred if the sample is to be generalizable. In this case the required sample size selected from each stratum should be determined independently so each stratum is appropriately represented. This may require a much larger number of participants compared to the number needed using simple random sampling.

Cluster Sampling

With cluster sampling the unit of analysis is based on intact groups rather than individuals. For example, all those in a particular school or classroom are selected, not specific individuals within each school or classroom. The intact units are however randomly selected. For this to produce a representative sample, it is assumed that the intact units will include a variety of individuals represented in the population or that an adequate number of heterogeneous intact groups selected will, as a whole, adequately represent the population. This may or may not be the case and may require a combination of stratified and cluster sampling. In practice, not all samples obtained in this manner are random samples. When a research study requires that the unit of analysis includes sampling of intact groups, special care needs to be taken to make sure that adequate representation is obtained.

Non-Random Sampling Techniques

While random sampling is preferred (and considered by some to be the gold standard), it is not always possible to obtain a random sample. And while the basic procedures used with non-random sampling often mirror sampling procedures used to obtain random samples, any method of sampling that does not allow for individuals (or units) to have an equal and independent chance of being selected is referred to as non-random sampling. Non-random sampling is considered inferior to random sampling because there is a greater chance that the sample will not represent the population adequately. However, for a variety of reasons, non-random sampling in the social sciences is quite common.

The most common reason for using non-random sampling is that of necessity. Random samples cannot be selected when the size of the population is unknown, individuals cannot be easily identified, access to the potential respondents is restricted, or contact information is unattainable. In addition, even when random selection is implemented, ethical consideration regarding the protection of human subjects’ rights may prevent the sample from being a true random sample. For example, randomly selected individuals may not be willing to provide information or allow their information to be used. If this happens in large numbers, or in a systematically unbalanced way, a potential random sample will, in practice, become a non-random sample. This could considerably diminish the chances that the sample adequately represents the population.

Regardless of the way a sample is obtained, the goal of any sampling technique is to allow the researcher to access information from those who can provide useful information. Useful in this case means providing information that helps answer the research questions in such a way that researchers can trust the results; this is an issue of validity. There are many ways to obtain a non-random sample.

Convenient Sampling

A convenient sample is comprised of individuals who are available and willing to complete the survey (i.e., volunteers who can be contacted and are willing to participate). Any time you send out a broad invitation to potential respondents asking them to volunteer to take a survey, you are creating a convenient sample. A convenient sample is less likely to adequately represent the population than a random sample, and the results are less likely to be generalizable without having a larger sample size. Even when a high response rate is obtained, if those available and willing to participate systematically do not represent those in the population, the results will not be valid. Unfortunately, we may never know the degree to which any sample is biased, but there is an increased probability that a convenient sample will not adequately represent the population compared to a random sample.

Quota Sampling

Like a stratified sample, quota sampling involves selecting individuals to participate based on identifiable characteristics of individuals within the population. With quota sampling, the researcher identifies major subgroups of interest within the population (strata), determines the number of individuals needed, and then attempts to obtain a sufficient number of willing and available participants from each subgroup. Like stratified sampling, the number of participants needed (i.e., the quota) may be based on equal or proportional requirements.

Purposive Sampling

Selecting participants using purposive sampling procedures requires the researcher to specify criterion for inclusion. As a result, purposive sampling has at times been called criterion-based sampling. Criteria are based on a set of characteristics individuals possess (i.e., things about the potential respondents that make them interesting because they would likely be able to provide useful information). Once the criterion for inclusion have been identified, participant selection will focus on getting a sufficient number of willing participants who meet the criterion. Because participation in a qualitative study often requires participants to willing submit to lengthy, involved data collection procedures, the sampling techniques used in qualitative studies are almost always purposive. There are several ways the inclusion criterion for purposive sampling may be established.

Comprehensive Sampling

Comprehensive sampling attempts to obtain data from individuals experiencing every possible condition or subgroups defined within the population. This usually isn’t possible, but when it is possible, it is not practical. More often researchers will use some form of homogeneous sampling where selection criteria are based on choosing individuals with similar experiences, situations, perspectives, interests, or circumstances. This is a more manageable approach and researchers often will refine inclusion criteria to match a particular research purpose. Following are examples of these inclusion criteria.

Maximum Variation (Intensity Sampling). Selection criteria are designed to obtain a wide range of participants based on a few specific variables. An example of maximum variation in a sample would be the selection of students with various levels of academic achievement from various years in school.

Extreme Case. In this situation, selection criteria are intended to include participants representing extreme situations. For example, those who participate in a regimented exercise routine every day without fail and those who claim to never exercise at all.

Typical Case. With this strategy, the researcher sets inclusion criteria to include people who typify the normal (most prominent) individuals in the population. To do this the researcher would consult experts or examine theory to determine characteristics of the “typical” person they wish to study, then set out to find a sample of these individuals. For example, the researcher might look for individuals described as being typical based on characteristics like age, experience, education, gender, behavior, or perspective. In cases where the purpose of the research is to define what is typical or normal, the sampling would need to be more comprehensive.

Critical Case. Sampling to include critical case individuals requires identifying individuals or intact groups who are important for some specific reason. For example, a researcher might select schools where conditions would likely result in greater resistance to planned reforms. The critical inclusion requirement being that if there is resistance, it will exist in those schools. The converse may also be a critical case; if there is little resistance to the proposed reforms, it will likely be at other schools.

Negative Case (Discrepancy Sampling). The selection criteria for a negative case are intended to identify respondents who are atypical, go against the norm, or provide examples that might disconfirm expected results. The sample is chosen to include those who appear to wholly disprove or refute a theory. For example, an intervention may be extremely effective for the vast majority of individuals; however, a small group of individuals tend to be negatively impacted by the intervention, meaning those individuals represent a negative case by going against expected outcomes.

Referral or Snowball Sampling. This type of sampling is based on practical purposes rather than research purposes. When those individuals matching a particular set of criteria are not readily identifiable, one way to locate participants is to ask for referrals. Once one individual is found and surveyed, they are asked if they know others who share similar characteristics. Thus, the selection process has a snowball effect (i.e., the sample gets larger as you go). With this technique it can be difficult to know when the number in the sample is sufficient. This is where data saturation decisions need to be made. Data saturation refers to situations where the information you obtain from participants begins to repeat. Saturation refers to the point where you don’t need more participant data because you are getting the same answers. Additional information in this case would not improve your understanding of the phenomenon, just substantiate the strength of the finding.

Need for Replication

It is important to understand the unlikelihood that any sample you obtain will perfectly represent the population from which it was drawn. Even with a random sample, there is a high probability that the sample will not exactly represent the population in some way. Given that sampling is not a precise science, the need to replicate a study should be evident. A carefully selected sample can provide valuable results, which is why we conduct research. However, the sample used to obtain a result may have been flawed in some way, thus you would need to redo the study with a different sample. In this sense, replication of a study is done to verify the results. Still, few studies are replicated in such a way that completely verify the results of previous studies. The outcomes obtained from any carefully constructed sample will likely be of some value, they just won’t be perfect.

Chapter Summary

The way in which a sample is obtained will affect the quality (or value) of the sample.
Quantitative surveys are typically designed to produce descriptive numerical statistics that can be used to describe various general characteristics found within the population.
Qualitative surveys are not interested as much with generalizability as they are with understanding a phenomenon. As a result, in a qualitative study, the sample must produce a good set (sample) of informants for the study.
Random sampling refers to sampling techniques that allow an equal and independent chance for participants to be selected.
Random sampling is believed to be the best way to alleviate sampling error. Sampling error affects the degree to which the sample represents those in the population and thus the generalizability of the results.
Random sampling is not a foolproof method, and any random sample has a high probability of being flawed in some way. Major flaws in the sample obtained have the potential to adversely affect the result. Minor flaws can be acknowledged and accounted for.
The ability to produce a true random sample will be dependent on whether the size of the population is known (finite), individuals can be easily identified, access to the potential respondents is unrestricted, and the contact information for potential participants is available. In addition, ethical consideration regarding the protection of human subjects’ rights and response refusal issues may prevent a true random sample from being obtained.
The most convincing reason for using random sampling is that it helps researchers avoid human bias in the selection process. Random sampling is often used when specific demographic and personal information about individual respondents is unavailable.
Non-random sampling is commonly used in the social sciences due to the difficulties in obtaining a true random sample.
Random sampling and non-random sampling techniques are similar with the exception of random selection.
The most common reason for using non-random sampling is one of necessity—there is no other way to proceed.
Convenient samples are based on participants' willingness and availability (i.e., volunteers).
Other selection parameters can help refine the sampling procedure (e.g., purposive sampling). This is only possible when specific attributes of the potential respondents are known.
The research purpose will often dictate the best sampling techniques to use; however, practical issues will also influence the decision.
The unit of analysis is often individual people in the population; however, sometimes intact groups are selected. When intact groups are used (i.e., cluster sampling) the degree to which adequate representation has been achieved must be carefully considered.
Because no sample is perfect, replication studies are useful to validate the results of any study.

Discussion Questions

What are the benefits of using a random sampling procedure over a non-random sampling procedure?
How likely is it that any sample you select will be perfect? Explain.
What is the purpose of a replication study? When and why are they needed?

Practice Tasks

Pretend you wish to make comparisons between specific groups of individuals within a population. What sampling techniques would best serve your needs? Explain the benefits and limitations of the sampling procedures you chose.
For a specific study you might consider completing, identify the population and chose a sampling technique that would serve your needs. Explain the benefits and limitations of the sampling procedures you chose. What particular challenges will you need to overcome in order to obtain the sample?

References

Davies, R. S., Williams, D. D., & Yanchar, S. (2008). The use of randomisation in educational research and evaluation: A critical analysis of underlying assumptions. Evaluation & Research in Education, 21(4), 303-317.

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/designing_surveys/sampling_techniquesp.