4.6 Validity Evidence and Inferences

So far, we hae explored two key aspects of a study’s design. Sampling referes to how participants are selected from a larger population (like sampling a soup). Assignment refers to how the selected participants are assigned to comparison groups. Both sampling and assignment play a large role in the inferences that can be drawn. These two facets are directly related to questions about external validity and internal validity, respectively.

Validity is the degree to which inferences and conclusions are meaningful and accurate. It is important that you note that the validation is about the inferences, not the study. (You should never claim a study is valid.) Unfortunately, it is almost impossible to know whether the inferences we draw are valid or not. Therefore, the best we can do is to provide evidence that supports the claims of validity and present that evidence to people. In research studies there are two major types of validity evidence that should be considered:

• Internal Validity Evidence: This is evidence that supports the drawing of cause-and-effect conclusions. Generally these types of conclusions are only appropriate when a study has employed random assignment. Read more at https://www.socialresearchmethods.net/kb/intval.php.

• External Validity Evidence: This is evidence that supports the drawing of generalizable conclusions. Generally these types of conclusions are only appropriate when a study has employed random sampling. Read more at https://www.socialresearchmethods.net/kb/external.php.

Much like evidence used in a court of law, it is useful to think about validity evidence on a continuum. For example, after reading about the sampling and design of a particular study you might envision the following two “barometers” of the internal and external validity evidence for a study’s inferences:

For the study depicted above, the inferences may generalize, but are probably not causal. As you read research studies (and work on the class activities) you will learn how to situate the yellow dot along these two continuums by considering the sampling and design plans used in the study.

Read more about random sampling, random assignment, and validity evidence at https://www.statology.org/random-selection-vs-random-assignment/