4 Comparing two groups: Experiments, observational studies, and causation (internal validity)

The nature of doing science, be it natural or social, inevitably calls for comparison. Statistical methods are at the heart of such comparison, for they not only help us gain understanding of the world around us but often define how our research is to be carried out.¹⁸

Drawing inferences about the differences between groups is an almost daily occurrence in the lives of most people. In any given hour of any given day, television, radio and social media abound with comparisons. For example, data scientists at OKCupid, an online dating site, examined whether frequent tweeters (users of Twitter) have shorter real-life relationships than others.¹⁹

In the readings, course activities, and assignments in Unit 4, you will explore the process of comparing groups. You will learn about and model experimental variation to be able to evaluate observed differences between groups. You will learn about the randomization test (a Monte Carlo method for evaluating whether an observed result in compatible with experimental variation from a hypothesized model) and how to carry out this test using TinkerPlots™. Finally, you will learn how statisticians evaluate cause-and-effect inferences.