Powerful things usually come with warning labels. Examples include pain medication, chemical drain clog removers, construction equipment, and electrical extension cables.
Much of what we’ve seen in this course is a kind of warning label for the data analysis tools. These are powerful; they are used for takes like guiding public policy, conducting scientific research, adjudicating legal disputes, and directing corporate spending. They need to have a warning label.
Existing social and cultural biases can be reinforced:
Recall Adrian Simpson’s critique of education meta-analysis. Meta-meta-analyses for ranking effect sizes get turned into government education policies about what teachers should be doing in their classrooms. These need to make sense!
Statistical tools in drug trials:
Turning data into medical advice: “results not significant” turns into “procedure does nothing”:
Statistical arguments in courts of law