Chapter 33 Bias and bad science

Motivating scenarios: We want to be critical consumers of statical claims.

Learning goals: By the end of this chapter you should be able to

  • Recognize how motivated reasoning can mislead statistics.
  • Recognize when people are studying a question with an incorrect lense / focus.
  • Articulate how societal bias can be reflected in sampling bias, confounded studies and the design of machine learning algorithms.

Required reading / Viewing:

Black pills. link. From Superior (Saini 2019).

The era of blind faith in big data must end link by Cathy ONeil.

How To Stop Artificial Intelligence From Marginalizing Communities? link by Timnit Gebru.

33.1 Review / Setup

  • So much of stats aims to learn the TRUTH.

  • A major goal of stats is to learn causation.

  • Confounds and bias limit our abilities to make causal claims.

  • Rejecting the null model means our null model poorly describes the observations.

  • Stats has a lot of math and computing, but it’s hard to math and compute your way out of bias or bad experimental design.

33.2 The dark origin of stats

Statistics was largely founded by people interests in the Eugenics project - A racist program with the goal of “improving the race”. Read more about this history here and watch this roundtable aboout past present and future concerns here if interested.

33.3 Poor statistical conclusions motivated by race are common

Required reading Read Chapter 11 of Superior here

Sadly, this is very poorly taught in medical schools Optional reading

33.4 Modern machine learning offers new opportunities for amplifying bias

Machine learning aims to classify and predict based on some quantitative metrics. This can solve or amplify issues of bias and confounds. Unfortunately, it seems like we’re heading for the later. Watch these two videos on the topic.

Figure 33.1: Watch this video on algorithmic bias by Cathy ONeil

Figure 33.2: Watch this video on How To Stop Artificial Intelligence From Marginalizing Communities? by Timnit Gebru

33.5 Quiz

Reflection questions on canvas.


Saini, A. 2019. Superior: The Return of Race Science. Beacon Press.