2.1 The concept of probability
Let’s begin with the following thought experiment: Assume that you are watching the international game show “Who Wants to Be a Millionaire?”. The contestant is asked to answer a very simple question: What is the last name of the brothers who are credited with inventing the world’s first successful motor-operated airplane?
- What is the probability that the contestant answers this question correctly?
Unless you have:
- watched this particular contestant participate in this show many times,
- seen him asked this same question each time,
- and computed the relative frequency with which he gives the correct answer,
you need to answer this question as a Bayesian!
Uncertainty about the event answering this question needs to be expressed as a “degree of belief,” informed by both data on the skill of the particular participant and how much he knows about inventors, as well as possibly prior knowledge of his performance in other game shows. Of course, your prior knowledge of the contestant may be minimal, or it may be well-informed. Either way, your final answer remains a degree of belief about an uncertain, and inherently unrepeatable, state of nature.
The point of this hypothetical, light-hearted scenario is simply to highlight that a key distinction between the Frequentist and Bayesian approaches to inference is not the use (or nature) of prior information, but the manner in which probability is used. To the Bayesian, probability is the mathematical construct used to quantify uncertainty about an unknown state of nature, conditional on observed data and prior knowledge about the context in which that state occurs. To the Frequentist, probability is intrinsically linked to the concept of a repeated experiment, and the relative frequency with which a particular outcome occurs, conditional on that unknown state. This distinction remains key whether the Bayesian chooses to be informative or subjective in the specification of prior information, or chooses to be non-informative or objective.
Frequentists consider probability to be a physical phenomenon, like mass or wavelength, whereas Bayesians stipulate that probability exists in the mind of scientists, as any scientific construct (Parmigiani and Inoue 2008).
It seems that the understanding of the concept of probability for the common human being is more associated with “degrees of belief” rather than relative frequency. Peter Diggle, President of The Royal Statistical Society between 2014 and 2016, was asked in an interview, “A different trend which has surged upwards in statistics during Peter’s career is the popularity of Bayesian statistics. Does Peter consider himself a Bayesian?” He replied, “… you can’t not believe in Bayes’ theorem because it’s true. But that doesn’t make you a Bayesian in the philosophical sense. When people are making personal decisions – even if they don’t formally process Bayes’ theorem in their mind – they are adapting what they think they should believe in response to new evidence as it comes in. Bayes’ theorem is just the formal mathematical machinery for doing that.”
However, we should mention that psychological experiments suggest that human beings suffer from anchoring, a cognitive bias that causes us to rely too heavily on previous information (the prior), so that the updating process (posterior) due to new information (likelihood) is not as strong as Bayes’ rule would suggest (Kahneman 2011).