2.2 Subjectivity is not the key
The concepts of subjectivity and objectivity indeed characterize both statistical paradigms in differing ways. Among Bayesians there are those who are immersed in subjective rationality ((Ramsey 1926), (Finetti 1937), (Savage 1954), (D. V. Lindley 2000)), but others who adopt objective prior distributions such as Jeffreys’, reference, empirical or robust ((T. Bayes 1763), (P. Laplace 1812), (Jeffreys 1961), (J. Berger 2006)) to operationalize Bayes’ rule, and thereby weight quantitative (data-based) evidence. Among Frequentists, there are choices made about significance levels which, if not explicitly subjective, are typically not grounded in any objective and documented assessment of the relative losses of Type I and Type II errors.14 In addition, both Frequentist and Bayesian statisticians make decisions about the form of the data generating process, or “model”, which – if not subject to rigorous diagnostic assessment – retains a subjective element that potentially influences the final inferential outcome. Although we all know that by definition a model is a schematic and simplified approximation to reality,
“Since all models are wrong the scientist cannot obtain a correct one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena.” (G. E. P. Box 1976)
We also know that “All models are wrong, but some are useful” (G. E. Box 1979), that is why model diagnostics are important. This task can be performed in both approaches. Particularly, the Bayesian framework can use predictive p–values for absolute testing ((A. Gelman and Meng 1996), (M. Bayarri and Berger 2000)) or posterior odds ratios for relative statements ((Jeffreys 1935), (R. E. Kass and Raftery 1995)). This is because the marginal density, conditional on data, is interpreted as the likelihood of the prior distribution (J. Berger 1993).
In addition, what is objectivity in a Frequentist approach? For example, why should we use a 5% or 1% significance level rather than any other value? As someone said, the apparent objectivity is really a consensus (D. V. Lindley 2000). In fact “Student” (William Gosset) saw statistical significance at any level as being “nearly valueless” in itself (Ziliak 2008). But, this is not just a situation in the Frequentist approach. The cut-offs given to “establish” scientific evidence against a null hypothesis in terms of \(log_{10}\) scale (Jeffreys 1961) or \(log_{e}\) scale (R. E. Kass and Raftery 1995) are also ad hoc.
Although the true state of nature in Bayesian inference is expressed in “degrees of belief”, the distinction between the two paradigms does not reside in one being more, or less, subjective than the other. Rather, the differences are philosophical, pedagogical, and methodological.