Appendix

Table 13.1: Libraries and commands in our graphical user interface.
mpg cyl disp hp drat wt qsec vs
Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0
Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0
Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1
Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1
Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0
Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1
Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0
Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1
Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1
Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1
Table 13.2: Datasets templates in folder DataSim.
mpg cyl disp hp drat wt qsec vs
Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0
Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0
Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1
Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1
Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0
Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1
Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0
Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1
Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1
Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1
Table 13.3: Real datasets in folder DataApp.
mpg cyl disp hp drat wt qsec vs
Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0
Mazda RX4 Wag 21.0 6 160.0 110 3.90 2.875 17.02 0
Datsun 710 22.8 4 108.0 93 3.85 2.320 18.61 1
Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1
Hornet Sportabout 18.7 8 360.0 175 3.15 3.440 17.02 0
Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1
Duster 360 14.3 8 360.0 245 3.21 3.570 15.84 0
Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1
Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1
Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1
Albert, James H, and Siddhartha Chib. 1993. “Bayesian Analysis of Binary and Polychotomous Response Data.” Journal of the American Statistical Association 88 (422): 669–79.
Bayarri, M. J., and J. Berger. 2004. “The Interplay of Bayesian and Frequentist Analysis.” Statistical Science 19 (1): 58–80.
Bayarri, M., and J. Berger. 2000. “P–Values for Composite Null Models.” Journal of American Statistical Association 95: 1127–42.
Bayes, T. 1763. “An Essay Towards Solving a Problem in the Doctrine of Chances.” Philosophical Transactions of the Royal Society of London 53: 370–416.
Bayes, Thomas. 1763. “LII. An Essay Towards Solving a Problem in the Doctrine of Chances. By the Late Rev. Mr. Bayes, FRS Communicated by Mr. Price, in a Letter to John Canton, AMFR S.” Philosophical Transactions of the Royal Society of London, no. 53: 370–418.
Benjamin, Daniel J, James O Berger, Magnus Johannesson, Brian A Nosek, E-J Wagenmakers, Richard Berk, Kenneth A Bollen, et al. 2018. “Redefine Statistical Significance.” Nature Human Behaviour 2 (1): 6–10.
Berger, J. 1993. Statistical Decision Theory and Bayesian Analysis. Third Edition. Springer.
———. 2006. “The Case for Objective Bayesian Analysis.” Bayesian Analysis 1 (3): 385–402.
Berger, James O. 2013. Statistical Decision Theory and Bayesian Analysis. Springer Science & Business Media.
Bernardo, J., and A. Smith. 1994. Bayesian Theory. Chichester: Wiley.
Bickel, Peter J, and Joseph A Yahav. 1969. “Some Contributions to the Asymptotic Theory of Bayes Solutions.” Zeitschrift für Wahrscheinlichkeitstheorie Und Verwandte Gebiete 11 (4): 257–76.
Box, G. E. P. 1976. “Science and Statistics.” Journal of the American Statistical Association 71: 791–99.
Box, George EP. 1979. “Robustness in the Strategy of Scientific Model Building.” In Robustness in Statistics, 201–36. Elsevier.
Chang, W. 2018. Web Application Framework for r: Package Shiny. R Studio. http://shiny.rstudio.com/.
Chernozhukov, V., and H. Hong. 2003. “An MCMC Approach to Classical Estimation.” Journal of Econometrics 115: 293–346.
Chib, Siddhartha. 1995. “Marginal Likelihood from the Gibbs Output.” Journal of the American Statistical Association 90 (432): 1313–21.
Chib, Siddhartha, and Ivan Jeliazkov. 2001. “Marginal Likelihood from the Metropolis–Hastings Output.” Journal of the American Statistical Association 96 (453): 270–81.
Clyde, M., and E. George. 2004. “Model Uncertatinty.” Statistical Science 19 (1): 81–94.
Dawid, A. P., M. Musio, and S. E. Fienberg. 2016. “From Statistical Evidence to Evidence of Causality.” Bayesian Analysis 11 (3): 725–52.
DeGroot, M. H. 1975. Probability and Statistics. London: Addison-Wesley Publishing Co.
Diaconis, Persi, Donald Ylvisaker, et al. 1979. “Conjugate Priors for Exponential Families.” The Annals of Statistics 7 (2): 269–81.
Efron, Bradley, and Trevor Hastie. 2016. Computer Age Statistical Inference. Vol. 5. Cambridge University Press.
Finetti, de. 1937. “Foresight: Its Logical Laws, Its Subjective Sources.” In Studies in Subjective Probability, edited by H. E. Kyburg and H. E. Smokler. New York: Krieger.
Fisher, R. 1958. Statistical Methods for Research Workers. 13th ed. New York: Hafner.
Gelfand, A. E., and A. F. M. Smith. 1990. “Sampling-Based Approaches to Calculating Marginal Densities.” Journal of the American Statistical Association 85: 398–409.
Gelfand, Alan E, and Dipak K Dey. 1994. “Bayesian Model Choice: Asymptotics and Exact Calculations.” Journal of the Royal Statistical Society: Series B (Methodological) 56 (3): 501–14.
Gelman, A., and X. Meng. 1996. “Model Checking and Model Improvement.” In In Markov Chain Monte Carlo in Practice, edited by Gilks, Richardson, and Speigelhalter. Springer US.
Gelman, A., X. Meng, and H. Stern. 1996. “Posterior Predictive Assessment of Model Fitness via Realized Discrepancies.” Statistica Sinica, 733–60.
Gelman, Andrew et al. 2006. “Prior Distributions for Variance Parameters in Hierarchical Models (Comment on Article by Browne and Draper).” Bayesian Analysis 1 (3): 515–34.
Gelman, Andrew, and Guido Imbens. 2013. “Why Ask Why? Forward Causal Inference and Reverse Causal Questions.” National Bureau of Economic Research.
Geman, S, and D. Geman. 1984. “Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images.” IEEE Transactions on Pattern Analysis and Machine Intelligence 6: 721–41.
Geweke, J. 1992. “Bayesian Statistics.” In. Clarendon Press, Oxford, UK.
Geweke, John. 2005. Contemporary Bayesian Econometrics and Statistics. Vol. 537. John Wiley & Sons.
Good, I. J. 1992. “The Bayes/Non Bayes Compromise: A Brief Review.” Journal of the American Statistical Association 87 (419): 597–606.
Goodman, S. N. 1999. “Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy.” Annals of Internal Medicine 130 (12): 995–1004.
Greenberg, Edward. 2012. Introduction to Bayesian Econometrics. Cambridge University Press.
Hastings, W. 1970. “Monte Carlo Sampling Methods Using Markov Chains and Their Application.” Biometrika 57: 97–109.
Heidelberger, P., and P. D. Welch. 1983. “Simulation Run Length Control in the Presence of an Initial Transient.” Operations Research 31 (6): 1109–44.
Jeffreys, H. 1935. “Some Test of Significance, Treated by the Theory of Probability.” Proccedings of the Cambridge Philosophy Society 31: 203–22.
———. 1961. Theory of Probability. London: Oxford University Press.
Jordan, Ghahramani, M. I., and L. Saul. 1999. “Introduction to Variational Methods for Graphical Models.” Machine Learning 37: 183–233.
Kahneman, Daniel. 2011. Thinking, Fast and Slow. Macmillan.
Kass, R E, and A E Raftery. 1995. “Bayes factors.” Journal of the American Statistical Association 90 (430): 773–95.
Kass, R. 2011. “Statistical Inference: The Big Picture.” Statistical Science 26 (1): 1–9.
Koop, Gary M. 2003. Bayesian Econometrics. John Wiley & Sons Inc.
Lancaster, Tony. 2004. An Introduction to Modern Bayesian Econometrics. Blackwell Oxford.
Laplace, P. 1812. Théorie Analytique Des Probabilités. Courcier.
Laplace, Pierre Simon. 1774. “Mémoire Sur La Probabilité de Causes Par Les évenements.” Mémoire de l’académie Royale Des Sciences.
Lehmann, E. L., and George Casella. 2003. Theory of Point Estimation. Second Edition. Springer.
Lindley, D. V. 2000. “The Philosophy of Statistics.” The Statistician 49 (3): 293–337.
Lindley, D. V., and L. D. Phillips. 1976. “Inference for a Bernoulli Process (a Bayesian View).” American Statistician 30: 112–19.
Lindley, Dennis V. 1957. “A Statistical Paradox.” Biometrika 44 (1/2): 187–92.
Martin, Andrew D., Kevin M. Quinn, and Jong Hee Park. 2011. “MCMCpack: Markov Chain Monte Carlo in R.” Journal of Statistical Software 42 (9): 1–21.
McGrayne, Sharon Bertsch. 2011. The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted down Russian Submarines, & Emerged Triumphant from Two Centuries of c. Yale University Press.
Metropolis, N., A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller. 1953. “Equations of State Calculations by Fast Computing Machines.” J. Chem. Phys 21: 1087–92.
Neyman, J., and E. Pearson. 1933. “On the Problem of the Most Efficient Tests of Statistical Hypotheses.” Philosophical Transactions of the Royal Society, Series A 231: 289–337.
Parmigiani, G., and L. Inoue. 2008. Decision Theory Principles and Approaches. John Wiley & Sons.
Pericchi, Luis, and Carlos Pereira. 2015. “Adaptative significance levels using optimal decision rules: Balancing by weighting the error probabilities.” Brazilian Journal of Probability and Statistics.
Petris, Giovanni, Sonia Petrone, and Patrizia Campagnoli. 2009. “Dynamic Linear Models.” In Dynamic Linear Models with r, 31–84. Springer.
R Core Team. 2021. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. http://www.R-project.org/.
Raftery, A. 1995. “Bayesian Model Selection in Social Research.” Sociological Methodology 25: 111–63.
Raftery, A. E., and S. M. Lewis. 1992. “One Long Run with Diagnostics: Implementation Strategies for Markov Chain Monte Carlo.” Statistical Science 7: 493–97.
Ramírez Hassan, A., J. Cardona Jiménez, and R. Cadavid Montoya. 2013. “The Impact of Subsidized Health Insurance on the Poor in Colombia: Evaluating the Case of Medellín.” Economia Aplicada 17 (4): 543–56.
Ramírez-Hassan, A., and M. Graciano-Londoño. 2020. “A GUIded Tour of Bayesian Regression.” Universidad EAFIT.
Ramsey, F. 1926. “Truth and Probability.” In The Foundations of Mathematics and Other Logical Essays, edited by Routledge and Kegan Paul. London: New York: Harcourt, Brace; Company.
Rossi, Peter E, Greg M Allenby, and Rob McCulloch. 2012. Bayesian Statistics and Marketing. John Wiley & Sons.
Savage, L. J. 1954. The Foundations of Statistics. New York: John Wiley & Sons, Inc.
Schlaifer, Robert, and Howard Raiffa. 1961. Applied Statistical Decision Theory.
Sellke, Thomas, MJ Bayarri, and James O Berger. 2001. “Calibration of p Values for Testing Precise Null Hypotheses.” The American Statistician 55 (1): 62–71.
Selvin, Steve. 1975a. “A Problem in Probability (Letter to the Editor).” The American Statistician 11 (1): 67–71.
———. 1975b. “A Problem in Probability (Letter to the Editor).” The American Statistician 11 (3): 131–34.
Serna Rodríguez, M., A. Ramírez Hassan, and A. Coad. 2019. “Uncovering Value Drivers of High Performance Soccer Players.” Journal of Sport Economics 20 (6): 819–49.
Smith, A. F. M. 1973. “A General Bayesian Linear Model.” Journal of the Royal Statistical Society. Series B (Methodological). 35 (1): 67–75.
Stigler, Stephen. 2018. “Richard Price, the First Bayesian.” Statistical Science 33 (1): 117–25.
Tanner, M. A., and W. H. Wong. 1987. “The Calculation of Posterior Distributions by Data Augmentation.” Journal of the American Statistical Association 82 (398): 528–40.
Tierney, Luke. 1994. “Markov Chains for Exploring Posterior Distributions.” The Annals of Statistics, 1701–28.
Tierney, Luke, and Joseph B Kadane. 1986. “Accurate Approximations for Posterior Moments and Marginal Densities.” Journal of the American Statistical Association 81 (393): 82–86.
Wasserstein, Ronald L., and Nicole A. Lazar. 2016. “The ASA’s Statement on p–Values: Context, Process and Purpose.” The American Statistician.
Zellner, Arnold. 1996. “Introduction to Bayesian Inference in Econometrics.”
Ziliak, S. 2008. “Guinnessometrics; the Economic Foundation of Student’s t.” Journal of Economic Perspectives 22 (4): 199–216.

  1. I strongly recommend to type the code line rather than copy and paste it. See below for a brief introduction to R software.↩︎

  2. Observe that I use the term “Bayes’ rule” rather than “Bayes’ theorem”. It was Laplace (P. S. Laplace 1774) who actually generalized the Bayes’ theorem (Thomas Bayes 1763). His generalization is named the Bayes’ rule.↩︎

  3. \(\lnot\) is the negation symbol. In addition, we have that \(P(B|A)=1-P(B|A^c)\) in this example, where \(A^c\) is the complement of \(A\). However, this is not true in general.↩︎

  4. https://www.wolframalpha.com/input/?i=number+of+people+who+have+ever+lived+on+Earth↩︎

  5. https://www.r-bloggers.com/2019/04/base-rate-fallacy-or-why-no-one-is-justified-to-believe-that-jesus-rose/↩︎

  6. From a Bayesian perspective \(\mathbf{\theta}\) is fixed, but unknown. Then, it is treated as a random object.↩︎

  7. \(\propto\) is the proportional symbol.↩︎

  8. See also (M. Bayarri and Berger 2000) to show potential flows due to using data twice in the construction of the predictive p values, and alternative proposals, for instance the partial posterior predictive p value.↩︎

  9. \(\perp\) is the independence symbol.↩︎

  10. Take into account that in the likelihood function the argument is \(\theta\). However, we keep the notation for facility in exposition.↩︎

  11. Independent and identically distributed draws.↩︎

  12. We should be aware that there may be technical problems using this king of hyperparameters in this setting (Andrew Gelman et al. 2006).↩︎

  13. (Chernozhukov and Hong 2003) propose Laplace type estimators (LTE) based on the quasi-posterior, \(p(\mathbf{\theta})=\frac{\exp\left\{L_n(\mathbf{\theta})\right\}\pi(\mathbf{\theta})}{\int_{\mathbf{\Theta}}\exp\left\{L_n(\mathbf{\theta})\right\}\pi(\mathbf{\theta})d\theta}\) where \(L_n(\mathbf{\theta})\) is not necessarily a log-likelihood function. The LTE minimizes the quasi-posterior risk.↩︎

  14. Type I error is rejecting the null hypothesis when this is true, and the Type II error is not rejecting the null hypothesis when this is false.↩︎

  15. A pivot quantity is a function of unobserved parameters and observations whose probability distribution does not depend on the unknown parameters.↩︎

  16. An ancillary statistic is a pivotal quantity that is also a statistic.↩︎

  17. https://fivethirtyeight.com/features/not-even-scientists-can-easily-explain-p-values/↩︎

  18. See https://joyeuserrance.wordpress.com/2011/04/22/proof-that-p-values-under-the-null-are-uniformly-distributed/ for a simple proof.↩︎

  19. Another parametrization of the gamma density is the scale parametrization where \(\kappa_0=1/\beta_0\). See the health insurance example in Chapter 1.↩︎

  20. A particular case of the Woodbury matrix identity↩︎

  21. Using this result \(({\bf{A}}+{\bf{B}}{\bf{D}}{\bf{C}})^{-1}={\bf{A}}^{-1}-{\bf{A}}^{-1}{\bf{B}}({\bf{D}}^{-1}+{\bf{C}}{\bf{A}}^{-1}{\bf{B}})^{-1}{\bf{C}}{\bf{A}}^{-1}\)↩︎

  22. \(vec\) denotes the vectorization operation, and \(\otimes\) denotes the kronecker product↩︎

  23. We can write down the former expression in a more familiar way using vectorization properties, \(\underbrace{vec(Y)}_{\bf{y}}=\underbrace{({\bf{I}}_M\otimes {\bf{X}})}_{{\bf{Z}}}\underbrace{vec({\bf{B}})}_{\beta}+\underbrace{vec({\bf{U}})}_{\mu}\), where \({\bf{y}}\sim N_{N\times M}({\bf{Z}}\beta,\bf{\Sigma}\otimes {\bf{I}}_N)\).↩︎

  24. I strongly recommend to type the code line rather than copy and paste it.↩︎

  25. Users should take into account that formal inference (hypothesis tests) in a Bayesian framework is based on Bayes factors.↩︎

  26. Tuning parameters should be set in a way such that one obtains reasonable diagnostic criteria and aceptation rates.↩︎

  27. Formulating the acceptance rate using \(\log\) helps to mitigate computational problems.↩︎

References

Bayarri, M., and J. Berger. 2000. “P–Values for Composite Null Models.” Journal of American Statistical Association 95: 1127–42.
Bayes, Thomas. 1763. “LII. An Essay Towards Solving a Problem in the Doctrine of Chances. By the Late Rev. Mr. Bayes, FRS Communicated by Mr. Price, in a Letter to John Canton, AMFR S.” Philosophical Transactions of the Royal Society of London, no. 53: 370–418.
Chernozhukov, V., and H. Hong. 2003. “An MCMC Approach to Classical Estimation.” Journal of Econometrics 115: 293–346.
Gelman, Andrew et al. 2006. “Prior Distributions for Variance Parameters in Hierarchical Models (Comment on Article by Browne and Draper).” Bayesian Analysis 1 (3): 515–34.
Laplace, Pierre Simon. 1774. “Mémoire Sur La Probabilité de Causes Par Les évenements.” Mémoire de l’académie Royale Des Sciences.