Chapter 8 Introduction to Continuous Prior and Posterior Distributions

Bayesian analysis is based on the posterior distribution of parameters \(\theta\) given data \(y\). The data \(y\) might be discrete (e.g., count data) or continuous (e.g., measurement data). However, parameters \(\theta\) almost always take values on a continuous scale, even when the data are discrete. For example, in a Binomial situation, the number of successes \(y\) takes values on a discrete scale, but the probability of success on any single trial \(\theta\) can potentially take any value in the continuous interval (0, 1).

Recall that the posterior distribution is proportional to the product of the prior distribution and the likelihood. Thus, there are two probability distributions which will influence the posterior distribution.

• The (unconditional) prior distribution of parameters \(\theta\), which is (almost always) a continuous distribution
• The conditional distribution of the data \(y\) given parameters \(\theta\), which determines the likelihood function. Viewed as a conditional distribution of \(y\) given \(\theta\), the distribution can be discrete or continuous, corresponding to the data type of \(y\). However, the likelihood function treats the data \(y\) as fixed and the parameters \(\theta\) as varying, and therefore the likelihood function is (almost always) a function of continuous \(\theta\).

This section provides an introduction to using continuous prior and posterior distributions to quantify uncertainty about parameters. Some general notation:

• \(\theta\) represents¹⁴ parameters of interest usually taking values on a continuous scale
• \(y\) denotes observed sample data (discrete or continuous)
• \(\pi(\theta)\) denotes the prior distribution of \(\theta\), usually a probability density function (pdf) over possible values of \(\theta\)
• \(f(y|\theta)\) denotes the likelihood function, a function of continuous \(\theta\) for fixed \(y\)
• \(\pi(\theta |y)\) denotes the posterior distribution of \(\theta\), the conditional distribution of \(\theta\) given the data \(y\).

Bayes rule works analogously for a continuous parameter \(\theta\), given data \(y\) \[\begin{align*} \pi(\theta|y) & = \frac{f(y|\theta)\pi(\theta)}{f_Y(y)}\\ & \\ \pi(\theta|y) & \propto f(y|\theta)\pi(\theta)\\ \text{posterior} & \propto \text{likelihood}\times \text{prior} \end{align*}\]

The continuous analog of the law of total probability is \[ f_Y(y) = \int_{-\infty}^{\infty}f(y|\theta)\pi(\theta) d\theta \]