How to Calculate Credible Interval From Posterior

What is a Credible Interval?

A credible interval is a range of values that contains a specified probability of the true parameter value, based on the posterior distribution. Unlike confidence intervals, which are based on repeated sampling, credible intervals reflect the uncertainty in the parameter estimate given the observed data and prior information.

For example, if you have a 95% credible interval for a parameter θ, it means there is a 95% probability that θ lies within this interval, given the observed data and prior information.

Understanding the Posterior Distribution

The posterior distribution combines the prior distribution (prior beliefs about the parameter) with the likelihood function (information from the data) using Bayes' theorem:

The posterior distribution represents the updated beliefs about the parameter after observing the data. To calculate a credible interval, you need to:

1. Define the prior distribution for the parameter of interest.
2. Specify the likelihood function based on the observed data.
3. Calculate the posterior distribution using Bayes' theorem.
4. Determine the credible interval by finding the range of values that contain the desired probability mass.

Common types of posterior distributions include normal, beta, and gamma distributions, depending on the choice of prior and likelihood.

Methods for Calculating Credible Intervals

There are several methods to calculate credible intervals from posterior distributions:

Equal-Tailed Interval

This method divides the probability mass equally on both tails of the distribution. For a 95% credible interval, 2.5% of the probability mass is excluded from each tail.

Highest Posterior Density (HPD) Interval

The HPD interval is the smallest interval that contains the specified probability mass. It is more efficient than the equal-tailed interval for many distributions.

Quantile-Based Interval

This method uses the quantiles of the posterior distribution to define the interval. For a 95% interval, you would use the 2.5th and 97.5th percentiles.

Example Calculation

Suppose you have a posterior distribution that is normally distributed with mean μ = 5 and standard deviation σ = 1. To calculate a 95% credible interval:

1. Identify the critical values for a 95% interval. For a normal distribution, these are approximately ±1.96 standard deviations from the mean.
2. Calculate the lower bound: 5 - (1.96 × 1) = 3.04
3. Calculate the upper bound: 5 + (1.96 × 1) = 6.96

The 95% credible interval is (3.04, 6.96). This means there is a 95% probability that the true parameter value lies within this range.

Interpreting Credible Intervals

Credible intervals provide a range of plausible values for a parameter, given the data and prior information. Key points to consider when interpreting credible intervals:

• The width of the interval reflects the uncertainty in the parameter estimate.
• A narrower interval indicates more precise estimates, while a wider interval indicates greater uncertainty.
• Credible intervals are not exact probabilities in the frequentist sense. They represent the probability of the parameter given the data and prior.
• Different credible intervals can be calculated for the same data, depending on the choice of prior and method.

For example, if you have a 95% credible interval for a treatment effect, it means you are 95% confident that the true effect lies within this range, based on your prior beliefs and the observed data.