How to Calculate Degrees of Freedom Likelihood Ratio

What is a Likelihood Ratio?

The likelihood ratio is a statistical test used to compare the fit of two nested models. It's commonly used in hypothesis testing, particularly in logistic regression and other generalized linear models. The test compares the likelihood of the data under two models: a restricted model (with fewer parameters) and an unrestricted model (with more parameters).

The likelihood ratio test statistic is calculated as -2 times the difference in the log-likelihoods of the two models. This statistic follows a chi-square distribution under the null hypothesis that the restricted model is correct.

Degrees of Freedom in Likelihood Ratio

The degrees of freedom for a likelihood ratio test is determined by the difference in the number of parameters between the unrestricted and restricted models. Specifically, it's calculated as:

For example, if you're comparing a model with 5 parameters to a model with 3 parameters, the degrees of freedom would be 2.

The degrees of freedom determine the shape of the chi-square distribution that the test statistic follows. A higher degrees of freedom means the distribution is more spread out, requiring larger values of the test statistic to be significant.

Calculation Method

The likelihood ratio test statistic is calculated using the following formula:

Where:

• LR is the likelihood ratio test statistic
• logL_restricted is the log-likelihood of the restricted model
• logL_unrestricted is the log-likelihood of the unrestricted model

The degrees of freedom for the test is the difference in the number of parameters between the two models.

Once you have the test statistic and degrees of freedom, you can compare it to the chi-square distribution to determine the p-value and assess the significance of the result.

Example Calculation

Let's consider an example where we're comparing two logistic regression models:

• Restricted model: 3 parameters (intercept + 2 predictors)
• Unrestricted model: 5 parameters (intercept + 4 predictors)

Suppose the log-likelihoods for the data are:

• logL_restricted = -120.5
• logL_unrestricted = -115.2

Calculating the likelihood ratio test statistic:

The degrees of freedom would be:

With a chi-square distribution table, we can find that a test statistic of 10.6 with 2 degrees of freedom corresponds to a p-value of approximately 0.005, which would typically be considered statistically significant.

Interpreting the Result

The likelihood ratio test helps determine whether the additional parameters in the unrestricted model provide a significantly better fit to the data than the restricted model. A significant result (typically p < 0.05) suggests that the unrestricted model is preferable.

The degrees of freedom indicate how many parameters are being tested. A higher degrees of freedom means more parameters are being tested, which generally requires a larger test statistic to be significant.

It's important to note that the likelihood ratio test assumes that the restricted model is nested within the unrestricted model. If this assumption is violated, the test may not be valid.