Tag Archives: Student’s t distribution

A note about the Student’s t distribution

Posted on by
1

The Stuent’s t distribution is an important distribution in statistics. It is the basis for the Student’s t statistic and arises in the problem of etimating the mean of a normal population. The Student’s t distribution (t distribution for short) is usually defined as the following ratio

\displaystyle T=\frac{Z}{\sqrt{\frac{U}{n}}}=\frac{Z \sqrt{n}}{\sqrt{U}}

where Z \sim N(0,1) and U has a chi-square distribution with n degrees of freedom. For this derivation, see [1].

In this post we discuss another way of deriving the t distribution. The alternative view is through the notion of mixture (compounding in some texts). Suppose that X has a normal distribution with mean 0 and variance \Theta^{-1}. That is, X \sim N(0,\Theta^{-1}). There is uncertainty in the variance \Theta^{-1}. Further suppose that \Theta follows a gamma distribution with parameters \alpha and \beta where \alpha=\beta and \alpha is a positive integer. Then the unconditional distribution of X has a Student’s t distribution with n=2 \alpha degrees of freedom. In the language of mixture in probability, we say that the Student’s t distribution is a mixture of normal distributions with gamma mixing weights.

The conditional distribution of X \lvert \Theta is f_{X \lvert \Theta}(x \lvert \theta) \thinspace h_{\Theta}(\theta) where

\displaystyle f_{X \lvert \Theta}(x \lvert \theta)=\frac{\sqrt{\theta}}{\sqrt{2 \pi}} \thinspace e^{-\frac{\Theta x^2}{2}} and

\displaystyle h_{\Theta}(\theta)=\frac{\alpha^{\alpha}}{\Gamma(\alpha)} \thinspace \theta^{\alpha-1} \thinspace e^{-\alpha \theta} \thinspace d \theta

The marginal (the unconditional) density function of X is obtained by integrating out the parameter \theta. The resulted density is that of the Student’s t distribution. The following is the derivation.

\displaystyle f_{X}(x)=\int_{0}^{\infty} f_{X \lvert \Theta}(x \lvert \theta) \thinspace h_{\Theta}(\theta) \thinspace d \theta

\displaystyle =\int_{0}^{\infty} \frac{\sqrt{\theta}}{\sqrt{2 \pi}} \thinspace e^{-\frac{\Theta x^2}{2}} \thinspace \frac{\alpha^{\alpha}}{\Gamma(\alpha)} \thinspace \theta^{\alpha-1} \thinspace e^{-\alpha \theta} \thinspace d \theta

\displaystyle =\frac{\alpha^{\alpha}}{\sqrt{2 \pi} \Gamma(\alpha)} \int_{0}^{\infty} \theta^{\alpha+\frac{1}{2}-1} \thinspace e^{-(\alpha+\frac{x^2}{2}) \theta} \thinspace d \theta

\displaystyle =\frac{\alpha^{\alpha}}{\sqrt{2 \pi} \Gamma(\alpha)} \frac{\Gamma(\alpha+\frac{1}{2})}{(\alpha+\frac{x^2}{2})^{\alpha+\frac{1}{2}}}\int_{0}^{\infty} \frac{(\alpha+\frac{x^2}{2})^{\alpha+\frac{1}{2}}}{\Gamma(\alpha+\frac{1}{2})} \theta^{\alpha+\frac{1}{2}-1} \thinspace e^{-(\alpha+\frac{x^2}{2}) \theta} \thinspace d \theta

\displaystyle =\frac{\alpha^{\alpha}}{\sqrt{2 \pi} \Gamma(\alpha)} \frac{\Gamma(\alpha+\frac{1}{2})}{(\alpha+\frac{x^2}{2})^{\alpha+\frac{1}{2}}}

Now, let n=2 \alpha. Then the above density function becomes:

\displaystyle f_X(x)=\frac{\Gamma(\frac{n+1}{2})}{\sqrt{\pi n} \thinspace \Gamma(\frac{n}{2})} \biggl(\frac{n}{n+x^2}\biggr)^{\frac{n+1}{2}}

The above density function is that of a Student’s t distribution with n degrees of freedom. It is interesting to note that because of the uncertainty in the parameter \Theta, the Student’s t distribution has a longer tail than the conditional normal distribution used in the beginning of the derivation.

Reference

  1. Feller W., An Introduction to Probability Theory and Its Applications, Vol II, Second Edition, John Wiley & Sons (1971)