A note about the Student’s t distribution

The Stuent’s t distribution is an important distribution in statistics. It is the basis for the Student’s t statistic and arises in the problem of etimating the mean of a normal population. The Student’s t distribution (t distribution for short) is usually defined as the following ratio

\displaystyle T=\frac{Z}{\sqrt{\frac{U}{n}}}=\frac{Z \sqrt{n}}{\sqrt{U}}

where Z \sim N(0,1) and U has a chi-square distribution with n degrees of freedom. For this derivation, see [1].

In this post we discuss another way of deriving the t distribution. The alternative view is through the notion of mixture (compounding in some texts). Suppose that X has a normal distribution with mean 0 and variance \Theta^{-1}. That is, X \sim N(0,\Theta^{-1}). There is uncertainty in the variance \Theta^{-1}. Further suppose that \Theta follows a gamma distribution with parameters \alpha and \beta where \alpha=\beta and \alpha is a positive integer. Then the unconditional distribution of X has a Student’s t distribution with n=2 \alpha degrees of freedom. In the language of mixture in probability, we say that the Student’s t distribution is a mixture of normal distributions with gamma mixing weights.

The conditional distribution of X \lvert \Theta is f_{X \lvert \Theta}(x \lvert \theta) \thinspace h_{\Theta}(\theta) where

\displaystyle f_{X \lvert \Theta}(x \lvert \theta)=\frac{\sqrt{\theta}}{\sqrt{2 \pi}} \thinspace e^{-\frac{\Theta x^2}{2}} and

\displaystyle h_{\Theta}(\theta)=\frac{\alpha^{\alpha}}{\Gamma(\alpha)} \thinspace \theta^{\alpha-1} \thinspace e^{-\alpha \theta} \thinspace d \theta

The marginal (the unconditional) density function of X is obtained by integrating out the parameter \theta. The resulted density is that of the Student’s t distribution. The following is the derivation.

\displaystyle f_{X}(x)=\int_{0}^{\infty} f_{X \lvert \Theta}(x \lvert \theta) \thinspace h_{\Theta}(\theta) \thinspace d \theta

\displaystyle =\int_{0}^{\infty} \frac{\sqrt{\theta}}{\sqrt{2 \pi}} \thinspace e^{-\frac{\Theta x^2}{2}} \thinspace \frac{\alpha^{\alpha}}{\Gamma(\alpha)} \thinspace \theta^{\alpha-1} \thinspace e^{-\alpha \theta} \thinspace d \theta

\displaystyle =\frac{\alpha^{\alpha}}{\sqrt{2 \pi} \Gamma(\alpha)} \int_{0}^{\infty} \theta^{\alpha+\frac{1}{2}-1} \thinspace e^{-(\alpha+\frac{x^2}{2}) \theta} \thinspace d \theta

\displaystyle =\frac{\alpha^{\alpha}}{\sqrt{2 \pi} \Gamma(\alpha)} \frac{\Gamma(\alpha+\frac{1}{2})}{(\alpha+\frac{x^2}{2})^{\alpha+\frac{1}{2}}}\int_{0}^{\infty} \frac{(\alpha+\frac{x^2}{2})^{\alpha+\frac{1}{2}}}{\Gamma(\alpha+\frac{1}{2})} \theta^{\alpha+\frac{1}{2}-1} \thinspace e^{-(\alpha+\frac{x^2}{2}) \theta} \thinspace d \theta

\displaystyle =\frac{\alpha^{\alpha}}{\sqrt{2 \pi} \Gamma(\alpha)} \frac{\Gamma(\alpha+\frac{1}{2})}{(\alpha+\frac{x^2}{2})^{\alpha+\frac{1}{2}}}

Now, let n=2 \alpha. Then the above density function becomes:

\displaystyle f_X(x)=\frac{\Gamma(\frac{n+1}{2})}{\sqrt{\pi n} \thinspace \Gamma(\frac{n}{2})} \biggl(\frac{n}{n+x^2}\biggr)^{\frac{n+1}{2}}

The above density function is that of a Student’s t distribution with n degrees of freedom. It is interesting to note that because of the uncertainty in the parameter \Theta, the Student’s t distribution has a longer tail than the conditional normal distribution used in the beginning of the derivation.

Reference

  1. Feller W., An Introduction to Probability Theory and Its Applications, Vol II, Second Edition, John Wiley & Sons (1971)
Advertisements

One thought on “A note about the Student’s t distribution

  1. Thanks, I knew the t-distribution was a mixture of normals, but I was using slightly the wrong formula to generate a t-distribution, and this set out what I needed to do very clearly. FWIW, I have found that a t-distribtion with >2 d.f. can be emulated extremely closely with just 4 suitably chosen and weighted (by a formula) normal distributions. I needed to do this as I wanted to add the square of a t-distributed variable with a low d.f. to a chi-squared varaible (actually a F distributed varaibel, with the second F d.f. large)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s