Skip to main content

Full text of "Theory of Probability- Characteristic Functions. Central Limit Theorem on R."

See other formats

MIT OpenCourseWare 

18.175 Theory of Probability 

Fall 2008 

For information about citing these materials or our Terms of Use, visit: . 

Section 9 

Characteristic Functions. Central 
Limit Theorem on R. 

Let X = (Xi, . . . , Xk) be a random vector on BJ^ with distribution P and let t = {t\,. . . , tk) € M''. Charac- 
teristic function of X is defined by 

If X has standard normal distribution A/'(0, 1) and A e M then 
For complex A = it, consider analytic function 

1 x2 

<^(a;) = e'*^^=e"^ for a; e C. 
V 27r 

By Cauchy's theorem, integral over a closed path is equal to 0. Let us take a closed path x + iO for x from 
— oo to +0O and x + it for x from +oo to — oo. Then 

-1 poo 2 1 

fit) = e'*"-'^rfx=^/ e^*('*+")-5('*+-)'dx 

v27r J-oo \/27r J-oo 

= ^ e-*'+^*^+5*'-**^-5^'dx = e-T / ^e-*da; = e-'^. (9.0.1) 
V^J-oo J ^ 

If y has normal distribution M(ra, a^) then 

Lemma 16 If X is a real-valued r.v. such that E|X|'" < oo for integer r then f{t) G C^{M.) and 


for j < r. 

Proof. If r = 0, then |e**''^| < 1 implies 

f{t) = Ee'*^ ^ Ee'"^ = f{s) if t ^ s, 


by dominated convergence theorem. This means that / e C(R). If r = 1, E|X| < oo, we can use 


and, therefore, by dominated convergence theorem, 

f'{t) = UmE 

t - S 

Also, by dominated convergence theorem, EiXe**^ e C'(IR), which means that / e C^(M). We proceed by 
induction. Suppose that we proved that 

and that r = j + 1, E|Xp+^ < oo. Then, we can use that 


so that by dominated convergence theorem = E{iXy+'^e'^^ e C(]R). 


The main goal of this section is to prove one of the most famous results in Probability Theory. 

Theorem 22 (Central Limit Theorem) Consider an i.i.d. sequence {Xi)i>i such thatEXi = 0,EX^ = < 
oo and let Sn = X]i<n -^i- Then Snl\fn converges in distribution to A/'(0, cr^). 

We will start with the following. 

Lemma 17 We have, 

lim Ee v« = e 


Proof. By independence. 


J^jEe = (Ee j . 


Since EXf < oo previous lemma implies that (fi{t) G C^{M.) and, therefore. 

ip{t) = Ee'*^i = ip{0) + ip'{0)t + ^^p"{0)t^ + o{t^) as t ^ 0. 

we get 


ip{0) = 1, (p'{0) = EiXe'-°-^ = iEX = 0, ^"{0) = E{iXf = -EX^ = -a^ 


v{t) = l-^+o{t^). 





e ^ , n ^ oo. 

Next, we want to show that characteristic function uniquely determines the distribution. Let X ~ 
P, F ~ Q be two independent random vectors on R*'. We denote by P * Q the convolution of P and Q which 
is the law C{X + Y) of the sum X + F. We have, 

= EI{X + y e A) = jjl{x + yG A)dF{x)dQ{y) 

= JJi{xgA- y)dF{x)dQ{y) = j ¥{A- y)dq{y). 


If P has density p then 

P * Q(A) = JJl{x + ye A)p{x)dxdQ{y) = JJKz& A)p{z - y)dz 
= jj P{z- y)dzdQ{y) = J{Jp{z- y)dQ{y)) dz 


which means that P * Q has density 
If, in addition, Q has density q then 

f{x) = J p{x-y)dQ{y). (9.0.2) 
= j p{x- 

f{x) = p{x- y)q{y)dy. 

Denote by 7V^(0, a^I) the law of the random vector X = {Xi , Xf.) of i.i.d. ^(0, cr^) random variables 
whose density on M*^ is 



1 X'^ ^ 1 l^|2 

e 2c 

For a distribution P denote P"" = P * J\f{0, a^I). 
Lemma 18 P'" = P * J\f{0, a'^I) has density 

P''ix)=[^y J me-'(''^^-'4w^dt 

where f{t) = / e'(*'^)tflP(a;). 

Proof. By (9.0.2), P * A/'(0, ci^/) has density 

p^ix) = i^-S' I e-5^^l^-^l'dP(2/). 

Using (9.0.1), we can write 

and taking a product over i < k we get 

(^^y Je-^^i^-y^^)e-'^\^\'dz. 

Then we can continue 

p^ix) = (^-^y j j e-'^^''-^''^-'^^'^' dzdV{y) 

(2^)'' // e"'^^''"'''^^"''^''<ffl"(2/)(^^ 

Let z = ta. 


Theorem 23 (Uniqueness) If 
then P = Q. 

Proof. By the above Lemma, P*" = Q'^. If X ~ P and ~ 7V(0, /) then X + a^^X almost surely as a ^ 
and, therefore, P"^ ^ P weakly Similarly, Q'^ Q. 


We proved that the characteristic function of Snl\fn converges to the c.f. of A/'(0, cr^). Also, the sequence 

ci—'^ ) - is uniformly tight. 

since by Chebyshev's inequality 

for large enough M. To finish the proof of the CLT on the real line we apply the following. 
Lemma 19 If (P„) is uniformly tight and 

fn{t) = J e"^dP„(a;) ^ f{t) 

then P„ ^ P and f{t) = J e**^(iP(x). 

Proof. For any sequence (n(fc)), by Selection Theorem, there exists a subsequence (n(fc(r))) such that 
lPra(fc(r)) converges weakly to some distribution P. Since e'^*'^^ is bounded and continuous. 

J e^(*'-)dP„(,(,)) ^ J e*(*'-)rfP(ar) 

as r — > oo and, therefore, / is a c.f. of P. By uniqueness theorem, distribution P does not depend on the 
sequence {n{k)). By Lemma 13, P„ — > P weakly.