# Full text of "Theory of Probability- Characteristic Functions. Central Limit Theorem on R."

## See other formats

```MIT OpenCourseWare
http://ocw.mit.edu

18.175 Theory of Probability

Fall 2008

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .

Section 9

Characteristic Functions. Central
Limit Theorem on R.

Let X = (Xi, . . . , Xk) be a random vector on BJ^ with distribution P and let t = {t\,. . . , tk) € M''. Charac-
teristic function of X is defined by

If X has standard normal distribution A/'(0, 1) and A e M then
For complex A = it, consider analytic function

1 x2

<^(a;) = e'*^^=e"^ for a; e C.
V 27r

By Cauchy's theorem, integral over a closed path is equal to 0. Let us take a closed path x + iO for x from
— oo to +0O and x + it for x from +oo to — oo. Then

-1 poo 2 1

fit) = e'*"-'^rfx=^/ e^*('*+")-5('*+-)'dx

v27r J-oo \/27r J-oo

= ^ e-*'+^*^+5*'-**^-5^'dx = e-T / ^e-*da; = e-'^. (9.0.1)
V^J-oo J ^

If y has normal distribution M(ra, a^) then

Lemma 16 If X is a real-valued r.v. such that E|X|'" < oo for integer r then f{t) G C^{M.) and

f^^\t)=E{iXye''^

for j < r.

Proof. If r = 0, then |e**''^| < 1 implies

f{t) = Ee'*^ ^ Ee'"^ = f{s) if t ^ s,

34

by dominated convergence theorem. This means that / e C(R). If r = 1, E|X| < oo, we can use

t-s

and, therefore, by dominated convergence theorem,

f'{t) = UmE

t - S

Also, by dominated convergence theorem, EiXe**^ e C'(IR), which means that / e C^(M). We proceed by
induction. Suppose that we proved that

f^^\t)=E{iXye''^
and that r = j + 1, E|Xp+^ < oo. Then, we can use that

t-s

so that by dominated convergence theorem = E{iXy+'^e'^^ e C(]R).

□

The main goal of this section is to prove one of the most famous results in Probability Theory.

Theorem 22 (Central Limit Theorem) Consider an i.i.d. sequence {Xi)i>i such thatEXi = 0,EX^ = <
oo and let Sn = X]i<n -^i- Then Snl\fn converges in distribution to A/'(0, cr^).

Lemma 17 We have,

lim Ee v« = e

n—*oo

Proof. By independence.

Ee

J^jEe = (Ee j .

i<n

Since EXf < oo previous lemma implies that (fi{t) G C^{M.) and, therefore.

ip{t) = Ee'*^i = ip{0) + ip'{0)t + ^^p"{0)t^ + o{t^) as t ^ 0.

Since
we get

Finally,

ip{0) = 1, (p'{0) = EiXe'-°-^ = iEX = 0, ^"{0) = E{iXf = -EX^ = -a^

0-2^2

v{t) = l-^+o{t^).

Ee

'n

2n

ft'

e ^ , n ^ oo.

Next, we want to show that characteristic function uniquely determines the distribution. Let X ~
P, F ~ Q be two independent random vectors on R*'. We denote by P * Q the convolution of P and Q which
is the law C{X + Y) of the sum X + F. We have,

= EI{X + y e A) = jjl{x + yG A)dF{x)dQ{y)

= JJi{xgA- y)dF{x)dQ{y) = j ¥{A- y)dq{y).

35

If P has density p then

P * Q(A) = JJl{x + ye A)p{x)dxdQ{y) = JJKz& A)p{z - y)dz
= jj P{z- y)dzdQ{y) = J{Jp{z- y)dQ{y)) dz

A

which means that P * Q has density
If, in addition, Q has density q then

f{x) = J p{x-y)dQ{y). (9.0.2)
= j p{x-

f{x) = p{x- y)q{y)dy.

Denote by 7V^(0, a^I) the law of the random vector X = {Xi , Xf.) of i.i.d. ^(0, cr^) random variables
whose density on M*^ is

k

\x\

1 X'^ ^ 1 l^|2

e 2c

For a distribution P denote P"" = P * J\f{0, a^I).
Lemma 18 P'" = P * J\f{0, a'^I) has density

P''ix)=[^y J me-'(''^^-'4w^dt

where f{t) = / e'(*'^)tflP(a;).

Proof. By (9.0.2), P * A/'(0, ci^/) has density

p^ix) = i^-S' I e-5^^l^-^l'dP(2/).

Using (9.0.1), we can write

and taking a product over i < k we get

(^^y Je-^^i^-y^^)e-'^\^\'dz.

Then we can continue

p^ix) = (^-^y j j e-'^^''-^''^-'^^'^' dzdV{y)

(2^)'' // e"'^^''"'''^^"''^''<ffl"(2/)(^^

Let z = ta.

36

Theorem 23 (Uniqueness) If
then P = Q.

Proof. By the above Lemma, P*" = Q'^. If X ~ P and ~ 7V(0, /) then X + a^^X almost surely as a ^
and, therefore, P"^ ^ P weakly Similarly, Q'^ Q.

□

We proved that the characteristic function of Snl\fn converges to the c.f. of A/'(0, cr^). Also, the sequence

ci—'^ ) - is uniformly tight.

since by Chebyshev's inequality

for large enough M. To finish the proof of the CLT on the real line we apply the following.
Lemma 19 If (P„) is uniformly tight and

fn{t) = J e"^dP„(a;) ^ f{t)

then P„ ^ P and f{t) = J e**^(iP(x).

Proof. For any sequence (n(fc)), by Selection Theorem, there exists a subsequence (n(fc(r))) such that
lPra(fc(r)) converges weakly to some distribution P. Since e'^*'^^ is bounded and continuous.

J e^(*'-)dP„(,(,)) ^ J e*(*'-)rfP(ar)

as r — > oo and, therefore, / is a c.f. of P. By uniqueness theorem, distribution P does not depend on the
sequence {n{k)). By Lemma 13, P„ — > P weakly.

37

``` 