# Full text of "Theory of Probability- Characteristic Functions. Central Limit Theorem on R."

## See other formats

MIT OpenCourseWare http://ocw.mit.edu 18.175 Theory of Probability Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms . Section 9 Characteristic Functions. Central Limit Theorem on R. Let X = (Xi, . . . , Xk) be a random vector on BJ^ with distribution P and let t = {t\,. . . , tk) € M''. Charac- teristic function of X is defined by If X has standard normal distribution A/'(0, 1) and A e M then For complex A = it, consider analytic function 1 x2 <^(a;) = e'*^^=e"^ for a; e C. V 27r By Cauchy's theorem, integral over a closed path is equal to 0. Let us take a closed path x + iO for x from — oo to +0O and x + it for x from +oo to — oo. Then -1 poo 2 1 fit) = e'*"-'^rfx=^/ e^*('*+")-5('*+-)'dx v27r J-oo \/27r J-oo = ^ e-*'+^*^+5*'-**^-5^'dx = e-T / ^e-*da; = e-'^. (9.0.1) V^J-oo J ^ If y has normal distribution M(ra, a^) then Lemma 16 If X is a real-valued r.v. such that E|X|'" < oo for integer r then f{t) G C^{M.) and f^^\t)=E{iXye''^ for j < r. Proof. If r = 0, then |e**''^| < 1 implies f{t) = Ee'*^ ^ Ee'"^ = f{s) if t ^ s, 34 by dominated convergence theorem. This means that / e C(R). If r = 1, E|X| < oo, we can use t-s and, therefore, by dominated convergence theorem, f'{t) = UmE t - S Also, by dominated convergence theorem, EiXe**^ e C'(IR), which means that / e C^(M). We proceed by induction. Suppose that we proved that f^^\t)=E{iXye''^ and that r = j + 1, E|Xp+^ < oo. Then, we can use that t-s so that by dominated convergence theorem = E{iXy+'^e'^^ e C(]R). □ The main goal of this section is to prove one of the most famous results in Probability Theory. Theorem 22 (Central Limit Theorem) Consider an i.i.d. sequence {Xi)i>i such thatEXi = 0,EX^ = < oo and let Sn = X]i<n -^i- Then Snl\fn converges in distribution to A/'(0, cr^). We will start with the following. Lemma 17 We have, lim Ee v« = e n—*oo Proof. By independence. Ee J^jEe = (Ee j . i<n Since EXf < oo previous lemma implies that (fi{t) G C^{M.) and, therefore. ip{t) = Ee'*^i = ip{0) + ip'{0)t + ^^p"{0)t^ + o{t^) as t ^ 0. Since we get Finally, ip{0) = 1, (p'{0) = EiXe'-°-^ = iEX = 0, ^"{0) = E{iXf = -EX^ = -a^ 0-2^2 v{t) = l-^+o{t^). Ee 'n 2n ft' e ^ , n ^ oo. Next, we want to show that characteristic function uniquely determines the distribution. Let X ~ P, F ~ Q be two independent random vectors on R*'. We denote by P * Q the convolution of P and Q which is the law C{X + Y) of the sum X + F. We have, = EI{X + y e A) = jjl{x + yG A)dF{x)dQ{y) = JJi{xgA- y)dF{x)dQ{y) = j ¥{A- y)dq{y). 35 If P has density p then P * Q(A) = JJl{x + ye A)p{x)dxdQ{y) = JJKz& A)p{z - y)dz = jj P{z- y)dzdQ{y) = J{Jp{z- y)dQ{y)) dz A which means that P * Q has density If, in addition, Q has density q then f{x) = J p{x-y)dQ{y). (9.0.2) = j p{x- f{x) = p{x- y)q{y)dy. Denote by 7V^(0, a^I) the law of the random vector X = {Xi , Xf.) of i.i.d. ^(0, cr^) random variables whose density on M*^ is k \x\ 1 X'^ ^ 1 l^|2 e 2c For a distribution P denote P"" = P * J\f{0, a^I). Lemma 18 P'" = P * J\f{0, a'^I) has density P''ix)=[^y J me-'(''^^-'4w^dt where f{t) = / e'(*'^)tflP(a;). Proof. By (9.0.2), P * A/'(0, ci^/) has density p^ix) = i^-S' I e-5^^l^-^l'dP(2/). Using (9.0.1), we can write and taking a product over i < k we get (^^y Je-^^i^-y^^)e-'^\^\'dz. Then we can continue p^ix) = (^-^y j j e-'^^''-^''^-'^^'^' dzdV{y) (2^)'' // e"'^^''"'''^^"''^''<ffl"(2/)(^^ Let z = ta. 36 Theorem 23 (Uniqueness) If then P = Q. Proof. By the above Lemma, P*" = Q'^. If X ~ P and ~ 7V(0, /) then X + a^^X almost surely as a ^ and, therefore, P"^ ^ P weakly Similarly, Q'^ Q. □ We proved that the characteristic function of Snl\fn converges to the c.f. of A/'(0, cr^). Also, the sequence ci—'^ ) - is uniformly tight. since by Chebyshev's inequality for large enough M. To finish the proof of the CLT on the real line we apply the following. Lemma 19 If (P„) is uniformly tight and fn{t) = J e"^dP„(a;) ^ f{t) then P„ ^ P and f{t) = J e**^(iP(x). Proof. For any sequence (n(fc)), by Selection Theorem, there exists a subsequence (n(fc(r))) such that lPra(fc(r)) converges weakly to some distribution P. Since e'^*'^^ is bounded and continuous. J e^(*'-)dP„(,(,)) ^ J e*(*'-)rfP(ar) as r — > oo and, therefore, / is a c.f. of P. By uniqueness theorem, distribution P does not depend on the sequence {n{k)). By Lemma 13, P„ — > P weakly. 37