Class Notes (921,692)
CA (542,563)
UTSG (45,898)
STA (305)
STA437H1 (10)
Lecture

Online notes

5 Pages
236 Views

Department
Statistical Sciences
Course Code
STA437H1
Professor
Radford Neal

This preview shows pages 1-2. Sign up to view the full 5 pages of the document.
Notes forSTA437/1005 Methods for MultivariateData
RadfordM. Neal, 26 November 2010
Random Vectors
Notation:
Let Xbearandom vectorwith pelements, so thatX=[X1,...,Xp],where denotes
transpose.(By convention, our vectors are column vectors unlessotherwise indicated.)
Wedenote aparticular realized value of Xbyx.
Expection:
The expectation (expected value, mean) ofarandomvector Xis E(X)=Rxf(x)dx,
where f(x)is the jointprobabilitydensityfunctionfor the distributionofX.
Weoften denote E(X)byµ,with µj=E(Xj)being the expectationof the j’th element
of X.
Variance:
The variance ofthe randomvariable Xjis Var(Xj)=E[(XjE(Xj))2], whichwesome-
times writeasσ2
j.
The standard deviation of Xjis pVar(Xj)=σj.
Covariance and correlation:
The covariance ofXjand Xkis Cov(Xj,Xk)=E[(XjE(Xj))(XkE(Xk))], whichwe
sometimes writeasσjk.Note that Cov(Xj,Xj)is the variance of Xj,soσjj =σ2
j.
The correlationof Xjand Xkis Cov(Xj,Xk)/(σjσk), whichwesometimes write as ρjk.
Notethatcorrelations are alwaysbetween 1and +1, and ρjj is alwaysone.
Covariance and correlation matrices:
The covariances for all pairs ofelements of X=[X1,...,Xp]can beput in amatrix called
the covariance matrix:
Σ=
σ11 σ12 · · · σ1p
σ21 σ22 · · · σ2p
.
.
..
.
..
.
..
.
.
σp1σp2· · · σpp
Notethatthe covariancematrix is symmetrical, with the variances of the elementson the
diagonal.
The covariance matrix can alsobewritten asΣ=E[(XE(X)) (XE(X))].
Similarly,the correlations can beput into a a symmetrical correlation matrix, whichwill
haveones on the diagonal.
1
www.notesolution.com
MultivariateSampleStatistics
Notation:
Suppose wehavenobservations, eachwith values for pvariables. Wedenote the value of
variable jin observation ibyxij,and the vector of all values for observationibyxi.
Weoften view the observed xias arandomsample ofrealizations of arandomvector X
with some (unknown) distribution.
The is potential ambiguitybetween the notation xifor observation i,and the notation xj
for arealization of the randomvariable Xj.(The textbook uses bold face forxi.)
Iwill (try to) reserveifor indexing observations, and usejand kfor indexing variables,
but the textbook somtimes uses ito index avariable.
Sample means:
The sample mean of variable jis ¯xj=1
n
n
P
i=1
xij.
The sample mean vector is ¯x=[¯x1,...,¯xp].
If the observations all havethe same distribution, the sample meanvector, ¯x,is an unbiased
estimate of the meanvector, µ,of the distribution from whichthese observations came.
Sample variances:
The sample varianceof variable jis s2
j=1
n1
n
P
i=1
(xij ¯xj)2.
If the observations all havethe same distribution, the sample variance, s2
j,is anestimate
of the variance, σ2
j,of the distribution for Xj,and will bean unbiasedestimate if the
observations are independent.
Sample covariance and correlation:
The sample covariance ofvariable jwith variable kis 1
n1
n
P
i=1
(xij ¯xj)(xik ¯xk).
The sample covariance is denoted bysjk.Note that sjj equals s2
j,the sample variance of
variable j.
The sample correlation of variable jwith variable kis sjk/(sjsk), often denoted byrjk.
Sample covariance and correlation matrices:
The sample covariances maybearranged as the sample covariance matrix:
S=
s11 s12 · · · s1p
s21 s22 · · · s2p
.
.
..
.
..
.
..
.
.
sp1sp2· · · spp
The sample covariance matrix can also becomputed asS=1
n1
n
P
i=1
(xi¯x)(xi¯x).
Similarly,the sample correlations maybearranged as the samplecorrelation matrix, some-
times denoted R(though the textbookalso uses Rfor the population correlationmatrix).
2
www.notesolution.com

Loved by over 2.2 million students

Over 90% improved by at least one letter grade.

Leah — University of Toronto

OneClass has been such a huge help in my studies at UofT especially since I am a transfer student. OneClass is the study buddy I never had before and definitely gives me the extra push to get from a B to an A!

Leah — University of Toronto
Saarim — University of Michigan

Balancing social life With academics can be difficult, that is why I'm so glad that OneClass is out there where I can find the top notes for all of my classes. Now I can be the all-star student I want to be.

Saarim — University of Michigan
Jenna — University of Wisconsin

As a college student living on a college budget, I love how easy it is to earn gift cards just by submitting my notes.

Jenna — University of Wisconsin
Anne — University of California

OneClass has allowed me to catch up with my most difficult course! #lifesaver

Anne — University of California
Description
Notes for STA 437/1005 — Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: ′ ′ Let X be a random vector with p elements, so that X = [X ,...,X ] , 1here p denotes transpose. (By convention, our vectors are column vectors unless otherwise indicated.) We denote a particular realized value of X by x. Expection: ▯ The expectation (expected value, mean) of a random vector X is E(X) = xf(x)dx, where f(x) is the joint probability density function for the distribution of X. We often denote E(X) by µ, with µ = E(X j being thj expectation of the j’th element of X. Variance: 2 The variance of the random variable X is Vaj(X ) = E[(j −E(X )) ], jhich wejsome- times write as σ 2. j ▯ The standard deviation of X is j Var(X )j= σ . j Covariance and correlation: The covariance of X and X is Cov(X ,X ) = E[(X −E(X ))(X −E(X ))], which we j k j k j j k k 2 sometimes write as σ . jkte that Cov(X ,X ) isjthejvariance of X , so σ j jj= σ j The correlation of X anj X is Cok(X ,X )/(σ σj), khichjweksometimes write as ρ . jk Note that correlations are always between −1 and +1, and ρ jj is always one. Covariance and correlation matrices: The covariances for all pairs of elements of X = [X ,..1,X ] p′ can be put in a matrix called the covariance matrix:   σ11 σ 12 ··· σ 1p    σ21 σ 22 ··· σ 2p  Σ =  . . . .   . . . .  σp1 σ p2 ··· σ pp Note that the covariance matrix is symmetrical, with the variances of the elements on the diagonal. ′ The covariance matrix can also be written as Σ = E [(X − E(X))(X − E(X)) ]. Similarly, the correlations can be put into a a symmetrical correlation matrix, which will have ones on the diagonal. 1 www.notesolution.com Multivariate Sample Statistics Notation: Suppose we have n observations, each with values for p variables. We denote the value of variable j in observation i by x ijand the vector of all values for observation i by x .i We often view the observed x as i random sample of realizations of a random vector X with some (unknown) distribution. The is potential ambiguity between the notation x for ibservation i, and the notation x j for a realization of the random variable X .j(The textbook uses bold face for x .) i I will (try to) reserve i for indexing observations, and use j and k for indexing variables, but the textbook somtimes uses i to index a variable. Sample means: ▯n The sample mean of variable j is x ¯ = 1 x . j n i=1 ij The sample mean vector is x ¯ = [x ¯ ,...,x¯ ] . 1 p If the observations all have the same distribution, the sample mean vector, x ¯, is an unbiased estimate of the mean vector, µ, of the distribution from which these observations came. Sample variances: ▯n The sample variance of variable j is s j = n−1 (xij x ¯j) . i=1 If the observations all have the same distribution, the sample variance, s , js an estimate 2 of the variance, σ j of the distribution for X , jnd will be an unbiased estimate if the observations are independent. Sample covariance and correlation: ▯n The sample covariance of variable j with variable k is 1 (x −x ¯ )(x −x ¯ ). n−1i=1 ij j ik k The sample covariance is denoted by s . Note that s equals s , the sample variance of jk jj j variable j. The sample correlation of variable j with variable k is s /jk s j koften denoted by r . jk Sample covariance and correlation matrices: The sample covariances may be arranged as the sample covariance matrix:   s11 s12 ··· s 1p  s21 s22 ··· s 2p  S =  . . . .   . . . .  sp1 sp2 ··· s pp 1 ▯n ′ The sample covariance matrix can also be computed as S = n−1 (xi− x¯)(x i x¯) . i=1 Similarly, the sample correlations may be arranged as the sample correlation matrix, some- times denoted R (though the textbook also uses R for the population correlation matrix). 2 www.notesolution.com Linear Combinations of Random Variables ′ Define the random variable Y = a X +a X +1··1a X 2 w2ich can be pripten as Y = a X, where a = [a ,1 ,.2.,a ] .p ′ ′ ′ Then one can show that E(Y ) = a µ and Var(Y ) = a Σa, where µ = E(X) and Σ is the covariance matrix for X. For a random vector of dimension q defined as Y = AX, with A being a q × p matrix, one can show that E(Y ) = Aµ and Var(Y ) = AΣA , where Var(Y ) is the covariance matrix of Y . Similarly, if x is the i’th observed vector, and we de
More Less
Unlock Document


Only pages 1-2 are available for preview. Some parts have been intentionally blurred.

Unlock Document
You're Reading a Preview

Unlock to view full version

Unlock Document

Log In


OR

Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


OR

By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.


Submit