Department

Statistical Sciences

Course Code

STA437H1

Professor

Radford Neal

Notes forSTA437/1005 —Methods for MultivariateData

RadfordM. Neal, 26 November 2010

Random Vectors

Notation:

Let Xbearandom vectorwith pelements, so thatX=[X1,...,Xp]′,where ′denotes

transpose.(By convention, our vectors are column vectors unlessotherwise indicated.)

Wedenote aparticular realized value of Xbyx.

Expection:

The expectation (expected value, mean) ofarandomvector Xis E(X)=Rxf(x)dx,

where f(x)is the jointprobabilitydensityfunctionfor the distributionofX.

Weoften denote E(X)byµ,with µj=E(Xj)being the expectationof the j’th element

of X.

Variance:

The variance ofthe randomvariable Xjis Var(Xj)=E[(Xj−E(Xj))2], whichwesome-

times writeasσ2

j.

The standard deviation of Xjis pVar(Xj)=σj.

Covariance and correlation:

The covariance ofXjand Xkis Cov(Xj,Xk)=E[(Xj−E(Xj))(Xk−E(Xk))], whichwe

sometimes writeasσjk.Note that Cov(Xj,Xj)is the variance of Xj,soσjj =σ2

j.

The correlationof Xjand Xkis Cov(Xj,Xk)/(σjσk), whichwesometimes write as ρjk.

Notethatcorrelations are alwaysbetween −1and +1, and ρjj is alwaysone.

Covariance and correlation matrices:

The covariances for all pairs ofelements of X=[X1,...,Xp]′can beput in amatrix called

the covariance matrix:

Σ=

σ11 σ12 · · · σ1p

σ21 σ22 · · · σ2p

.

.

..

.

..

.

..

.

.

σp1σp2· · · σpp

Notethatthe covariancematrix is symmetrical, with the variances of the elementson the

diagonal.

The covariance matrix can alsobewritten asΣ=E[(X−E(X)) (X−E(X))′].

Similarly,the correlations can beput into a a symmetrical correlation matrix, whichwill

haveones on the diagonal.

1

www.notesolution.com

MultivariateSampleStatistics

Notation:

Suppose wehavenobservations, eachwith values for pvariables. Wedenote the value of

variable jin observation ibyxij,and the vector of all values for observationibyxi.

Weoften view the observed xias arandomsample ofrealizations of arandomvector X

with some (unknown) distribution.

The is potential ambiguitybetween the notation xifor observation i,and the notation xj

for arealization of the randomvariable Xj.(The textbook uses bold face forxi.)

Iwill (try to) reserveifor indexing observations, and usejand kfor indexing variables,

but the textbook somtimes uses ito index avariable.

Sample means:

The sample mean of variable jis ¯xj=1

n

n

P

i=1

xij.

The sample mean vector is ¯x=[¯x1,...,¯xp]′.

If the observations all havethe same distribution, the sample meanvector, ¯x,is an unbiased

estimate of the meanvector, µ,of the distribution from whichthese observations came.

Sample variances:

The sample varianceof variable jis s2

j=1

n−1

n

P

i=1

(xij −¯xj)2.

If the observations all havethe same distribution, the sample variance, s2

j,is anestimate

of the variance, σ2

j,of the distribution for Xj,and will bean unbiasedestimate if the

observations are independent.

Sample covariance and correlation:

The sample covariance ofvariable jwith variable kis 1

n−1

n

P

i=1

(xij −¯xj)(xik −¯xk).

The sample covariance is denoted bysjk.Note that sjj equals s2

j,the sample variance of

variable j.

The sample correlation of variable jwith variable kis sjk/(sjsk), often denoted byrjk.

Sample covariance and correlation matrices:

The sample covariances maybearranged as the sample covariance matrix:

S=

s11 s12 · · · s1p

s21 s22 · · · s2p

.

.

..

.

..

.

..

.

.

sp1sp2· · · spp

The sample covariance matrix can also becomputed asS=1

n−1

n

P

i=1

(xi−¯x)(xi−¯x)′.

Similarly,the sample correlations maybearranged as the samplecorrelation matrix, some-

times denoted R(though the textbookalso uses Rfor the population correlationmatrix).

2

www.notesolution.com

Over 90% improved by at least one letter grade.

OneClass has been such a huge help in my studies at UofT especially since I am a transfer student. OneClass is the study buddy I never had before and definitely gives me the extra push to get from a B to an A!

Leah — University of Toronto

Balancing social life With academics can be difficult, that is why I'm so glad that OneClass is out there where I can find the top notes for all of my classes. Now I can be the all-star student I want to be.

Saarim — University of Michigan

As a college student living on a college budget, I love how easy it is to earn gift cards just by submitting my notes.

Jenna — University of Wisconsin

OneClass has allowed me to catch up with my most difficult course! #lifesaver

Anne — University of California

Join OneClass

Access over 10 million pages of study

documents for 1.3 million courses.

Sign up

Join to view

OR

By registering, I agree to the
Terms
and
Privacy Policies

Already have an account?
Log in

Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.