Study Guides (256,509)
CA (124,674)
UTSG (8,518)
STA (81)
STA261H1 (2)
N/ A (1)

Summary notes

23 Pages
287 Views

Department
Statistical Sciences
Course Code
STA261H1
Professor
N/ A

This preview shows pages 1-3. Sign up to view the full 23 pages of the document.
STA261H1.doc
Page 1 of 23
Statistical Inference
RANDOM SAMPLE
Definition: Random Sample
n
XX ,,
1 is called a random sample from a distribution with pdf
(
)
xf (or pf
(
)
xP) if n
XX ,,
1 are
independent and have identical distribution with pdf
(
)
xf (or pf
(
)
xP). It is often denoted iid
(independent-identically-distributed).
Note
Let n
XX ,,
1 be a sample from a distribution with pdf
(
)
xf. The joint pdf of
(
)
n
XXX ,,
1= is
(
)
(
)
(
)
(
)
(
)
nnnn xfxfxfxfxxf1111 ,== .
SAMPLE MEAN
Definition: Sample Mean and Sample Variance
Let n
XX ,,
1 be a random sample. The sample mean is defined by n
XX
n
X
Xn
n
i
i
n
++
==
=
11 , and the
sample variance is defined by
( )
1
12
=
=
n
XX
S
n
i
ni
.
Theorem
If n
XX ,,
1 are iid each with a
(
)
2
,
σµ
N distribution, then nn XkXk++
11 has a normal distribution
with mean
(
)
µµµ
nn kkkk ++=++ 11 and variance
(
)
2
22
1
σ
n
kk ++.
More generally, if n
XX ,,
1 are independent and each i
X has a
(
)
2
,ii
N
σµ
, then nn XkXk++
11 has a
normal distribution mean nn
kk
µµ
++
11 and variance 222
1
2
1nn
kk
σσ
++.
USEFUL DISTRIBUTION
Theorem
Suppose n
XX ,,
1 is a random sample from
(
)
2
,
σµ
N distribution. Then
n
X and
( )
=
n
i
niXX
1
2 are
independent, and
( )
2
1
2
σ
=
n
i
niXX
has a 2
1n
χ
distribution.
www.notesolution.com
STA261H1.doc
Page 2 of 23
The t and F Distribution
Two important distributions useful in statistical inference are:
1) If
(
)
1,0~NW and 2
~r
V
χ
are independent, then
r
V
W has a t-distribution.
2) 2
1
~r
U
χ
and 2
2
~r
V
χ
are independent, then
2
1
r
V
r
U
has a F-distribution.
THE CENTRAL LIMIT THEOREM
Let n
XX ,,
1 is a random sample with finite mean
µ
and variance 0
2>
σ
. Let nn XXS++=
1. Then
(
)
( )
(
)
( )
( )
1,0
var
var 2
N
n
X
X
XEX
S
SES
n
n
n
nn
n
nn
=
=
σ
µ
.
STATISTICAL MODEL
Consider a random sample n
XX ,,
1 from a distribution with pdf
(
)
xf
θ
. The family
(
)
{
}
θ
θ
|xf (where
(
)
xf
θ
is pdf (or
(
)
xP
θ
a pf),
θ
is an unknown parameter, is a parameter space) is a statistical model.
We know that the distribution under investigation is in the family, but dont know which one. Based on the
sample values n
xx ,,
1, we find an estimate for
θ
; Once we find
θ
, we know the distribution.
LIKELIHOOD FUNCTION
Definition: The Likelihood Function
Let n
XX ,,
1 be a random sample from a distribution with pdf
(
)
xf
θ
(or pf
(
)
xP
θ
). The likelihood is
defined by R
:L given by
(
)
(
)
(
)
(
)
nnn xfxcfxxcfxxL
θθθ
θ
111 ,,,|== where 0
>
c(or
(
)
(
)
(
)
(
)
nnn xPxcPxxcPxxL
θθθ
θ
111 ,,,|== ).
Definition: The Maximum Likelihood Estimate (MLE)
The function S:
ˆ
θ
is called the maximum likelihood estimator.
(
)
s
θ
ˆ is called the maximum likelihood estimate of
θ
if for each
θ
,
(
)
(
)
(
)
nn xxLxxsL,,|,,|
ˆ11
θθ
.
The Algorithm
This suggests that in order to obtain the MLE of
θ
, we maximum the likelihood function. Since a version of
the likelihood version with 1
=
c gives the same maximum value, we use this version. In most cases, this is
done by differentiation.
1) Write the likelihood function
( ) ( )
=
=
n
i
inxfxxL
1
1,|
θ
θ
.
www.notesolution.com
STA261H1.doc
Page 3 of 23
2) Write the log likelihood function defined by
( ) ( )( ) ( )
=
==
n
i
inn xfxxLxxl
1
11 ln,|ln,|
θ
θθ
.
3) Write the score function
( )
(
)
θ
θ
θ
=n
n
xxl
xxS,|
,|1
1
.
4) Write the score equation
(
)
0,|1=
n
xxS
θ
and solve for
θ
.
5) Check that the solution is the global maximum. If it is, then it is the MLE of
θ
.
Theorem
If
(
)
n
xx ,
ˆ1
θ
is the MLE in , and
1-1
:
φ
, then the MLE in the new parameterization is
(
)
(
)
(
)
nn xxxx ,
ˆ
,
ˆ11
θθφ
=.
Proof:
(
)
(
)
( )
(
)
( )
( )
(
)
( )
( ) ( )
( )
( ) ( ) ( ) ( )
nnnn
nnn
xx
n
xx
n
xx
nn
xxLxxgxxfxxL
xxxxLxxf
xxgxxgxxxxL
n
nn
,,|,,,,,,|
,,|,,
ˆ
,,
,,,,,,|,,
ˆ
1
*
111
111
,,
ˆ
1
,,
ˆ
1
,,
ˆ
11
*
1
11
θθ
θ
θ
θθ
θ
θφθ
===
==
==
Hence for every
θ
,
(
)
(
)
(
)
nnn xxLxxxxL,,|,,|,,
ˆ1
*
11
*
θθ
, and so
(
)
n
xx ,,
ˆ1
θ
is the MLE
of the new parameterization.
The Algorithm: The Multidimensional Case
In the multidimensional case, the parameter space is
(
)
{
}
1,,,
1>=k
k
θθ
.
1) Write the likelihood function
(
)
(
)
nkxxL,|,, 11
θθ
.
2) Write the log likelihood function defined by
(
)
(
)
(
)
(
)
(
)
nknkxxLxxl,|,,ln,|,, 1111
θθθθ
=.
3) Write the score function
( )( )
(
)
(
)
( )( )
=
k
nk
nk
nkxxl
xxl
xxS
θ
θθ
θ
θθ
θθ
,|,,
,|,,
,|,,
11
1
11
11
.
4) Write the score equation
( )( )
(
)
(
)
( )( )
=
=
0
0
,|,,
,|,,
0,|,,
11
1
11
11
k
nk
nk
nkxxl
xxl
xxS
θ
θθ
θ
θθ
θθ
and solve.
5) Check that the solutions are the global maximum (the matrix of the second partial derivatives evaluated at
(
)
k
θθ
ˆ
,,
ˆ1 must be negative definite, or equivalently, all eigenvalues negative).
STANDARD ERROR AND BIAS
Suppose
θ
ˆ is the MLE;
(
)
(
)
(
)
nn xxxx ,,
ˆ
,,
ˆ11
θφφ
= is the estimate of
(
)
θφ
. How reliable are the estimates?
One measure of accuracy commonly used is MSE (mean squared error).
Definition: Mean Squared Error
www.notesolution.com

Loved by over 2.2 million students

Over 90% improved by at least one letter grade.

Leah — University of Toronto

OneClass has been such a huge help in my studies at UofT especially since I am a transfer student. OneClass is the study buddy I never had before and definitely gives me the extra push to get from a B to an A!

Leah — University of Toronto
Saarim — University of Michigan

Balancing social life With academics can be difficult, that is why I'm so glad that OneClass is out there where I can find the top notes for all of my classes. Now I can be the all-star student I want to be.

Saarim — University of Michigan
Jenna — University of Wisconsin

As a college student living on a college budget, I love how easy it is to earn gift cards just by submitting my notes.

Jenna — University of Wisconsin
Anne — University of California

OneClass has allowed me to catch up with my most difficult course! #lifesaver

Anne — University of California
Description
STA261H1.doc Statistical Inference R ANDOM SAMPLE Definition: Random Sample X 1, X n is called a random sample from a distribution with pxf)(or pf Px )) if 1 ,, Xn are independent and have identical distribution with pxf)(or pf Px )). It is often denoted iid (independent-identically-distributed). Note Let X 1, X n be a sample from a distribution with pdx). The joint pdf of X (X1,, X n )is f(x1,x n )= f1(x1 f n(xn) = f(x1 f (xn ). S AMPLE M EAN Definition: Sample Mean and Sample Variance n X i i=1 X1 + + X n Let X 1, X n be a random sample. The sample mean is defined by n = n = n , and the n (X i X n ) 2 i= sample variance is defined bS = n 1 . Theorem 2 If X 1, X nare iid each with a )distribution, thk1X 1+k X n n has a normal distribution 2 2 2 with mean k1++k n = k 1+k n) and variance k1 ++k n . 2 More generally, ifX1,, X n are independent and eachX ihas a N i, i ), thenk1X1++k X n nhas a 2 2 2 2 normal distribution meank1 1 ++k n n and variancek1 1 ++k n n . U SEFUL D ISTRIBUTION Theorem n Suppose X ,, X is a random sample fromN , 2 )distribution. Then X and (X X 2 )re 1 n n i n i=1 n (X i Xn 2 ) i 1 2 independent, and 2 has a n1distribution. Page 1 of 23 www.notesolution.com STA261H1.doc The t and F Distribution Two important distributions useful in statistical inference are: 1) IfW ~ N 0,1)and V ~ 2 are independent, thenW has a t-distribution. r V r U 2) U ~ r and V ~ r are independent, then r1 has a F-distribution. 1 2 V r2 T HE C ENTRAL L IMITT HEOREM Let X 1 , Xn is a random sample with finite meanand variance 2> 0 . Let n =X 1++ X n Then Sn E S n = Xn E X)n = X n N(0,1 ) var S ) 2 n n var Xn ) n S TATISTICAL M ODEL Consider a random sample X 1 , n from a distribution with pf(x). The family{f(x) } (where f(x) is pdf (oP(x)a pf), is an unknown parameter, is a parameter space) is a statistical model. We know that the distribution under investigation is in the family, but dont know which one. Based on the sample values x1, , n, we find an estimate for ; Once we find , we know the distribution. L IKELIHOOD FUNCTION Definition: The Likelihood Function Let X , , X be a random sample from a distribution with fd(x)(or pfP x )). The likelihood is 1 n defined by L: R given by L 1 , n)= c (x1, , n)= c (x1 f(xn where c > 0(or L x , x)= cP x , , x)= cP x( ) P x ) ). 1 n 1 n 1 n Definition: The Maximum Likelihood Estimate (MLE) The function : S is called the maximum likelihood estimator. (s)is called the maximum likelihood estimate of if fo ch,L (s 1 , , n) L 1 , , n). The Algorithm This suggests that in order to obtain the MLE of , we maximum the likelihood function. Since a version of the likelihood version wic =1 gives the same maximum value, we use this version. In most cases, this is done by differentiation. n 1) Write the likelihood function 1 , xn = ) f xi . ( ) i1 Page 2 of 23 www.notesolution.com
More Less
Unlock Document


Only pages 1-3 are available for preview. Some parts have been intentionally blurred.

Unlock Document
You're Reading a Preview

Unlock to view full version

Unlock Document

Log In


OR

Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


OR

By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.


Submit