MGSC 372 Lecture Notes - Lecture 1: Maximum Likelihood Estimation, Random Variable, Probability Distribution

• School: McGill University
• Department: Management Science
• Course: MGSC 372
• Professor: Brian Smith

Document Summary

Example: estimate the mean of a distribution based on a sample of values (cid:2869),(cid:2870),(cid:2871), ,. Then the least squares estimate of is the value that minimizes the sum of squares. Intuitively, the quantity measures total (squared) deviation from the mean of the population, so its minimum value will be the least squares estimate. A special example of the chain rule is: (cid:4666) (cid:4667)(cid:2870)=(cid:884)(cid:4666) (cid:4667)(cid:4666) (cid:4667)=(cid:884)(cid:4666) (cid:4667)(cid:4666) (cid:883)(cid:4667) To find the value of that minimizes we first calculate the derivative with respect to : (cid:4666)(cid:4667)=(cid:884)(cid:4666)(cid:2869) (cid:4667)(cid:4666) (cid:883)(cid:4667)+(cid:884)(cid:4666)(cid:2870) (cid:4667)(cid:4666) (cid:883)(cid:4667)+(cid:884)(cid:4666)(cid:2871) (cid:4667)(cid:4666) (cid:883)(cid:4667)+ +(cid:884)(cid:4666) (cid:4667)(cid:4666) (cid:883)(cid:4667) (cid:4666)(cid:4667)= (cid:884)[(cid:4666)(cid:2869) (cid:4667)+(cid:4666)(cid:2870) (cid:4667)+(cid:4666)(cid:2871) (cid:4667)+ +(cid:4666) (cid:4667)] (cid:4666)(cid:4667)= (cid:884)[(cid:2869)+(cid:2870)+(cid:2871)+ ] Next, set the derivative to equal 0: (cid:4666)(cid:4667)=(cid:882) (cid:2869)+(cid:2870)+(cid:2871)+ + =(cid:882) =(cid:2869)+(cid:2870)+(cid:2871)+ + Therefore, the least squares estimate of is the sample mean: For a sample of observations, the lse of the population mean is given by the sample mean: Notation: it is common practice in statistics to denote an estimator of a parameter by .