# ECE 313 Chapter 1-5: probabilityJan15

139 views289 pages
Published on 16 Jul 2015
School
University of Illinois
Department
Electrical and Computer Engineering
Course
ECE 313
Professor
Probability with Engineering Applications
ECE 313 Course Notes
Bruce Hajek
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
January 2015
c
ī2015 by Bruce Hajek
All rights reserved. Permission is hereby given to freely print and circulate copies of these notes so long as the notes are left
intact and not reproduced for commercial purposes. Email to b-hajek@illinois.edu, pointing out errors or hard to understand
passages or providing comments, is welcome.
Unlock document

This preview shows pages 1-3 of the document.
Unlock all 289 pages and 3 million more documents.

Contents
1 Foundations 3
1.1 Embracinguncertainty................................... 3
1.2 Axiomsofprobability ................................... 6
1.3 Calculating the size of various sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Probability experiments with equally likely outcomes . . . . . . . . . . . . . . . . . . 13
1.5 Sample spaces with inļ¬nite cardinality . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7 Problems .......................................... 21
2 Discrete-type random variables 25
2.1 Random variables and probability mass functions . . . . . . . . . . . . . . . . . . . . 25
2.2 The mean and variance of a random variable . . . . . . . . . . . . . . . . . . . . . . 27
2.3 Conditionalprobabilities.................................. 32
2.4 Independence and the binomial distribution . . . . . . . . . . . . . . . . . . . . . . . 34
2.4.1 Mutually independent events . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4.2 Independent random variables (of discrete-type) . . . . . . . . . . . . . . . . 36
2.4.3 Bernoulli distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.4.4 Binomial distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.5 Geometricdistribution................................... 41
2.6 Bernoulli process and the negative binomial distribution . . . . . . . . . . . . . . . . 43
2.7 The Poisson distributionāa limit of binomial distributions . . . . . . . . . . . . . . . 45
2.8 Maximum likelihood parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . 47
2.9 Markov and Chebychev inequalities and conļ¬dence intervals . . . . . . . . . . . . . . 50
2.10 The law of total probability, and Bayes formula . . . . . . . . . . . . . . . . . . . . . 53
2.11 Binary hypothesis testing with discrete-type observations . . . . . . . . . . . . . . . 60
2.11.1 Maximum likelihood (ML) decision rule . . . . . . . . . . . . . . . . . . . . . 61
2.11.2 Maximum a posteriori probability (MAP) decision rule . . . . . . . . . . . . . 62
2.12Reliability.......................................... 67
2.12.1 Unionbound .................................... 67
2.12.2 Network outage probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.12.3 Distribution of the capacity of a ļ¬ow network . . . . . . . . . . . . . . . . . . 70
2.12.4 Analysis of an array code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
iii
Unlock document

This preview shows pages 1-3 of the document.
Unlock all 289 pages and 3 million more documents.

iv CONTENTS
2.12.5 Reliability of a single backup . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.14Problems .......................................... 76
3 Continuous-type random variables 95
3.1 Cumulative distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2 Continuous-type random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.3 Uniformdistribution....................................103
3.4 Exponentialdistribution..................................104
3.5 Poissonprocesses......................................106
3.5.1 Time-scaled Bernoulli processes . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.5.2 Deļ¬nition and properties of Poisson processes . . . . . . . . . . . . . . . . . . 108
3.5.3 The Erlang distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.6 Linear scaling of pdfs and the Gaussian distribution . . . . . . . . . . . . . . . . . . 113
3.6.1 Scalingruleforpdfs ................................113
3.6.2 The Gaussian (normal) distribution . . . . . . . . . . . . . . . . . . . . . . . 115
3.6.3 The central limit theorem and the Gaussian approximation . . . . . . . . . . 119
3.7 ML parameter estimation for continuous-type variables . . . . . . . . . . . . . . . . . 124
3.8 Functions of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
3.8.1 The distribution of a function of a random variable . . . . . . . . . . . . . . . 125
3.8.2 Generating a random variable with a speciļ¬ed distribution . . . . . . . . . . 135
3.8.3 The area rule for expectation based on the CDF . . . . . . . . . . . . . . . . 137
3.9 Failureratefunctions....................................138
3.10 Binary hypothesis testing with continuous-type observations . . . . . . . . . . . . . . 140
3.12Problems ..........................................147
4 Jointly Distributed Random Variables 161
4.1 Joint cumulative distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . 161
4.2 Joint probability mass functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
4.3 Joint probability density functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
4.4 Independence of random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
4.4.1 Deļ¬nition of independence for two random variables . . . . . . . . . . . . . . 175
4.4.2 Determining from a pdf whether independence holds . . . . . . . . . . . . . . 176
4.5 Distribution of sums of random variables . . . . . . . . . . . . . . . . . . . . . . . . . 178
4.5.1 Sums of integer-valued random variables . . . . . . . . . . . . . . . . . . . . . 178
4.5.2 Sums of jointly continuous-type random variables . . . . . . . . . . . . . . . . 181
4.6 Additional examples using joint distributions . . . . . . . . . . . . . . . . . . . . . . 184
4.7 Joint pdfs of functions of random variables . . . . . . . . . . . . . . . . . . . . . . . 188
4.7.1 Transformation of pdfs under a linear mapping . . . . . . . . . . . . . . . . . 189
4.7.2 Transformation of pdfs under a one-to-one mapping . . . . . . . . . . . . . . 190
4.7.3 Transformation of pdfs under a many-to-one mapping . . . . . . . . . . . . . 194
4.8 Correlation and covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Unlock document

This preview shows pages 1-3 of the document.
Unlock all 289 pages and 3 million more documents.