CSE 150 Lecture Notes - Lecture 11: List Of Zeppelins, Product Rule, Bayes Estimator
9 May 2018
School
Department
Course
Professor
Estimating parameters (Hidden Values)
Noisy-OR Model
X1X2… XnXi ∈ {0,1} diseases
\ \ | / P(Y = 1|X1,...,Xn) = 1 - ∏i = 1
n (1 - pi)Xi
Y Y ∈ {0,1} symptom
How to estimate pi ∈ [0,1] from complete data {(Xt,Yt)}t = 1
T
Alternative Model:
X1X2..., Xn
||||
Z1Z2… ZnZi = “noisy copy” of Xi
\ \ | / P(Zi = 0|Xi = 0) = 1P(Zi = 0|Xi) = (1 - pi)Xi = 1 if Xi = 0
Y P(Zi = 1|Xi = 1) = pi1 - pi if Xi = 1
Y = OR(Z1,...,Zn)
P(Y = 1|Z1…,Zn) = 0 if ALL Zi = 0
1 otherwise
What is P(Y = 1|X’) in this new model? X’ = (X1…,Xn), Z’ = (Z1,...Zn)
P(Y = 1|X’) = Σz’ (Y = 1,Z’|X’) [marginalization]
= Σz’ P(Z’|X’) P(Y = 1|Z’,X’) [product rule]
= Σz’ P(Z’|X’) P(Y = 1|Z’) [cond. Ind.]
= [Σz’ != 0’ P(Z’|X’) * 1] + P(Z’ = 0’|X’) P(Y = 1|Z’ = 0’) [logical OR]
= [Σz’ P(Z’|X’)] - P(Z’ = 0|X’) 0
= 1 - P(Z’ = 0’|X’) [probs are normalized]
P(Y = 1|X’) = 1 - ∏i = 1
nP(Zi = 0|Xi)
= 1 - ∏i = 1
n (1 - pi)Xi
This matches P(Y = 1|X’) in the original noisy-or model!
Insight: use hidden variable and EM to estimate pi ∈ [0,1]
First, consider inference in this model:
Idea: let’s swap y and z, but keep x on the RHS (it’s a parent)
Notes:
- P(y|zi = 1,x’) = 0 if y = 0, and 1 if y = 1!
- P(zi = 1|x’) = P(zi = 1|xi) [all other xi’s are d-separated by rule 3]
P(zi = 1|y, x’) = P(y|zi = 1,x’) P(zi = 1|x’) [Bayes Rule]
P(y|x’)
= y pixi
P(y = 1|x’) ← ONLY consider y = 1 b/c numerator vanishes when y = 0
= y pi xi
1 - ∏i = 1
n (1 - pi)Xi
,
Log-likelihood Data {(x’t,y’t)}t = 1
T
Document Summary
How to estimate p i [0,1] from complete data {(x t ,y t )} t = 1 . = [ z" != 0" p(z"|x") * 1] + p(z" = 0"|x") p(y = 1|z" = 0") = 1 - i = 1 n p(z i = 0|x i ) n (1 - p i ) xi. P(z i = 0|x i ) = (1 - p i ) xi = This matches p(y = 1|x") in the original noisy-or model! Insight : use hidden variable and em to estimate p i [0,1] Idea: let"s swap y and z, but keep x on the rhs (it"s a parent) P(y|z i = 1,x") = 0 if y = 0, and 1 if y = 1! P(z i = 1|x") = p(z i = 1| x i ) P(z i = 1|y, x") = p(y|z i = 1,x") p(z i = 1|x") [all other x i "s are d-separated by rule 3]
