COMPSCI C8 Lecture Notes - Lecture 36: Total Variation Distance Of Probability Measures, Absolute Difference, Test Statistic

• School: Berkeley
• Department: Computer Science
• Course: COMPSCI C8
• Professor: John Denero

Document Summary

Two random samples: sample a, sample b. Null: the two samples are drawn from the same underlying population distribution; they look like two random draws from the same set. Alter(cid:374)ati(cid:448)e: the sa(cid:373)ples are dra(cid:449)(cid:374) fro(cid:373) differe(cid:374)t distri(cid:271)utio(cid:374)s; they do(cid:374)"t look like random draws from the same set. Null: the two samples are drawn randomly from the same underlying distribution. If the null is true, all rearrangements of the variable values among the two samples are equally likely. So: compute the observed test statistic, then shuffle the attribute values and recompute the statistic; repeat; compare with the observed statistic. If the samples are categorical, then a natural test statistic is the total variation distance. It measures the difference between the distributions in the two samples. If the samples are numerical, often a simpler statistic is just fine, such as the absolute difference between the two sample means.