Computer Science 4442A/B Lecture Notes - Lecture 5: Linear Classifier, Joule, Wfor-Tv

• School: Western University
• Department: Computer Science
• Course: Computer Science 4442A/B
• Professor: Olga Veksler

Document Summary

Optimization: how to minimize a function of a single variable. Gradient direction: j(x) is the direction of the steepest increase of function j(x) Gradient direction in 1d: - j(x) points in the direction of the steepest decrease, gradient is just derivative in 1d. Gradient direction: 1d: just a derivative (example above) Let x = 3 d ( ) 4. Let a = [10, 5] ( ) 10 aj. J(x1, x2) =(x1-5)2+(x2-10)2 ( ) 10 global min aj. J(x1, x2) =(x1-5)2+(x2-10)2 x2: given j(x1, x2) = (x1 - 5)2 + (x2 - 10)2, which step size to take, which step size to take, how do we select the step size, controlled by parameter . 5: controlled by parameter a (the learning rate, controlled by parameter . From previous example: a = [10 5] a = [10 5] Let = 0. 2 a - j(a) = [10 5]+0. 2 [-10 10]=[8 7] a - j(a) = [10 5]+0. 2 [-10 10]=[8 7]