Perceptron

• Perceptron is a linear classifier
  • So it is used only when data are linearly separable
• Initially unit step activation function was used
• In short, it's a simple model of $y = wx + b$ where
  • prediction = 1 if y > 0
  • otherwise, prediction = 0

Steps

1. For d=2 dimensional data,
   1. W = ($w_0, w_1, w_2$) - (d+1) dimension
   2. X = ($x_0, x_1, x_2$) - (d+1) dimension
   3. Here $x_0$ is always $1$
2. Now calculation $\hat{y} = w_0 + w_1 x_1 + w_2 x_2$
   1. if $\hat{y} > 0$, then upper side
   2. else, lower side
3. Update:
   1. For all incorrectly classified ones:
4. $w_{new} = w_{new} + \eta d x$
   1. d = 1, if it needs to be on upper side
   2. d = 0, if it needs to be on lower side