Naive Bayes

  • Naive bayes is formed from Bayes Theorem
  • Naive bayes is called naive as it assumes all the features are independent of each other, which is really no the case in real life

For example,
Given msg = "Dear Friends", predict if it is Spam or Not-Spam

P(Spam|Dear Friend) &= P(Dear Friend|Spam) P(Spam) \\&= P(Dear|Spam) P(Friend|Spam) P(Spam)

[!def] Naive Bayes
P(X|x_1, x_2, ... x_n) = \prod_i p(x_i|X) P(X)


  1. Works well with small dataset
  2. Lesser overfitting
  3. Smaller in size
  4. Faster Processing