RNN

• RNN = Recurrent Neural Network
• The weights and biases of every cell is shared, which is the main reason it can work with variable size of input and output
• RNN uses Tanh Activation Function
• Issue with RNN are
  • Vanishing Gradient
  • Exploding Gradient
  • Single context for long sequence
    • Solution: Attention

[!def] Equation of RNN
$$
h_t = tanh(x_t W_x + b_x + h_{t-1} W_h + b_h)
$$