Softsign

  • Similar to Tanh, Range: $[-1, 1]$
  • Smoother than tanh, so less saturation; hence fewer Vanishing Gradient
  • Faster convergence
  • Complex derivative so computationally expensive

[!def] Softsign Formula
$$
f(x) = \frac{x}{1 + |x|}
$$

Pasted image 20231103173856.png