Learning Rate Scheduler

  • Usually learning rate scheduler starts with big value
    • for quick jumping to convergence
  • Go to small value, when close to convergence
    • For not to overshoot