Gradient Boost (Regression)

Steps:

1. Get initial guess by taking the average of the dataset
2. Get $residual_output = ground_truth - guess$
3. Build a tree to fit the $residual_output$
4. Now the prediction will be $prediction = lr * new_tree_prediction$
5. Go to step 2 unless predetermined limit is reached

• $LR$ is learning rate and an hyperparameter.