Yes indeed. Very small gradients and hessians can lead to some rather unwanted behaviour in gradient boosting models (due to Newton descent during boosting iterations), so alpha=1 is more stable during training. Small alpha values results in a very large change in the gradient/hessian function between positive and negative directions.
This is a bit different from inference synthesis or model validation/evaluation, where large gradients can overly punish predictions in the wrong direction (relative to the reward for getting the direction right). So in these cases we want alpha ~ 0. It’s also preferable to use the logarithm of the loss in these cases because it provides better contrast for accurate predictions (see the true vs predicted returns figures above), but it cannot be used for training as log(loss) is not a convex function.
I’ll also add here that we have slightly modified the gradient and hessian functions (mainly in “Region 2” between zero and the true value) to achieve a smooth function for gradient boosting. This doesn’t affect cases where we only use the loss function (inference synthesis, model validation/evaluation).
Without this boosting steps would increase when approaching the true value from zero:
