Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Breaking] Remove max_delta_step from Poisson. #7267

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Sep 27, 2021

User can use the parameter with same name max_delta_step in tree method to control leaf weights, no
need to manipulate hessian values.

User can use the parameter with same name in `max_delta_step` to control leaf weights, no
need to manipulate hessian values.
@codecov-commenter
Copy link

Codecov Report

Merging #7267 (004fd0c) into master (d8a549e) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #7267   +/-   ##
=======================================
  Coverage   82.64%   82.64%           
=======================================
  Files          13       13           
  Lines        4067     4067           
=======================================
  Hits         3361     3361           
  Misses        706      706           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8a549e...004fd0c. Read the comment docs.

Copy link
Member

@RAMitchell RAMitchell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that this parameter is different from the other max_delta_step. It looks like it was put here to stop numerical instability problems. In particular the default value means the hessian cannot drop below 2 (assuming weights are 1.0). Can we be sure this change will not cause problems?

@trivialfis
Copy link
Member Author

trivialfis commented Oct 12, 2021

@RAMitchell It might cause problems in hessian as you mentioned due to small values. But having it might also cause trouble if the hessian is supposed to be small. It's a hyper-parameter that needed tuning.

To me, both max_depth_step serve the same purpose. The weight of the leaf is roughly this:

min(G/H, delta)

If you have a small hessian, G/H will be large, then max_delta_step in tree param chunks the output weight (G/H). On the other hand, the max_delta_step in the objective increases the hessian which is also to reduce G/H. So they are both trying to reduce the leaf weights.

We should use only one of them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants