You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to add a parameter, last_dataset_only or similar naming, to lightgbm.callback.early_stopping that would set early stopping to use the last item of eval_set only.
Motivation
There are situations where it's desirable to have multiple evaluation sets. Sometimes we want to record the evaluation results at each iteration for multiple datasets, but only use one for early stopping. The way XGBoost deals with this is by using only the last item of the eval_set to determine early stopping. We could score the model at each iteration to recreate the evaluation history, but this is inefficient.
This is also important for us when we are developing tools or pipelines that we want to be compatible with both LightGBM and XGBoost, like implementing feature selection or model selection algorithms/utilities that we want to be able to work with both.
Description
In the early stopping callback, LightGBM will use all datasets provided for early stopping. This would add a parameter, last_dataset_only or similar naming, to lightgbm.callback.early_stoppingthat would set early stopping to use the last item ofeval_set` only to determine when to early stop.
I would like the following to create an early stopping callback that would use only the first metric from the last dataset in eval_set to early stop, but would still score on every dataset in eval_set:
I'm not super familiar with the LGBM codebase and what, if anything, would need to be changed in the codebase besides the early stopping callback, but for what it's worth, I have a working version of a modified early stopping callback that I'm happy to work with you to contribute.
The text was updated successfully, but these errors were encountered:
Summary
I would like to add a parameter,
last_dataset_only
or similar naming, tolightgbm.callback.early_stopping
that would set early stopping to use the last item ofeval_set
only.Motivation
There are situations where it's desirable to have multiple evaluation sets. Sometimes we want to record the evaluation results at each iteration for multiple datasets, but only use one for early stopping. The way XGBoost deals with this is by using only the last item of the
eval_set
to determine early stopping. We could score the model at each iteration to recreate the evaluation history, but this is inefficient.This is also important for us when we are developing tools or pipelines that we want to be compatible with both LightGBM and XGBoost, like implementing feature selection or model selection algorithms/utilities that we want to be able to work with both.
Description
In the early stopping callback, LightGBM will use all datasets provided for early stopping. This would add a parameter,
last_dataset_only
or similar naming, to lightgbm.callback.early_stoppingthat would set early stopping to use the last item of
eval_set` only to determine when to early stop.I would like the following to create an early stopping callback that would use only the first metric from the last dataset in
eval_set
to early stop, but would still score on every dataset ineval_set
:I'm not super familiar with the LGBM codebase and what, if anything, would need to be changed in the codebase besides the early stopping callback, but for what it's worth, I have a working version of a modified early stopping callback that I'm happy to work with you to contribute.
The text was updated successfully, but these errors were encountered: