Custom objective does not have the same loss curves.

[nd7141]

I see that if I use custom L2 implementation instead of default one, I get much slower convergence. Why is that? I would expect two lines will be exactly the same.

Plot:

Code:

X, y = sklearn.datasets.fetch_california_housing(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, train_size=0.05, random_state=0)
print(f"Train Data: {X_train.shape[0]} examples, {X_train.shape[1]} features")

params = {
    'boosting_type': 'gbdt',
    'metric': {'l2'},
}

evals_result = {}
model = lightgbm.train(params, train_data,
                       num_boost_round=200,
                       valid_sets=[train_data, test_data],
                       evals_result=evals_result,
                      )

train_rmse_default = evals_result['training']['l2']

def l2_loss(y, data):
    t = data.get_label()
    grad = y - t 
    hess = np.ones_like(y)
    return grad, hess

evals_result = {}
model = lightgbm.train(params, train_data,
                       fobj = l2_loss,
                       num_boost_round=200,
                       valid_sets=[train_data, test_data],
                       evals_result=evals_result,
                      )

train_rmse_custom = evals_result['training']['l2']

[jmoralez]

Hi @nd7141, I believe this is due to the different init scores, as explained in #5114 (comment). If you set boost_from_average to false with the built-in objective or set the init_score to the mean of the labels when using the custom one you should see identical results.

[nd7141]

Thank you @jmoralez Is there a way to set up init_scores for python API or it's only possible for CLI version?

[jmoralez]

Yes, there's an example in #5114 (comment) passing it to Dataset but you can also pass it to the fit method in the scikit-learn API.

Please let us know if you have further doubts.

[github-actions]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.