Implement advanced early stopping in train()

[ummel]

It appears a min_delta parameter was recently added to lightGBM to allow early stopping if the score does not improve by some minimum value (in addition to the current stopping criterion provided by early_stopping_rounds) -- discussion here. This would be useful, since nrounds could be set quite high without risk of excessive runtime.

However, as of today, there is no mention of min_delta in the lightGBM parameter documentation or the R package API. But it is documented in the Python API under callbacks.

[ummel]

This would help stop lightgbm training earlier than currently (current method is: early_stopping_rounds = 1L) without any significant loss of accuracy.

[ummel]

This is a new feature in LightGBM and may not be totally ready, yet. But the idea here is that a model should stop "growing" (adding iterations/trees) when the prediction performance on the hold-out "test" set is no longer increasing my some minimum delta; i.e. at some point, the real-world significance of further improvement is effectively zero. At present, we can't do this -- instead, LightGBM keeps adding trees to the model until the hold-out performance stops improving at all. So LightGBM will keep building out the model even if the improvement is 0.00001%, for example (so num_iterations = 5000 can be really expensive). Obviously, that is a big waste of computation. Implementing more aggressive early stopping could save a lot of time. Wouldn't be surprised if it cuts compute time by 1/3 in many cases.