[LightGBM] [Fatal] [CUDA] invalid argument /workspace/LightGBM/src/treelearner/cuda_tree_learner.cpp 414

[pseudotensor]

#3450

I noticed this discussion, but I'm already including the commit "Ensure..." for that PR that supposedly fixed it. i.e. I'm on hash f2695da

Python 3.6.10 | packaged by conda-forge | (default, Apr 24 2020, 16:44:11) 
[GCC 7.3.0] on linux
Ubuntu 18.04 LTS
RTX2080.

(base) jon@pseudotensor:~/h2oai.fullcondatest$ python lgb_prefit_54d6a97e-dfed-4f18-96fb-286642619db8.py
2021-01-22 04:55:28,746 C: NA  D:  NA    M:  NA    NODE:SERVER      31430  FDEBUG | openblas library at /home/jon/minicondadai/lib/libopenblas.so.0
2021-01-22 04:55:28,777 C: NA  D:  NA    M:  NA    NODE:SERVER      31430  FDEBUG | No MKL library loaded (normal)
2021-01-22 04:55:28,782 C: NA  D:  NA    M:  NA    NODE:SERVER      31430  FDEBUG | Registered SegFault library for signals 'all'
2021-01-22 04:55:28,790 C: NA  D:  NA    M:  NA    NODE:SERVER      31430  FDEBUG | omp library at libomp.so.5
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/basic.py:1566: UserWarning: Using categorical_feature in Dataset.
  warnings.warn('Using categorical_feature in Dataset.')
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/basic.py:1092: UserWarning: categorical_feature keyword has been found in `params` and will be ignored.
Please use categorical_feature argument of the Dataset constructor to pass this parameter.
  .format(key))
[LightGBM] [Warning] num_threads is set=4, n_jobs=4 will be ignored. Current value: num_threads=4
[LightGBM] [Fatal] [CUDA] invalid argument /workspace/LightGBM/src/treelearner/cuda_tree_learner.cpp 414

Traceback (most recent call last):
  File "lgb_prefit_54d6a97e-dfed-4f18-96fb-286642619db8.py", line 8, in <module>
    model.fit(X, y, sample_weight=sample_weight, init_score=init_score, eval_set=eval_set, eval_names=valid_X_features, eval_sample_weight=eval_sample_weight, eval_init_score=init_score, eval_metric=eval_metric, early_stopping_rounds=early_stopping_rounds, feature_name=X_features, categorical_feature=categorical_feature, verbose=verbose_fit)
  File "/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/sklearn.py", line 774, in fit
    categorical_feature=categorical_feature, callbacks=callbacks, init_model=init_model)
  File "/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/sklearn.py", line 617, in fit
    callbacks=callbacks, init_model=init_model)
  File "/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/engine.py", line 229, in train
    booster = Booster(params=params, train_set=train_set)
  File "/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/basic.py", line 2076, in __init__
    ctypes.byref(self.handle)))
  File "/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/basic.py", line 52, in _safe_call
    raise LightGBMError(_LIB.LGBM_GetLastError().decode('utf-8'))
lightgbm.basic.LightGBMError: [CUDA] invalid argument /workspace/LightGBM/src/treelearner/cuda_tree_learner.cpp 414

using python 3.6 with this

fatal_cuda.zip

[StrikerRUS]

Should be completely fixed by #3752.

[pseudotensor]

I see. Any reason it's not merged yet?

[pseudotensor]

I confirm it no longer fails with that fix.

[StrikerRUS]

Any reason it's not merged yet?

This open discussion I guess.
#3752 (review)

I confirm it no longer fails with that fix.

Great! Thanks a lot for the try!

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.