Issue on Google Colab

[jacons]

Hi,
I was trying to install ilmart with the following steps (on google colab):

!git clone https://github.com/veneres/ilmart.git
!git clone https://github.com/veneres/LightGBM.git
%cd LightGBM/python-package
!python setup.py install

when the errors occurred.
Which are the problem?

In order to debug I've attached the log-error.

CMake Deprecation Warning at CMakeLists.txt:35 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Success
-- Using _mm_prefetch
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Success
-- Using _mm_malloc
-- Configuring done (1.9s)
-- Generating done (0.0s)
-- Build files have been written to: /content/LightGBM/python-package/build_cpp
[  2%] Building CXX object CMakeFiles/lightgbm_capi_objs.dir/src/c_api.cpp.o
[  4%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/boosting.cpp.o
[  6%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt.cpp.o
[  8%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt_model_text.cpp.o
In file included from /content/LightGBM/python-package/compile/include/LightGBM/config.h:16,
                 from /content/LightGBM/python-package/compile/include/LightGBM/boosting.h:8,
                 from /content/LightGBM/python-package/compile/src/boosting/boosting.cpp:5:
/content/LightGBM/python-package/compile/include/LightGBM/utils/common.h:37:10: fatal error: ../../../external_libs/fmt/include/fmt/format.h: No such file or directory
   37 | #include "../../../external_libs/fmt/include/fmt/format.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [CMakeFiles/lightgbm_objs.dir/build.make:76: CMakeFiles/lightgbm_objs.dir/src/boosting/boosting.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from /content/LightGBM/python-package/compile/include/LightGBM/config.h:16,
                 from /content/LightGBM/python-package/compile/include/LightGBM/boosting.h:8,
                 from /content/LightGBM/python-package/compile/src/c_api.cpp:7:
/content/LightGBM/python-package/compile/include/LightGBM/utils/common.h:37:10: fatal error: ../../../external_libs/fmt/include/fmt/format.h: No such file or directory
   37 | #include "../../../external_libs/fmt/include/fmt/format.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /content/LightGBM/python-package/compile/include/LightGBM/config.h:16,
                 from /content/LightGBM/python-package/compile/src/boosting/gbdt_model_text.cpp:5:
/content/LightGBM/python-package/compile/include/LightGBM/utils/common.h:37:10: fatal error: ../../../external_libs/fmt/include/fmt/format.h: No such file or directory
   37 | #include "../../../external_libs/fmt/include/fmt/format.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [CMakeFiles/lightgbm_objs.dir/build.make:104: CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt_model_text.cpp.o] Error 1
make[3]: *** [CMakeFiles/lightgbm_capi_objs.dir/build.make:76: CMakeFiles/lightgbm_capi_objs.dir/src/c_api.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:141: CMakeFiles/lightgbm_capi_objs.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
In file included from /content/LightGBM/python-package/compile/include/LightGBM/config.h:16,
                 from /content/LightGBM/python-package/compile/include/LightGBM/boosting.h:8,
                 from /content/LightGBM/python-package/compile/src/boosting/gbdt.h:8,
                 from /content/LightGBM/python-package/compile/src/boosting/gbdt.cpp:5:
/content/LightGBM/python-package/compile/include/LightGBM/utils/common.h:37:10: fatal error: ../../../external_libs/fmt/include/fmt/format.h: No such file or directory
   37 | #include "../../../external_libs/fmt/include/fmt/format.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [CMakeFiles/lightgbm_objs.dir/build.make:90: CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:89: CMakeFiles/lightgbm_objs.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:175: CMakeFiles/_lightgbm.dir/rule] Error 2
make: *** [Makefile:208: _lightgbm] Error 2

[veneres]

Hi @jacons!
From your console log, it seems that you are missing the external modules of Lightgbm, including fmt.
If you want to separately clone Ilmart and Lightgbm, do not forget to add the option --recurse-submodules
I've just tried:

!git clone https://github.com/veneres/ilmart.git
!git clone --recurse-submodules https://github.com/veneres/LightGBM.git
%cd LightGBM/python-package
!python setup.py install

On Colab and it works.
Let me know if this solves your problems.

[jacons]

Now it's works. Thank you so much!

[jacons]

Hi!
I'm sorry if write again. Your code is very interesting and I'm trying to use it for my work. However, I'm utilizing Win10.
I was installing the framework when the execution stopped.

After downloading the LightGBM using
git clone --recurse-submodules https://github.com/veneres/LightGBM.git
And moved in the right directory:
LightGBM/python-package
I'm not able to install the package with:
python setup.py install

Can you give me some advice?

Thank you for your help in this matter.
Jacons

[veneres]

Ciao Jacons!
I am actively working on a completely new version of Ilmart, so stay tuned, I have everything implemented and will commit a new version in a week or two.
In the meantime, you can check the new addition to lightgbm by looking at my pull request to the original repository: microsoft/LightGBM#6335

Since I have modified a lot of the library in the last few months, you cannot install it as before.
One of the problems that you are facing is probably due to the fact that now I am using the new building script provided by lightgbm (https://github.com/veneres/LightGBM/blob/master/build-python.sh).
I will be back to this issue as soon as I update this repository with the whole new infrastructure.

[jacons]

Thank you so much!

[veneres]

Dear @jacons, the new release has finally been uploaded.
Feel free to open a new issue if needed.

[jacons]

Hi!
Thank you for ping me.

In your installation pipeline, I think that you've forgot to git clone the ILMART code.
Is it correct?

[veneres]

Yes, of course!
I briefly modified the README, but keep in mind that the best way to reuse the code provided in this repo is now to use the dockerized version.
Feel free to close the issue when you think that everything is solved.