Linking Ipopt with linear solvers

[odow]

There are a lot of issues relating to linear solvers with Ipopt. #236 was a start at providing some documentation on how to use Ipopt with different solvers: https://github.com/jump-dev/Ipopt.jl#custom-installation. However, most are untested and I don't have a Windows machine. I've opened this issue to track which solver-platform combinations we have instructions for.

If you have clarifications for existing instructions, or if you have instructions for a missing solver-platform pair, please open a pull request.

Here are solver/platform combinations I have tested and got working so far:

• Pardiso Project
  • Windows
  • Mac
  • Linux (Related issues: Compiling with pardiso on ubuntu #106, pardiso, ipopt, undefined symbol: sgetrf_ #177)
• MKL Pardiso
  • Windows
  • Mac
  • Linux (Related issues: Error building 'Ipopt' when compiled to use with MKL Pardiso Solver #195)
• HSL for Ipopt (MA27)
  • Windows (Related issues: Error linking IPOPT with HSL built on Windows #229)
  • Mac
  • Linux
• HSL (MA86, MA97)
  • Windows (Related issues: HSL linear solver on windows and admin privileges #192)
  • Mac (Related issues: BUG: Tried to obtain HSL_MA86 from shared library "libhsl.dylib" #197, Trying to load MA97 when HSL is not available crashes Julia #213)
  • Linux

[metab0t]

I try it on Windows
When I put libhsl.dll in a directory in my PATH and use Ipopt bundled with Ipopt_jll, following error is encountered.

Exception of type: OPTION_INVALID in file "IpAlgBuilder.cpp" at line 268:
 Exception message: Selected linear solver MA27 not available.
Tried to obtain MA27 from shared library "libhsl.dll", but the following error occured:
loadLib error: Do not know how to handle shared libraries on this operating system

EXIT: Invalid option encountered.

I suspect that dynamic loading of linear solver is unavailable for Ipopt_jll on Windows.

[ksun46]

I use Mac 11.0.1. I have a customized Ipopt installed in ~/CoinOR/Ipopt, and all third party lib files can be found in ~/CoinOR/Ipopt/build/lib (including libhsl.dylib).

I followed the custom installation section by setting the lib path to ~/CoinOR/Ipopt/build/lib and executable path to ~/CoinOR/Ipopt/build/bin, build, using Ipopt, and then call Ipopt with MA27 in JuMP. But It didn't work. I still saw

Exception of type: OPTION_INVALID in file "IpAlgBuilder.cpp" at line 268:
 Exception message: Selected linear solver MA27 not available.
Tried to obtain MA27 from shared library "libhsl.dylib", but the following error occured:
dlopen(libhsl.dylib, 2): image not found

or even a weird segmentation fault several times (I still don't know why).

So I followed https://github.com/jump-dev/Ipopt.jl#mac-2, configure and re-install HSL separately (I followed the instruction here and the dylib file is installed under /usr/local/lib). And then it worked.

[Fisheryu1234]

In Pardiso Project -> Linux, I am working with lapack and blas in MKL libraries in cloud computing, and I'm not the superuser. Is it possible to make it work through MKL libraries, without public BLAS/LAPACK packages? Btw, I want to stick to Pardiso project, not MKL Pardiso.

[odow]

@Fisheryu1234 as a rule of thumb, you just need libpardiso.so to be somewhere that dlopen can find it, and then set set_optimizer_attribute(model, "linear_solver", "pardiso").

The specifics of how to compile pardiso and the needed dependencies can vary greatly depending on the environment, so I don't have any suggestions. If you get something working and have a suggestion to improve the documentation, let me know.

[Fisheryu1234]

@odow You are totally right! Later I figured out something similar in my case

using Libdl
Libdl.dlopen("/opt/apps/software/imkl/2022.1.0/mkl/2022.1.0/lib/intel64/libmkl_rt.so",
             RTLD_GLOBAL)
Libdl.dlopen("/opt/apps/software/GCCcore/11.3.0/lib64/libgfortran.so",RTLD_GLOBAL)
Libdl.dlopen("/opt/apps/software/GCCcore/11.3.0/lib64/libgomp.so", RTLD_GLOBAL)

[Fisheryu1234]

For HSL MA solver, how should I install and switch between solvers like ma27 and ma86 in JuMP? I am using Mac, and copied all files in </full/path/somewhere/lib> to /usr/local/lib, then rename the .dylib one. Since both solvers are renamed the same "libhsl.dylib", does it mean I could not have ma27 and ma86 at the same time and switch in set_optimizer_attribute(model, "linear_solver", "ma27"/"ma86")?

[odow]

More recent versions of Ipopt provide the hsllib option, https://coin-or.github.io/Ipopt/OPTIONS.html#OPT_hsllib, but I haven't used it.

You should be able to set it as the absolute path:

set_optimizer_attribute(model, "hsllib", "/usr/local/lib/libhsl_ma27.dylib")
set_optimizer_attribute(model, "linear_solver", "ma27")

set_optimizer_attribute(model, "hsllib", "/usr/local/lib/libhsl_ma86.dylib")
set_optimizer_attribute(model, "linear_solver", "ma86")

[Fisheryu1234]

Thanks, It works! It turns out that ma86 is terribly slower than ma27, which the opposite should be true. Does importing openblas32.jll matter? ma27 does not need while ma86 does in my case.

[odow]

which the opposite should be true

Not necessarily. The performance of the linear solvers can vary a lot between problems. See, e.g., https://arxiv.org/pdf/1909.08104.pdf.