Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Julia threads and HiGHS threads interaction #181

Closed
bachdavi opened this issue Oct 10, 2023 · 11 comments
Closed

Julia threads and HiGHS threads interaction #181

bachdavi opened this issue Oct 10, 2023 · 11 comments

Comments

@bachdavi
Copy link

Hello again :)

I have a question about the interaction between Julia threads and the HiGHS threads option. It appears that HiGHS initializes itself for each Julia thread where I'm optimizing a model. Is that intended behavior?

Some background:
Via gdb we have recently observed a large number of logical threads associated with HiGHS (HighsTaskExecutor::HighsTaskExecutor). We are solving a linear program that via the documentation does not use any parallelism. By default, HiGHS will initialize its scheduler with half the machine threads, which we want to avoid.
HiGHS supports the threads option, which we can set via MOI.RawOptimizerAttribute("threads") or MOI.NumberOfThreads(). The documentation mentioned that this is a global setting and can only be changed after calling resetGlobalScheduler.

I can set the number of threads to 1 and can observe the expected increase of threads in the process. Here is a simple example:

using JuMP, HiGHS, MathOptInterface; begin 
           model = Model(HiGHS.Optimizer)
           MathOptInterface.set(model, MathOptInterface.NumberOfThreads(), 1)
           optimize!(model)
end

If I change 1 to 2, HiGHS mentions the need to reset the global scheduler.

But if I run the above snippet, with the changed threads count, in another Julia thread, via Threads.@spawn HiGHS sets the number of threads, apparently initializing a new scheduler. I can also observe an increase in the thread count of the Julia process.

Thank you very much for helping us out!

@odow
Copy link
Member

odow commented Oct 11, 2023

See #130 (comment)

@odow
Copy link
Member

odow commented Oct 11, 2023

@bachdavi is #182 helpful?

@chriscoey
Copy link

are there any changes we ought to suggest to the HiGHS devs directly?

@odow
Copy link
Member

odow commented Oct 12, 2023

They're aware of the issue. I don't know if there is the desire or engineering time to change it.

@bachdavi
Copy link
Author

@odow Sorry for taking such a long time to answer. I was a little swamped with work!
Thank you for clarifying the Readme, I think that is great ❤️
I'm wondering what to make of the behavior of the HiGHS scheduler related to Julia threads. It appears that on each Julia thread where I'm running the HiGHS optimizer we are initializing the scheduler again. Looking at the ps output it adds half the machine's threads to the thread count of the Julia process for each unique Julia thread I'm using the solver in.

@odow
Copy link
Member

odow commented Oct 19, 2023

cc @jajhall is there a way to limit HiGHS to one thread before we initialize it?

(@bachdavi Julian is traveling for the next week or two, so I don't know when we should expect a reply.)

@jajhall
Copy link

jajhall commented Oct 20, 2023

Yes, set the option 'threads' to 1. There should be little noticeable performance degradation as parallelism is only used in symmetry detection in the MIP solver.

If this is done before a call to 'Highs::run()' then the scheduler is initialised to use one thread.

The default value for 'threads' of 0 results in half the possible machine threads being initialised for use by HiGHS

@odow
Copy link
Member

odow commented Oct 20, 2023

Hmm that's what @bachdavi had tried to do. I'll take a look at this next week.

@bachdavi
Copy link
Author

bachdavi commented Oct 20, 2023

@odow and @jajhall To clarify, I can set the number of threads to 1 and it works properly. I do have to do that on each Julia thread though.

I'm running the following in an __init__:

function __init__()
    _g_set_highs_threads!()
end

# Set the number of threads used by HiGHS _globally_ to `1`.
function _g_set_highs_threads!()
    Threads.@threads :static for i in 1:Threads.threadpoolsize()
        _set_highs_threads!()
    end
end

# Set the number of threads used by HiGHS to `1`. This applies to the
# current _Julia_ thread only!
function _set_highs_threads!()
    # Reset, otherwise we cannot change the number of threads.
    HiGHS.Highs_resetGlobalScheduler(1)

    model = Model(HiGHS.Optimizer; add_bridges = false)

    set_silent(model)

    # HiGHS will by default initialize its task executor with half the
    # available threads on a machine, but won't use them to solve LPs:
    # https://ergo-code.github.io/HiGHS/dev/parallel/. Set the number
    # of threads to 1 to avoid it. This is a _Julia_ thread-local
    # setting and can only be changed after calling
    # `HiGHS.Highs_resetGlobalScheduler`.
    MathOptInterface.set(model, MathOptInterface.RawOptimizerAttribute("threads"), 1)

    # We have to call `optimize!` to apply the setting above.
    optimize!(model)
end

I'm wondering if that is intended or expected :)

@odow
Copy link
Member

odow commented Oct 20, 2023

@bachdavi what about something like this:

function __init__()
    _set_all_highs_threads_to_serial()
    return
end

function _set_all_highs_threads_to_serial()
    HiGHS.Highs_resetGlobalScheduler(1)
    Threads.@threads :static for _ in 1:Threads.threadpoolsize()
        _set_thread_local_highs_threads_to_serial()
    end
    return
end

"""
    _set_thread_local_highs_threads_to_serial()

Set the number of threads used by HiGHS to `1`. This applies to the current
_Julia_ thread only!

HiGHS will by default initialize its task executor with half the available
threads on a machine, but won't use them to solve LPs:
https://ergo-code.github.io/HiGHS/dev/parallel/. 

Set the number of threads to 1 to avoid it. This is a _Julia_ thread-local
setting and can only be changed after calling `Highs_resetGlobalScheduler`.
"""
function _set_thread_local_highs_threads_to_serial()
    highs = HiGHS.Highs_create()
    HiGHS.Highs_setIntOptionValue(highs, "threads", 1)
    HiGHS.Highs_destroy(highs)
    return
end

@bachdavi
Copy link
Author

bachdavi commented Dec 5, 2023

@odow I tried your suggestion, but I think the threads option is only applied when presolve is executed: https://github.com/ERGO-Code/HiGHS/blob/77583e20470b2ed5dad05a8eb04511034710ddd2/src/lp_data/Highs.cpp#L708-L737

In any case, I think #182 is a good addition to the README.

I'll close this issue, thank you :)

@bachdavi bachdavi closed this as completed Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

4 participants