Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi-thread support #72

Open
rana opened this issue Dec 15, 2019 · 7 comments
Open

multi-thread support #72

rana opened this issue Dec 15, 2019 · 7 comments

Comments

@rana
Copy link

rana commented Dec 15, 2019

Multi-thread support post v1.3 would be helpful.
Can't use without it.
Thank you

@KristofferC
Copy link
Owner

I think support for running a callback when a task gets scheduled to stop / start would be needed for this, Otherwise, for example if you have

const to = TimerOutput()
function foo()
    @timeit to "some section"
        some_work()
    end
end

for i in 1:200
    Threads.@spawn foo()
end

creating 200 tasks running concurrently doing work in foo then I don't see a way to give a sensible result for how much time is spent inside foo unless I can only record the times where the task is actually executing.

@racinmat
Copy link

racinmat commented Jan 14, 2020

Maybe it would be useful to add some merging function so I could have one TimerOutput per thread, and then sum them up to see how they all performed.
When I tried the approach above, I obtained some race conditions, probably, and crashes like:

ERROR: TaskFailedException:
UndefRefError: access to undefined reference
Stacktrace:
 [1] getindex at .\array.jl:744 [inlined]
 [2] pop! at .\array.jl:1080 [inlined]
 [3] pop!(::TimerOutput) at C:\Users\racinsky\.julia\packages\TimerOutputs\7Id5J\src\TimerOutput.jl:74
 [4] macro expansion at C:\Users\racinsky\.julia\packages\TimerOutputs\7Id5J\src\TimerOutput.jl:218 [inlined]
 [5] load_shas(::SubArray{String,1,Array{String,1},Tuple{UnitRange{Int64}},true}) at C:\Projects\SkunkAV\scripts\bench_schema_creation.jl:93
 [6] (::ThreadTools.var"#8#10"{Tuple{SubArray{String,1,Array{String,1},Tuple{UnitRange{Int64}},true}},typeof(load_shas)})() at .\threadingconstructs.jl:113
Stacktrace:
 [1] wait at .\task.jl:251 [inlined]
 [2] fetch at .\task.jl:266 [inlined]
 [3] _broadcast_getindex_evalf at .\broadcast.jl:630 [inlined]
 [4] _broadcast_getindex at .\broadcast.jl:603 [inlined]
 [5] getindex at .\broadcast.jl:563 [inlined]
 [6] copy at .\broadcast.jl:853 [inlined]
 [7] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(fetch),Tuple{Array{Task,1}}}) at .\broadcast.jl:819
 [8] tmap(::Function, ::Base.Iterators.PartitionIterator{Array{String,1}}) at C:\Users\racinsky\.julia\packages\ThreadTools\5hMgn\src\ThreadTools.jl:61
 [9] top-level scope at none:0

@antoine-levitt
Copy link

I think a barebones support for this where basically every logging call is prefixed by if Threads.threadid() == 1 would already be super helpful for most workflows.

@ericphanson
Copy link
Contributor

as a simple alternative, you could try my (not yet registered) https://github.com/ericphanson/TrackingTimers.jl, which supports the same @timeit API as TimerOutputs and allows multiprocess/multithreading, but doesn't keep track of nesting or anything like that (making parallelism very easy to support).

@rana
Copy link
Author

rana commented Feb 28, 2021

Nice @ericphanson

@IanButterworth
Copy link
Contributor

It requires a bit of manual organization but this seems to work for straightforwardly structured multi-threading #128

@oschulz
Copy link

oschulz commented May 28, 2023

If I understand #128 correctly this could be closed now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants