Julia plot hline! doesn't work with multiple threads

Question

I have a simple simulation and I want to plot errors on 3 distinct figures. To speed things up I wanted to introduce a little bit of parallel computing.

Threads.@threads for i in 1:3
plt = plot(t, err[:, i], linecolor=:blue, label=[""], linewidth=3)
hline!(plt, [0], linestyle=:dash, linecolor=:black, label=[""])
xlabel!(L"t")
ylabel!(L"e_%$i")
savefig(plt, "fig/Staubli_Slotine_Li$i.pdf")
end

Everything stops working when I try running julia with -t3 flag

ERROR: LoadError: TaskFailedException
Stacktrace:
 [1] wait
   @ ./task.jl:334 [inlined]
 [2] threading_run(func::Function)
   @ Base.Threads ./threadingconstructs.jl:38
 [3] top-level scope
   @ ./threadingconstructs.jl:97

    nested task error: KeyError: key :annotations not found
    Stacktrace:
      [1] pop!(h::Dict{Symbol, Any}, key::Symbol)
        @ Base ./dict.jl:587
      [2] pop_kw!(dd::RecipesPipeline.DefaultsDict, k::Symbol)
        @ RecipesPipeline ~/.julia/packages/RecipesPipeline/F2mWY/src/utils.jl:57
      [3] _update_subplot_args(plt::Plots.Plot{Plots.PGFPlotsXBackend}, sp::Plots.Subplot{Plots.PGFPlotsXBackend}, plotattributes_in::Dict{Symbol, Any}, subplot_index::Int64, remove_pair::Bool)
        @ Plots ~/.julia/packages/Plots/nzdhU/src/args.jl:2058
      [4] _subplot_setup(plt::Plots.Plot{Plots.PGFPlotsXBackend}, plotattributes::Dict{Symbol, Any}, kw_list::Vector{Dict{Symbol, Any}})
        @ Plots ~/.julia/packages/Plots/nzdhU/src/pipeline.jl:277
      [5] plot_setup!(plt::Plots.Plot{Plots.PGFPlotsXBackend}, plotattributes::Dict{Symbol, Any}, kw_list::Vector{Dict{Symbol, Any}})
        @ Plots ~/.julia/packages/Plots/nzdhU/src/pipeline.jl:138
      [6] recipe_pipeline!(plt::Any, plotattributes::Any, args::Any)
        @ RecipesPipeline ~/.julia/packages/RecipesPipeline/F2mWY/src/RecipesPipeline.jl:87
      [7] _plot!(plt::Plots.Plot, plotattributes::Any, args::Any)
        @ Plots ~/.julia/packages/Plots/nzdhU/src/plot.jl:208
      [8] plot!(::Plots.Plot; kw::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
        @ Plots ~/.julia/packages/Plots/nzdhU/src/plot.jl:198
      [9] plot!(; kw::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
        @ Plots ~/.julia/packages/Plots/nzdhU/src/plot.jl:188
     [10] #xlabel!#484
        @ ~/.julia/packages/Plots/nzdhU/src/shorthands.jl:416 [inlined]
     [11] xlabel!
        @ ~/.julia/packages/Plots/nzdhU/src/shorthands.jl:416 [inlined]
     [12] macro expansion
        @ ~/Documents/studia/master_thesis/master_thesis_code/sym_scripts/Staubli_Slotine_Li.jl:29 [inlined]
     [13] (::var"#88#threadsfor_fun#1"{UnitRange{Int64}})(onethread::Bool)
        @ Main ./threadingconstructs.jl:85
     [14] (::var"#88#threadsfor_fun#1"{UnitRange{Int64}})()
        @ Main ./threadingconstructs.jl:52
in expression starting at /home/jcebulsk/Documents/studia/master_thesis/master_thesis_code/sym_scripts/Staubli_Slotine_Li.jl:26

If I comment out hline! the script runs without any issue.

It looks like I can't have both hline and parallel operation.

Answer 1

On my machine I am getting an EXCEPTION_ACCESS_VIOLATION which is even uglier. For sure in your code xlabel!(L"t") and ylabel!(L"e_%$i") mutate the global state which is very bad for any kind parallelism and should be xlabel!(plt, L"t") and ylabel!(plt, L"e_%$i").

However, in many scenarios this might still not work because Plots.jl is also maintaining it's global state and it might be not thread safe. Hence the best way to go forward is through distributed computing:

using Distributed
addprocs(3)
@everywhere using Plots, LaTeXStrings
Distributed.@distributed for i in 1:3
plt = plot(t, err[:, i], linecolor=:blue, label=[""], linewidth=3)
hline!(plt, [0], linestyle=:dash, linecolor=:black, label=[""])
xlabel!(L"t")
ylabel!(L"e_%$i")
savefig(plt, "fig/Staubli_Slotine_Li$i.pdf")
end

Note that you will have a better performance if the plots are very complex as each of those subprocesses will experience the "Julia time to first plot" issue.

Hence before going multi-process you might want to parametrize the GR backend and see if the performance is sufficient:

using Plots
ENV["GKSwstype"]="nul"  # this parameter significantly speeds up generation of GR-based plots
gr()