Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo: Remove SparseArrays from sysimage & set up as weak dep of Statistics #47745

Closed

Conversation

IanButterworth
Copy link
Sponsor Member

@IanButterworth IanButterworth commented Nov 30, 2022

Reverts commit 237c8d3 to remove SparseArrays from sysimage

Uses #47695 to set up SparseArrays as a weak dependency of Statistics, via a glue package.

See JuliaStats/Statistics.jl@86454ce for the relatively simple and minimal change. All code was copied across to the glue package untouched, with the addition of some imports.

This PR

% ./julia --startup-file=no

julia> @time using Statistics
[ Info: Precompiling Statistics [10745b16-79ce-11e8-11f9-7d13ad32a3b2]
  0.574351 seconds (15.58 k allocations: 1.226 MiB)

julia> @time using SparseArrays
[ Info: Precompiling SparseArrays [2f01184e-e22b-5df5-ae63-d93ebab69eaf]
[ Info: Precompiling SparseArraysGlue [b66d789c-28ca-57c3-870e-2abdfb76150c]
 15.779174 seconds (7.06 M allocations: 981.552 MiB, 0.48% compilation time)

julia> 
% ./julia --startup-file=no

julia> @time using Statistics
  0.011658 seconds (13.76 k allocations: 1.030 MiB)

julia> @time using SparseArrays
  4.618848 seconds (7.05 M allocations: 980.989 MiB, 1.46% compilation time)

Master

% ./julia --startup-file=no

julia> @time using Statistics
[ Info: Precompiling Statistics [10745b16-79ce-11e8-11f9-7d13ad32a3b2]
  0.516886 seconds (6.90 k allocations: 685.641 KiB)

julia> 
% ./julia --startup-file=no

julia> @time using Statistics
  0.013421 seconds (13.87 k allocations: 1.043 MiB)

4.6 seconds to load SparseArrays & the glue package seems a bit high. Worth checking on other non-macos platforms

cc. @KristofferC

This allows packages to define "glue packages" which
are modules that are automatically loaded when
a set of other packages are loaded into the Julia
session.
@IanButterworth
Copy link
Sponsor Member Author

On linux

julia> @time using Statistics
  0.008449 seconds (12.34 k allocations: 961.833 KiB)

julia> @time using SparseArrays
  2.505745 seconds (7.05 M allocations: 980.824 MiB, 1.23% compilation time)

@brenhinkeller brenhinkeller added domain:arrays:sparse Sparse arrays compiler:latency Compiler latency labels Nov 30, 2022
@KristofferC KristofferC deleted the branch JuliaLang:kc/glue December 7, 2022 15:01
@KristofferC KristofferC closed this Dec 7, 2022
@IanButterworth IanButterworth deleted the ib/stats_sparse_glue branch December 7, 2022 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:latency Compiler latency domain:arrays:sparse Sparse arrays
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants