Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some improvements to summarysize, fixes #32881 #32886

Merged
merged 1 commit into from
Aug 15, 2019

Conversation

JeffBezanson
Copy link
Member

@JeffBezanson JeffBezanson commented Aug 13, 2019

  • 0-field mutable structs take 1 word
  • include alignment in object sizes
  • take uniqueness into account for Strings
  • include union selector bytes for Arrays

This leaves types alone for now, but as a further enhancement we might want to take hash-consing of types into account.

fixes #32881

@vtjnash
Copy link
Member

vtjnash commented Aug 14, 2019

I don’t like the addition of gc-padding (alignment) into the mix here. I think that’s an implementation detail that has no place here. Other than that though, the corrections here to account for some types more correctly sound good.

@sjoelund
Copy link

Regarding GC padding, yes this is a matter of taste. For other languages I have seen several values, two of which would be sizes with and without padding (which often differ by quite a lot); when possible I have also seen sizes including literal constants being referenced (where this is tracked by the GC).

@JeffBezanson
Copy link
Member Author

Yes, I can see two sides here. But I think including alignment is defensible, since people want to know the actual amount of memory consumed. The fact that it's an implementation detail is kind of the point --- the actual resources consumed by something is the ultimate implementation detail, but when you ask we might as well tell you.

@sjoelund
Copy link

sjoelund commented Aug 15, 2019

The changes in my testcase (a large-ish immutable AST created using ANLTR from the C API):

Version summarysize GC allocated
1.1 103279999 154.621 MiB
1.3 alpha.0 103279999 152.953 MiB
#32886 140046304 152.953 MiB
#32886 without string sharing in the tree 163277424 175.108 MiB

The GC allocated I believe includes some objects that are converted at times, so some garbage is expected. The size for the strings even seems to line up with the GC-reported numbers:

julia> (163277424-140046304)/1024. ^ 2
22.154922485351562
julia> 175.108-152.953
22.155

So with these changes we would have a much more reliable summarysize it seems.

@sjoelund
Copy link

sjoelund commented Aug 15, 2019

There might still be room for improvement as it seems:

julia> struct ABC end
julia> Base.summarysize(ABC())
0

Is the size of a header for a struct perhaps missing?

Edit: Or perhaps this was intentional as it says only mutable 0-field takes 1 word now.

@JeffBezanson
Copy link
Member Author

Right; immutable empty structs are singletons, only one instance is ever heap allocated, so each additional one takes no space.

@vtjnash
Copy link
Member

vtjnash commented Aug 15, 2019

know the actual amount of memory consumed

Yeah, but it's also not necessarily that either. In fact, it could be either an over or underestimate, depending on what the allocator decided to do with the request.

@JeffBezanson
Copy link
Member Author

Ok, I might as well separate those changes anyway.

- 0-field mutable structs take 1 word
- take uniqueness into account for Strings
- include union selector bytes for Arrays
@JeffBezanson JeffBezanson force-pushed the jb/better_summarysize branch from dc450be to 6258d08 Compare August 15, 2019 17:14
@JeffBezanson
Copy link
Member Author

Note there is no final decision on whether to include alignment; I'm just putting in the changes everybody agrees with first so we can at least get some improvements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

should Base.summarysize include alignment?
3 participants