Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transform BST to gapless bytecode inplace #1391

Merged
merged 5 commits into from
Nov 10, 2016
Merged

Conversation

undingen
Copy link
Contributor

@undingen undingen commented Nov 3, 2016

this is a modified version of #1381 which creates the bytecode in-place and does not evolve duplicating some CFG structures.

  • all instructions of the CFG are directly emitted a single memory region.
  • all instructions are packed and follow each other directly
  • the CFGBlock just point into the bytecode block where the first instruction of the block is located
  • invokes are only a bit in the opcode field which when set means that the pointers to the normal and exc CFGBlocks* directly follow the normal instruction

@undingen undingen added the wip label Nov 3, 2016
- all instructions of the CFG are directly emitted a single memory region
- all instructions are packed and follow each other directly
- the CFGBlock just stores the offset into the bytecode where the first instruction of the block is located
- invokes are only a bit in the opcode field which when set means that the pointers to the normal and exc CFGBlocks* directly follow the normal instruction
@undingen
Copy link
Contributor Author

undingen commented Nov 5, 2016

memory improvements are:

django_template3.py:          131.4 --> 108.5 MB
sqlalchemy_imperative2.py:     97.0 -->  85.0 MB

@undingen undingen changed the title WIP: transform BST to gapless bytecode inplace transform BST to gapless bytecode inplace Nov 5, 2016
@undingen undingen closed this Nov 7, 2016
@undingen undingen reopened this Nov 7, 2016
@undingen
Copy link
Contributor Author

undingen commented Nov 8, 2016

this also improves max rss for:

s = """
class C(object):
    def f(self):
        return 'lalala'
C().f()
"""
for i in xrange(50000):
    exec s

from 124MB to 38MB.
But it is still growing (but much slower) with the number of the iterations so needs further investigation

@@ -4054,6 +4055,8 @@ void CodeConstants::dealloc() const {
owned_refs.clear();
for (auto&& e : funcs_and_classes) {
Py_DECREF(e.second);
assert(e.first->type() == BST_TYPE::FunctionDef || e.first->type() == BST_TYPE::ClassDef);
delete[] e.first;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be just delete?

this simplifies the code and removes a lot of ugliness
@undingen
Copy link
Contributor Author

undingen commented Nov 9, 2016

I fixed the delete ugliness by merging MakeFunction with FunctionDef and MakeClass with ClassDef which removes the whole special allocation and CodeConstant handling for FunctionDef and ClassDef nodes I added previously.
I think after this this PR should be ready for merging, please let me know what you think :-)

@kmod
Copy link
Collaborator

kmod commented Nov 10, 2016

lgtm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants