pure_eval misses the variable on the crash line

[vmarkovtsev]

(Related to #748)

I keep experiencing the same situation for various crash reports in my project: the new "interesting variables" logic includes everything but the entities on the line where the crash happens. For example,

def foo(arg: Dict[str, str]):
    bar = arg[5]
    x = func()
    x.attr = "AttributeError"  # crash here
    ...

The visible variables in the stack trace are arg, bar, and even Dict[str, str] from typing (pufe_eval loves typing for some reason), but x - never. Yet the most important local stack information for me is the variables from the line where I raise an exception. I guess the variable suggestion logic can be hacked to prioritize the variables on that line.

[untitaker]

@alexmojaki can you look into this?

[vmarkovtsev]

Sorry, I've just remembered another concern: we've got the number of max variables reported per stack frame (10 by default AFAIK). It frequently happens that there are many random noisy names before x-es, so even if the priority is fixed, such variables can still be discarded after truncation of the alphabetically sorted keys sequence.

[untitaker]

Yes. I think the solution here is that pure_eval does its own trimming, such that it does not run into limits later, as it knows better what to trim first.

[alexmojaki]

Hi @vmarkovtsev, thanks for the report. I'm unable to reproduce this. Using your code:

from typing import Dict

from sentry_sdk.integrations.pure_eval import PureEvalIntegration
import sentry_sdk

sentry_sdk.init(
    dsn="...",
    integrations=[PureEvalIntegration()],
    send_default_pii=True
)


class X:
    __slots__ = ["none"]


def func():
    return X()


def foo(arg: Dict[int, str]):
    bar = arg[5]
    x = func()
    x.attr = "AttributeError"  # crash here


foo({5: "asd"})

I see x as expected:

Of course x.attr doesn't exist yet, but it sounds like you're talking about x itself. Do you think you can put together a reproducible example?

I had no idea sentry trims the number of variables. That's definitely something pure_eval can improve if the trimming is currently done alphabetically. Are you saying it's trimming x for you right now because there's too many variables, or is x missing even when there's only a few variables?

[vmarkovtsev]

Sorry, that was a schematic example 😄 The real ones are in my production 😞 I'll try to rip out some good case but it'll take some time... For now, here is a screenshot:

Are you saying it's trimming x for you right now because there's too many variables, or is x missing even when there's only a few variables?

It does both. I've got tens of variables, and sometimes the line is down the first 10, sometimes it is up the first 10 but still not included.

[vmarkovtsev]

Maybe it has something to do with the "variable pressure" and asyncio - all my cases are coroutines.

[alexmojaki]

@untitaker can you point me to docs/code relating to variable trimming? I can't find anything. If it happens on the server I don't see what I can do unless you want to rely on JSON ordering.

[untitaker]

sentry-python/sentry_sdk/serializer.py

Line 71 in 4d91fe0

MAX_DATABAG_BREADTH = 10

I was suggesting that you apply your own "smart trimming" in pure_eval such that you never run into this limit.

[alexmojaki]

OK thanks, that helped. I've opened a PR to deal with trimming.

@vmarkovtsev I'm still not clear on whether this happens when you've got less than 10 variables. Your screenshot shows more than 10 so trimming is expected. It seems that the trimming has always been somewhat arbitrary so I'm surprised this wasn't a problem before pure_eval, although adding more values probably exacerbated it.

[vmarkovtsev]

Before pure_eval, only function arguments were printed, and there wasn't such a thing as the argument's importance, so even if they were trimmed, nobody knew how to do better.

[untitaker]

@vmarkovtsev can you test from master or should I release?

[vmarkovtsev]

Can you please release, it is hard for me to switch to master today.

[untitaker]

Ok 0.17.3 is out. Let me know how it goes.

…

On Wed, Sep 2, 2020 at 10:47 AM Vadim Markovtsev ***@***.***> wrote: Can you please release, it is hard for me to switch to master today. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#805 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGMPRJW7QYP5HNMPSQTJIDSDYBBDANCNFSM4QQNFFDQ> .

[vmarkovtsev]

It works well so far.

[vmarkovtsev]

I confirm that the priorities are set correctly, thanks @alexmojaki @untitaker