Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make it possible to collect pystats of parts of runs #100143

Closed
mdboom opened this issue Dec 9, 2022 · 2 comments
Closed

Make it possible to collect pystats of parts of runs #100143

mdboom opened this issue Dec 9, 2022 · 2 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@mdboom
Copy link
Contributor

mdboom commented Dec 9, 2022 •
edited by bedevere-bot
Loading

The --enable-pystats system has a couple of important shortcomings:

  • When stats is turned off, stats aren't dumped (on quit or explicitly). Therefore, it's impossible to put sys._stats_on/sys._stats_off calls around tested code and get the results out.
  • Since stats collection is on by default, it is very fiddly to exclude code for stats, especially for things that fire off subprocesses, e.g. pyperformance.

@markshannon's suggestion is to:

  • Having stats off by default
  • Add an -Xstats flag to turn it on at startup
  • Always dump out stats

Linked PRs
@mdboom mdboom added the type-bug An unexpected behavior, bug, or error label Dec 9, 2022
@bedevere-bot bedevere-bot mentioned this issue Dec 9, 2022
Merged
markshannon pushed a commit that referenced this issue Dec 12, 2022
@mdboom
GH-100143: Improve collecting pystats for parts of runs (GH-100144)
1583c6e 
* pystats off by default

* Add -Xpystats flag

* Always dump pystats, even if turned off
@markshannon
Copy link
Member

We also need to add a mechanism to always have stats on, for gathering stats for applications which cannot be modified or spawn subprocesses.

Adding an environment variable analog for -Xstats, e.g. PYTHONSTATS should be sufficient.

carljm added a commit to carljm/cpython that referenced this issue Dec 14, 2022
@carljm
Merge branch 'main' into typedocs
53d33a1 
* main: (103 commits)
  pythongh-100248: Add missing `ssl_shutdown_timeout` parameter in `asyncio` docs (python#100249)
  Assorted minor fixes for specialization stats. (pythonGH-100219)
  pythongh-100176: venv: Remove redundant compat code for Python <= 3.2 (python#100177)
  pythonGH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (pythonGH-100223)
  pythongh-99955: undef ERROR and SUCCESS before redefining (fixes sanitizer warning) (python#100215)
  pythonGH-100206: use versionadded for the addition of sysconfig.get_default_scheme (python#100207)
  pythongh-81057: Move _Py_RefTotal to the "Ignored Globals" List (pythongh-100203)
  pythongh-81057: Move Signal-Related Globals to _PyRuntimeState (pythongh-100085)
  pythongh-81057: Move faulthandler Globals to _PyRuntimeState (pythongh-100152)
  pythongh-81057: Move tracemalloc Globals to _PyRuntimeState (pythongh-100151)
  pythonGH-100143: Improve collecting pystats for parts of runs (pythonGH-100144)
  pythongh-99955: standardize return values of functions in compiler's code-gen (python#100010)
  pythongh-79218: Define `MS_WIN64` macro for Mingw-w64 64bit on Windows (pythonGH-100137)
  Fix: typo (Indention) (pythonGH-99904)
  pythongh-96715 Remove redundant NULL check in `profile_trampoline` function (python#96716)
  pythongh-100176: remove incorrect version compatibility check from argument clinic (python#100190)
  clarify the 4300-digit limit on int-str conversion (python#100175)
  pythongh-70393: Clarify mention of "middle" scope (python#98839)
  pythongh-99688: Fix outdated tests in test_unary (python#99712)
  pythongh-100174: [Enum] Correct PowersOfThree example. (pythonGH-100178)
  ...
@hauntsaninja
Copy link
Contributor

Thanks, looks like this is completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mdboom @markshannon @hauntsaninja