Gracefully error out when there are concurrent profilers

[albertyw]

Fixes #682
Unblocks #686

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (23ff43b) 86.65% compared to head (6537b6e) 86.67%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #692      +/-   ##
==========================================
+ Coverage   86.65%   86.67%   +0.02%     
==========================================
  Files          52       52              
  Lines        2113     2117       +4     
==========================================
+ Hits         1831     1835       +4     
  Misses        282      282              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[50-Course]

Hello again @albertyw,

this patch permanently fixes the Django==5.1 & python^=3.12 pipeline issue?

[50-Course]

Update: Great progress! The "Gracefully error out when there are concurrent profilers" job (#722) is now passing, which has also unblocked #686.

[robinchow]

I'd like to point out while this stop silk from interfering with processing a request, it also ends up not profiling that particular request. Prior to Python 3.11, the effect was similar but different (it would disable/stop the first profiler, and then start this current one). Neither outcome seems great, especially since this problem will happen when Django is processing multiple long-running requests at the same time (I've made a minimal repo here to highlight this issue)

I think at a minimum, the documentation should tell users that this issue exists. In terms of a proper fix, I'm not sure at the moment if that's possible without using another profiler that properly supports multithreaded applications (such as https://github.com/sumerc/yappi)

[50-Course]

Thank you for pointing out this oversight, @robinchow. I'd take a look into yappi

[albertyw]

Prior to Python 3.11, the effect was similar but different (it would disable/stop the first profiler, and then start this current one)

3.11 or 3.12? This code gives an error in python 3.12 but no error in 3.11:

import cProfile
p1 = cProfile.Profile()
p1.enable()
p2 = cProfile.Profile()
p2.enable()

[robinchow]

@albertyw ah yes, apologies, prior to Python 3.12.