-
Notifications
You must be signed in to change notification settings - Fork 691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backports for 2.0.27 #2678
Backports for 2.0.27 #2678
Commits on Sep 8, 2024
-
pyuwsgi: avoid interleaving pywsgi threadstate
In all versions of pyuwsgi at the moment the first fork has a NULL threadstate due to uwsgi_python_master_fixup which calls UWSGI_RELEASE_GIL (expanded to PyEval_SaveThread -- which drops the GIL and sets threadstate to NULL). This is called during uwsgi_setup. After uwsgi_setup was returning, PyThreadState_Swap was restoring the pyuwsgi threadstate (in both the original and worker processes) Future forks would have the pyuwsgi threadstate active (from the restoration at PyThreadState_Swap) in python versions < 3.12 this wasn't an issue. In 3.12+ the PyEval_RestoreThread would attempt to take_gil and then block forever on the GIL mutex (despite it actually holding it? due to the fork state from the parent process). Bisecting cpython showed that python/cpython@92d8bff slightly changed behaviour of PyThreadState_Swap (it now additionally manages GIL state: unlocking the previous threadstate and locking the new threadstate). Putting a log line in the PyThreadState_Swap showed a suspicious swapping from oldts=123123 to newts=123123 (swapping from its own threadstate to itself?); this is because after forking control would be given back to the original threadstate (which mostly worked but was in UB territory given the GIL state). In 3.11 the threadstate that was restored after the PyThreadState_Swap did not have the GIL locked (technically this could have allowed a data race if threads existed before starting uwsgi via pyuwsgi). In 3.12 since PyThreadState_Swap was changed to release the old threadstate's GIL and acquire the GIL in the new threadstate this meant that the saved threadstate had ->locked = 1 (which is sort of an invalid state?). As far as I can tell there aren't any public apis to undo this and "restore" the 3.11 behaviour precisely. Then later it would try and lock (despite already being -> locked = 1) and deadlock against itself this is actually called out on the docs: If the lock has been created, the current thread must not have acquired it, otherwise deadlock ensues. To fix this once we call uwsgi_setup we never give control back to the original pyuwsgi threadstate avoiding the Swap dance entirely.
Configuration menu - View commit details
-
Copy full SHA for 7206318 - Browse repository at this point
Copy the full SHA 7206318View commit details -
Fix gracefully_kill_them_all with running requests
With the following: ``` wsgi = app:app http = :8000 master = true processes = 2 harakiri = 15 harakiri-verbose = true harakiri-graceful-timeout = 15 harakiri-graceful-signal = 15 max-requests = 100000 memory-report = true enable-threads = true threads = 4 enable-thread = true showconfig = true listen = 1024 post-buffering = 8192 buffer-size = 32768 lazy = true http-keepalive = 1 add-header = Connection: Keep-Alive http-timeout = 70 socket-timeout = 75 hook-master-start = unix_signal:15 gracefully_kill_them_all vacuum = true hook-master-start = unix_signal:15 gracefully_kill_them_all ``` kill -s 15 master-pid while request does not complete. fllowing is uwsgi log: ``` running "unix_signal:15 gracefully_kill_them_all" (master-start)... WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55de52cb68f0 pid: 143521 (default app) WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55de52cb68f0 pid: 143520 (default app) graceful shutdown triggered... Gracefully killing worker 1 (pid: 143520)... gateway "uWSGI http 1" has been buried (pid: 143522) Gracefully killing worker 2 (pid: 143521)... worker 1 buried after 1 seconds {address space usage: 277147648 bytes/264MB} {rss usage: 34459648 bytes/32MB} [pid: 143521|app: 0|req: 1/1] 127.0.0.1 () {28 vars in 291 bytes } [Mon Jul 22 09:13:34 2024] GET / => generated 11 bytes in 6036 msecs (HTTP/1.1 200) 3 headers in 103 bytes (1 switches on core 0) worker 2 buried after 4 seconds goodbye to uWSGI. ``` The gateway process(pid=143522) is closed prematurely, causing the client to be unable to correctly receive the request result. I think you should wait for the worker process to shut down before shutting down the gateway process Fix unbit#2656
Configuration menu - View commit details
-
Copy full SHA for 8f1d0e5 - Browse repository at this point
Copy the full SHA 8f1d0e5View commit details -
Fix --catch-exceptions causing a segfault in Python 3.5.
This bug arose due to the fact that the traceback module was overhauled in python 3.5: https://hg.python.org/cpython/rev/73afda5a4e4c The crash was due to PyTuple_GetItem returning NULL as a traceback is no longer a list of tuples but rather a list of objects (which support indexing).
Configuration menu - View commit details
-
Copy full SHA for 738030d - Browse repository at this point
Copy the full SHA 738030dView commit details -
PHP: Add support for uwsgi.disconnect() function
This allows to run some code after finishing request. As described in https://symfony.com/doc/current/components/http_kernel.html#8-the-kernel-terminate-event
Configuration menu - View commit details
-
Copy full SHA for 0fd9f8b - Browse repository at this point
Copy the full SHA 0fd9f8bView commit details -
plugins/python: use PyOS_*Fork stable API functions on 3.7+
To avoid calling functions made private from 3.13+. And probably fixing issues with C extensions.
Configuration menu - View commit details
-
Copy full SHA for 92f207c - Browse repository at this point
Copy the full SHA 92f207cView commit details -
core/uwsgi: set enable threads by default
As it should have been since years :_)
Configuration menu - View commit details
-
Copy full SHA for 1caea16 - Browse repository at this point
Copy the full SHA 1caea16View commit details -
plugins/python: handle cframe removal from CPython thread state
Use current_frame instead
Configuration menu - View commit details
-
Copy full SHA for d8ea43b - Browse repository at this point
Copy the full SHA d8ea43bView commit details -
That is still in beta but removed private C APIs we were using.
Configuration menu - View commit details
-
Copy full SHA for 79fc49e - Browse repository at this point
Copy the full SHA 79fc49eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 65577d6 - Browse repository at this point
Copy the full SHA 65577d6View commit details
Commits on Sep 15, 2024
-
Minor rework of Python 3.13 support
As mentioned in unbit#2655, this changes the way support for python 3.13 is handled. Instead of handling python 3.13 as a minor change from 3.12, and handling support for it under the 3.12 `#ifdef` blocks, this breaks out 3.13 into its own block, apart from 3.12. This makes the code a bit more verbose, but makes it easier to see what the structures look like for different python versions.
Configuration menu - View commit details
-
Copy full SHA for dbeeaf8 - Browse repository at this point
Copy the full SHA dbeeaf8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 555304d - Browse repository at this point
Copy the full SHA 555304dView commit details -
Configuration menu - View commit details
-
Copy full SHA for c5e6ce2 - Browse repository at this point
Copy the full SHA c5e6ce2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 86f2499 - Browse repository at this point
Copy the full SHA 86f2499View commit details -
[xrmx: add 36e045c to reduce delta with master]
Configuration menu - View commit details
-
Copy full SHA for d6f2e89 - Browse repository at this point
Copy the full SHA d6f2e89View commit details