[bug] uwsgi.opt doesn't return list for multi-item options in python 3 #1374
Labels
Comments
|
Is there any fix or workaround for this issue? It seems it's still present in uwsgi 2.0.18. Thanks! |
xrmx
added a commit
that referenced
this issue
Sep 18, 2019
python: use consistent key type for opt dict manipulations - fixes #1374
outscale-fne
pushed a commit
to outscale-fne/uwsgi
that referenced
this issue
Oct 2, 2019
…bit#1374 PyString_FromString is defined as PyBytes_FromString in uwsgi_python.h What happens in Python 3 during the population of the opt_dict is that we first check if a byte object, representing the key is in the dict. If there is none, we use PyDict_SetItemString to set it. However, as the docs say, PyDict_SetItemString will convert the key using PyUnicode_FromString and we actually put a unicode key in the dict[1]. Therefore, when we check the "same" key again, we check again for the "same" key as bytes object we don't find it and end up overwriting it instead of doing the list promotion dance. Attached patch fixes this by using PyDict_SetItem and PyDict_GetItem with a consistent key type. For Python 3, a unicode object is used as key as this is the backwards compatible thing to do. Mini tester: import uwsgi def application(env, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) for k, v in uwsgi.opt.items(): yield "{} {!r} ({})\n".format(k, v, type(v)).encode("utf-8") yield "{} {}\n".format(uwsgi.version, type(uwsgi.version)).encode("utf-8") yield "{} {}\n".format(uwsgi.hostname, type(uwsgi.hostname)).encode("utf-8") yield b"END" What is a bit rough is that in Python 3 the actual values for uwsgi.opt entries end-up being all bytes, but that has always been like this... [1] https://docs.python.org/3.5/c-api/dict.html#c.PyDict_SetItemString
Fran6nd
pushed a commit
to outscale-fne/uwsgi
that referenced
this issue
Oct 2, 2019
…bit#1374 PyString_FromString is defined as PyBytes_FromString in uwsgi_python.h What happens in Python 3 during the population of the opt_dict is that we first check if a byte object, representing the key is in the dict. If there is none, we use PyDict_SetItemString to set it. However, as the docs say, PyDict_SetItemString will convert the key using PyUnicode_FromString and we actually put a unicode key in the dict[1]. Therefore, when we check the "same" key again, we check again for the "same" key as bytes object we don't find it and end up overwriting it instead of doing the list promotion dance. Attached patch fixes this by using PyDict_SetItem and PyDict_GetItem with a consistent key type. For Python 3, a unicode object is used as key as this is the backwards compatible thing to do. Mini tester: import uwsgi def application(env, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) for k, v in uwsgi.opt.items(): yield "{} {!r} ({})\n".format(k, v, type(v)).encode("utf-8") yield "{} {}\n".format(uwsgi.version, type(uwsgi.version)).encode("utf-8") yield "{} {}\n".format(uwsgi.hostname, type(uwsgi.hostname)).encode("utf-8") yield b"END" What is a bit rough is that in Python 3 the actual values for uwsgi.opt entries end-up being all bytes, but that has always been like this... [1] https://docs.python.org/3.5/c-api/dict.html#c.PyDict_SetItemString
Fran6nd
pushed a commit
to outscale-fne/uwsgi
that referenced
this issue
Oct 3, 2019
…bit#1374 PyString_FromString is defined as PyBytes_FromString in uwsgi_python.h What happens in Python 3 during the population of the opt_dict is that we first check if a byte object, representing the key is in the dict. If there is none, we use PyDict_SetItemString to set it. However, as the docs say, PyDict_SetItemString will convert the key using PyUnicode_FromString and we actually put a unicode key in the dict[1]. Therefore, when we check the "same" key again, we check again for the "same" key as bytes object we don't find it and end up overwriting it instead of doing the list promotion dance. Attached patch fixes this by using PyDict_SetItem and PyDict_GetItem with a consistent key type. For Python 3, a unicode object is used as key as this is the backwards compatible thing to do. Mini tester: import uwsgi def application(env, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) for k, v in uwsgi.opt.items(): yield "{} {!r} ({})\n".format(k, v, type(v)).encode("utf-8") yield "{} {}\n".format(uwsgi.version, type(uwsgi.version)).encode("utf-8") yield "{} {}\n".format(uwsgi.hostname, type(uwsgi.hostname)).encode("utf-8") yield b"END" What is a bit rough is that in Python 3 the actual values for uwsgi.opt entries end-up being all bytes, but that has always been like this... [1] https://docs.python.org/3.5/c-api/dict.html#c.PyDict_SetItemString
|
Hi! @xrmx |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi there,
If I set a multi-item option in
uwsgi.ini, e.g.In python 2,
uwsgi.opt['cache2']would return a list of 2 items:["name=1,items=100,blocksize=100", "name=2,items=200,blocksize=200"];While in python 3,
uwsgi.opt['cache2']only returns a string with the last item:"name=2,items=200,blocksize=200"Is there other ways that can get this option as a list? Otherwise I'll read
uwsgi.inifile directly.Thanks!
The text was updated successfully, but these errors were encountered: