cgitb sends a bogus HTTP header if the app crashes before finishing headers #52950

stutzbach · 2010-05-13T14:12:51Z

BPO	8704
Nosy	@orsenthil, @cthart

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2010-05-13.14:12:50.822>
labels = ['easy', '3.8', 'type-bug', 'library']
title = 'cgitb sends a bogus HTTP header if the app crashes before finishing headers'
updated_at = <Date 2022-03-15.14:23:55.676>
user = 'https://bugs.python.org/stutzbach'

bugs.python.org fields:

activity = <Date 2022-03-15.14:23:55.676>
actor = 'cthart'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2010-05-13.14:12:50.822>
creator = 'stutzbach'
dependencies = []
files = []
hgrepos = []
issue_num = 8704
keywords = ['easy']
message_count = 9.0
messages = ['105633', '105690', '105705', '193189', '294027', '364186', '376811', '389031', '415246']
nosy_count = 10.0
nosy_names = ['orsenthil', 'stutzbach', 'igs', 'meatballhat', 'ysj.ray', 'p0lar_bear', '\xd0\x90\xd1\x80\xd1\x82\xd1\x83\xd1\x80 \xd0\x9a\xd0\xbb\xd0\xb5\xd1\x81\xd1\x83\xd0\xbd', 'coyot', 'Ryan Tu', 'cthart']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8704'
versions = ['Python 3.8']

stutzbach · 2010-05-13T14:12:50Z

If the CGI script crashes before finishing the headers, cgitb will emit invalid HTTP headers before showing the error message. Below are HTTP headers I received, captured with a packet sniffer. Note the "<--: spam".

HTTP/1.1 200 OK
Date: Thu, 13 May 2010 14:00:42 GMT
Server: Apache/2.2.9
<!--: spam
Vary: Accept-Encoding
Cache-Control: max-age=0
Expires: Thu, 13 May 2010 14:00:42 GMT
Set-Cookie: ref=; path=/; HttpOnly
Transfer-Encoding: chunked
Content-Type: text/html

That string it emitted by cgitb.reset(), which is trying to reset the browser to a sane state so the error message will be shown. The problem can be easily fixed by having cgitb.reset() emit two CRLF pairs first, to ensure that we're done with the headers and emitting content:

return '''<!--: spam
+ return '''\r\n\r\n<!--: spam

ysjray · 2010-05-14T07:57:48Z

Yes, I saw the "<!--: spam" string in headers, but it seems that this string doesn't make problems. The displaying page is correct.

But after I apply the changes you mentioned:

return '''<!--: spam
+ return '''\r\n\r\n<!--: spam

I got text/plain output, and the response headers are like this:

Date	Fri, 14 May 2010 07:30:03 GMT
Server	Apache/2.2.15 (Unix)
Keep-Alive	timeout=5, max=100
Connection	Keep-Alive
Transfer-Encoding	chunked
Content-Type	text/plain

And the content is like this:

<!--: spam
Content-Type: text/html

......

So the hole page is not displayed correctly!

Is there any problem with me?

stutzbach · 2010-05-14T13:06:35Z

It displays correctly in some browsers, yes, but not everything that speaks HTTP is a browser. For example, the invalid header makes C#'s WebRequest throw an exception.

I hadn't noticed the 'Content-Type' on the next line of the string output by reset(). That does make things more complicated.

We could put the "Content-Type: text/html" first, but the downside is that it will be output as visible content if a script crashes after the headers have been emitted.

I'm not sure if that's better or worse than emitting an invalid header.

p0larbear · 2013-07-16T20:44:41Z

I get similar results if my CGI script sends a Content-Type header of anything besides "text/html", e.g. print('Content-Type: text/json').

ghost · 2017-05-20T10:40:20Z

Apache started strict check of headers ch
aracters to be valid recently. That causes it fail on "<--: spam".

[Sat May 20 13:09:23.056673 2017] [http:error] [pid 26379] [client 12.34.567.41:60988] AH02429: Response header name '<!--' contains invalid characters, aborting request, referer: http://example.com/

http://apache-http-server.18135.x6.nabble.com/Bug-60863-New-Apache-proxy-2-4-25-can-disable-header-check-Set-Cookie-td5036120.html

The workaround is to put:
HttpProtocolOptions Unsafe
line into your apache .conf

RyanTu · 2020-03-14T15:51:46Z

#Maybe not a good solution
I do not know the should we delete the code in cgitb.py or adjust the configration of apache httpd. My solution is deleting some code as follows:

        return '''
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->
</font> </font> </font> </script> </object> </blockquote> </pre>
</table> </table> </table> </table> </table> </font> </font> </font>'''

Then it works very well, and it has good view.Anyone know what is the situation in ngix?

igs · 2020-09-12T19:25:29Z

As mentioned above standard Apache does not accept the extra characters anymore and produces '500 internal error'. So the normal behaviour of this module makes things worse in most cases instead of being helpful.

coyot · 2021-03-18T17:21:55Z

Ran into this also, got:

AH02429: Response header name '<!--' contains invalid characters, aborting request

cthart · 2022-03-15T14:23:56Z

This module is scheduled to be removed by Python 3.13 (although I preseonally am of the opinion that it is a useful module and would like to see it brought up-to-date).
Is reset() even necessary anymore? Can't the same results be achieved with CSS since we are in the third decade of the 2000s after all?

hugovk · 2022-04-11T11:28:37Z

Yep, deprecated in 3.11 and removed in 3.13: see PEP 594 – Removing dead batteries from the standard library, #91217 and #32410.

@cthart Some good news: a fork has been made by @jackrosenthal:

Contributions are accepted, but should be focused on bug fixes instead of new features or major refactoring.

Because there's no maintainer for this module in CPython, perhaps it would be better to close this issue and instead put energy into the fork?

AlexWaygood · 2022-04-12T09:20:00Z

Because there's no maintainer for this module in CPython, perhaps it would be better to close this issue and instead put energy into the fork?

I think so. I'm going to close this issue for now, but I'm happy to reopen if somebody wants to persuasively argue for why this issue should be fixed in CPython despite the deprecation.

Cc. @cthart

cthart · 2022-04-12T10:37:03Z

Good call. A fork available via GitHub / PyPi is a perfect solution for this library. My interest is only in the cgitb part, which I believe could be modernised to use CSS for displaying Python stack traces in a marked up format.

stutzbach mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error easy labels May 13, 2010

RyanTu mannequin added the 3.8 only security fixes label Mar 14, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

AlexWaygood closed this as completed Apr 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cgitb sends a bogus HTTP header if the app crashes before finishing headers #52950

cgitb sends a bogus HTTP header if the app crashes before finishing headers #52950

stutzbach mannequin commented May 13, 2010

stutzbach mannequin commented May 13, 2010

ysjray mannequin commented May 14, 2010

stutzbach mannequin commented May 14, 2010

p0larbear mannequin commented Jul 16, 2013

ghost commented May 20, 2017

RyanTu mannequin commented Mar 14, 2020

igs mannequin commented Sep 12, 2020

coyot mannequin commented Mar 18, 2021

cthart mannequin commented Mar 15, 2022

hugovk commented Apr 11, 2022 •

edited

Loading

AlexWaygood commented Apr 12, 2022

cthart commented Apr 12, 2022

cgitb sends a bogus HTTP header if the app crashes before finishing headers #52950

cgitb sends a bogus HTTP header if the app crashes before finishing headers #52950

Comments

stutzbach mannequin commented May 13, 2010

stutzbach mannequin commented May 13, 2010

ysjray mannequin commented May 14, 2010

stutzbach mannequin commented May 14, 2010

p0larbear mannequin commented Jul 16, 2013

ghost commented May 20, 2017

RyanTu mannequin commented Mar 14, 2020

igs mannequin commented Sep 12, 2020

coyot mannequin commented Mar 18, 2021

cthart mannequin commented Mar 15, 2022

hugovk commented Apr 11, 2022 • edited Loading

AlexWaygood commented Apr 12, 2022

cthart commented Apr 12, 2022

hugovk commented Apr 11, 2022 •

edited

Loading