Skip to content

Commit 2ed0f68

Browse files
authored
chore: do not log stack traces for HttpException (#235)
This improves on #224. My first attempt dumps the stack for any HttpException, which means any explicit 400, etc, that we raise. What we really want is the stack trace for unexpected errors, that is, anything that isn't HttpException. * Move the stack dump to the base Exception handler * Log `e.detail` in the HttpException handler. This is also important - we want to see what the user ran into. * Disable the `uvicorn.error` logger, which also dumps stack on any exception. We want that to go through our handler. To test, use the curl commands from the last pr. The HttpException (filetype not supported error) should cleanly show as a one liner: ``` 2023-09-15 12:06:13,570 unstructured_api ERROR Unable to process logger_config.yaml: File type None is not supported. 2023-09-15 12:06:13,570 127.0.0.1:58003 POST /general/v0/general HTTP/1.1 - 400 Bad Request ``` The 500 error should show this one liner along with *a single* instance of the stack: ``` File "/Users/austin/repos/pipeline-api/prepline_general/api/general.py", line 386, in pipeline_api raise e File "/Users/austin/repos/pipeline-api/prepline_general/api/general.py", line 367, in pipeline_api elements = partition( File "/Users/austin/.pyenv/versions/pipeline-api/lib/python3.10/site-packages/unstructured/partition/auto.py", line 348, in partition raise ValueError( ValueError: Detected a JSON file that does not conform to the Unstructured schema. partition_json currently only processes serialized Unstructured output. 2023-09-15 12:00:09,585 unstructured_api ERROR Detected a JSON file that does not conform to the Unstructured schema. partition_json currently only processes serialized Unstructured output. 2023-09-15 12:00:09,585 127.0.0.1:57735 POST /general/v0/general HTTP/1.1 - 500 Internal Server Error ```
1 parent ee82bb7 commit 2ed0f68

File tree

2 files changed

+18
-10
lines changed

2 files changed

+18
-10
lines changed

CHANGELOG.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
1-
## 0.0.44-dev1
1+
## 0.0.44
22

33
* Bump unstructured to 0.10.14
44
* Improve parallel mode retry handling
5+
* Improve logging during error handling. We don't need to log stack traces for expected errors.
56

67
## 0.0.43
78

prepline_general/api/app.py

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,18 +18,16 @@
1818
openapi_url="/general/openapi.json",
1919
)
2020

21+
# Note(austin) - This logger just dumps exceptions
22+
# We'd rather handle those below
23+
uvicorn_logger = logging.getLogger("uvicorn.error")
24+
uvicorn_logger.disabled = True
25+
2126

2227
# Catch all HTTPException for uniform logging and response
2328
@app.exception_handler(HTTPException)
2429
async def http_error_handler(request: Request, e: HTTPException):
25-
trace = traceback.format_exc()
26-
27-
# Note(austin) - If ENV is set, dump the stack in json
28-
# for nicer parsing. Soon we'll just have a json logger do this.
29-
if os.environ.get("ENV") in ["dev", "prod"]:
30-
trace = json.dumps(trace)
31-
32-
logger.error(trace)
30+
logger.error(e.detail)
3331

3432
return JSONResponse(
3533
status_code=e.status_code,
@@ -38,9 +36,18 @@ async def http_error_handler(request: Request, e: HTTPException):
3836

3937

4038
# Note(austin) - Convert any other errors to HTTPException
41-
# to be handled above
39+
# to be handled above, and log the stack trace
4240
@app.exception_handler(Exception)
4341
async def error_handler(request: Request, e: Exception):
42+
trace = traceback.format_exc()
43+
44+
# Note(austin) - If ENV is set, dump the stack in json
45+
# for nicer parsing. Soon we'll just have a json logger do this.
46+
if os.environ.get("ENV") in ["dev", "prod"]:
47+
trace = json.dumps(trace)
48+
49+
logger.error(trace)
50+
4451
error = HTTPException(
4552
status_code=500,
4653
detail=str(e)

0 commit comments

Comments
 (0)