Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: print unexpected error traceback in debug logs #344

Merged
merged 1 commit into from
Jan 7, 2025

Conversation

JamesGuthrie
Copy link
Member

@JamesGuthrie JamesGuthrie commented Jan 7, 2025

When debugging #334 the first thing I needed was a backtrace of the reported error. I added this code to print the backtrace to the debug logs. The resulting log looks as follows:

2025-01-06 14:51:23 [error    ] unexpected error: expected string or bytes-like object
2025-01-06 14:51:23 [debug    ] Traceback (most recent call last):
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/cli.py", line 319, in vectorizer_worker
2025-01-06 14:51:23 [debug    ]     run_vectorizer(
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/cli.py", line 153, in run_vectorizer
2025-01-06 14:51:23 [debug    ]     results = event_loop.run_until_complete(
2025-01-06 14:51:23 [debug    ]   File "/nix/store/53syhg1by396dm23a1in31xqfb62yz0d-python3-3.10.15/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
2025-01-06 14:51:23 [debug    ]     return future.result()
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/cli.py", line 148, in run_workers
2025-01-06 14:51:23 [debug    ]     return await asyncio.gather(*tasks)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/vectorizer/vectorizer.py", line 470, in run
2025-01-06 14:51:23 [debug    ]     items_processed = await self._do_batch(conn)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/.devenv/state/venv/lib/python3.10/site-packages/ddtrace/internal/compat.py", line 178, in func_wrapper
2025-01-06 14:51:23 [debug    ]     result = await coro(*args, **kwargs)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/vectorizer/vectorizer.py", line 550, in _do_batch
2025-01-06 14:51:23 [debug    ]     raise e
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/vectorizer/vectorizer.py", line 511, in _do_batch
2025-01-06 14:51:23 [debug    ]     num_chunks = await self._embed_and_write(conn, items)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/.devenv/state/venv/lib/python3.10/site-packages/ddtrace/internal/compat.py", line 178, in func_wrapper
2025-01-06 14:51:23 [debug    ]     result = await coro(*args, **kwargs)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/vectorizer/vectorizer.py", line 619, in _embed_and_write
2025-01-06 14:51:23 [debug    ]     records, errors = await self._generate_embeddings(items)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/vectorizer/vectorizer.py", line 711, in _generate_embeddings
2025-01-06 14:51:23 [debug    ]     chunks = self.vectorizer.config.chunking.into_chunks(item)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/pgai/vectorizer/chunking.py", line 131, in into_chunks
2025-01-06 14:51:23 [debug    ]     return self._chunker.split_text(text)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/.devenv/state/venv/lib/python3.10/site-packages/langchain_text_splitters/character.py", line 118, in split_text
2025-01-06 14:51:23 [debug    ]     return self._split_text(text, self._separators)
2025-01-06 14:51:23 [debug    ]   File "/home/james/workspace/pgai/projects/pgai/.devenv/state/venv/lib/python3.10/site-packages/langchain_text_splitters/character.py", line 88, in _split_text
2025-01-06 14:51:23 [debug    ]     if re.search(_separator, text):
2025-01-06 14:51:23 [debug    ]   File "/nix/store/53syhg1by396dm23a1in31xqfb62yz0d-python3-3.10.15/lib/python3.10/re.py", line 200, in search
2025-01-06 14:51:23 [debug    ]     return _compile(pattern, flags).search(string)
2025-01-06 14:51:23 [debug    ] TypeError: expected string or bytes-like object

@JamesGuthrie JamesGuthrie requested a review from a team as a code owner January 7, 2025 09:25
@JamesGuthrie JamesGuthrie merged commit d9bdcd6 into main Jan 7, 2025
5 checks passed
@JamesGuthrie JamesGuthrie deleted the jg/print-unexpected-backtrace branch January 7, 2025 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants