🐛 Enhance tests for long SQL statements and fix errorOffset handling

[hoshinotsuyoshi]

The main commit of this PR is 28fe282.

Issue

• resolve: Parsing long Postgres Stored Procedures crashes the cli tool #874

Why is this change needed?

What would you like reviewers to focus on?

Testing Verification

manually tested on

• https://github.com/mastodon/mastodon/blob/v4.3.2/db/schema.rb
• https://gitlab.com/gitlab-org/gitlab-foss/-/raw/b94eb57589bd639078b783c296642e68dfb91780/db/structure.sql

What was done

🤖 Generated by PR Agent at 6253721

• Fixed parsing issues for long SQL statements exceeding CHUNK_SIZE.
• Enhanced error handling with improved retryOffset logic.
• Added tests for long "create function" SQL statements.
• Improved error messages for unexpected conditions in SQL parser.

Detailed Changes

	Relevant files
Tests	index.test.ts Add and enhance tests for long SQL statements                        frontend/packages/db-structure/src/parser/sql/postgresql/index.test.ts Updated test description for long "create table" statements. Added a new test for long "create function" statements. Ensured tests validate parsing without errors for statements exceeding CHUNK_SIZE. +23/-1    processSQLInChunks.test.ts Update tests for processSQLInChunks with retryOffset          frontend/packages/db-structure/src/parser/sql/postgresql/processSQLInChunks.test.ts Updated test cases to use retryOffset instead of errorOffset. Enhanced test descriptions for chunk processing scenarios. +5/-5
Bug fix	index.ts Refactor error handling and improve SQL parser logic          frontend/packages/db-structure/src/parser/sql/postgresql/index.ts Replaced errorOffset with retryOffset for better error handling. Improved error messaging for unexpected conditions. Added comments explaining readOffset and retryOffset usage. +14/-6    processSQLInChunks.ts Refactor chunk processing logic with retryOffset                  frontend/packages/db-structure/src/parser/sql/postgresql/processSQLInChunks.ts Replaced errorOffset with retryOffset in chunk processing logic. Added detailed comments for return values and retry logic. Improved error handling for incomplete SQL statements. +8/-4
Documentation	large-ligers-warn.md Add changeset for SQL parsing bug fix                                        .changeset/large-ligers-warn.md Added changeset entry for fixing long SQL statement parsing. Referenced issue Parsing long Postgres Stored Procedures crashes the cli tool #874 in the changeset. +5/-0

Additional Notes

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[changeset-bot]

🦋 Changeset detected

Latest commit: 6253721

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@liam-hq/db-structure	Patch
@liam-hq/cli	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
liam-app	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 21, 2025 10:17am
liam-erd-sample	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 21, 2025 10:17am

4 Skipped Deployments

Name	Status	Preview	Comments	Updated (UTC)
liam-docs	⬜️ Ignored (Inspect)	Visit Preview		Mar 21, 2025 10:17am
test-liam-app	⬜️ Ignored (Inspect)			Mar 21, 2025 10:17am
test-liam-docs	⬜️ Ignored (Inspect)			Mar 21, 2025 10:17am
test-liam-erd-sample	⬜️ Ignored (Inspect)			Mar 21, 2025 10:17am

[supabase]

Updates to Preview Branch (fix-874-error) ↗︎

Deployments	Status	Updated
Database	✅	Fri, 21 Mar 2025 10:13:42 UTC
Services	✅	Fri, 21 Mar 2025 10:13:42 UTC
APIs	✅	Fri, 21 Mar 2025 10:13:42 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Fri, 21 Mar 2025 10:13:42 UTC
Migrations	✅	Fri, 21 Mar 2025 10:13:43 UTC
Seeding	✅	Fri, 21 Mar 2025 10:13:43 UTC
Edge Functions	✅	Fri, 21 Mar 2025 10:13:43 UTC

---

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

[qodo-merge-pro-for-open-source]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅ 874 - Fully compliant Compliant requirements: Fix bug where parsing long Postgres Stored Procedures crashes the CLI tool Ensure the CLI tool can handle approximately 700-line functions in PostgreSQL Fix the "UnexpectedCondition" error that occurs during parsing

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Error Message Clarity The error message for unexpected conditions has been improved, but it might be helpful to include more context about what caused the condition to help with future debugging. throw new Error( 'UnexpectedCondition. parse_tree.stmts.length > 0 && parseError !== null', )

[qodo-merge-pro-for-open-source]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Handle errors during retry When retryOffset is not null and the chunk size is being decreased, the code should also check if there are errors and handle them appropriately. Currently, errors from the callback are only collected when increasing chunk size exceeds a threshold, but not when decreasing reaches zero. frontend/packages/db-structure/src/parser/sql/postgresql/processSQLInChunks.ts [47-53] const [retryOffset, readOffset, errors] = await callback(chunk) if (retryOffset !== null) { if (retryDirection === retryDirectionValues.decrease) { currentChunkSize-- if (currentChunkSize === 0) { + if (errors.length > 0) { + processErrors.push(...errors) + break + } retryDirection = retryDirectionValues.increase Apply this suggestion Suggestion importance[1-10]: 8 __ Why: This suggestion addresses an important error handling gap in the retry logic. When decreasing chunk size reaches zero, errors from the callback aren't collected, potentially hiding parsing issues. The fix ensures errors are properly propagated in all retry scenarios.	Medium
	Prevent potential NaN assignment The code sets readOffset to lastStmt?.stmt_location - 1 without checking if stmt_location is undefined, which could result in readOffset being set to NaN if stmt_location is undefined. This would happen in cases where the statement is incomplete but doesn't have a location defined. frontend/packages/db-structure/src/parser/sql/postgresql/index.ts [52-59] if (lastStmt?.stmt_len === undefined) { isLastStatementComplete = false if (lastStmt?.stmt_location === undefined) { retryOffset = 0 // no error, but the statement is not complete return [retryOffset, readOffset, errors] } - readOffset = lastStmt?.stmt_location - 1 + readOffset = lastStmt.stmt_location - 1 } Apply this suggestion Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential bug where readOffset could be assigned NaN if stmt_location is undefined. The fix uses non-optional chaining since we've already verified stmt_location is defined, improving type safety and preventing runtime errors.	Medium
Learned best practice	Include actual values in error messages to improve debugging and error diagnosis The error message provides information about the condition but doesn't include details about the actual values that caused the error. For better debugging, include the actual values of parse_tree.stmts.length and parseError in the error message to help diagnose issues more effectively. frontend/packages/db-structure/src/parser/sql/postgresql/index.ts [35-39] if (parse_tree.stmts.length > 0 && parseError !== null) { throw new Error( - 'UnexpectedCondition. parse_tree.stmts.length > 0 && parseError !== null', + `UnexpectedCondition: parse_tree.stmts.length (${parse_tree.stmts.length}) > 0 && parseError (${JSON.stringify(parseError)}) !== null`, ) } Apply this suggestion Suggestion importance[1-10]: 6	Low
More