Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Pg-kit] Fix malformed array literal error on indexes #2884

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Kratious
Copy link

@Kratious Kratious commented Aug 31, 2024

Fix #2715, fix #2744

The issue is the expression text to array conversion happening in (('{' || pg_get_expr(i.indexprs, i.indrelid)|| '}')::text[])[k.i]. Commas in an expression become delimiters and split the expression up prematurely. Some special characters like double quotes can cause the malformed array literal errors.

The postgres function pg_get_indexdef does what the snippet above is trying to do, but safely.

I tried adding a test to introspect/pg.test.ts, but some of the indexes get added to list of "alteredIndexes" as the indexes get transformed a little bit and don't pass the json diff snapshots check (e.g. lower(first_name) becomes lower(\"first_name\") and gets flagged as altered).

Which is why the test below has a expect(statements.length).toBe(10);. I decided to leave the test out of my PR for now. This test fails in the main branch but passes with this fix at the very least.

test('basic index test', async () => {
	const client = new PGlite();

	const schema = {
		users: pgTable('users', {
			firstName: text('first_name'),
			lastName: text('last_name'),
			data: jsonb('data'),
		}, (table) => ({
			singleColumn: index('single_column').on(table.firstName),
			multiColumn: index('multi_column').on(table.firstName, table.lastName),
			singleExpression: index('single_expression').on(sql`lower(${table.firstName})`),
			multiExpression: index('multi_expression').on(sql`lower(${table.firstName})`, sql`lower(${table.lastName})`),
			expressionWithComma: index('expression_with_comma').on(sql`(lower(${table.firstName}) || ', '::text || lower(${table.lastName}))`),
			expressionWithDoubleQuote: index('expression_with_double_quote').on(sql`('"'::text || ${table.firstName})`),
			expressionWithJsonbOperator: index('expression_with_jsonb_operator').on(sql`(${table.data} #>> '{a,b,1}'::text[])`),
		})),
	};

	const { statements, sqlStatements } = await introspectPgToFile(
		client,
		schema,
		'basic-index-introspect',
	);

	expect(statements.length).toBe(10);
	expect(sqlStatements.length).toBe(10);
});

The main issue is the expression text to array conversion happening in the edited line.
Commas in an expression become delimiters and split the expression up prematurely.
Some special characters like double quotes can cause the malformed array literal errors.

The postgres function pg_get_indexdef does what the snippet above is trying to do, but safely.
@juspky
Copy link

juspky commented Sep 5, 2024

@AndriiSherman could you please review this when you have spare time?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants