Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: get rid of nested parametrization in vectorizer tests #394

Merged

Conversation

Askir
Copy link
Contributor

@Askir Askir commented Jan 24, 2025

Refactors the vectorizer test suite to be more easily readable by:

  • Removing nested test_params fixture (num_items, concurrency, batch_size, chunking, formatting) in favor of direct parameters that are then passed to setup functions
  • Creating dedicated configuration functions (configure_openai_vectorizer, configure_ollama_vectorizer, etc.) to replace the fixtures
  • Using a common run_vectorizer_worker helper function standardizing CLI invocation

I also want to split the file up, but wondering how and if I should do that separately.

@Askir Askir force-pushed the jascha/ai-413-refactor-vectorizer-tests-to-use-less-fixtures-and branch 5 times, most recently from f9cb895 to fd3952e Compare January 27, 2025 11:38
@Askir Askir marked this pull request as ready for review January 27, 2025 11:52
@Askir Askir requested a review from a team as a code owner January 27, 2025 11:52
@@ -2053,7 +2053,6 @@ wheels = [

[[package]]
name = "pgai"
version = "0.5.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when i ran uv sync --all-extras this was removed, I didn't do that 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya I don't quite get what's going on there. I suspect that different minor versions (0.4 vs 0.5) of uv treat this parameter differently, and some write it in and others remove it.

Copy link
Member

@JamesGuthrie JamesGuthrie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow - this is so much better. Thank you!

cur.executemany(
"INSERT INTO blog (id, id2, content) VALUES (%s, %s, %s)", values
)
return table_name


@pytest.fixture(scope="session")
def ollama_connection_url():
def ollama_container_url_or_url():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me, what about just ollama_url?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeh fair, I was just confused why this is a fixture and not a constant 😅 which is why i put the container in there, but tbh fair to encapsulate and hide the details.

Copy link
Contributor

@smoya smoya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a late-Christmas present 💯 💯

@Askir Askir force-pushed the jascha/ai-413-refactor-vectorizer-tests-to-use-less-fixtures-and branch from fd3952e to 5b44a00 Compare January 27, 2025 12:13
@Askir Askir merged commit 0a399e2 into main Jan 27, 2025
5 checks passed
@Askir Askir deleted the jascha/ai-413-refactor-vectorizer-tests-to-use-less-fixtures-and branch January 27, 2025 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants