[WIP] RefactoringAICaller class for better using reasoning models

[KennyDizi]

PR Type

Enhancement, Tests

---

Description

• Introduced model-specific behavior lists to AICaller.

  • Added USER_MESSAGE_ONLY_MODELS, NO_SUPPORT_TEMPERATURE_MODELS, and NO_SUPPORT_STREAMING_MODELS.
  • Adjusted logic to handle models with specific constraints.

• Refactored call_model to dynamically adapt parameters based on model capabilities.

  • Removed unsupported parameters like temperature or stream for certain models.
  • Adjusted message structure for models supporting only user messages.

• Enhanced test coverage for AICaller.

  • Added parameterized tests for model-specific behaviors.
  • Verified handling of API base inclusion, message structures, and non-streaming responses.

• Added a new module-level initialization for model-specific lists in cover_agent/__init__.py.

---

Changes walkthrough 📝

	Relevant files
Enhancement	AICaller.py Refactored `AICaller` for model-specific behavior                cover_agent/AICaller.py Added model-specific behavior lists for parameter handling. Refactored call_model to adapt to model constraints. Adjusted message structure for user-message-only models. Removed unsupported parameters dynamically. +21/-12  __init__.py Added model-specific behavior lists                                            cover_agent/init.py Added USER_MESSAGE_ONLY_MODELS list for user-message-only models. Added NO_SUPPORT_TEMPERATURE_MODELS list for models without temperature support. Added NO_SUPPORT_STREAMING_MODELS list for models without streaming support. +23/-0
Tests	test_AICaller.py Enhanced test coverage for `AICaller`                                        tests/test_AICaller.py Added parameterized tests for model-specific behaviors. Verified handling of API base inclusion and message structures. Tested non-streaming responses and max completion tokens. Ensured environment variable models are correctly initialized. +120/-47

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro-for-open-source]

PR Reviewer Guide 🔍

(Review updated until commit fd3a2f3)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Inconsistent Behavior The code adds model-specific behavior lists but still has hardcoded model names in some places. For example, line 106-108 still have model-specific adjustments without checking against the new lists. completion_params["max_completion_tokens"] = 2 * self.max_tokens # completion_params["reasoning_effort"] = "high" completion_params.pop("max_tokens", None) # Remove 'max_tokens' if present Duplicate Models Some models appear in multiple lists which could lead to maintenance issues if model capabilities change. Consider implementing a more structured approach with model capability flags instead of separate lists. USER_MESSAGE_ONLY_MODELS = [ "deepseek/deepseek-reasoner", "o1-mini", "o1-mini-2024-09-12", "o1-preview" ] NO_SUPPORT_TEMPERATURE_MODELS = [ "deepseek/deepseek-reasoner", "o1-mini", "o1-mini-2024-09-12", "o1", "o1-2024-12-17", "o3-mini", "o3-mini-2025-01-31", "o1-preview" ] NO_SUPPORT_STREAMING_MODELS = [ "deepseek/deepseek-reasoner", "o1", "o1-2024-12-17", ]

[qodo-merge-pro-for-open-source]

PR Code Suggestions ✨

Latest suggestions up to fd3a2f3

Category	Suggestion	Impact
General	Remove unused variable assignment The local variable stream is modified but never used afterward. This modification has no effect on the function's behavior since the stream parameter is only used to initially set completion_params["stream"]. cover_agent/AICaller.py [102-108] # Model-specific adjustments if self.model in self.no_support_streaming_models: - stream = False completion_params["stream"] = False completion_params["max_completion_tokens"] = 2 * self.max_tokens # completion_params["reasoning_effort"] = "high" completion_params.pop("max_tokens", None) # Remove 'max_tokens' if present Apply this suggestion Suggestion importance[1-10]: 5 __ Why: The suggestion correctly identifies that the local variable stream is modified but never used afterward. Removing this unnecessary assignment improves code cleanliness without affecting functionality. This is a minor improvement to code quality.	Low
More

---

Previous suggestions

✅ Suggestions up to commit 99d10ea

Category	Suggestion	Impact
Possible issue	✅ Prevent token limit overflow Suggestion Impact: The commit addressed the token limit concern by refactoring how max_tokens is handled. Instead of hardcoding max_tokens in the method parameter, it's now a class attribute that can be set during initialization. This changes how max_completion_tokens is calculated, though it doesn't add the explicit limit check suggested. code diff: - def __init__(self, model: str, api_base: str = "", enable_retry=True): + def __init__( + self, model: str, api_base: str = "", enable_retry=True, max_tokens=16384 + ): """ Initializes an instance of the AICaller class. @@ -49,19 +51,19 @@ self.model = model self.api_base = api_base self.enable_retry = enable_retry + self.max_tokens = max_tokens self.user_message_only_models = USER_MESSAGE_ONLY_MODELS self.no_support_temperature_models = NO_SUPPORT_TEMPERATURE_MODELS self.no_support_streaming_models = NO_SUPPORT_STREAMING_MODELS @conditional_retry # You can access self.enable_retry here - def call_model(self, prompt: dict, max_tokens=16384, stream=True): + def call_model(self, prompt: dict, stream=True): """ Call the language model with the provided prompt and retrieve the response. Parameters: prompt (dict): The prompt to be sent to the language model. - max_tokens (int, optional): The maximum number of tokens to generate in the response. Defaults to 16384. stream (bool, optional): Whether to stream the response or not. Defaults to True. Returns: @@ -90,7 +92,7 @@ "messages": messages, "stream": stream, # Use the stream parameter passed to the method "temperature": 0.2, - "max_tokens": max_tokens, + "max_tokens": self.max_tokens, } # Remove temperature for models that don't support it @@ -101,7 +103,7 @@ if self.model in self.no_support_streaming_models: stream = False completion_params["stream"] = False - completion_params["max_completion_tokens"] = 2*max_tokens + completion_params["max_completion_tokens"] = 2 * self.max_tokens The code doubles max_tokens for max_completion_tokens without considering model limits. This could exceed model context windows. Add a check to ensure max_completion_tokens doesn't exceed model-specific limits. cover_agent/AICaller.py [101-106] if self.model in self.no_support_streaming_models: stream = False completion_params["stream"] = False - completion_params["max_completion_tokens"] = 2*max_tokens + max_completion = min(2*max_tokens, 32768) # Example limit, adjust per model + completion_params["max_completion_tokens"] = max_completion # completion_params["reasoning_effort"] = "high" completion_params.pop("max_tokens", None) # Remove 'max_tokens' if present [Suggestion has been applied] Suggestion importance[1-10]: 8 __ Why: The suggestion addresses a critical issue where doubling max_tokens could exceed model context limits, potentially causing runtime errors. Adding a safety cap is important for preventing failures.	Medium
General	Fix stream parameter inconsistency The code modifies the stream parameter after it's already been used in completion_params. Move the stream parameter modification before setting completion_params to maintain consistency. cover_agent/AICaller.py [90-103] +# Adjust stream parameter based on model support +if self.model in self.no_support_streaming_models: + stream = False + completion_params = { "model": self.model, "messages": messages, - "stream": stream, # Use the stream parameter passed to the method + "stream": stream, # Use the adjusted stream parameter "temperature": 0.2, "max_tokens": max_tokens, } -# Model-specific adjustments -if self.model in self.no_support_streaming_models: - stream = False - completion_params["stream"] = False - Suggestion importance[1-10]: 7 __ Why: The suggestion improves code reliability by fixing a logical flow issue where the stream parameter is modified after being used, which could lead to inconsistent behavior in streaming functionality.	Medium
Learned best practice	Add validation for numeric calculations to prevent potential issues with negative or zero values The calculation of max_completion_tokens should validate that max_tokens is positive to avoid potential issues with negative or zero values. Add a validation check before the multiplication. cover_agent/AICaller.py [104] -completion_params["max_completion_tokens"] = 2*max_tokens +completion_params["max_completion_tokens"] = 2 * max(0, max_tokens) Suggestion importance[1-10]: 6	Low

[KennyDizi]

Hi @EmbeddedDevops1 @qododavid, How do I enable the PR's CI/CD pipeline?

[EmbeddedDevops1]

I just enabled the CI pipeline to run. To run tests locally you can simply make test.

You will also want to run regression tests locally: https://github.com/qodo-ai/qodo-cover/blob/main/tests_integration/test_all.sh

[KennyDizi]

Thanks, @EmbeddedDevops1. I will continue to work to finish it soon.

[KennyDizi]

Sorry for the delay. I will update this PR with support reasoning effort. I will continue to work next weekend.

[KennyDizi]

@EmbeddedDevops1 I've adjusted the PR and ran the test command ./tests_integration/test_all.sh; it looks good on my end. However, would you please enable the pipeline for this PR again? Let's see.

[EmbeddedDevops1]

Looks like tests are failing. Also, do you have any sample runs with the new LLMs you’re adding? Perhaps you can add it in a comment?

[EmbeddedDevops1]

/review

[EmbeddedDevops1]

/improve

[qodo-merge-pro-for-open-source]

Persistent review updated to latest commit fd3a2f3