Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed no max output tokens #175

Merged
merged 4 commits into from
Jan 22, 2025

Conversation

Puchaczov
Copy link
Contributor

@Puchaczov Puchaczov commented Jan 18, 2025

Hey, thanks a lot for creating this project! I think I found a bug related to missing MaxOutputTokens field mapping, I fixed it both in the code and in the unit test.

I think i found another one but it's within Semantic Kernel connector library. The OllamaPromptExecutionSettings doesn't contain and thus you cannot send max output tokens, I will try to report it them

    [JsonPropertyName("num_predict")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public double? NumPredict
    {
        get => this._numPredict;

        set
        {
            this.ThrowIfFrozen();
            this._numPredict = value;
        }
    }

@awaescher
Copy link
Owner

Nice spot. Would you add a line to TryAddOllamaOption(...) for it as well? Just in case these settings are delivered as untyped argument.

@Puchaczov
Copy link
Contributor Author

Not sure if that's what you meant but I've extended TryAddOllamaOption to support max_output_tokens and added new test to cover all the options included

TryAddOllamaOption<bool?>(options, OllamaOption.UseMmap, v => request.Options.UseMmap = (bool?)v);
TryAddOllamaOption<bool?>(options, OllamaOption.VocabOnly, v => request.Options.VocabOnly = (bool?)v);
}
if (!(options?.AdditionalProperties?.Any() ?? false)) return request;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand you prefer jumping out here. I am okay with that but the combination of not and the coalesce operator is not that easy to recognize so far. Also I personally don't like having the return in the same line.

I would prefer something like this, what do you think?

var hasAdditionalProperties = options?.AdditionalProperties?.Any() ?? false;
if (!hasAdditionalProperties)
    return request;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I totally agree with you! fixed!

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Jakub

@awaescher awaescher merged commit 71ccc5e into awaescher:main Jan 22, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants