Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ollama trying to pull custom model #145

Open
shirofaii opened this issue Jan 14, 2025 · 2 comments
Open

ollama trying to pull custom model #145

shirofaii opened this issue Jan 14, 2025 · 2 comments
Labels
bug Something isn't working

Comments

@shirofaii
Copy link

Describe the bug

When you add custom model for ollama (with id "custom" for example) provider trying to pull this model and failed

Steps to reproduce the bug

Steps to reproduce:

  1. Create custom model with id="custom" for ollama
  2. Run clean ollama server without any model loaded
  3. Compile and run this code
using LangChain.Providers.Ollama;
using Ollama;

var provider = new OllamaProvider("http://127.0.0.1:11434/api");
var llm = new OllamaChatModel(provider, "custom");

var result = await llm.GenerateAsync("Hi!");

Actual result: provider trying to pull model "custom" and fail

Expected behavior

ollama load existed model "custom" and generate valid response

Screenshots

No response

NuGet package version

No response

Additional context

No response

@shirofaii shirofaii added the bug Something isn't working label Jan 14, 2025
@HavenDV
Copy link
Contributor

HavenDV commented Jan 14, 2025

Now it can be disabled in provider using CanPullModelsAutomatically property

@fyziktom
Copy link

I think it will help if you will use ListModelsAsync instad of ListRunningModelsAsync in OllamaProvider.cs. Like this:

public async Task PullModelIfRequiredAndAllowedAsync(
    string id,
    CancellationToken cancellationToken = default)
{
    if (!CanPullModelsAutomatically)
    {
        return;
    }

    try
    {
        // Pull the model if it is not running
        var runningModels = await Api.Models.ListModelsAsync(cancellationToken).ConfigureAwait(false);
        if (runningModels.Models != null &&
            runningModels.Models.All(x => x.Model1?.Contains(id) != true))
        {
            await Api.Models.PullModelAsync(id, cancellationToken: cancellationToken)
                .EnsureSuccessAsync().ConfigureAwait(false);
        }
    }
    catch (HttpRequestException)
    {
        // Ignore
    }
}

I discovered that previous version worked only when the model was really running. So it was possible to use custom models, but only when you will "wake them up" to running state. With change to ListModelsAsync I can see it list all the available models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants