Add magpie support llama cpp ollama

[davidberenstein1957]

I added extended support for Ollama and Llamacpp.

• ollama support
• llamacpp support
• minor refactors w.r.t. import of disitlabel.models module.

I looked into adding OpenAI API format for other providers but this does not work because tokenization is handled server-side and cannot be disabled out of the box.

• litellm - https://docs.litellm.ai/docs/text_completion
• vllm server -https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html?ref=blog.mozilla.ai#id4
• openai - completion is deprecated https://platform.openai.com/docs/api-reference/completions

Perhaps we can refactor the HF InferenceClient a bit to make this work.

Ollama

from distilabel.models import LlamaCppLLM
from distilabel.steps.tasks import Magpie

llm = LlamaCppLLM(
    model_path="smollm2-360m-instruct-q8_0.gguf",
    tokenizer_id="HuggingFaceTB/SmolLM2-360M-Instruct",
    magpie_pre_query_template="qwen2",
)
magpie = Magpie(
    llm=llm,
)
magpie.load()

print(next(magpie.process(inputs=[{"system": "You are a helpful assistant."}])))

Llamacpp

from distilabel.models import OllamaLLM
from distilabel.steps.tasks import Magpie

llm = OllamaLLM(
    model="llama3.1",
    tokenizer_id="meta-llama/Meta-Llama-3-8B-Instruct",
    magpie_pre_query_template="llama3",
)
magpie = Magpie(llm=llm)
magpie.load()

print(next(magpie.process(inputs=[{"system_prompt": "You are a helpful assistant."}])))

[codspeed-hq]

CodSpeed Performance Report

Merging #1086 will not alter performance

_{Comparing feat/add-magpie-support-llama-cpp-ollama (4e291e7) with develop (f1f7d77)}

Summary

✅ 1 untouched benchmarks