SUPPORTED_MODELS.md

OpenVINO™ GenAI: Supported Models

Architecture	Models	Example HuggingFace Models
ChatGLMModel	ChatGLM	THUDM/chatglm2-6b THUDM/chatglm3-6b
GemmaForCausalLM	Gemma	google/gemma-2b-it
GPTNeoXForCausalLM	Dolly	databricks/dolly-v2-3b
	RedPajama	ikala/redpajama-3b-chat
LlamaForCausalLM	Llama 3	meta-llama/Meta-Llama-3-8B meta-llama/Meta-Llama-3-8B-Instruct meta-llama/Meta-Llama-3-70B meta-llama/Meta-Llama-3-70B-Instruct
	Llama 2	meta-llama/Llama-2-13b-chat-hf meta-llama/Llama-2-13b-hf meta-llama/Llama-2-7b-chat-hf meta-llama/Llama-2-7b-hf meta-llama/Llama-2-70b-chat-hf meta-llama/Llama-2-70b-hf microsoft/Llama2-7b-WhoIsHarryPotter
	OpenLLaMA	openlm-research/open_llama_13b openlm-research/open_llama_3b openlm-research/open_llama_3b_v2 openlm-research/open_llama_7b openlm-research/open_llama_7b_v2
	TinyLlama	TinyLlama/TinyLlama-1.1B-Chat-v1.0
MistralForCausalLM	Mistral	mistralai/Mistral-7B-v0.1
	Notus	argilla/notus-7b-v1
	Zephyr	HuggingFaceH4/zephyr-7b-beta
PhiForCausalLM	Phi	microsoft/phi-2 microsoft/phi-1_5
QWenLMHeadModel	Qwen	Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat-Int4 Qwen/Qwen1.5-7B-Chat Qwen/Qwen1.5-7B-Chat-GPTQ-Int4

The pipeline can work with other similar topologies produced by optimum-intel with the same model signature. The model is required to have the following inputs after the conversion:

1. input_ids contains the tokens.
2. attention_mask is filled with 1.
3. beam_idx selects beams.
4. position_ids (optional) encodes a position of currently generating token in the sequence and a single logits output.

Note

Models should belong to the same family and have the same tokenizers.

Some models may require access request submission on the Hugging Face page to be downloaded.

If https://huggingface.co/ is down, the conversion step won't be able to download the models.