TypeError: MistralForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'

[edchengg]

System Info

I got this error when I tried to use sentiment classification pipeline with "nvidia/Mistral-NeMo-Minitron-8B-Base". It works fine with llama 3.1.

TypeError: MistralForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'

• transformers 4.43.4

See code below:

Who can help?

@ArthurZucker

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

import torch 
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline 

torch.random.manual_seed(0) 

model_id = "nvidia/Mistral-NeMo-Minitron-8B-Base" # not working
# model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct" # working
model = AutoModelForSequenceClassification.from_pretrained( 
    model_id,  
    device_map="auto",  
    torch_dtype=torch.bfloat16 
) 

tokenizer = AutoTokenizer.from_pretrained(model_id) 



pipe = pipeline( 
    "sentiment-analysis", 
    model=model, 
    tokenizer=tokenizer, 
) 


output = pipe("hello how are you today?")
print(output)
# TypeError: MistralForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'

Expected behavior

expected to be able to run the code

[LysandreJik]

Thanks for your PR @edchengg! Does it work with the default 7b mistral model on your side? So for example with this checkpoint: mistralai/Mistral-7B-Instruct-v0.1

[ArthurZucker]

(You either need to delete the token_type_ids, or update tokenizer.model_input_names 🤗