ValueError: Please specify target_modules in peft_config, issue exist with gemma

[yananchen1989]

System Info

peft version: 0.12.0
trl version: 0.9.6

I am following official script from huggingface trl: https://github.com/huggingface/trl/blob/main/examples/scripts/sft.py
with the given demo execution.

it works fine with most models such as llama-3.1, llama-3.2 etc, but not for gemma (such as unsloth/gemma-2-9b-it-bnb-4bit, google/gemma-2-9b-it)

Actually, I do see that lora_target_modules=None in model_config, therefore after peft_config=get_peft_config(model_config)
the target_modules in peft_config is None which can cause error.

But what confuses me is the for other models, there is no error, under the same circumstances.

May I ask your opinions ?

Thanks.

detailed error message:

Traceback (most recent call last):
File "/home/ubuntu/trl/examples/scripts/sft.py", line 95, in
trainer = SFTTrainer(
File "/opt/conda/envs/trl/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
return f(*args, **kwargs)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/trl/trainer/sft_trainer.py", line 265, in init
model = get_peft_model(model, peft_config)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/mapping.py", line 183, in get_peft_model
return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/peft_model.py", line 1542, in init
super().init(model, peft_config, adapter_name, **kwargs)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/peft_model.py", line 155, in init
self.base_model = cls(model, {adapter_name: peft_config}, adapter_name)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 139, in init
super().init(model, config, adapter_name)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 175, in init
self.inject_adapter(self.model, adapter_name)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 394, in inject_adapter
peft_config = self._prepare_adapter_config(peft_config, model_config)
File "/opt/conda/envs/trl/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 459, in _prepare_adapter_config
raise ValueError("Please specify target_modules in peft_config")
ValueError: Please specify target_modules in peft_config

Who can help?

@BenjaminBossan @sayakpaul

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

CUDA_VISIBLE_DEVICES=0 python /home/ubuntu/trl/examples/scripts/sft.py
--model_name_or_path="google/gemma-2-9b-it"
--dataset_text_field="text"
--report_to="none"
--learning_rate=1.41e-5
--per_device_train_batch_size=64
--gradient_accumulation_steps=16
--output_dir="sft_openassistant-guanaco"
--logging_steps=1
--num_train_epochs=3
--max_steps=-1
--push_to_hub
--gradient_checkpointing
--use_peft
--lora_r=64
--lora_alpha=16

Expected behavior

the official script should works for all models, including gemma.

[yananchen1989]

@qgallouedec could you please take a look ? Merci.

[yananchen1989]

after I upgrade trl to 0.11.1, this error for sft gemma, disappeared.

But according to the script, there is no target_modules passed in, either externally from the python run, or internally from anywhere of default config. This applies to all models, such as llama, gemma, mistral etc.

Is there anything I miss ? that set a default target_modules of Lora to the trainer ?

Thanks a lot.

[yananchen1989]

test again for trl-0.12.0.dev0

google/gemma-2-9b-it works again.

do not know the reason.

in trl/trainer/sft_trainer.py, line: model = get_peft_model(model, peft_config), it should follow peft.get_peft_model in https://huggingface.co/docs/peft/v0.13.0/en/package_reference/peft_model#peft.get_peft_model
where The name of the adapter to be injected, if not provided, the default adapter name is used (“default”).

close for now.

[BenjaminBossan]

Just to help you understand, if target_modules is not specified, PEFT will check if the model architecture is one of the standard architectures defined here and choose the right target modules from there. As an example, the gemma2 architecture was added just recently, so for older PEFT versions, you need to specify target_modules if you want to use gemma2.

Regarding the sft script you mentioned, you can pass lora_target_modules as a command line argument to specify the target_modules.