PEFT fix: T5 prefix-tuning with new cache format

[zucchini-nlp]

What does this PR do?

Fixes prefix tuning for T5 models which started breaking after the recent compile-compatibility PR. However, note this will not enable correct prefix tuning unless huggingface/peft#2096 (review) is merged

The main issue was that in new cache format we don't concatenate new key/values with cached key/values if past_key_values.is_updated. That issue is fixed on PEFT side by init a cache object and setting is_updated=False.

Another issue is the mask shape mismatch since we have to extend cross_attention_mask to account for new virtual tokens. Current PR adds enables it but I am also thinking if it will be possible to do in PEFT and pass already extended attention masks?

Tested with the below code for T5:

import torch
from transformers import AutoModelForSeq2SeqLM
from peft import PrefixTuningConfig, get_peft_model

inputs = {
    "input_ids": torch.tensor([[1, 2, 3, 4, 5, 6, 7]]),
    "decoder_input_ids": torch.tensor([[1, 2, 3, 4, 5]]),
    "attention_mask": torch.tensor([[1, 1, 1, 1, 1, 1, 1]]),
}
model_id = "ybelkada/tiny-random-T5ForConditionalGeneration-calibrated"
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
model(**inputs)

config = PrefixTuningConfig(num_virtual_tokens=20, task_type="SEQ_2_SEQ_LM")
model = get_peft_model(model, config)

output = model(**inputs)

cc @BenjaminBossan wdyt?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

Just to update, we're internally discussing other options.