`device_map='auto' support for BertForSequenceClassification for inference using large models

[tnnandi]

Feature request

Currently, from_pretrained() for BertForSequenceClassification does not support device_map='auto', and is preventing inference using large models that don't fit into a single GPU. I'd also appreciate it if you could suggest the best alternative right now.

Motivation

Inference using large models that don't fit into a single GPU is a big issue.

Your contribution

N/A

[amyeroberts]

Closing, as this is a duplicate of #25296.