[Core] Support image processor

[DarkLight1337]

I have implemented a plugin architecture (MultiModelPlugin) over MultiModalData to define how each modality type should be preprocessed before being passed to the model as keyword arguments. This preserves the contract between the output of HuggingFace processor and the input into the HuggingFace model. As long as those keyword arguments do not conflict with the ones we have in vLLM, I think this is a good way to make the framework flexible enough to support other multi-modal architectures.

FIX #4054 (the data is now automatically converted into the model's device)

Related Contributions

This PR is part of #3978.

This PR also implements Proposals 1 and 3 of #4194.

Features

1. Refactor of MultiModalData
   • Image input is now split into two subclasses:
     • ImageFeatureData represents the image features of LLaVA after being passed through the vision tower, but before the multi-modal projection is applied.
     • ImagePixelData represents the raw image (using PIL.Image class). AutoImageProcessor from HuggingFace is loaded from config.json to pre-process input images before being passed to the model as pixel_values. As with the tokenizer, you can override the default one and specify the version of image processor via EngineConfig; you can even disable image preprocessing altogether, which is useful if you want to pass in images that have already been preprocessed.
   • The LLaVA implementation has been updated accordingly to accept the new inputs.
2. A new documentation page for using VLMs can be found under (dev/multimodal).

Compatibility Changes

• pillow will be upgraded to a common dependency (from dev) to process the images.

[DarkLight1337]

The LLaVA test passes on my end (with both outputs matching the HF output shown in CI). Does anyone have a clue what might cause it to fail in CI? Perhaps a case of floating-point error in GPU computation?

[ywang96]

Per offline discussion - waiting for #5118 to be merged first.

[ywang96]

@DarkLight1337 Could you resolve the merge conflicts? Once that's done I think this PR is ready to merge.

[ywang96]

I did a final pass and left a note, but everything LGTM! Thank you for the hard work on this. @DarkLight1337