Is vllm willing to support models based on other AI frameworks, such as mindspore?

[wxsIcey]

Hello,

we are MindSpore eco-developers! MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.(You can get a more detailed introduction to MindSpore at: https://www.mindspore.cn/en)

We want to add a MindSpore backend to vllm to expand the MindSpore ecosystem, and at the same time, also make the vllm ecosystem better.

Our team has several software engineers with years of development experience, and we are supported by the team of MindSpore framework developers.

We would like to know if your community accepts models based on MindSpore? and if it is possible, we can discuss the technical details further.

[WangErXiao]

如果支持这些框架，那vllm就走不远了。

[ywang96]

Personally I think it makes more sense for the support for alternative frameworks to stay as a fork of vLLM. For now the scope of vLLM is still limited to PyTorch and we don't have plan to expand to other frameworks.

#3620 as you mentioned is adding support for TPUs with PyTorch XLA rather than supporting JAX directly, and general speaking supporting different hardware is a much different story than supporting other DL frameworks.

[wxsIcey]

Personally I think it makes more sense for the support for alternative frameworks to stay as a fork of vLLM. For now the scope of vLLM is still limited to PyTorch and we don't have plan to expand to other frameworks.

#3620 as you mentioned is adding support for TPUs with PyTorch XLA rather than supporting JAX directly, and general speaking supporting different hardware is a much different story than supporting other DL frameworks.

Thank you very much for your patient reply. We will continue to pay attention to the technical development of vllm.