Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I want to add mamba_chat (2.8b) model #2001

Closed
SafeyahShemali opened this issue Dec 9, 2023 · 5 comments
Closed

I want to add mamba_chat (2.8b) model #2001

SafeyahShemali opened this issue Dec 9, 2023 · 5 comments
Labels
new model Requests to new models

Comments

@SafeyahShemali
Copy link

Hello vllm Team,

havenhq/mamba-chat seems not supported with vllm. Will you be able to help me out to add it?

Sincerely,
@RonanKMcGovern
Copy link
Contributor

+1 for this.

Would be big to have vLLM support for Mamba type models. I expect there will be even bigger Mamba models out soon.

@hmellor hmellor added the new model Requests to new models label Apr 4, 2024
@DarkLight1337 DarkLight1337 changed the title I want to add mambo_chat (2.8b) model I want to add mamba_chat (2.8b) model Jul 9, 2024
@DarkLight1337
Copy link
Member

FYI Jamba is now supported in vLLM (#4115), so it should be easier to implement this model now.

@K-Mistele
Copy link
Contributor

+1 on Mamba models, Mistral just dropped a Mamba-based code model that looks awesome
https://mistral.ai/news/codestral-mamba/

@DarkLight1337
Copy link
Member

I think this should be solved by #6484

@tlrmchlsmth
Copy link
Collaborator

Mamba support just landed (still TODO to support Mamba2, so Codestral Mamba is not yet supported)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new model Requests to new models
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@tlrmchlsmth @K-Mistele @hmellor @DarkLight1337 @SafeyahShemali @RonanKMcGovern