Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llamamodel: add DeepSeek-V2 to whitelist #2702

Merged
merged 1 commit into from
Jul 22, 2024
Merged

llamamodel: add DeepSeek-V2 to whitelist #2702

merged 1 commit into from
Jul 22, 2024

Conversation

cebtenzzre
Copy link
Member

@cebtenzzre cebtenzzre commented Jul 19, 2024

This model actually works fine. I tried it with llama-cli before, but was thrown off by its huge default context size. With GPT4All's context size of 2K, this model works fine with CUDA. It doesn't work with Kompute due to the lack of MoE support.


The changelog is not merged yet, but this patch should be applied:

diff --git a/gpt4all-chat/CHANGELOG.md b/gpt4all-chat/CHANGELOG.md
index b56b993c..bc75cb6c 100644
--- a/gpt4all-chat/CHANGELOG.md
+++ b/gpt4all-chat/CHANGELOG.md
@@ -28,8 +28,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 - Support translation of settings choices ([#2667](https://github.com/nomic-ai/gpt4all/pull/2667), [#2690](https://github.com/nomic-ai/gpt4all/pull/2690))
 - Improve LocalDocs view's error message (by @cosmic-snow in [#2679](https://github.com/nomic-ai/gpt4all/pull/2679))
 - Ignore case of LocalDocs file extensions ([#2642](https://github.com/nomic-ai/gpt4all/pull/2642), [#2684](https://github.com/nomic-ai/gpt4all/pull/2684))
-- Update llama.cpp to commit 87e397d00 from July 19th ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694))
+- Update llama.cpp to commit 87e397d00 from July 19th ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694), [#2702](https://github.com/nomic-ai/gpt4all/pull/2702))
   - Add support for GPT-NeoX, Gemma 2, OpenELM, ChatGLM, and Jais architectures (all with Vulkan support)
+  - Add support for DeepSeek-V2 architecture (no Vulkan support)
   - Enable Vulkan support for StarCoder2, XVERSE, Command R, and OLMo
 - Show scrollbar in chat collections list as needed (by [@cosmic-snow](https://github.com/cosmic-snow) in [#2691](https://github.com/nomic-ai/gpt4all/pull/2691))
 

Signed-off-by: Jared Van Bortel <jared@nomic.ai>
@cebtenzzre cebtenzzre requested a review from manyoso July 19, 2024 23:12
@manyoso manyoso merged commit 4ca1d04 into main Jul 22, 2024
6 of 20 checks passed
@cosmic-snow
Copy link
Collaborator

I'm assuming this includes deepseek-coder-v2? Because there's an open issue for that: #2527

@cebtenzzre
Copy link
Member Author

I'm assuming this includes deepseek-coder-v2? Because there's an open issue for that: #2527

This change is required for DeepSeek-Coder-V2-Instruct support, but I haven't tested that model yet - only DeepSeek-V2-Lite-Chat. I believe the different DeepSeek models use different pretokenizer configurations which have to be individually supported by llama.cpp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants