Discussions

• 1 You must be logged in to vote

  🙏

  请教一个关于 asyncio 的问题

  sleepwalker2017 asked Jul 30, 2024 in Q&A · Unanswered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  lmdeploy的kv cache管理

  sitabulaixizawaluduo asked Aug 30, 2024 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  请问支持官网的Qwen1.5 32B awq模型吗？

  lai-serena asked Aug 23, 2024 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How to boot up the inference server faster?

  chicsfever asked Aug 19, 2024 in Q&A · Unanswered

  

  

  4
• 1 You must be logged in to vote

  🙏

  fastapi封装stream_infer会遇到event_loop问题

  Aaron2117 asked Aug 13, 2024 in Q&A · Unanswered

  

  

  

  

  5
• 2 You must be logged in to vote

  🙏

  question about implements LRU policy

  xiaoxiangshusheng asked Aug 15, 2024 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  关于 pre-fill 和 decode 阶段输入长度的疑问

  SeibertronSS asked Aug 19, 2024 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  有关输入拆分计算的问题

  SeibertronSS asked Aug 12, 2024 in Q&A · Closed · Unanswered

  

  

  4
• 1 You must be logged in to vote

  🙏

  请问一下在prefill计算时，ProcessKV和FlattenKV的作用分别是什么？

  sitabulaixizawaluduo asked Jul 17, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  使用 lmdeploy 部署 OpenGVLab/InternVL2-40B 需要多少显存

  navono asked Aug 9, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  💬

  发布到 Quay 或者 GHCR

  thiner started Aug 8, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  lmdeploy serve is more than two times slower than normal transformers code

  paniabhisek asked Aug 6, 2024 in Q&A · Unanswered

  

  

  

  

  8
• 1 You must be logged in to vote

  🙏

  lmdeploy是否支持qwen2的工具调用？

  lai-serena asked Aug 2, 2024 in Q&A · Closed · Answered

  

  

  3
• 1 You must be logged in to vote

  💬

  CUDA11.4 安装报错,提示404

  MdcGIt started Jul 29, 2024 in General · Closed

  

  

  

  

  20
• 1 You must be logged in to vote

  🙏

  Triton test failed when running models with Pytorch backend

  cmpute asked Jul 23, 2024 in Q&A · Closed · Answered

  

  

  4
• 1 You must be logged in to vote

  🙏

  [Bug] 如何使用python代码加载turbomind模型并对话（菜鸡请求大佬支援）

  quanfeifan asked Jul 21, 2024 in Q&A · Closed · Unanswered

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  lmdeploy的设计是和vllm一样先来先服务（FCFS），后来先抢占，gpu不够就先swap到cpu上吗？

  sunzx8 asked Jul 19, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  lmdeploy serve api_server 可以把两个模型部署到同一个端口吗？

  thomZ1 asked Jul 17, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  How multimodal batch inference works?

  bonre asked Jul 16, 2024 in Q&A · Answered

  

  

  4
• 1 You must be logged in to vote

  🙏

  Q: Continuous Batching without Turbomind?

  awaiting response

  SinanAkkoyun asked Jul 15, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  💬

  Does lmdeploy support InternVL2?

  nickmitchko started Jul 11, 2024 in General

  

  

  2
• 1 You must be logged in to vote

  💬

  LMDeploy有相关的论文或者技术报告吗

  IcyFeather233 started Jul 9, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  MPI related undefined reference errors while lmdeploy build from source

  anisingh1 asked Sep 28, 2023 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  KV Cache int8 cooperate with Flash-attention2

  yhyu13 asked Nov 15, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  What's the difference between Persistent Batch and continuous batching?

  luweigen asked Nov 20, 2023 in Q&A · Unanswered

  

  

  1