Skip to content

Issues: triton-inference-server/server

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

628 Open 3,232 Closed
628 Open 3,232 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Can't build r25.01 (r24.12 builds okay) on ubuntu-22.04 (unclear build errors). Also can't build r24.12 on ubuntu-24.04 (C++ errors)
#7997 opened Feb 8, 2025 by vadimkantorov
2
The system looks the same, but errors occur on some machines, but the reason is unknown
#7996 opened Feb 8, 2025 by coder-2014
[BUG] [GenAI-Perf] openai-fronted server with --endpoint-type completions openai OpenAI related
#7995 opened Feb 7, 2025 by jihyeonRyu
Batching module: backends Issues related to the backends python Python related, whether backend, in-process API, client, etc question Further information is requested
#7994 opened Feb 7, 2025 by riyajatar37003
build.py setting docker build args for secrets even when build-secret flag is not present build Issues pertaining to builds
#7992 opened Feb 6, 2025 by BenjaminBraunDev
@mc-nv
2
libtriton_fil.so missing on Arm64 containers 24.12 and 25.01 module: backends Issues related to the backends module: platforms Issues related to platforms, hardware, and support matrix
#7991 opened Feb 5, 2025 by dagardner-nv
Performance issue - High queue times in perf_analyzer performance A possible performance tune-up question Further information is requested
#7986 opened Feb 4, 2025 by asaff1
3
Something like "model instance index" inside python backend enhancement New feature or request module: backends Issues related to the backends python Python related, whether backend, in-process API, client, etc
#7984 opened Feb 3, 2025 by vadimkantorov
Expected model dimensions when expected shape is not suitable to batch
#7981 opened Jan 31, 2025 by codeofdutyAI
[Question] triton-client numpy 2 support
#7979 opened Jan 30, 2025 by john-pixforce
Pytorch backend: Model is run in no_grad mode even with INFERENCE_MODE=false
#7974 opened Jan 28, 2025 by hakanardo
Method 'forward' is not defined error !
#7968 opened Jan 26, 2025 by MHmi1
1
vLLM backend Hugging Face feature branch model loading enhancement New feature or request
#7963 opened Jan 23, 2025 by knitzschke
1
unexpected throughput results - Increasing instance group count VS deploying the count distributed on the same card using shared computing windows performance A possible performance tune-up
#7956 opened Jan 21, 2025 by ariel291888
2
How to start/expose the metrics end point of the Triton Server via openai_frontend/main.py arguments
#7954 opened Jan 21, 2025 by shuknk8s
3
Segmentation Fault error when crafting pb_utils.Tensor object Triton BLS model bug Something isn't working
#7953 opened Jan 18, 2025 by carldomond7
1
Failed to launch triton-server：”error: creating server: Internal - failed to load all models“ module: backends Issues related to the backends
#7950 opened Jan 17, 2025 by pzydzh
2
build.py broken in r24.11 bug Something isn't working
#7939 opened Jan 15, 2025 by prm-james-hill
5
Triton crashes with SIGSEGV crash Related to server crashes, segfaults, etc.
#7938 opened Jan 15, 2025 by ctxqlxs
2
[Question] Are the libnvinfer_builder_resources necessary in the triton image ? question Further information is requested
#7932 opened Jan 14, 2025 by MatthieuToulemont
1
Server build with python BE failing due to missing Boost lib
#7925 opened Jan 9, 2025 by buddhapuneeth
6
running triton as a inference service on host
#7915 opened Jan 3, 2025 by sriram-dsl
2
OpenAI-Compatible Frontend should support world_size larger than 1 enhancement New feature or request
#7914 opened Jan 3, 2025 by cocodee
1
vllm_backend: What is the right way to use downloaded model + model.json together? question Further information is requested
#7912 opened Jan 2, 2025 by kyoungrok0517
1
Python backend with multiple instances cause unexpected and non-deterministic results bug Something isn't working
#7907 opened Dec 25, 2024 by NadavShmayo
@pskiran1
3
ProTip! Add no:assignee to see everything that’s not assigned.