Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand
-
Updated
Feb 22, 2025 - Go
#
large-models
Here is 1 public repository matching this topic...
Improve this page
Add a description, image, and links to the large-models topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the large-models topic, visit your repo's landing page and select "manage topics."