From bab8960de13961de6189a20dd9d8294dcfd01836 Mon Sep 17 00:00:00 2001 From: Aniket Maurya Date: Tue, 19 Mar 2024 17:39:24 +0000 Subject: [PATCH] fix docs --- docs/CHANGELOG.md | 11 ++- docs/fastserve/index.md | 0 .../llms/{local_llm.md => local_llms.md} | 0 docs/index.md | 67 +++++++++++-------- mkdocs.yml | 14 ++-- 5 files changed, 59 insertions(+), 33 deletions(-) create mode 100644 docs/fastserve/index.md rename docs/fastserve/models/llms/{local_llm.md => local_llms.md} (100%) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 495f5e0..6433bb6 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -1,4 +1,13 @@ # Release Notes -## 0.0.1 +## 0.0.3 * Setup repo +* Feat/refactor api by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/8 +* deploy lightning by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/9 +* add face recognition by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/10 +* add image classification by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/11 +* document youtube video by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/12 +* [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in https://github.com/aniketmaurya/fastserve-ai/pull/13 +* Serve UI by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/14 +* Redirect `/ui ` to web app by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/15 +* add vLLM by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/21 diff --git a/docs/fastserve/index.md b/docs/fastserve/index.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/fastserve/models/llms/local_llm.md b/docs/fastserve/models/llms/local_llms.md similarity index 100% rename from docs/fastserve/models/llms/local_llm.md rename to docs/fastserve/models/llms/local_llms.md diff --git a/docs/index.md b/docs/index.md index 6681679..19592ac 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,42 +1,38 @@ -# FastServe +

+ logo +
+ Machine Learning Serving focused on GenAI & LLMs with simplicity as the top priority. +

+

+ Docs | + Examples +

+ +--- -Machine Learning Serving focused on GenAI & LLMs with simplicity as the top priority. ## Installation + +**Stable:** ```shell -pip install git+https://github.com/aniketmaurya/fastserve.git@main +pip install FastServeAI ``` -## Run locally - -```bash -python -m fastserve +**Latest:** +```shell +pip install git+https://github.com/gradsflow/fastserve-ai.git@main ``` -## Usage/Examples - -### Serve Mistral-7B with Llama-cpp -```python -from fastserve.models import ServeLlamaCpp - -model_path = "openhermes-2-mistral-7b.Q5_K_M.gguf" -serve = ServeLlamaCpp(model_path=model_path, ) -serve.run_server() -``` - -or, run `python -m fastserve.models --model llama-cpp --model_path openhermes-2-mistral-7b.Q5_K_M.gguf` from terminal. +## Usage/Examples -### Serve SDXL Turbo + + + -```python -from fastserve.models import ServeSDXLTurbo +> YouTube: How to serve your own GPT like LLM in 1 minute with FastServe. -serve = ServeSDXLTurbo(device="cuda", batch_size=2, timeout=1) -serve.run_server() -``` -or, run `python -m fastserve.models --model sdxl-turbo --batch_size 2 --timeout 1` from terminal. ### Serve Custom Model @@ -46,6 +42,7 @@ returns the response as a list. ```python from fastserve import FastServe + class MyModelServing(FastServe): def __init__(self): super().__init__(batch_size=2, timeout=0.1) @@ -56,28 +53,42 @@ class MyModelServing(FastServe): response = self.model(inputs) return response + app = MyModelServing() app.run_server() ``` You can run the above script in terminal, and it will launch a FastAPI server for your custom model. +## Deploy + +### Lightning AI Studio ⚡️ + +```shell +python fastserve.deploy.lightning --filename main.py \ + --user LIGHTNING_USERNAME \ + --teamspace LIGHTNING_TEAMSPACE \ + --machine "CPU" # T4, A10G or A10G_X_4 +``` + ## Contribute **Install in editable mode:** + ```shell -git clone https://github.com/aniketmaurya/fastserve.git +git clone https://github.com/gradsflow/fastserve-ai.git cd fastserve pip install -e . ``` **Create a new branch** + ```shell git checkout -b ＜new-branch＞ ``` -**Make your changes, commit and [create a PR](https://github.com/aniketmaurya/fastserve/compare).** +**Make your changes, commit and [create a PR](https://github.com/gradsflow/fastserve-ai/compare).**