Skip to content

Commit

Permalink
fix docs
Browse files Browse the repository at this point in the history
  • Loading branch information
aniketmaurya committed Mar 19, 2024
1 parent fe2c8fe commit bab8960
Show file tree
Hide file tree
Showing 5 changed files with 59 additions and 33 deletions.
11 changes: 10 additions & 1 deletion docs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,13 @@
# Release Notes

## 0.0.1
## 0.0.3
* Setup repo
* Feat/refactor api by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/8
* deploy lightning by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/9
* add face recognition by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/10
* add image classification by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/11
* document youtube video by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/12
* [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in https://github.com/aniketmaurya/fastserve-ai/pull/13
* Serve UI by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/14
* Redirect `/ui ` to web app by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/15
* add vLLM by @aniketmaurya in https://github.com/aniketmaurya/fastserve-ai/pull/21
Empty file added docs/fastserve/index.md
Empty file.
File renamed without changes.
67 changes: 39 additions & 28 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,38 @@
# FastServe
<p align="center">
<img width="250" alt="logo" src="https://ik.imagekit.io/gradsflow/logo/v2/Gradsflow-gradient_TPwd2H3s4.png?updatedAt=1710283252606"/>
<br>
<strong>Machine Learning Serving focused on GenAI & LLMs with simplicity as the top priority.</strong>
</p>
<p align="center">
<a href="https://fastserve.gradsflow.com">Docs</a> |
<a href="https://github.com/gradsflow/fastserve-ai/tree/main/examples">Examples</a>
</p>

---

Machine Learning Serving focused on GenAI & LLMs with simplicity as the top priority.

## Installation

**Stable:**
```shell
pip install git+https://github.com/aniketmaurya/fastserve.git@main
pip install FastServeAI
```

## Run locally

```bash
python -m fastserve
**Latest:**
```shell
pip install git+https://github.com/gradsflow/fastserve-ai.git@main
```

## Usage/Examples

### Serve Mistral-7B with Llama-cpp

```python
from fastserve.models import ServeLlamaCpp

model_path = "openhermes-2-mistral-7b.Q5_K_M.gguf"
serve = ServeLlamaCpp(model_path=model_path, )
serve.run_server()
```

or, run `python -m fastserve.models --model llama-cpp --model_path openhermes-2-mistral-7b.Q5_K_M.gguf` from terminal.
## Usage/Examples

### Serve SDXL Turbo
<a href="https://www.youtube.com/watch?v=GfcmyfPB9qY">
<img src="https://img.youtube.com/vi/GfcmyfPB9qY/0.jpg" width=350px>
</a>

```python
from fastserve.models import ServeSDXLTurbo
> YouTube: How to serve your own GPT like LLM in 1 minute with FastServe.
serve = ServeSDXLTurbo(device="cuda", batch_size=2, timeout=1)
serve.run_server()
```

or, run `python -m fastserve.models --model sdxl-turbo --batch_size 2 --timeout 1` from terminal.

### Serve Custom Model

Expand All @@ -46,6 +42,7 @@ returns the response as a list.
```python
from fastserve import FastServe


class MyModelServing(FastServe):
def __init__(self):
super().__init__(batch_size=2, timeout=0.1)
Expand All @@ -56,28 +53,42 @@ class MyModelServing(FastServe):
response = self.model(inputs)
return response


app = MyModelServing()
app.run_server()
```

You can run the above script in terminal, and it will launch a FastAPI server for your custom model.

## Deploy

### Lightning AI Studio ⚡️

```shell
python fastserve.deploy.lightning --filename main.py \
--user LIGHTNING_USERNAME \
--teamspace LIGHTNING_TEAMSPACE \
--machine "CPU" # T4, A10G or A10G_X_4
```


## Contribute

**Install in editable mode:**

```shell
git clone https://github.com/aniketmaurya/fastserve.git
git clone https://github.com/gradsflow/fastserve-ai.git
cd fastserve
pip install -e .
```

**Create a new branch**

```shell
git checkout -b <new-branch>
```

**Make your changes, commit and [create a PR](https://github.com/aniketmaurya/fastserve/compare).**
**Make your changes, commit and [create a PR](https://github.com/gradsflow/fastserve-ai/compare).**


<!-- ## FAQ
Expand Down
14 changes: 10 additions & 4 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
site_name: FastServe
site_description: 'Machine Learning Serving focused on GenAI & LLMs with simplicity as the top priority.'
site_author: Aniket Maurya
copyright: 'Copyright &copy; 2023 Aniket Maurya'
copyright: 'Copyright &copy; 2024 Aniket Maurya'

banner_url: https://IMAGE_URL
repo_url: https://github.com/aniketmaurya/fastserve/
banner_url: https://ik.imagekit.io/gradsflow/logo/v2/gf-cover-hd_N3naW2FHA.png?updatedAt=1710282045836
repo_url: https://github.com/gradsflow/fastserve-ai/
repo_name: aniketmaurya/fastserve

theme:
Expand Down Expand Up @@ -60,11 +60,17 @@ plugins:
show_source: false

extra:
homepage: https://github.com/aniketmaurya/fastserve
homepage: https://github.com/gradsflow/fastserve-ai
analytics:
provider: google
property: G-B8FPDYGVPR

nav:
- Introduction: 'index.md'
- Image GenAI: fastserve/models/image_gen.md
- LLMs:
- vLLM: fastserve/models/llms/vllm.md
- Local LLMs: fastserve/models/llms/local_llms.md
- HF LLMs: fastserve/models/llms/hf.md
- Containerization: fastserve/containerization.md
- Release Notes: 'CHANGELOG.md'

0 comments on commit bab8960

Please sign in to comment.