Skip to content

Commit

Permalink
deploy lightning (#9)
Browse files Browse the repository at this point in the history
* deploy lightning

* fixes

* add docs

* fixes
  • Loading branch information
aniketmaurya authored Dec 26, 2023
1 parent dbd9f52 commit 5ef62d5
Show file tree
Hide file tree
Showing 3 changed files with 64 additions and 0 deletions.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,17 @@ app.run_server()
You can run the above script in terminal, and it will launch a FastAPI server for your custom model.


## Deploy

### Lightning AI Studio ⚡️

```shell
python fastserve.deploy.lightning --filename main.py \
--user LIGHTNING_USERNAME \
--teamspace LIGHTNING_TEAMSPACE \
--machine "CPU" # T4, A10G or A10G_X_4
```

## Contribute

**Install in editable mode:**
Expand Down
1 change: 1 addition & 0 deletions src/fastserve/deploy/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from fastserve.deploy.lightning import lightning_job as lightning_job
52 changes: 52 additions & 0 deletions src/fastserve/deploy/lightning.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
try:
from lightning_sdk import Machine, Studio
except ModuleNotFoundError:
Machine = None
Studio = None


def lightning_job(
user: str,
teamspace: str = "default",
command="python main.py",
machine=Machine.CPU,
):
# Start the studio
s = Studio(name="fastserve", teamspace=teamspace, user=user, create_ok=True)
print("starting Studio...")
s.start()

# Install plugin if not installed (in this case, it is already installed)
s.install_plugin("jobs")

jobs_plugin = s.installed_plugins["jobs"]

jobs_plugin.run(command, name="serve", machine=machine)

print("Stopping Studio")
s.stop()


if __name__ == "__main__":
import argparse

parser = argparse.ArgumentParser(description="Deploy FastServe")
parser.add_argument("--filename", type=str, required=True, help="Python filename")
parser.add_argument("--user", type=str, required=True, help="Lightning AI username")
parser.add_argument(
"--teamspace", type=str, required=True, help="Lightning AI teamspace"
)
parser.add_argument(
"--machine",
type=Machine,
required=False,
default="CPU",
help="Lightning AI job plugin machine type",
)

args = parser.parse_args()

command = f"python {args.filename}"
lightning_job(
command=command, user=args.user, teamspace=args.teamspace, machine=args.machine
)

0 comments on commit 5ef62d5

Please sign in to comment.