This project aims to build a Rust micro-server that summarizes text, based on the common task of reading and summarizing books among students. The project uses the rust actix and libtorch to run a pre-trained hugging-face model for summarization.
Compared with the Individual Project 2, this project add more functions such as onnx model with albert models, GPU support for rust-bert, etc.
- Develop my Rust micro-service with kubernetes
- Use Github Codespaces and Copilot
- Integrate libtorch and 'hugging-face pretrained models' into a Rust web project
- go to the directory
K8s
cd K8s
- create the namespace
resume-prod
kubectl create namespace resume-prod
- apply the yaml files
kubectl apply -f .
- This repo main branch is automatically published to Dockerhub with CI/CD, you can pull the image from here
docker pull szheng3/sz-rust-ml:latest
- Run the docker image.
docker run -d -p 8000:8000 szheng3/sz-rust-ml:latest
- Run the docker image with GPU.
docker run -d -p 8000:8000 szheng3/sz-rust-ml:latest-gpu
- Install rust via rustup
- Install the libtorch (for Mac M1), Intel chips users can skip this step
- node
brew install pytorch@1.13.1
- Build frontend
make frontend
- Run, you can run the rust web app.
make run
- Release
make releasex86
- Bench
make benchx86
- Build frontend
make frontend
- change the path in the Makefile to your libtorch path
export LIBTORCH=/opt/homebrew/Cellar/pytorch/1.13.1 &&export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH
- Run, you can run the rust web app.
make runarm
- Release
make release
- Bench
make bench
Github Actions configured in .github/workflows
The binary could be downloaded from the release pages. release
- Configure Github Codespaces.
- Initialise Rust project with pretrained model from hugging-face
- CI/CD with Github Actions
- Tag and Releases
- Benchmark
- kubernetes deployment
- logging