diff --git a/README.md b/README.md index 21b8f1d456..4c7a7a752f 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,46 @@ [![Build Status](https://travis-ci.com/volcano-sh/volcano.svg?token=sstuqFE81ukmNz9cEEtd&branch=master)](https://travis-ci.com/volcano-sh/volcano) [![slack](https://img.shields.io/badge/Volcano-%23SLACK-red.svg)](https://volcano-sh.slack.com/messages/CGET876H5/) -Volcano is a Kubernetes-based system for high performance workload, providing mechanisms for applications -which would like to run high performance workload leveraging Kubernetes, e.g. Tensorflow, Spark, MPI. - -Volcano builds upon a decade and a half of experience on running high performance workload workloads at scale -using several systems, combined with best-of-breed ideas and practices from the open source community. +Volcano is system for runnning high performance workloads on +Kubernetes. It provides a suite of mechanisms currently missing from +Kubernetes that are commonly required by many classes of high +performance workload including: + +1. machine learning/deep learning, +2. bioinformatics/genomics, and +3. other "big data" applications. + +These types of applications typically run on generalized domain +frameworks like Tensorflow, Spark, PyTorch, MPI, etc, which Volcano integrates with. + +Some examples of the mechanisms and features that Volcano adds to Kubernetes are: + +1. Job management extensions and improvements, e.g: + 1. Multi-pod jobs + 2. Lifecycle management extensions including suspend/resume and + restart. + 3. Improved error handling + 4. Indexed jobs + 5. Task dependencies +2. Scheduling extensions, e.g: + 1. Co-scheduling + 2. Fair-share scheduling + 3. Queue scheduling + 4. Preemption and reclaims + 5. Reservartions and backfills + 6. Topology-based scheduling +3. Runtime extensions, e.g: + 1. Support for specialized continer runtimes like Singularity, + with GPU accelerator extensions and enhanced security features. +4. Other + 1. Data locality awareness and intelligent scheduling + 2. Optimizations for data throughput, round-trip latency, etc. + + +Volcano builds upon a decade and a half of experience running a wide +variety of high performance workloads at scale using several systems +and platforms, combined with best-of-breed ideas and practices from +the open source community. ## Overall Architecture @@ -18,23 +53,32 @@ The easiest way to use Volcano is to use the Helm chart. ### 1. Volcano Image -Official images now are not available on DockerHub, however you can build them locally with command: + +Official images are not yet available on DockerHub, however you can +build them locally with the command: + ``` make docker ``` + **NOTE**: You need ensure the images are correctly loaded in your kubernetes cluster, for example, if you are using [kind cluster](https://github.com/kubernetes-sigs/kind), try command ```kind load docker-image : ``` for each of the images. ### 2. Helm charts -First of all, clone repo to your local path +First of all, clone the repo to your local path: + ``` # mkdir -p $GOPATH/src/volcano.sh/ # cd $GOPATH/src/volcano.sh/ # git clone https://github.com/volcano-sh/volcano.git ``` -Second, install required helm plugin and generate valid certificate, volcano uses a helm plugin **gen-admission-secret** -to generate certificate for admission service to communicate with kubernetes API server. + +Second, install the required helm plugin and generate valid +certificate, volcano uses a helm plugin **gen-admission-secret** to +generate certificate for admission service to communicate with +kubernetes API server. + ``` #1. Install helm plugin helm plugin install installer/chart/volcano/plugins/gen-admission-secret @@ -42,14 +86,18 @@ helm plugin install installer/chart/volcano/plugins/gen-admission-secret #2. Generate secret within service name helm gen-admission-secret --service -admission-service --namespace ``` + Finally, install helm chart. + ``` helm install installer/chart/volcano --namespace --name ``` + **NOTE**:The `````` used in the two commands above should be identical. -To Verify your installation run the below commands +To Verify your installation run the following commands: + ``` #1. Verify the Running Pods # kubectl get pods --namespace