Skip to content

Commit

Permalink
docs: added sustainability and scaling docs. Various other changes to…
Browse files Browse the repository at this point in the history
… the docs
  • Loading branch information
CS76 committed Sep 1, 2023
1 parent 9ddca88 commit b9ce98b
Show file tree
Hide file tree
Showing 16 changed files with 153 additions and 58 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,4 @@ RUN pip3 install --no-cache-dir chembl_structure_pipeline --no-deps

COPY ./app /code/app

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80", "--workers", "4"]
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80", "--workers", "4", "--reload"]
5 changes: 3 additions & 2 deletions app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
from fastapi.responses import RedirectResponse
from fastapi_versioning import VersionedFastAPI

from .routers import chem, converters, depict, tools, ocsr
from .routers import chem, converters, depict, tools
#, ocsr
from fastapi.middleware.cors import CORSMiddleware

from prometheus_fastapi_instrumentator import Instrumentator
Expand All @@ -28,7 +29,7 @@
app.include_router(converters.router)
app.include_router(depict.router)
app.include_router(tools.router)
app.include_router(ocsr.router)
# app.include_router(ocsr.router)

app = VersionedFastAPI(
app,
Expand Down
6 changes: 4 additions & 2 deletions docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { defineConfig } from 'vitepress'
// https://vitepress.dev/reference/site-config
export default defineConfig({
title: "Cheminformatics Python Microservice",
description: "This set of essential and valuable microservices is designed to be accessed via API calls to support cheminformatics. Generally, it is designed to work with SMILES-based inputs and could be used to translate between different machine-readable representations, get Natural Product (NP) likeliness scores, visualize chemical structures, and generate descriptors. In addition, the microservices also host an instance of STOUT and another instance of DECIMER (two deep learning models for IUPAC name generation and optical chemical structure recognition, respectively).",
description: "This set of essential and valuable microservices is designed to be accessed via API calls to support cheminformatics. Generally, it is designed to work with SMILES-based inputs and could be used to translate between different machine-readable representations, get Natural Product (NP) likeliness scores, visualize chemical structures, and generate descriptors. In addition, the microservice also host an instance of STOUT and another instance of DECIMER (two deep learning models for IUPAC name generation and optical chemical structure recognition, respectively).",

themeConfig: {
search: {
Expand All @@ -30,6 +30,7 @@ export default defineConfig({
{ text: 'Introduction', link: '/introduction' },
{ text: 'Versions', link: '/versions' },
{ text: 'Architecture', link: '/architecture' },
{ text: 'Sustainability', link: '/sustainability' },
]
},
{
Expand All @@ -38,6 +39,7 @@ export default defineConfig({
{ text: 'Docker', link: '/docker' },
{ text: 'Cluster Deployment (K8S)', link: '/cluster-deployment' },
{ text: 'Standalone (Python virtual environment)', link: '/standalone' },
{ text: 'Scaling', link: '/scaling' },
]
},
{
Expand All @@ -53,7 +55,7 @@ export default defineConfig({
{
text: 'Development',
items: [
{ text: 'Local Installation', link: '/installation' },
{ text: 'Local Setup', link: '/installation' },
{ text: 'License', link: '/license' },
{ text: 'Issues/Feature requests', link: '/issues' },
{ text: 'Contributors', link: '/contributors' }
Expand Down
2 changes: 1 addition & 1 deletion docs/chem.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@ Under the CHEM module, CPM exposes functions and routines offered by cheminforma
width="100%"
height="720"
style="border:none"
src="https://dev.api.naturalproducts.net/v1/docs#/chem">
src="https://dev.api.naturalproducts.net/latest/docs#/chem">
</iframe>
</div>
36 changes: 22 additions & 14 deletions docs/cluster-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,25 @@
outline: deep
---

# Kubernetes deployment via Helm.
Cheminformatics Python Microservices comes packaged with [Helm](https://helm.sh/docs/) chart which simplifies the deployment and management of applications on [Kubernetes](https://kubernetes.io/) by providing a convenient package manager interface. By following the steps outlined in this documentation, you can easily deploy this microservice using [Helm](https://helm.sh/docs/), enabling efficient and reproducible deployments in your [Kubernetes](https://kubernetes.io/) cluster.
# Kubernetes deployment (K8S)

Please refer to Helm’s [documentation](https://helm.sh/docs/) to get started.
## Helm Charts - v0.1.6

Cheminformatics Python Microservices comes packaged with a [Helm](https://helm.sh/docs/) chart, that makes it easy to deploy and manage (scale) containers on [Kubernetes](https://kubernetes.io/) via a convenient package manager interface.

By following the steps outlined in this documentation, you can easily deploy this microservice using [Helm](https://helm.sh/docs/).

For more information about Helm Charts based deployment please refer to official [Helm documentation](https://helm.sh/docs/).

**Prerequisites:**

Before proceeding with the deployment, ensure that you have the following:

Kubernetes cluster: Set up a functioning Kubernetes cluster with kubectl configured to interact with the cluster.

Helm: [Install Helm](https://helm.sh/docs/docs/intro/install/) on your local machine or the machine from which you'll be deploying the application.

### Deploy the chart:
## Deployment using CPM Helm Chart
* **Add repo:** Once Helm has been set up correctly, add the repo as follows:
```bash
helm repo add repo-helm-charts https://nfdi4chem.github.io/repo-helm-charts/
Expand All @@ -35,7 +41,7 @@ kubectl get services
```
* **Upgrading and Managing Deployments:** To upgrade an existing deployment, Use the `helm upgrade` command to apply the changes to the existing release e.g.
```bash
helm upgrade myrelease repo-helm-charts/cheminfo-microservice-0.0.2
helm upgrade myrelease repo-helm-charts/cheminfo-microservice-0.1.6
```

* **Uninstalling the Chart:** To remove a deployed chart and associated resources, use the helm uninstall command:
Expand All @@ -45,10 +51,12 @@ helm uninstall myrelease
This will delete all resources created by the chart, including pods, services, and any other Kubernetes objects.


# Deployment to Google Kubernetes Engine(GKE) using Helm.

## Google Kubernetes Engine(GKE)

This documentation will guide you through the process of deploying the CPM application in [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) using [Helm](https://helm.sh/docs/). Here we provide a step-by-step instructions to help you set up your environment, install Helm, and dpeloy the CPM application in the Google Kubernetes Cluster.

## Prerequisites
### Prerequisites
Before you begin, ensure you have the following prerequisites in place:
* Google Cloud Platform (GCP) Account: You need a GCP account to create a GKE cluster and use other GCP services.
* In the Google Cloud console, on the project selector page, select or [create a Google Cloud project](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
Expand All @@ -57,7 +65,7 @@ Before you begin, ensure you have the following prerequisites in place:
* Go to [Google Cloud Console](https://console.cloud.google.com/)
* Click on the 'Activate Cloud Shell' button located at the top of the Google Cloud console window.

### Step 1: Create a GKE Cluster
#### Step 1: Create a GKE Cluster
A GKE cluster consists of a pool of Compute Engine VM instances running [Kubernetes](https://kubernetes.io/), the open source cluster orchestration system that powers GKE.
1. Navigate to [Google Kubernetes Engine](https://console.cloud.google.com/kubernetes) page in Google Cloud console.
2. Click on the Create icon.
Expand All @@ -77,11 +85,11 @@ A GKE cluster consists of a pool of Compute Engine VM instances running [Kuberne

6. Click on `Create` button and wait for the Cluster to be ready.

### Step2: Connect to Cluster
#### Step2: Connect to Cluster
1. After you cluster is created you can see the green check next to it. Once you see the check, click on the three dots next to your cluster to click on `Connect`.
2. Click on `Run in Cloud Shell` option and press enter.

### Step3: Deploy via Helm Chart
#### Step3: Deploy via Helm Chart
1. Add the helm repo by running below command.
```bash
helm repo add repo-helm-charts https://nfdi4chem.github.io/repo-helm-charts/
Expand All @@ -102,14 +110,14 @@ kubectl get services
5. To access your service you may have to expose it either by setting the type to `Load Balancer` under service in values.yml file or by an Ingress depending upon your requirement.
To learn more about how to configure Nginx Ingress you can click on the link [here](https://github.com/GoogleCloudPlatform/community/blob/master/archived/nginx-ingress-gke/index.md#deploying-the-nginx-ingress-controller-with-helm).

### Step4: Clean up
#### Step4: Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete resources by running below command.
```bash
helm delete myrelease
```

## Scaling
In case of performance issue, the CPM application can be scaled accordingly. The Helm chart for CPM comes packaged with Horizontal Pod Autoscaler. Horizontal Pod Autoscaler (HPA) is a Kubernetes feature that automatically adjusts the number of replica pods in a deployment or replica set based on observed CPU utilization or other select metrics. This allows your Kubernetes cluster to dynamically respond to changes in application load, ensuring optimal resource utilization and application availability.
### Scaling
In case of performance issue, the CPM application can be scaled automatically using different strategies. The Helm chart for CPM comes packaged with Horizontal Pod Autoscaler. Horizontal Pod Autoscaler (HPA) is a Kubernetes feature that automatically adjusts the number of replica pods in a deployment or replica set based on observed CPU utilization or other select metrics. This allows your Kubernetes cluster to dynamically respond to changes in application load, ensuring optimal resource utilization and application availability.
You can set the `targetCPUUtilizationPercentage` and `targetMemoryUtilizationPercentage` values in values.yml file according to your need and demand, which is the deciding factor to trigger the scaling. To learn more about different types of scaling in Kubernetes and GKE follow the official [documentation](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler) of Google Cloud.

::: info
Expand All @@ -118,7 +126,7 @@ For Docker Compose based deployments follow the documentation here for [scaling]

:::

### Contribute or Report an issue
### Contribute or Report an issue with CPM Helm Chart
Thank you for your valuable assistance in enhancing our deployment process. If you would like to contribute, kindly create a pull request in our [GitHub](https://github.com/NFDI4Chem/repo-helm-charts) repository. For any issues or bugs you have encountered, please feel free to create an [issue](https://github.com/NFDI4Chem/repo-helm-charts/issues) in the same or write to us at caffeine-devs@uni-jena.de.
Your feedback is greatly appreciated.

Expand Down
2 changes: 1 addition & 1 deletion docs/conversions.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@ CONVERT module offers endpoints for converting molecular representations from on
width="100%"
height="670"
style="border:none"
src="https://dev.api.naturalproducts.net/v1/docs#/convert">
src="https://dev.api.naturalproducts.net/latest/docs#/convert">
</iframe>
</div>
4 changes: 2 additions & 2 deletions docs/depict.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ Depict module exposes endpoints for 2D and 3D molecular rendering using differen
<iframe id="inlineFrameExample"
title="Inline Frame Example"
width="100%"
height="230"
height="190"
style="border:none"
src="https://dev.api.naturalproducts.net/v1/docs#/depict">
src="https://dev.api.naturalproducts.net/latest/docs#/depict">
</iframe>
</div>
13 changes: 8 additions & 5 deletions docs/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,15 @@ docker run -d -p 8080:80 --name [image-name] caffeinejena/cheminfo-microservice:

```

# Docker Compose
[Docker Compose](https://docs.docker.com/get-started/08_using_compose/) is a tool that helps you define and share multi-container applications. With Compose, you can create a YAML file to define the services and with a single command, you can spin everything up or tear it all down.
The Cheminformatics Python Microservice comes packaged with docker-compose file which you can use to deploy your application in your server. To deploy using Docker compose follow the steps as described below.
## Docker Compose

[Docker Compose](https://docs.docker.com/get-started/08_using_compose/) is a handy tool that allows you to define and manage multi-container applications. By creating a YAML file to define the services, you can use a simple command to start or stop everything at once.
Check out https://docs.docker.com/get-started/08_using_compose/ for more information.

The Cheminformatics Python Microservice is readily equipped with a docker-compose file that enables you to effortlessly deploy your application on your server. To deploy your application using Docker compose, simply follow the steps described below.The Cheminformatics Python Microservice comes packaged with a docker-compose file, which you can use to deploy your application on your server. To deploy using Docker compose, follow the steps described below.

### Installation
1. Before you run the command make sure you've installed Docker including docker-compose support. If not then follow the link [here](https://docs.docker.com/compose/install/).
1. Before you run the command make sure you've installed Docker including docker-compose support. More details [here](https://docs.docker.com/compose/install/).
2. Clone the repository
```bash
git clone https://github.com/Steinbeck-Lab/cheminformatics-python-microservice.git
Expand All @@ -41,7 +44,7 @@ docker-compose -f ops/docker-compose-prod.yml up -d

## Scaling

Scaling in case of performance issues - This docker-compose file supports the builtin scaling. Request are load balanced by [Traefik](https://doc.traefik.io/traefik/). For example to scale upto 3 additional application containers you can simply invoke:
For better performance you can scale your application using docker-compose builtin scaling support. Requests are load balanced by [Traefik](https://doc.traefik.io/traefik/). For example to scale upto 3 additional application containers you can simply invoke:
```bash
docker-compose -f ops/docker-compose-prod.yml up -d --scale web=3 --no-recreate
```
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Welcome to Cheminformatics Python Microservice's documentation!
:width: 500
:align: center

This set of essential and valuable microservices is designed to be accessed via API calls to support cheminformatics. Generally, it is designed to work with SMILES-based inputs and could be used to translate between different machine-readable representations, get Natural Product (NP) likeliness scores, visualize chemical structures, and generate descriptors. In addition, the microservices also host an instance of STOUT and another instance of DECIMER (two deep learning models for IUPAC name generation and optical chemical structure recognition, respectively).
This set of essential and valuable microservices is designed to be accessed via API calls to support cheminformatics. Generally, it is designed to work with SMILES-based inputs and could be used to translate between different machine-readable representations, get Natural Product (NP) likeliness scores, visualize chemical structures, and generate descriptors. In addition, the microservice also host an instance of STOUT and another instance of DECIMER (two deep learning models for IUPAC name generation and optical chemical structure recognition, respectively).

How to access
=============
Expand Down
4 changes: 2 additions & 2 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,10 @@ In any case, you would run it like this:
$ uvicorn main:app --host 0.0.0.0 --port 8080 --workers 4
```

Update the Dockerfile in case you are running via docker-compose and rebuild the image for the changes to reflect.
Update the Dockerfile to watch for code changes

```
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80", "--workers", "4"]
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80", "--workers", "4", "--reload"]
```
</div>
Expand Down
Loading

0 comments on commit b9ce98b

Please sign in to comment.