Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infra configs #224

Merged
merged 3 commits into from
Jun 30, 2023
Merged

Infra configs #224

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b4985aa
initial kube configs created via kompose
tejash-jl Mar 29, 2023
a635d27
Add helm chart for all rc service
tejash-jl May 23, 2023
06cbcbf
Update infra readme
tejash-jl May 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 24 additions & 21 deletions imports/realm-export.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -587,14 +587,14 @@
"clientId": "account",
"name": "${client_account}",
"rootUrl": "${authBaseUrl}",
"baseUrl": "/realms/divoc/account/",
"baseUrl": "/realms/sunbird-rc/account/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"/realms/divoc/account/*"
"/realms/sunbird-rc/account/*"
],
"webOrigins": [],
"notBefore": 0,
Expand All @@ -607,7 +607,9 @@
"publicClient": false,
"frontchannelLogout": false,
"protocol": "openid-connect",
"attributes": {},
"attributes": {
"login_theme": "sunbird-rc"
},
"authenticationFlowBindingOverrides": {},
"fullScopeAllowed": false,
"nodeReRegistrationTimeout": 0,
Expand All @@ -629,14 +631,14 @@
"clientId": "account-console",
"name": "${client_account-console}",
"rootUrl": "${authBaseUrl}",
"baseUrl": "/realms/divoc/account/",
"baseUrl": "/realms/sunbird-rc/account/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"/realms/divoc/account/*"
"/realms/sunbird-rc/account/*"
],
"webOrigins": [],
"notBefore": 0,
Expand All @@ -650,7 +652,8 @@
"frontchannelLogout": false,
"protocol": "openid-connect",
"attributes": {
"pkce.code.challenge.method": "S256"
"pkce.code.challenge.method": "S256",
"login_theme": "sunbird-rc"
},
"authenticationFlowBindingOverrides": {},
"fullScopeAllowed": false,
Expand Down Expand Up @@ -681,24 +684,24 @@
{
"id": "cd59dfa9-6289-40f9-b5c1-d76b1565501d",
"clientId": "admin-api",
"rootUrl": "https://divoc.xiv.in/keycloak/auth/realms/divoc/account/",
"adminUrl": "https://divoc.xiv.in/keycloak/auth/realms/divoc/account/",
"rootUrl": "https://sunbird-rc.xiv.in/keycloak/auth/realms/sunbird-rc/account/",
"adminUrl": "https://sunbird-rc.xiv.in/keycloak/auth/realms/sunbird-rc/account/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"https://localhost:4202/*",
"https://divoc.xiv.in",
"https://divoc.xiv.in/keycloak/auth/realms/divoc/account/*",
"https://sunbird-rc.xiv.in",
"https://sunbird-rc.xiv.in/keycloak/auth/realms/sunbird-rc/account/*",
"https://localhost:4200/*",
"https://oauth.pstmn.io/v1/callback",
"http://ndear.xiv.in/*"
],
"webOrigins": [
"https://localhost:4202/*",
"https://divoc.xiv.in",
"https://sunbird-rc.xiv.in",
"https://localhost:4200/*"
],
"notBefore": 0,
Expand Down Expand Up @@ -878,7 +881,7 @@
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"https://divoc-portal.xiv.in/*"
"https://sunbird-rc-portal.xiv.in/*"
],
"webOrigins": [],
"notBefore": 0,
Expand All @@ -896,7 +899,7 @@
"saml.force.post.binding": "false",
"saml.multivalued.roles": "false",
"saml.encrypt": "false",
"login_theme": "divoc",
"login_theme": "sunbird-rc",
"saml.server.signature": "false",
"saml.server.signature.keyinfo.ext": "false",
"exclude.session.state.from.auth.response": "false",
Expand Down Expand Up @@ -964,23 +967,23 @@
{
"id": "ac354645-3c3c-4592-879b-20a2f4f48302",
"clientId": "certificate-login",
"rootUrl": "https://divoc.xiv.in/certificate/",
"rootUrl": "https://sunbird-rc.xiv.in/certificate/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"http://192.168.0.103:3000/*",
"https://divoc.xiv.in/*",
"https://sunbird-rc.xiv.in/*",
"http://localhost:3000/*",
"*",
"https://divoc.xiv.in/certificate/",
"https://sunbird-rc.xiv.in/certificate/",
"http://localhost/certificate/*",
"https://divoc.xiv.in/certificate/*"
"https://sunbird-rc.xiv.in/certificate/*"
],
"webOrigins": [
"https://divoc.xiv.in",
"https://sunbird-rc.xiv.in",
"*",
"http://localhost",
"http://192.168.0.103:3000",
Expand Down Expand Up @@ -1284,14 +1287,14 @@
"clientId": "security-admin-console",
"name": "${client_security-admin-console}",
"rootUrl": "${authAdminUrl}",
"baseUrl": "/admin/divoc/console/",
"baseUrl": "/admin/sunbird-rc/console/",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"/admin/divoc/console/*"
"/admin/sunbird-rc/console/*"
],
"webOrigins": [
"+"
Expand Down Expand Up @@ -1351,7 +1354,7 @@
"clientAuthenticatorType": "client-secret",
"secret": "**********",
"redirectUris": [
"https://divoc.xiv.in",
"https://sunbird-rc.xiv.in",
"https://oauth.pstmn.io/v1/callback"
],
"webOrigins": [],
Expand Down
180 changes: 112 additions & 68 deletions infra/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,91 +1,135 @@
# SunbirdRC Infra setup

### Docker compose

##### Prerequisite
java 8

##### Start Postgres and Elastic Search

```sh
cd java/registry
docker-compose up -d db es
# Production deployment using Helm charts
The below scripts will help the adopters to deploy SunbirdRC services in kubernetes environment.

## Prerequisites
- Kubernetes Cluster with minimum 3 nodes
- [Helm](https://helm.sh/docs/intro/install/)
- kubectl
- Ingress
- Postgres DB (create database for `keycloak` and `registry`)
- ElasticSearch (Optional)
- Kafka (Optional)
- Redis (Optional)
- Minio (Optional)
- Domain URL (domain url mapped to kubernetes cluster)

The above optional services are not mandatory for SunbirdRC services. It can be installed based on the requirement on the project. For more details https://docs.sunbirdrc.dev/learn/readme-1/high-level-architecture

## Deployment steps

### Clone the repo
```bash
git clone https://github.com/Sunbird-RC/sunbird-rc-core.git
cd infra
```
```sh
sh configure-dependencies.sh
cd java/
./mvnw clean install -DskipTests
java -jar registry/target/registry.jar
```

### dependencies
* Elastic search https://hub.kubeapps.com/charts/bitnami/elasticsearch
* Postgres : https://hub.kubeapps.com/charts/cetic/postgresql


#Infra setup

### Install docker
sudo apt-get install docker.io

### Give sudo access to docker
sudo groupadd docker

echo $USER

sudo usermod -aG docker $USER

id

## Minikube setup
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

sudo install minikube-linux-amd64 /usr/local/bin/minikube

minikube start
### Pre check
Make sure from the current directory you're able to run the below commands
```bash
kubectl cluster-info
kubectl get nodes
kubectl get ns
helm version
```

### Create namespace
```bash
kubectl create ns demo-registry
```
`Feel free to use a different name for the namespace. Use the same name in the reset of the commands.`

### Install kubectl
sudo snap install kubectl --classic
### Create secrets
Convert all the passwords/secrets into base64 format and update these values in `values.yaml` file
**Secrets**
- DB_PASSWORD: Postgres database password
- KEYCLOAK_ADMIN_PASSWORD: Keycloak admin password used to login to admin console
- KEYCLOAK_DEFAULT_USER_PASSWORD: Default password to be set for new users created by registry
- MINIO_SECRET_KEY: Minio secret key
- ELASTIC_SEARCH_PASSWORD: Elastic search connection password
- KEYCLOAK_ADMIN_CLIENT_SECRET: Client secret of keycloak admin client for registry

### Helm setup
curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -
`DB_PASSWORD, KEYCLOAK_ADMIN_PASSWORD and KEYCLOAK_DEFAULT_USER_PASSWORD are mandotry secrets to be set. Other secrets can be set to empty `

sudo apt-get install apt-transport-https --yes
### Modify configuration values
Configuration values like database address, elastic search address etc should be modified in values.yaml file.

echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list

sudo apt-get update
### Schemas
All schema files should be placed in the schemas directory located at `sunbird-rc-core/infra/helm_charts/charts/registry/schemas`.

sudo apt-get install helm
### Configure signing keys
The signing keys should be placed in the below directories

### Add postgres kubeapps
helm repo add bitnami https://charts.bitnami.com/bitnami
Both public and private keys for signing

helm install ```APP_NAME``` bitnami/postgresql
`sunbird-rc-core/infra/helm_charts/charts/certificate-signer/keys`

### Install postgres client
sudo apt install postgresql-client-common
Only public key for exposing to verifiers

sudo apt-get install postgresql-client-12
`sunbird-rc-core/infra/helm_charts/charts/public-key-service/keys`
# Please note that by default a sample key is added. It is highly recommended to update this key before going to production.

Use helm status ```APP_NAME``` to get the details about the app.
### Deploy helm charts
```bash
helm upgrade --install --namespace=demo-registry demo-registry helm_charts --create-namespace
```
**Output**
```
Release "demo-registry" does not exist. Installing it now.
NAME: demo-registry
LAST DEPLOYED: Thu May 4 17:02:08 2023
NAMESPACE: demo-registry
STATUS: deployed
REVISION: 1
```

### Add elasticsearch kubeapps
helm repo add bitnami https://charts.bitnami.com/bitnami
**Check if all the pods are running**
```bash
kubectl get pods -n demo-registry
```

helm install bitnami/elasticsearch --version 15.9.1 --generate-name
### Import keycloak realm

NOTE: ES setup might take sometime to use
- Goto keycloak admin console `<host>/auth/`
- Login with username `admin` and use the same password configured in secrets
- Click on `Master` and select `Add realm`
- Select `https://github.com/Sunbird-RC/sunbird-rc-core/blob/main/imports/realm-export.json` file
- And click on `Create`

Use helm status ```APP_NAME``` to get the details about the app.

### Kubectl commands
To create namespace: kubectl create ns <name>
### Configure keycloak secret

To apply deployment file: kubectl -n <namespace> apply -f <deployment yaml>
**Get keycloak secret from keycloak admin console**
- Goto keycloak admin console `<host>/auth/`
- Login with username `admin` and use the same password configured in secrets
- Goto `clients` page and click on `admin-api`
- Goto `Credentials` tab and click on `Regenerate Secret`
- Copy the secret

To get all the pods: kubectl -n <namespace> get pods
**Configure secret in registry**
- Get all secrets created
```bash
kubectl get secret -n demo-registry
```
- Encode the secret in base64 format
```bash
echo -n "secret copied from keycloak" | base64
```
- Open the secret in edit mode
```bash
kubectl edit secret rc-secret -n demo-registry
```
Replace empty string for `KEYCLOAK_ADMIN_CLIENT_SECRET` with the base64 encoded secret
- Restart registry
```bash
kubectl rollout restart deploy/demo-registry -n demo-registry
```
- Check the pods status
```bash
kubectl get pods -n demo-registry
```

To get all the services: kubectl -n <namespace> get svc
### Check registry apis
Open the below url in browser and check if you're able to get the swagger json
`<host>/registry/api/docs/swagger.json`

Loading