Merge pull request #62 from WolframResearch/was/removing-minio-azure

[WAS-637] - Removed minio from Azure

EnvironmentSetup/Azure/README.md

@@ -51,7 +51,32 @@ Once your total region CPU of 40 or higher is confirmed available, authenticate
 
 	cd WAS-Kubernetes/EnvironmentSetup/Azure/
 
-**Step 3.** Replace the tag with your tenant ID and run the following command to set up AKS and deploy WAS:
+**Step 3.** Create resource group and store name
+
+	az group create -l <LOCATION> -n <RESOURCE_GROUP_NAME>
+```
+LOCATION: The region of the resource group will use, eastus
+RESOURCE_GROUP_NAME: The name of the resource group, WAS-rg
+```
+**Step 4.** Create storage account and store account key
+
+	az storage account create --resource-group <RESOURCE_GROUP_NAME> --name <SAN> --sku Standard_LRS --encryption-services blob
+
+`**SAN**: The storage account name, there shouldn't be any space and characters between letters`
+
+	az storage account keys list --resource-group <RESOURCE_GROUP_NAME> --account-name <SAN> --query '[0].value' -o tsv
+`The command will return the **SAN_ACCOUNT_KEY**`
+
+**Step 5.** Update the **containers** files with proper values
+
+	RESOURCE_GROUP_NAME:<resource group name, WAS-rg>
+	REGION:<location of the resource group, eastus>
+	SAN:<storage account name that created in step 3, azurewasa3280>
+	SAN_ACCOUNT_KEY:<storage account key that created in step3, dGVzdDEyMyMkwr3CvntbXX0uIQo=>
+	RESOURCEINFO_BUCKET:<blob name that WAS will create to store resources, was-resources-3280, name can be 	like given format>
+	NODEFILEINFO_BUCKET:<blob name that WAS will create to store nodefiles, was-nodefiles-3280, name can be 	like given format>
+
+**Step 6.** Replace the tag with your tenant ID and run the following command to set up AKS and deploy WAS:
 
 	YOURTENANTID=<your-tenant-id> && rm -rf Source/terraform/tenant-id.config && echo $YOURTENANTID >> Source/terraform/tenant-id.config && mkdir -p ~/.kube && docker-compose up --build -d && clear && docker exec -it azure-setup-manager bash setup --create && sudo chown -R $USER ~/.kube
 
@@ -63,7 +88,7 @@ Example:
 **Note:** This can take approximately 25 minutes to complete.
 
 
-**Step 4.** Run the following command to retrieve your base URL and application URLs:
+**Step 7.** Run the following command to retrieve your base URL and application URLs:
 
 	docker-compose up --build -d && clear && docker exec -it azure-setup-manager bash setup --endpoint-info
 
@@ -84,15 +109,15 @@ The output of this command will follow this pattern:
 
 
 
-**Step 5.** After completion, run this command to shutdown the azure-setup-manager:
+**Step 8.** After completion, run this command to shutdown the azure-setup-manager:
 
 	docker-compose down
 
 
-**Step 6.** Get a license file from your Wolfram Research sales representative.
+**Step 9.** Get a license file from your Wolfram Research sales representative.
 
 
-**Step 7.** This file needs to be deployed to WAS as a node file in the conventional location `.Wolfram/Licensing/mathpass`. From a Wolfram Language client, this may be achieved using the following code: 
+**Step 10.** This file needs to be deployed to WAS as a node file in the conventional location `.Wolfram/Licensing/mathpass`. From a Wolfram Language client, this may be achieved using the following code: 
 
     was = ServiceConnect["WolframApplicationServer", "http://<your-base-url>"];
 	ServiceExecute[was, "DeployNodeFile",
@@ -106,7 +131,7 @@ Alternatively you may use the [node files REST API](../../Documentation/API/Node
     Needs["WolframApplicationServer`"]
 
 
-**Step 8.** Restart the application using the [restart API](../../Documentation/API/Utilities.md) to enable your Wolfram Engines.
+**Step 11.** Restart the application using the [restart API](../../Documentation/API/Utilities.md) to enable your Wolfram Engines.
 
 URL: `http://<your-base-url>/.applicationserver/kernel/restart`
 

EnvironmentSetup/Azure/Source/Dockerfile

@@ -1,11 +1,11 @@
 FROM ubuntu:focal
 ENV KUBECTL_VERSION='v1.27.0'
 ENV DEBIAN_FRONTEND=noninteractive 
-ENV TERM xterm-256color
-ENV COLORTERM truecolor
+ENV TERM=xterm-256color
+ENV COLORTERM=truecolor
 ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
 ENV TF_DATA_DIR=/terraform-state
-RUN apt-get update && apt-get install software-properties-common apt-transport-https locales curl tar git procps libncurses-dev -y && \
+RUN apt-get update && apt-get install software-properties-common apt-transport-https locales curl tar git procps libncurses-dev --reinstall ca-certificates -y && \
 	locale-gen en_US.utf8 && locale -a  && \
 	update-locale LANG=en_US.UTF-8 LANG=en_US.UTF8 LC_CTYPE=en_US.UTF8 LC_COLLATE=en_US.UTF8 && \
 	curl -sL https://aka.ms/InstallAzureCLIDeb | bash && \

EnvironmentSetup/Azure/Source/containers

@@ -0,0 +1,6 @@
+RESOURCE_GROUP_NAME:<resource group name, WAS-rg>
+REGION:<location of the resource group, eastus>
+SAN:<storage account name that created in step 3, azurewasa3280>
+SAN_ACCOUNT_KEY:<storage account key that created in step3, dGVzdDEyMyMkwr3CvntbXX0uIQo=>
+RESOURCEINFO_BUCKET:<blob name that WAS will create to store resources, was-resources-3280, name can be 	like given format>
+NODEFILEINFO_BUCKET:<blob name that WAS will create to store nodefiles, was-nodefiles-3280, name can be 	like given format>

EnvironmentSetup/Azure/Source/deployments/active-web-elements-server-deployment.yaml

@@ -31,8 +31,14 @@ spec:
           value: /tmp/.wolframcache
         - name: KAFKA.BOOTSTRAP-SERVERS
           value: kafka-persistent-kafka-bootstrap.kafka.svc.cluster.local:9092
-        - name : API_REQUEST_TOPIC
+        - name: API_REQUEST_TOPIC
           value: "api-request"
+        - name: RESOURCE_INFO_TOPIC
+          value: "resource-info"
+        - name: NODEFILE_INFO_TOPIC
+          value: "nodefile-info"
+        - name: ENDPOINT_INFO_TOPIC
+          value: "endpoint-info"
         - name: applicationserver.nodefiles.cachedirectory
           value: /opt/.wolframcache/nodefiles/
         - name: applicationserver.version

EnvironmentSetup/Azure/Source/deployments/endpoint-manager-deployment.yaml

@@ -65,6 +65,10 @@ spec:
           value: docker
         - name: LOG_LOCATION
           value: /opt/app/logs
+        - name: RESOURCE_INFO_TOPIC
+          value: "resource-info"
+        - name: ENDPOINT_INFO_TOPIC
+          value: "endpoint-info"
         - name: KAFKA.BOOTSTRAP-SERVERS
           value: kafka-persistent-kafka-bootstrap.kafka.svc.cluster.local:9092
         image: wolframapplicationserver/endpoint-manager:1.2.5
@@ -92,9 +96,6 @@ spec:
         - mountPath: "/opt/app/logs"
           name: endpoint-logs-storage
       initContainers:
-      - name: init-minio
-        image: bash
-        command: ["bash", "-c", "for i in $(seq 1 3000); do nc -zvw1 minio 9000 && exit 0 || sleep 3; done; exit 1"]
       - name: init-kafka
         image: bash
         command: ["bash", "-c", "for i in $(seq 1 3000); do nc -zvw1 kafka-persistent-kafka-bootstrap.kafka.svc.cluster.local 9092 && exit 0 || sleep 3; done; exit 1"]

EnvironmentSetup/Azure/Source/deployments/resource-manager-deployment.yaml

@@ -29,16 +29,26 @@ spec:
           value: docker
         - name: LOG_LOCATION
           value: /opt/app/logs
+        - name: RESOURCE_INFO_TOPIC
+          value: "resource-info"
+        - name: NODEFILE_INFO_TOPIC
+          value: "nodefile-info"
         - name: KAFKA.BOOTSTRAP-SERVERS
           value: kafka-persistent-kafka-bootstrap.kafka.svc.cluster.local:9092
-        - name: MINIOACCESSKEY
-          value: <your-access-key>
-        - name: MINIOSECRETKEY
-          value: <your-secret-key>
+        - name: STORAGE_PROVIDER
+          value: "azure"
+        - name: SPRING_CLOUD_AZURE_STORAGE_BLOB_ENABLED
+          value: "true"
+        - name: SPRING_CLOUD_AZURE_STORAGE_BLOB_ACCOUNT_NAME
+          value: "STORAGE_ACCOUNT_NAME"
+        - name: SPRING_CLOUD_AZURE_STORAGE_BLOB_ACCOUNT_KEY
+          value: "STORAGE_ACCOUNT_KEY"
+        - name: SPRING_CLOUD_AZURE_STORAGE_BLOB_ENDPOINT
+          value: "https://SAN.blob.core.windows.net/"
         - name: RESOURCEINFO.BUCKET
-          value: <your-resourceinfo-bucketname>
+          value: RESOURCEINFO.BUCKETNAME
         - name: NODEFILES.BUCKET
-          value: <your-nodefiles-bucketname>
+          value: NODEFILES.BUCKETNAME
         image: wolframapplicationserver/resource-manager:1.2.6
         name: resource-manager
         ports:
@@ -53,20 +63,17 @@ spec:
         readinessProbe:
           tcpSocket:
             port: 9090
-          initialDelaySeconds: 5
-          periodSeconds: 10
+          initialDelaySeconds: 10
+          periodSeconds: 30
         livenessProbe:
           tcpSocket:
             port: 9090
-          initialDelaySeconds: 15
+          initialDelaySeconds: 30
           periodSeconds: 20   
         volumeMounts:
         - mountPath: "/opt/app/logs"
           name: resources-logs-storage
       initContainers:
-      - name: init-minio
-        image: bash
-        command: ["bash", "-c", "for i in $(seq 1 3000); do nc -zvw1 minio 9000 && exit 0 || sleep 3; done; exit 1"]
       - name: init-kafka
         image: bash
         command: ["bash", "-c", "for i in $(seq 1 3000); do nc -zvw1 kafka-persistent-kafka-bootstrap.kafka.svc.cluster.local 9092 && exit 0 || sleep 3; done; exit 1"]