[tf] COS (and Minio) terraform modules (#227)

* s3 fix as a local exec

* sleep longer

* Update

* Move null resource outside

* Fixes before merge

* Add gagent

* Fix

* Fix

* add self-monitoring relations

* add mimir and loki traicing relations

* add Grafana tracing relation

* grafana-agent send traces to Tempo

* migrate to issue bot

* add repos and gitignore

* migrate to issue bot

* add terraform ignres to .gitignore

* grafana agent scrapes mimir's metrics endpoint

* add loki's bucket name as variable

* add missing output

* fix output

* add buckets and s3_integrator names

* linting

* minio variables added

* add bucket_name output for tempo and mimir

* add outputs for se-integrator

* fix outputs

* fix relations

* add minio

* add s3-workaround

* fix charm-tracing endpoints names

* fix traicing endpoits

* linting

* mc_binary_url variable added

* s3-workaround.tf refactor

* mote linting...

* segregation of responsibilities: script -> logic, terraform -> execution

* linting

* refactor s3

* add missing versions.tf and fix url

* add missing model_name

* general fixes to s3 and cos models

* linting

* make s3fix.sh executable

* fix bucket names

* refactor tempo outputs.tf

* add tempo send-remote-write relation to mimir

* rename and make s3management.sh idempotent

* refactor and execute s3management always

* fiz relations and remove ref

* rename integration names

* fix description strinf

* add sensitive = true to minio password

* rename output.tf -> outputs.tf

* fix README.md

* add missing variables

* improve descriptions

* add units for mimir, loki and tempo

* fix grafana (charm-tracing) relation

* linting

* fix sample deployment in README.md

* more README.md improvements

* rename s3->minio and decouple from cos module

* refactor s3management.sh

* add missing output

---------

Signed-off-by: Jose C. Massón <939888+Abuelodelanada@users.noreply.github.com>
Co-authored-by: sed-i <82407168+sed-i@users.noreply.github.com>
Co-authored-by: Simon Aronsson <simme@arcticbit.se>

.gitignore

@@ -1,19 +1,23 @@
 **/dist/**
 
-# Local .terraform directories
+# Terraform
 **/.terraform/*
-
-# .tfstate files
 *.tfstate
 *.tfstate.*
-
-# Crash log files
+*.tfplan
+.terraform.lock.hcl
+crash.log
+.terraform/
+terraform.tfvars
+terraform.tfvars.json
+*.backup
+*.bak
 crash.log
 crash.*.log
 
 # Exclude all .tfvars files, which are likely to contain sensitive data, such as
-# password, private keys, and other secrets. These should not be part of version 
-# control as they are data points which are potentially sensitive and subject 
+# password, private keys, and other secrets. These should not be part of version
+# control as they are data points which are potentially sensitive and subject
 # to change depending on the environment.
 *.tfvars
 *.tfvars.json

terraform/modules/cos/README.md

@@ -1,6 +1,6 @@
 Terraform module for COS HA solution
 
-This is a Terraform module facilitating the deployment of COS HA solution, using the [Terraform juju provider](https://github.com/juju/terraform-provider-juju/). For more information, refer to the provider [documentation](https://registry.terraform.io/providers/juju/juju/latest/docs). 
+This is a Terraform module facilitating the deployment of COS HA solution, using the [Terraform juju provider](https://github.com/juju/terraform-provider-juju/). For more information, refer to the provider [documentation](https://registry.terraform.io/providers/juju/juju/latest/docs).
 
 The HA solution consists of the following Terraform modules:
 - [grafana-k8s](https://github.com/canonical/grafana-k8s-operator): Visualization, monitoring,and dashboards.
@@ -10,7 +10,7 @@ The HA solution consists of the following Terraform modules:
 
 This Terraform module deploys COS with Mimir and Loki in their microservices modes, and grafana, prometheus, and loki in monolithic mode.
 
-> [!NOTE]  
+> [!NOTE]
 > `s3-integrator` itself doesn't act as an S3 object storage system. For the HA solution to be functional, `s3-integrator` needs to point to an S3-like storage. See [this guide](https://discourse.charmhub.io/t/cos-lite-docs-set-up-minio/15211) to learn how to connect to an S3-like storage for traces.
 
 ## Requirements
@@ -23,8 +23,26 @@ The module offers the following configurable inputs:
 
 | Name | Type | Description | Required |
 | - | - | - | - |
-| `channel`| string | Channel that the charms are deployed from | latest/edge |
-| `model_name`| string | Name of the model that the charm is deployed on |  |
+| `channel` | string | Channel that the charms are deployed from | latest/edge |
+| `model_name` | string | Name of the model that the charm is deployed on |  |
+| `use_tls` | bool | Specify whether to use TLS or not for coordinator-worker communication |
+| `minio_user` | string | User for MinIO |
+| `minio_password` | string | Password for MinIO |
+| `loki_backend_units` | number | Number of Loki worker units with backend role |
+| `loki_backend_units` | number | Number of Loki worker units with backend role |
+| `loki_read_units` | number | Number of Loki worker units with read role |
+| `loki_write_units` | number | Number of Loki worker units with write role |
+| `mimir_backend_units` | number | Number of Mimir worker units with backend role |
+| `mimir_read_units` | number | Number of Mimir worker units with read role |
+| `mimir_write_units` | number | Number of Mimir worker units with write role |
+| `tempo_compactor_units` | number | Number of Tempo worker units with compactor role |
+| `tempo_distributor_units` | number | Number of Tempo worker units with distributor role |
+| `tempo_ingester_units` | number | Number of Tempo worker units with ingester role |
+| `tempo_metrics_generator_units` | number | Number of Tempo worker units with metrics_generator role |
+| `tempo_querier_units` | number | Number of Tempo worker units with querier role |
+| `tempo_query_frontend_units` | number | Number of Tempo worker units with query_frontend role |
+
+
 
 ### Outputs
 Upon application, the module exports the following outputs:
@@ -44,7 +62,69 @@ Users should ensure that Terraform is aware of the `juju_model` dependency of th
 
 To deploy this module with its needed dependency, you can run `terraform apply -var="model_name=<MODEL_NAME>" -auto-approve`. This would deploy all COS HA solution modules in the same model.
 
-### High Availability 
+### High Availability
+
+By default, this Terraform module will deploy each worker with `1` unit. If you want to scale each Loki, Mimir or Tempo worker unit please check the variables available for that purpose in `variables.tf`. For instance to deploy 3 units of each Loki worker, you can run:
+
+```shell
+terraform apply -var='minio_password=Password' -var='minio_user=User' -var='model_name=test'\
+-var='loki_backend_units=3' -var='loki_read_units=3' -var='loki_write_units=3'
+```
+
+
+### Minimal sample deployment.
+
+In orrder to deploy COS with just one unit per worker charm create a `main.rf` file with the following content.
+
+```hcl
+# COS module that deploy the whole Canonical Observability Stack
+module "cos" {
+  source         = "git::https://github.com/canonical/observability//terraform/modules/cos"
+  model_name     = var.model_name
+  minio_password = var.minio_password
+  minio_user     = var.minio_user
+}
+
+# S3 module that deploy the Object Storage MinIO required by COS
+module "minio" {
+  source         = "git::https://github.com/canonical/observability//terraform/modules/minio"
+  model_name     = var.model_name
+  channel        = var.channel
+  minio_user     = var.minio_user
+  minio_password = var.minio_password
+
+  loki  = module.cos.loki
+  mimir = module.cos.mimir
+  tempo = module.cos.tempo
+}
+
+variable "channel" {
+  description = "Charms channel"
+  type        = string
+  default     = "latest/edge"
+}
+
+variable "model_name" {
+  description = "Model name"
+  type        = string
+}
+variable "minio_user" {
+  description = "User for MinIO"
+  type        = string
+}
+
+variable "minio_password" {
+  description = "Password for MinIO"
+  type        = string
+  sensitive   = true
+}
+```
+
+
+Then execute:
+
+```shell
+$ tofu init
 
-By default, this Terraform module will deploy each worker with `1` unit. To configure the module to run `x` units of any worker role, you can run `terraform apply -var="model_name=<MODEL_NAME>" -var="<ROLE>_units=<x>" -auto-approve`.
-See each ... for the recommended scale for each role.
+$ tofu apply -var='minio_password=Password' -var='minio_user=User' -var='model_name=test'
+```