All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- add Kubernetes Descheduler - https://github.com/kubernetes-sigs/descheduler/blob/master/charts/descheduler/README.md
- refactor Kubernetes Karpenter
- remove manual settings for eks managed node group. these are now fully managed by Karpenter.
- upgrade all EBS Volumes from gp2 to gp3
- replace kubectl_manifest with kubernetes_manifest
- relax most openedx-actions version pins to "v1"
- mandatory refactor of AWS S3 ACL config
- Olive release
This is a reference archive of the final code base for build-deploy of Open edX open-release/nutmeg.master version 14.
Lots of new functionality added related to gathering and storing meta data about the exact environment that was used to create AWS resources via Terraform. Also added functionality to gather and persist as much information as possible about build and deploy CI work flows.
- add module to gather environment state data of current user
- add cookiecutter meta tags for AWS resources
- revert to installing nutmeg.2 by default
- gather and persist CI build and deploy meta data in new k8s secrets
- add scorm proxy service to backend file storage based on eduNEXT prototype
Added lots of settings to settings_merge.yml in order to smooth out deployment hurdles as well as to expose as many of the easily-configurable features of Open edX as possible. These include:
- AUTH_PASSWORD_VALIDATORS
- COURSE_ABOUT_VISIBILITY_PERMISSION
- COURSE_CATALOG_VISIBILITY_PERMISSION
- CSRF_TRUSTED_ORIGINS
- Lots of additions to the FEATURES list
- HEARTBEAT_EXTENDED_CHECKS
- LANGUAGE_CODE
- LOGIN_REDIRECT_WHITELIST
- MKTG_URL_OVERRIDES
- PARENTAL_CONSENT_AGE_LIMIT
- PROFILE_IMAGE_SIZES_MAP
- THIRD_PARTY_AUTH_BACKENDS
- bug fix: settings_merge.yml PREVIEW_LMS_BASE
- refactor MySQL and MongoDB backup scripts
- add sticky sessions to openedx and wordpress ingresses
- add annotation to redirect http to https in openedx and wordpress ingresses
- bump deploy CI kubectl to 1.25/stable
- create one NAT gateway per availability zone
- set settings.ENABLE_HTTPS=true in deployment
- add a CORS policy to to openedx storage S3 bucket
- bump AWS EKS to release 1.25
- bump AWS EKS Add-on versions
- parameterize aws-auth.mapUsers
- refactor AWS EKS Managed Node groups into two groups, service and hosting. Default the service group to 3 nodes and the hosting group to 0.
- remove AWS EKS service node taints. replace these with node affinity for service pods to encourage isolation of these from nodes running mostly application software.
- create AWS EBS volumes for Wordpress deployments so that we can control lifecycle, naming and drive volume attributes.
- refactor MySQL and MongoDB remote backup solutions
- add backup cron jobs to Bastion
- add lifecycle retention policy to AWS S3 backup bucket
- refactor Kubernetes Dashboard
- Add Kubecost to stack
- Add resource quota to Wordpress environment
- add phpMyAdmin as an optional Wordpress installation feature
- tune Wordpress pod resource limits
- add Varnish cache as a stack installation option
- add VPC-CNI EKS Addon
- remove wildcards from all certificate manifests and ingresses
- consolidate all Wordpress config params into a single secret
- add multi-site Wordpress environment
- adds an optional Wordpress site to environment
- parameterize cluster issuer email address
- name cluster issuers and certificates by domain name
- add MFE url endpoint to CORS whitelist
- use a common TLS secret for service url endpoints
- bug fix MONGODB_HOST
- bump aws-ebs-csi-driver=v1.15.0-eksbuild.1
- refactor MFE treatment
Adds the following feature toggles:
- stack_add_remote_mysql: Y to create and auto-configure AWS RDS for MySQL, N to use tutor's k8s pod
- stack_add_remote_mongodb: Y to create and auto-configure AWS EC2 remote MongoDB, N to use tutor's k8s pod
- stack_add_remote_redis: Y to create and auto-configure AWS ElastiCache Redis, N to use tutor's k8s pod
- stack_add_k8s_dashboard: Y to install and configure Kubernetes Dashboard
- stack_add_k8s_kubeapps: Y to install and configure VMWare Kubeapps
- stack_add_k8s_karpenter: Y to install and configure Karpenter
- stack_add_k8s_prometheus: Y to install and configure Prometheus
- support for MFE's
- convert defalt stack certificate.yml to a template to parameterize name and namespace
- ensure that the secret name of all cert requests matches the domain name of the request itself
- add aws eks update-kubeconfig call ahead of annotating service account for AWS EBS CSI Driver
- add Terraform outputs to environments/modules/acm so that Terragrunt run-all won't complain
- parameterize environment ingress and certificate manifests
- parameterize REDIS_KEY_PREFIX in redis environment configuration
- add a kms_key_owners list to AWS EKS stack
- add Cookiecutter parameter documentation
- move redis module from environment to stack
- add tags to all redis resources
- fix all redis module deprecation warnings
- refactor redis security group from module to direct terraform resource declaration
- set stack mysql k8s secret HOST to route53 subdomain
- add a more complete set of outputs to each stack module
- add missing cluster name and namespace to build workflow
- add complete mock inputs and dependency declarations through environment hcl files.
- minor bug fixes after fully testing a build from scratch.
- ensure that sudo apt get install operations do not solicit input
- refactor environment tags
- parameterize stack name references in environment modules
- remove nginx vpa manifest from environment ingress module
- refactor AWS resource tags
- set global_platform_shared_resource_identifier=service
- bugs fixes related to refactoring of bastion, MongoDB, and Kubernetes
- refine Terragrunt Kubernetes dependencies
- refactor endpoints for all stack services: mysql, mongodb, redis, grafana, dashboard, kubeapps
- refactor aws resource tags to format of "cookiecutter/name-of-the-tag"
- rename Cookiecutter default global_platform_shared_resource_identifier=service
- move VPA manifest for metrics-server to kubernetes_vpa, since its a dependency
- pin each EKS Add-On version
- enhance Terragrunt stack dependency tree
- remove nginx ingress CLB DNS records from root domain
- refactor Prometheus into its own module
- refactor metrics-server into its own module
- refactor Vertical Pod Autoscaler into its own module
- bump all Helm chart versions
- add more Cookiecutter parameters
Note: this concludes the Kubernetes refactoring exercise. Happy new year!
- refactor karpenter into its own module.
- parameterize helm chart version of vertical-pod-autoscaler
- add Cookiecutter Y/N install parameters to toggle optional Kubernetes features: Karpenter, Prometheus, Dashboard, Kubeapps
- refactor cert-manager into its own module and move from environment to stack.
- Move cert-manager to its own namespace
- bump cert-manager to v1.8.2
- bump ingress-nginx-controller to 4.4.2 and parameterize version
- standardize and consolidate ssl cert to a single secret
- refactor all non-core kubernetees packages into a new Terraform module named kubernetes_monitoring
- created new subdomain to host all admin software packages
- add Kubernetes Dashboard web app
- add kubeapps by VMWare + Bitnami
- move grafana to new admin subdomain
- deprecated Github Action openedx-actions/tutor-plugin-enable-mfe
- deprecated Github Action openedx-actions/tutor-plugin-build-mfe
- version bumps to all Terraform AWS modules https://registry.terraform.io/namespaces/terraform-aws-modules
- version bumps to all Open edX Github Actions https://github.com/openedx-actions
- version bumps
- add sql db migration scripts
- Fix openedx_backup resource configuration
- refactor build-deploy workflows for tutor upgrade
- disable tutor web proxy to elminate unused EBS volume
- add IAM Role for EKS add-on EBS CSI
- add AmazonEBSCSIDriverPolicy to the karpenter node group
- add more parameters to openedx-actions/tutor-k8s-init
- bump to K8s 1.24
- version bumps to all Terraform AWS modules https://registry.terraform.io/namespaces/terraform-aws-modules
- version bumps to all Open edX Github Actions https://github.com/openedx-actions
- refactor cert-manager for v1.9
- bump to nutmeg.2
- bump tutor to 14.0.5
- tie optional repo build & deployment features to new Y/N flags in Cookiecutter
- add docker ce and python3 to bastion install.sh script
- add installed application versions to bastion login screen
- add an option to create a remote MongoDB server running on a standalone EC2 instance.
- reconfigure k8s node groups to use AWS SPOT Pricing for EC2 instances
- add k8s metrics-server
- add Prometheus
- add Grafana
- add Karpenter
- add k8s Horizontal Pod Autoscaling for all Open edX pods
- add k8s Vertical Pod Autoscaling for all Open edX pods
- add AWS S3 encrypted bucket to store e-commerce payment processor api keys and secrets
- add per-environment mysql db names
- add openedx-actions/tutor-plugin-configure-courseware-mfe
- add openedx-actions/tutor-plugin-enable-k8s-deploy-tasks
- misc security patches
- add a bastion setup script to install tutor, kubectl, terraform, terragrunt
- bump most openedx-actions to production
- enhanced k8s administration documentation
General production release
- refactor for tutor 14.x
- bump to open-release/nutmeg.1
- Refactor Github workflows to use openedx-actions
- bump all Terraform versions
- Adds the plugin installation
- adds Terraform code to create a dedicated private S3 bucket for backups
- Terraform
- bumped all version
- Deployment workflow
- bumped all versions
- Added installation options for Credentials, Ecommerce, MFE, Discovery, Notes, Forum, Xqueue
- Stacks
- Introduced shared infrastructure stacks consisting of a private VPC, EKS K8S and an option EC2 Bastion. This collection of resources is configured to host external non-openedx platforms such as for example, your custom micro services or a content management system.
- AWS Services
- Bastion: full Bastion management including creation and storage of ssh key
- K8s: added a namespace for shared secrets: Bastion ssh key, MySQL root credentials
- RDS: Added storage auto-scaling
- Mongo: reverted to Tutor-installed MongoDB pod on k8s
- removed subdomains list
- parameterized deployment yaml manifests with cookiecutter
- refactored VPC and EKS modules based on the current latest version of terraform-aws-modules modules
- upgraded AWS RDS Terraform module to v4
- added AWS certficates in us-east-1 and the aws region specified in environments/global.hcl
- added new module for Cloudfront distribution and DNS record for 'cdn' subdomain
- added new optional module for EC2 Bastion and DNS record for subdomain
- added version constraint parameters to cookiecutter for all terraform-aws-modules
- added mock outputs to terragrunt scripts to facilitate
run-allinit and validate operations in environments - added this change log
- resolved deprecation warnings in all modules
- restructured terraform folders
- fixed a bug that was causing multiple SSL/TLS certificates to be created in both us-east-1 as well as the environment region
- added the text 'cookiecutter' to the descriptions of all security groups, IAM roles, and IAM policies resources that are explicitly created by this repository
- added Cookiecutter parameters for environment_subdomain, ci_openedx_release_tag, ci_build_tutor_version, all teraform version constraints
- split environment_name and environment_subdomain
- added Cookiecutter post hook to process selection of EKS Load Balancer configuration
- added scripts to make, test, lint
- more sensible defaults in cookiecutter.json
- expanded README.md documentation
- added git pre-commit
- added AUTHORS.md
- Additional documentation
Initial release