Revert Flatcar Linux Azure to manual upload images

* Initial support for Flatcar Linux on Azure used the Flatcar
Linux Azure Marketplace images (e.g. `flatcar-stable`) in
#664
* Flatcar Linux Azure Marketplace images have some unresolved
items #703
* Until the Marketplace items are resolved, revert to requiring
Flatcar Linux's images be manually uploaded (like GCP and
DigitalOcean)

CHANGES.md

@@ -8,6 +8,7 @@ Notable changes between versions.
 * Choose Fedora CoreOS or Flatcar Linux (**action required**)
   * Use a `fedora-coreos` module for Fedora CoreOS
   * Use a `container-linux` module for Flatcar Linux
+* Change Container Linux modules' defaults from CoreOS Container Linux to [Flatcar Container Linux](https://typhoon.psdn.io/architecture/operating-systems/)
   * CoreOS Container Linux [won't receive updates](https://coreos.com/os/eol/) after May 2020
 
 ### Fedora CoreOS
@@ -16,27 +17,29 @@ Notable changes between versions.
 
 * Add support for Fedora CoreOS ([#704](https://github.com/poseidon/typhoon/pull/704))
 
-### Flatcar Linux / Container Linux
+### Container Linux
 
 #### AWS
 
-* Change Container Linux `os_image` default from `coreos-stable` to `flatcar-stable` ([#702](https://github.com/poseidon/typhoon/pull/702))
+* Change `os_image` default from `coreos-stable` to `flatcar-stable` ([#702](https://github.com/poseidon/typhoon/pull/702))
 
 #### Azure
 
-* Change Container Linux `os_image` default from `coreos-stable` to `flatcar-stable` ([#702](https://github.com/poseidon/typhoon/pull/702))
+* Change `os_image` to be required. Recommend uploading a Flatcar Linux image (**action required**) ([#702](https://github.com/poseidon/typhoon/pull/702))
+* Disable Flatcar Linux Azure Marketplace image [support](https://github.com/poseidon/typhoon/pull/664) (**breaking**)
+  * Revert to manual uploading until marketplace issue is closed ([#703](https://github.com/poseidon/typhoon/issues/703))
 
 #### Bare-Metal
 
-* Container Linux users should change [os_channel](https://typhoon.psdn.io/cl/bare-metal/#required) from a CoreOS channel to a Flatcar channel
+* Recommend changing [os_channel](https://typhoon.psdn.io/cl/bare-metal/#required) from `coreos-stable` to `flatcar-stable`
 
 #### Google
 
-* Change Container Linux `os_image` to be required. Container Linux users should upload a Flatcar Linux image and set it (**action required**) ([#702](https://github.com/poseidon/typhoon/pull/702))
+* Change `os_image` to be required. Recommend uploading a Flatcar Linux image (**action required**) ([#702](https://github.com/poseidon/typhoon/pull/702))
 
 #### DigitalOcean
 
-* Change Container Linux `os_image` to be required. Container Linux users should upload a Flatcar Linux image and set it (**action required**) ([#702](https://github.com/poseidon/typhoon/pull/702))
+* Change `os_image` to be required. Recommend uploading a Flatcar Linux image (**action required**) ([#702](https://github.com/poseidon/typhoon/pull/702))
 
 ## v1.18.1
 

azure/container-linux/kubernetes/controllers.tf

@@ -53,23 +53,18 @@ resource "azurerm_linux_virtual_machine" "controllers" {
     storage_account_type = "Premium_LRS"
   }
 
-  source_image_reference {
-    publisher = local.flavor == "flatcar" ? "Kinvolk" : "CoreOS"
-    offer     = local.flavor == "flatcar" ? "flatcar-container-linux" : "CoreOS"
-    sku       = local.channel
-    version   = "latest"
-  }
-
-  # Gross hack just for Flatcar Linux
-  dynamic "plan" {
-    for_each = local.flavor == "flatcar" ? [1] : []
+  // CoreOS Container Linux or Flatcar Container Linux (manual upload)
+  dynamic "source_image_reference" {
+    for_each = local.flavor == "coreos" ? [1] : []
 
     content {
-      name      = local.channel
-      publisher = "kinvolk"
-      product   = "flatcar-container-linux"
+      publisher = "CoreOS"
+      offer     = "CoreOS"
+      sku       = local.channel
+      version   = "latest"
     }
   }
+  source_image_id = local.flavor == "coreos" ? null : var.os_image
 
   # network
   network_interface_ids = [

azure/container-linux/kubernetes/variables.tf

@@ -48,8 +48,7 @@ variable "worker_type" {
 
 variable "os_image" {
   type        = string
-  description = "Channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta)"
-  default     = "flatcar-stable"
+  description = "Channel for a Container Linux derivative (/subscriptions/some-flatcar-upload, coreos-stable, coreos-beta, coreos-alpha)"
 }
 
 variable "disk_size" {

azure/container-linux/kubernetes/workers/workers.tf

@@ -24,23 +24,18 @@ resource "azurerm_linux_virtual_machine_scale_set" "workers" {
     caching              = "ReadWrite"
   }
 
-  source_image_reference {
-    publisher = local.flavor == "flatcar" ? "Kinvolk" : "CoreOS"
-    offer     = local.flavor == "flatcar" ? "flatcar-container-linux" : "CoreOS"
-    sku       = local.channel
-    version   = "latest"
-  }
-
-  # Gross hack just for Flatcar Linux
-  dynamic "plan" {
-    for_each = local.flavor == "flatcar" ? [1] : []
+  // CoreOS Container Linux or Flatcar Container Linux (manual upload)
+  dynamic "source_image_reference" {
+    for_each = local.flavor == "coreos" ? [1] : []
 
     content {
-      name      = local.channel
-      publisher = "kinvolk"
-      product   = "flatcar-container-linux"
+      publisher = "CoreOS"
+      offer     = "CoreOS"
+      sku       = local.channel
+      version   = "latest"
     }
   }
+  source_image_id = local.flavor == "coreos" ? null : var.os_image
 
   # Azure requires setting admin_ssh_key, though Ignition custom_data handles it too
   admin_username = "core"

docs/cl/azure.md

@@ -57,6 +57,38 @@ provider "ct" {
 
 Additional configuration options are described in the `azurerm` provider [docs](https://www.terraform.io/docs/providers/azurerm/).
 
+### Flatcar Linux Images
+
+Flatcar Linux publishes images for Azure. Azure allows custom images to be uploaded to a storage account bucket and imported.
+
+[Download](https://www.flatcar-linux.org/releases/) a Flatcar Linux Azure VHD image and upload it to an Azure storage account container (i.e. bucket).
+
+Azure requires fixed VHDs and Flatcar Linux provides dynamic VHDs, so uploads require Azure tools and cannot be done through the UI. Azure's tool compilation requires old versions, so Flatcar Linux has packaged a container image you may choose to use. See their [docs](https://docs.flatcar-linux.org/os/booting-on-azure/#uploading-your-own-image).
+
+```
+bzip2 -d flatcar_production_azure_image.vhd.bz2
+```
+
+```
+podman run -it --entrypoint=/bin/bash quay.io/kinvolk/azure-flatcar-image-upload
+...
+
+# az login
+# az storage account keys list --resource-group GROUP	--account-name BUCKET | jq -r '.[0].value'
+# azure-vhd-utils upload --localvhdpath /data/flatcar_production_azure_image.vhd --stgaccountname BUCKET --containername flatcar-linux --blobname flatcar-stable-2345.3.1 --stgaccountkey "KEYFROMABOVE"
+# exit
+```
+
+Create an Azure disk (note disk ID) and create an Azure image from it (note image ID).
+
+```
+az disk create --name flatcar-stable-2345.3.1 -g GROUP --source https://BUCKET.blob.core.windows.net/flatcar-linux/flatcar_production_azure_image.vhd
+
+az image create --name flatcar-stable-2345.3.1 -g GROUP --os-type=linux --source /subscriptions/some/path/providers/Microsoft.Compute/disks/flatcar-stable-2345.3.1
+```
+
+Set the [os_image](#variables) in the next step.
+
 ## Cluster
 
 Define a Kubernetes cluster using the module `azure/container-linux/kubernetes`.
@@ -72,6 +104,7 @@ module "ramius" {
   dns_zone_group = "example-group"
 
   # configuration
+  os_image           = "/subscriptions/some/path/Microsoft.Compute/images/flatcar-stable-2345.3.1"
   ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
 
   # optional
@@ -185,6 +218,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/azure/c
 | region | Azure region | "centralus" |
 | dns_zone | Azure DNS zone | "azure.example.com" |
 | dns_zone_group | Resource group where the Azure DNS zone resides | "global" |
+| os_image | Container Linux image for instances | "/subscriptions/..../some-flatcar-image", coreos-stable, coreos-beta, coreos-alpha |
 | ssh_authorized_key | SSH public key for user 'core' | "ssh-rsa AAAAB3NZ..." |
 
 !!! tip
@@ -225,7 +259,6 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | Machine type for controllers | "Standard_B2s" | See below |
 | worker_type | Machine type for workers | "Standard_DS1_v2" | See below |
-| os_image | Channel for a Container Linux derivative | "flatcar-stable" | coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta |
 | disk_size | Size of the disk in GB | 40 | 100 |
 | worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
 | controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |

docs/fedora-coreos/azure.md

@@ -67,7 +67,7 @@ Fedora CoreOS publishes images for Azure, but does not yet upload them. Azure al
 xz -d fedora-coreos-31.20200323.3.2-azure.x86_64.vhd.xz
 ```
 
-Create an Azure disk (note its ID) and create an Azure image from it (note its ID).
+Create an Azure disk (note disk ID) and create an Azure image from it (note image ID).
 
 ```
 az disk create --name fedora-coreos-31.20200323.3.2 -g GROUP --source https://BUCKET.blob.core.windows.net/fedora-coreos/fedora-coreos-31.20200323.3.2-azure.x86_64.vhd