From 25c6284bee94f35da471e0599218d0d45f068bdd Mon Sep 17 00:00:00 2001 From: Nathaniel Stickman <19861991+nasanos@users.noreply.github.com> Date: Wed, 15 Nov 2023 13:27:46 -0600 Subject: [PATCH] Guides on using cloud-init (#6603) * Initial draft of the guide on using cloud-init to configure and secure a new server. * Initial draft of the guide on installing and updating software with cloud-init. * Initial draft of the guide on adding and managing users with cloud-init. * General editing. * Initial draft of the guide on writing files with cloud-init. * Initial draft of the guide on running shell commands with cloud-init. * General revising. * Tech Edit 1 * CI Tests Fix 1 * Tech Edit 2 * Tech Edit 3 * Copy edit --------- Co-authored-by: Adam Overa Co-authored-by: Matt Wildman --- .../cloud-init/_index.md | 9 + .../index.md | 262 +++++++++++++++ .../index.md | 312 ++++++++++++++++++ .../manage-users-with-cloud-init/index.md | 195 +++++++++++ .../index.md | 113 +++++++ .../write-files-with-cloud-init/index.md | 147 +++++++++ .../laravel-forge/_index.md | 1 - 7 files changed, 1038 insertions(+), 1 deletion(-) create mode 100644 docs/guides/applications/configuration-management/cloud-init/_index.md create mode 100644 docs/guides/applications/configuration-management/cloud-init/configure-and-secure-servers-with-cloud-init/index.md create mode 100644 docs/guides/applications/configuration-management/cloud-init/install-and-update-software-with-cloud-init/index.md create mode 100644 docs/guides/applications/configuration-management/cloud-init/manage-users-with-cloud-init/index.md create mode 100644 docs/guides/applications/configuration-management/cloud-init/run-shell-commands-with-cloud-init/index.md create mode 100644 docs/guides/applications/configuration-management/cloud-init/write-files-with-cloud-init/index.md diff --git a/docs/guides/applications/configuration-management/cloud-init/_index.md b/docs/guides/applications/configuration-management/cloud-init/_index.md new file mode 100644 index 00000000000..99bbfbbbf21 --- /dev/null +++ b/docs/guides/applications/configuration-management/cloud-init/_index.md @@ -0,0 +1,9 @@ +--- +description: 'Cloud-init is an industry standard method for automating cloud instance initialization, with support across distributions and platforms. Cloud-init manages initialization using a combination of instance metadata and configuration scripts (user data) to automate the process of setting up a new server.' +keywords: ["cloud-init","cloud-config"] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +published: 2023-11-15 +title: Cloud-Init +show_in_lists: true +authors: ["Linode"] +--- \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/cloud-init/configure-and-secure-servers-with-cloud-init/index.md b/docs/guides/applications/configuration-management/cloud-init/configure-and-secure-servers-with-cloud-init/index.md new file mode 100644 index 00000000000..19a5b22fbb2 --- /dev/null +++ b/docs/guides/applications/configuration-management/cloud-init/configure-and-secure-servers-with-cloud-init/index.md @@ -0,0 +1,262 @@ +--- +slug: configure-and-secure-servers-with-cloud-init +title: "Use Cloud-Init to Automatically Configure and Secure Your Servers" +description: "Learn how you can use cloud-init to automate the process of configuring and securing a new cloud instance." +keywords: ['cloud-init','cloudinit','metadata'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +authors: ["Nathaniel Stickman"] +published: 2023-11-15 +modified_by: + name: Nathaniel Stickman +--- + +By using Akamai's Metadata service, you can automatically configure your new Compute Instances through cloud-init. This guide walks you through building a cloud-config file (for use with cloud-init) and deploying a Compute Instance using that configuration. It covers a series of recommended options for initializing and securing a Compute Instance. These configurations parallel the steps in our guide on [Setting Up and Securing a Compute Instance](/docs/products/compute/compute-instances/guides/set-up-and-secure/). Toward the end of the guide, you can see the [Complete Cloud-Config File](#complete-cloud-config-file) as well as steps for how to [Deploy an Instance with User Data](#deploy-an-instance-with-user-data). + +## What is cloud-init? + +[Cloud-init](https://cloudinit.readthedocs.io/en/latest/index.html) is an industry standard method for automating cloud instance initialization, with support across distributions and platforms. Cloud-init manages initialization using a combination of instance metadata and configuration scripts (user data) to automate the process of setting up a new server. + +Akamai's [Metadata](/docs/products/compute/compute-instances/guides/metadata/) service provides an API for cloud-init to consume, offering the relevant instance and user data to initialize your server. When your new instance spins up, cloud-init starts running locally, accesses the metadata, and automatically configures your system accordingly. + +{{< note title="Metadata Availability" >}} +Akamai’s Metadata service is now available in select data centers. Use Metadata to automate system configuration by adding directives or scripts when deploying Compute Instances. This user data can then be consumed by cloud-init, an industry standard system initialization tool, or accessed directly using the Metadata API. For instructions on using the Metadata service and for a list of supported regions and distributions, reference our [documentation](/docs/products/compute/compute-instances/guides/metadata/#availability). +{{< /note >}} + +## Create a Cloud-Config File + +Cloud-init consumes instance metadata from Akamai's Metadata server. This gives cloud-init relevant information about the Compute Instance. From there, cloud-init derives specific initialization steps from supplied *user data*, which comes in the form of *cloud-config* scripts. + +Cloud-config scripts use declarative YAML formatting to define configuration and other steps cloud-init needs to initialize the new instance. Learn more about cloud-config scripts in our guide on [Using Cloud-Config Files to Configure a Server](/docs/products/compute/compute-instances/guides/metadata-cloud-config/). + +When creating an Akamai Compute Instance, you can add the cloud-config via the Cloud Manager, Linode CLI, or Linode API when deploying the instance. Refer to our [Metadata](/docs/products/compute/compute-instances/guides/metadata/) guide for details on when and how to add the cloud-config *user data*. + +To start, create an initial cloud-config file to design your desired server initialization. The examples that follow name the file `cloud-config.yaml`. All cloud-config files begin with the following line: + +```file {title="cloud-config.yaml" lang="yaml"} +#cloud-config +``` + +From there, you need to fill out the cloud-config with specific options matching your needs for the server. Follow the steps within this guide to build your file or skip to end of the guide to see the [a completed Cloud-Config file](#complete-cloud-config-file). + +## Update Your System + +Cloud-config includes a module for handling system updates using two keys: `package_update` and `package_upgrade`. Including both of these with values of `true` ensures that package updates and upgrades are run as part of the initialization. + +```file {title="cloud-config.yaml" lang="yaml"} +package_update: true +package_upgrade: true +``` + +The section on how to [Install Any Additional Required Software](#install-any-additional-required-software) provides further resources for working with packages in cloud-config. + +## Set the Hostname and Timezone + +Cloud-config has a range of options for setting up server details. This section applies two of those options, providing your server with recommended details by setting up the timezone and hostname. + +The `timezone` key provides a straightforward method for setting your server's timezone. It takes any valid timezone for your system as an argument. Typically, you can find these using `timedatectl list-timezones` or by navigating the paths in `/usr/share/zoneinfo`. + +```file {title="cloud-config.yaml" lang="yaml"} +timezone: 'US/Central' +``` + +The `hostname` key conveniently assigns a hostname for your instance. In this example, the cloud-init initializes the server with the hostname `examplehost`: + +```file {title="cloud-config.yaml" lang="yaml"} +hostname: examplehost +``` + +{{< note >}} +Cloud-config's `hostname` key does not modify the `/etc/hosts` file by default. That lets you take control over the server's hostname by the usual route of modifying the `/etc/hosts` file after initialization. + +You can, alternatively, have cloud-init fully manage the `/etc/hosts` file by setting the `manage_etc_hosts` key to `true`. On each boot, cloud-init ensures that the contents of the `/etc/hosts` matches the contents of `/etc/cloud/templates/hosts.tmpl`. + +Learn more in cloud-init's module reference for [Update Etc Hosts](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#update-etc-hosts). +{{< /note >}} + +## Add a Limited User Account + +Your instance should have at least one limited user account to prevent remote root access and the associated security risks. Cloud-config's `users` key can define one or more new users for your system, including features like `sudo` access. + +For example, below is the typical configuration for initializing a limited user with `sudo` access. The setup also includes the default user, which is recommended. + +```file {title="cloud-config.yaml" lang="yaml"} +users: + - default + - name: example-user + groups: + - sudo + sudo: + - ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash +``` + +However, the example is incomplete, since the user does not have a password or SSH key. You can create a password using the `passwd` option, but this is not recommended. Instead, you should set up an SSH key for the user, as shown in the next section. + +For more on creating and managing users with cloud-init, refer to our guide [Use Cloud-Init to Manage Users on New Servers](/docs/guides/manage-users-with-cloud-init/). The guide includes more information on setting up user passwords, should you need to. + +## Add an SSH Key to Your Limited User Account + +Rather than using a password for access, the more secure approach is setting up your limited user, or users, with SSH key authentication. If you do not yet have an SSH key pair, get one by following the relevant section of our guide on how to [Use SSH Public Key Authentication](/docs/guides/use-public-key-authentication-with-ssh/#generate-an-ssh-key-pair). + +Once you have an SSH key pair, you can add an SSH public key to a user defined in the cloud-config with the `ssh_authorized_keys` option. The option accepts a list of SSH public keys to authorize for access to this user. + +```file {title="cloud-config.yaml" lang="yaml"} +users: + - default + - name: example-user + groups: + - sudo + sudo: + - ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + ssh_authorized_keys: + - +``` + +## Harden SSH + +To increase the security of SSH connections into your Compute Instance, you should generally disable password authentication and root logins via SSH. This way, access is restricted to limited users and connections authenticated by SSH key pairs. + +By default, the cloud-config `users` setup assumes `lock_passwd: true`, automatically disabling password authentication. You can learn more about user setup and managing such features in our guide [Use Cloud-Init to Manage Users on New Servers](/docs/guides/manage-users-with-cloud-init/). + +To disable root logins, you need to modify the SSH configuration file. Cloud-config does not have a direct option for this, but you can use its versatile `runcmd` key to automate the necessary commands. Learn more about the `runcmd` option in our guide [Use Cloud-Init to Run Commands and Bash Scripts on First Boot](/docs/guides/run-commands-and-bash-scripts-with-cloud-init). + +The example below removes any existing `PermitRootLogin` configuration and adds a new configuration disabling `PermitRootLogin`. The last command restarts the `sshd` service for the changes to take effect. + +```file {title="cloud-config.yaml" lang="yaml"} +runcmd: + - sed -i '/PermitRootLogin/d' /etc/ssh/sshd_config + - echo "PermitRootLogin no" >> /etc/ssh/sshd_config + - systemctl restart sshd +``` + +{{< note >}} +The example above assumes your system uses `systemctl` to manage the SSH service. While that is the case with the latest versions of the most popular distributions, it is not the case for all distributions. You may need to modify the commands above depending on how your system manages the SSH service. + +For instance, systems like CentOS 6, Debian 7, and Ubuntu 14.04 use `service` instead of `systemctl`. So you would need to replace the `systemctl` command above with the following: + +```command +service sshd restart +``` +{{< /note >}} + +## Install Any Additional Required Software + +With cloud-config's `packages` key, you can automate software installation and management as part of server initialization. For thorough coverage of cloud-init's package management features, and examples of how to use it, see our guide [Use Cloud-Init to Install and Update Software on New Servers](/docs/guides/install-and-update-software-with-cloud-init/). + +As a basic illustration, the snippet below shows how to install a set of software during instance initialization. The example installs software for a LEMP web stack (NGINX, MySQL, and PHP) a popular setup for web applications. You can learn more about LEMP stacks in our guide on how to [Install a LEMP Stack](/docs/guides/how-to-install-a-lemp-stack-on-ubuntu-22-04/). + +```file {title="cloud-config.yaml" lang="yaml"} +packages: + - mysql-server + - nginx + - php +``` + +{{< note >}} +Software packages have different names depending on the distribution you are using. Reference your distribution's package repository to identify the package names for the software you wish to install. +{{}} + +## Complete Cloud-Config File + +What follows is a complete example cloud-config file, summarizing all of the initialization options covered in this tutorial. You can use this as a basis for initializing your own Compute Instances. For example, remove the `packages` section and customize the limited user and server details, and you have a script following our [Setting Up and Securing a Compute Instance](/docs/products/compute/compute-instances/guides/set-up-and-secure/) guide. Feel free to add other features to fine-tune the instance to your needs. + +```file {title="cloud-config.yaml" lang="yaml" hl_lines="6,13,20,21,31,32,33"} +#cloud-config + +# Configure a limited user +users: + - default + - name: example-user + groups: + - sudo + sudo: + - ALL=(ALL) NOPASSWD:ALL + shell: /bin/bash + ssh_authorized_keys: + - "SSH_PUBLIC_KEY" + +# Perform System Updates +package_update: true +package_upgrade: true + +# Configure server details +timezone: 'US/Central' +hostname: examplehost + +# Harden SSH access +runcmd: + - sed -i '/PermitRootLogin/d' /etc/ssh/sshd_config + - echo "PermitRootLogin no" >> /etc/ssh/sshd_config + - systemctl restart sshd + +# Install additional software packages +packages: + - nginx + - mysql-server + - php +``` + +## Deploy an Instance with User Data + +There are three paths to deploy a new Compute Instance using your cloud-config initialization script. These options are summarized below, with links to relevant deployment guides. For more coverage of cloud-init deployments, review our [Overview of the Metadata Service](/docs/products/compute/compute-instances/guides/metadata/) guide. + +- **Cloud Manager**: You can conveniently configure and deploy new instances using a web browser. Follow along with our [Create a Compute Instance](/docs/products/compute/compute-instances/guides/create/) guide to deploy a new instance. The guide includes a section on how to **Add User Data**, showing where you can input your cloud-config content. + +- **Linode CLI**: The command-line tool provides a command for creating a new Compute Instance, and that command can take a `--metdata.user_data` option with your cloud-config script. To learn more about using the Linode CLI, see our guides [Getting Started with the Linode CLI](/docs/products/tools/cli/get-started/) and [Linode CLI Commands for Compute Instances](/docs/products/tools/cli/guides/linode-instances/). + + What follows is an example set of commands to deploy an instance from a cloud-config file. The example assumes you have set up the Linode CLI tool and that your cloud-config is stored as `cloud-config.yaml` in the current directory. Review the guide linked above for more on the other options used in this example command. + + The CLI command requires that the cloud-config be encoded as a base64 string, which you can make using the `cat cloud-config.yaml | base64 -w 0` command. The first command below does this and assigns the value to a shell variable for convenience. + + ```command + cloudconfigvar="$(cat cloud-config.yaml | base64 -w 0)" + + linode-cli linodes create \ + --label cloud-init-example \ + --region us-iad \ + --type g6-nanode-1 \ + --image linode/ubuntu22.04 \ + --root_pass EXAMPLE_ROOT_PASSWORD \ + --metadata.user_data "$cloudconfigvar" + ``` + +- **Linode API**: Within the `instances/` endpoint of the API, you have access to a `metadata.user_data` option for inputting a cloud-config. Using this, you can initialize a new Compute Instance in a convenient `POST` request. Learn more about the Linode API in our documentation on the [Linode API](/docs/api/) and the [Linode Instances API](/docs/api/linode-instances/) documentation. + + With versatility being one of its main advantages, there are numerous ways to use the Linode API to deploy a server. The steps below show a simple approach using just the command line. This example is easily adaptable for other contexts as well. + + 1. Create a payload file. This is a convenient approach for a `POST` request's data, as it makes the data easier to craft and reuse. You can create the file using your preferred text editor, but this example uses a `>` command to do so. + + ```command + cat > cloud-init-example-deployment.json <<'EOF' + { + "label": "cloud-init-example", + "region": "us-iad", + "type": "g6-nanode-1", + "image": "linode/ubuntu22.04", + "root_pass": "EXAMPLE_ROOT_PASSWORD", + "metadata": { + "user_data": "" + } + } + EOF + ``` + + 1. Insert the cloud-config as `metadata.user_data`. The API, like the CLI, requires that the cloud-config be encoded as a base64 string. Below, that is accomplished using the command `cat cloud-config.yaml | base64 -w 0`, and the value is assigned to a shell variable for convenience. + + ```command + cloudconfigvar="$(cat cloud-config.yaml | base64 -w 0)" + sed -i "s,,$cloudconfigvar," cloud-init-example-deployment.json + ``` + + Alternatively, you can copy the base64 string, open the file, and paste the string into the `user_data` field. + + 1. Make the `POST` request to the `instances` API endpoint. Include your Linode API personal access token (covered in the guide linked above) as an `Authorization: Bearer` header. + + ```command + curl -H "Content-Type: application/json" \ + -H "Authorization: Bearer API_ACCESS_TOKEN" \ + -X POST \ + -d @cloud-init-example-deployment.json \ + https://api.linode.com/v4/linode/instances + ``` \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/cloud-init/install-and-update-software-with-cloud-init/index.md b/docs/guides/applications/configuration-management/cloud-init/install-and-update-software-with-cloud-init/index.md new file mode 100644 index 00000000000..7b7938b8716 --- /dev/null +++ b/docs/guides/applications/configuration-management/cloud-init/install-and-update-software-with-cloud-init/index.md @@ -0,0 +1,312 @@ +--- +slug: install-and-update-software-with-cloud-init +title: "Use Cloud-Init to Install and Update Software on New Servers" +description: "Learn how you can use cloud-init to upgrade and install software automatically when initializing new servers." +keywords: ['cloud-init','cloudinit','apt','yum'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +authors: ["Nathaniel Stickman"] +published: 2023-11-15 +modified_by: + name: Nathaniel Stickman +external_resources: +- '[Cloud-init Documentation - Module Reference: Package Update Upgrade Install](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#package-update-upgrade-install)' +- '[Cloud-init Documentation - Cloud-config Examples: Install Arbitrary Packages](https://cloudinit.readthedocs.io/en/latest/reference/examples.html#install-arbitrary-packages)' +--- + +[Cloud-init](https://cloudinit.readthedocs.io/en/latest/index.html) offers a cross-platform, cross-distribution approach to automating server initialization. With Akamai's [Metadata](/docs/products/compute/compute-instances/guides/metadata/) service, you can leverage cloud-init to deploy Compute Instances, employing custom user data scripts to define your desired setup. + +In this guide, learn how to manage packages on new servers using cloud-init. Whether you want to upgrade system packages, install packages during initialization, or manage your repositories, this tutorial shows you how. + +Before getting started, you should review our guide on how to [Use Cloud-Init to Automatically Configure and Secure Your Servers](/docs/guides/configure-and-secure-servers-with-cloud-init/). There, you can see how to create a cloud-config file, which you need to follow along with the present guide. When you are ready to deploy your cloud-config, the guide linked above shows you how. + +## Upgrade Packages + +Cloud-init includes a module for managing package updates and upgrades during initialization. The `package_update` option, when set to `true`, applies updates to installed packages and updates the package repositories. This option is generally useful to ensure the server is working from the most up-to-date package references. + +```file {title="cloud-config.yaml" lang="yaml"} +package_update: true +``` + +The `package_upgrade` option upgrades installed packages to their latest versions. Unless you need specific package versions, running this option during initialization keeps your system more stable and secure. + +```file {title="cloud-config.yaml" lang="yaml"} +package_upgrade: true +``` + +Additionally, cloud-config has an option to ensure that the system reboots for any package upgrades or installations that require it. With this option, you ensure that package upgrades and installations are ready to use immediately after the cloud-init process has finished. + +```file {title="cloud-config.yaml" lang="yaml"} +package_update: true +package_upgrade: true +package_reboot_if_required: true +``` + +## Install Packages + +To install packages with cloud-init, use the `packages` option in your cloud-config. Provide the option a list of package names, and cloud-init handles installation during the initialization. + +Below are examples installing the main components of a LAMP stack, a popular web application setup. Cloud-config requires exact package names, which can vary between distributions, as can the overall prerequisites for a setup. To demonstrate, the examples below show how the setup would look between two different distributions. + +Learn more about the LAMP stack, and its package prerequisites, in our guide on [How to Install a LAMP Stack](/docs/guides/how-to-install-a-lamp-stack-on-ubuntu-22-04/). Use the drop down at the top of that guide to see different distributions. + +{{< tabs >}} +{{< tab "Ubuntu 22.04" >}} +```file {title="cloud-config.yaml" lang="yaml"} +packages: + - apache2 + - mysql-server + - php + - libapache2-mod-php + - php-mysql +``` +{{< /tab >}} +{{< tab "CentOS 8" >}} +```file {title="cloud-config.yaml" lang="yaml"} +packages: + - httpd + - mariadb-server + - php + - php-pear + - php-mysqlnd +``` +{{< /tab >}} +{{< /tabs >}} + +{{< note >}} +The `package_reboot_if_required` option covered in the previous section also affects package installations. If set to `true`, it ensures that the system restarts if any newly installed packages require it. +{{< /note >}} + +## Add Software Repositories + +Among the more advanced package manager tools within cloud-init is the ability to add custom repositories during initialization. Cloud-init uses specific modules for managing different package managers, so the steps vary depending on your distribution. What follows covers two of the most popular: [APT](/docs/guides/apt-package-manager/), most often found on Debian and Ubuntu systems, and [Yum](/docs/guides/yum-package-manager/)/[DNF](/docs/guides/dnf-package-manager/), mostly found on CentOS, Fedora, and other RHEL-based distributions. + +Other than these, cloud-init also supports the [Zypper](/docs/guides/zypper-package-manager/) package manager, used on openSUSE distributions. You can learn about adding repositories for Zypper in cloud-init's [Zypper Add Repo](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#zypper-add-repo) module reference documentation. + +{{< tabs >}} +{{< tab "APT" >}} +Within cloud-config, the `apt` option allows for fine-grained management of the APT package manager. You can learn more about the range of features through cloud-init's [APT Configure](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#apt-configure) module reference. + +To add third-party repositories to APT, use the `sources` option within an `apt` block. The `sources` option is a dictionary, with one or more repository entries. Each repository entry needs a `source` string, indicating the repository location, and a set of key-related options, providing the GPG key for the repository. + +There are two means of adding the repository, based on how you want to supply the GPG key: + +- To use a GPG key server, you can supply the key ID and the server location, as in this example. + + ```file {title="cloud-config.yaml" lang="yaml"} + apt: + sources: + docker: + source: deb [arch="amd64"] https://download.docker.com/linux/ubuntu $RELEASE stable + keyid: 8D81803C0EBFCD88 + keyserver: 'https://download.docker.com/linux/ubuntu/gpg' + ``` + + In this case, the key ID was obtained from the GPG public key using this set of commands: + + ```command + wget https://download.docker.com/linux/ubuntu/gpg -O docker.gpg.pub.key + gpg --list-packets docker.gpg.pub.key | awk '/keyid:/{ print $2 }' + ``` + +- The other option is to manually add the GPG key, using the `key` option, as shown in this example. Replace the example GPG string with a full GPG public key, like the one retrieved with the `wget` command above. + + ```file {title="cloud-config.yaml" lang="yaml"} + apt: + sources: + docker: + source: deb [arch="amd64"] https://download.docker.com/linux/ubuntu $RELEASE stable + key: | + -----BEGIN PGP PUBLIC KEY BLOCK----- + + mQINBFit2ioBEADhWpZ8/wvZ6hUTiXOwQHXMAlaFHcPH9hAtr4F1y2+OYdbtMuth + lqqwp028AqyY+PRfVMtSYMbjuQuu5byyKR01BbqYhuS3jtqQmljZ/bJvXqnmiVXh + 38UuLa+z077PxyxQhu5BbqntTPQMfiyqEiU+BKbq2WmANUKQf+1AmZY/IruOXbnq + ... + jCxcpDzNmXpWQHEtHU7649OXHP7UeNST1mCUCH5qdank0V1iejF6/CfTFU4MfcrG + YT90qFF93M3v01BbxP+EIY2/9tiIPbrd + =0YYh + -----END PGP PUBLIC KEY BLOCK----- + ``` + +With either method, you can verify the added repository on the new system with a command like this: + +```command +sudo apt-cache policy +``` + +```output +Package files: + 100 /var/lib/dpkg/status + release a=now + 500 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages + release o=Docker,a=jammy,l=Docker CE,c=stable,b=amd64 + origin download.docker.com +... +``` + +{{< /tab >}} +{{< tab "Yum/DNF" >}} +Cloud-config uses a dedicated `yum_repos` option for managing repositories in Yum and DNF. Each repository to be added to the package manager gets an entry beneath the `yum_repos` option, with an identifier, URL, and other information. + +The example here adds the Extra Packages for Enterprise Linux (EPEL) repository, a popular repository for accessing a wider range of packages on RHEL-based systems: + +```file {title="cloud-config.yaml" lang="yaml"} +yum_repos: + epel-release: + name: Extra Packages for Enterprise Linux 8 - Release + baseurl: http://download.fedoraproject.org/pub/epel/8/Everything/$basearch + enabled: true + failovermethod: priority + gpgcheck: true + gpgkey: http://download.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 +``` + +The first three options (`name`, `baseurl`, and `enabled`) are specific to cloud-init's `yum_repos`. However, `yum_repos` also supports the use of Yum's own repository configuration options, which the rest of the options above leverage. + +Once initialization has finished, you can verify the added repository using the `repolist` command for Yum/DNF: + +```command +sudo dnf repolist +``` + +```output +repo id repo name +... +epel-release Extra Packages for Enterprise Linux 8 - Release +... +``` + +Learn more about the various features the `yum_repos` option can manage in cloud-init's [Yum Add Repo](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#yum-add-repo) module reference documentation. +{{< /tab >}} +{{< /tabs >}} + +## Verify Update and Installation + +Cloud-init stores a log at `/var/log/cloud-init-output.log` with all of the output from cloud-init's initialization steps. For instance, the example output below shows the portion of the logs for APT installing the `apache2` package. + +```command +sudo cat /var/log/cloud-init-output.log +``` + +```output +... +The following additional packages will be installed: + apache2-bin apache2-data apache2-utils libapache2-mod-php8.1 libapr1 + libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libcgi-fast-perl + libcgi-pm-perl libclone-perl libencode-locale-perl libevent-pthreads-2.1-7 + libfcgi-bin libfcgi-perl libfcgi0ldbl libhtml-parser-perl + libhtml-tagset-perl libhtml-template-perl libhttp-date-perl + libhttp-message-perl libio-html-perl liblua5.3-0 liblwp-mediatypes-perl + libmecab2 libprotobuf-lite23 libtimedate-perl liburi-perl mecab-ipadic + mecab-ipadic-utf8 mecab-utils mysql-client-8.0 mysql-client-core-8.0 + mysql-common mysql-server-8.0 mysql-server-core-8.0 php-common php8.1 + php8.1-cli php8.1-common php8.1-mysql php8.1-opcache php8.1-readline + ssl-cert +Suggested packages: + apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser + php-pear libdata-dump-perl libipc-sharedcache-perl libbusiness-isbn-perl + libwww-perl mailx tinyca +The following NEW packages will be installed: + apache2 apache2-bin apache2-data apache2-utils libapache2-mod-php + libapache2-mod-php8.1 libapr1 libaprutil1 libaprutil1-dbd-sqlite3 + libaprutil1-ldap libcgi-fast-perl libcgi-pm-perl libclone-perl + libencode-locale-perl libevent-pthreads-2.1-7 libfcgi-bin libfcgi-perl + libfcgi0ldbl libhtml-parser-perl libhtml-tagset-perl libhtml-template-perl + libhttp-date-perl libhttp-message-perl libio-html-perl liblua5.3-0 + liblwp-mediatypes-perl libmecab2 libprotobuf-lite23 libtimedate-perl + liburi-perl mecab-ipadic mecab-ipadic-utf8 mecab-utils mysql-client-8.0 + mysql-client-core-8.0 mysql-common mysql-server mysql-server-8.0 + mysql-server-core-8.0 php php-common php-mysql php8.1 php8.1-cli + php8.1-common php8.1-mysql php8.1-opcache php8.1-readline ssl-cert +0 upgraded, 49 newly installed, 0 to remove and 11 not upgraded. +... +``` + +While the logs' high level of detail is useful for debugging, it makes verifying upgraded and installed packages somewhat cumbersome. Instead, use commands specific to your system's package manager to verify package upgrades and installations. Below you can find steps for doing that on Debian/Ubuntu systems (using APT) and RHEL-based systems like CentOS and Fedora (using DNF or Yum). + +{{< tabs >}} +{{< tab "Debian, Ubuntu" >}} +The APT package manager includes a `list` command that provides useful functions for reviewing packages. Using the `--upgradable` option with the command displays a list of packages that have available upgrades. + +``` command +sudo apt list --upgradable +``` + +Ideally, the output would be empty, but your system usually has a few packages that do not get upgraded with `apt upgrade`. Typically, this is due to dependencies. If this is the case, the `upgrade` command, and the cloud-init logs, should indicate the un-upgraded packages. + +```file {title="/var/log/cloud-init-output.log"} +... +0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded. +... +``` + +The `list` command's `--installed` option provides a comprehensive list of packages installed with the APT package manager. However, this list can be hard to navigate for verification as it also includes all packages installed as dependencies. To only see the packages that have been installed explicitly, use the `--manual-installed` option instead. + +```command +sudo apt list --manual-installed +``` + +The package installation example earlier in this guide only had a few packages, so a short text filter could even further shorten the output. The command below does this by piping to `grep`. Each `\|` separates a search term, and each search term identifies one or more of the packages indicated in the cloud-config. + +```command +sudo apt list --manual-installed | grep 'apache2\|mysql-server\|php' +``` + +```output +apache2/jammy-updates,now 2.4.52-1ubuntu4.6 amd64 [installed] +libapache2-mod-php/jammy,now 2:8.1+92ubuntu1 all [installed] +mysql-server/jammy-updates,jammy-security,now 8.0.34-0ubuntu0.22.04.1 all [installed] +php-mysql/jammy,now 2:8.1+92ubuntu1 all [installed] +php/jammy,now 2:8.1+92ubuntu1 all [installed] +``` + +{{< /tab >}} +{{< tab "AlmaLinux, CentOS, Fedora, Rocky Linux" >}} +With the Yum/DNF package manager, the dedicated `check-update` command shows any packages with available upgrades. After the cloud-init initialization, the output should be empty, indicating that all installed packages are up to date. + +{{< note >}} +The examples in this section explicitly use `dnf`, as newer systems most often use the DNF package manager rather than Yum. However, if your system uses Yum instead, the same commands should work, just replace `dnf` with `yum`. +{{< /note >}} + +```command +sudo dnf check-update +``` + +Typically, just after initialization, the most straightforward way to verify installed packages is through the `history` command. The output shows recent commands run by the package manager, including updates and installations. + +```command +sudo dnf history +``` + +```output +ID | Command line | Date and time | Action(s) | Altered +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ + 2 | -y install httpd mariadb-server php php-pear php-mysqlnd | 2023-08-09 12:12 | Install | 76 + 1 | -y upgrade | 2023-08-09 12:11 | I, U | 132 +``` + +With Yum/DNF, you can also more specifically check on installed packages with the `list installed` command. Below is an example that also pipes the results to a `grep` text search, allowing you to narrow the output to just matching package names. Here, each search term is separated by `\|`. This may be useful when you only want to verify a limited range of installed packages. + +``` command +sudo dnf list installed | grep 'httpd\|mariadb-server\|php' +``` + +```output +httpd.x86_64 2.4.37-62.module_el8+657+88b2113f @appstream +httpd-filesystem.noarch 2.4.37-62.module_el8+657+88b2113f @appstream +httpd-tools.x86_64 2.4.37-62.module_el8+657+88b2113f @appstream +mariadb-server.x86_64 3:10.3.28-1.module_el8.3.0+757+d382997d @appstream +mariadb-server-utils.x86_64 3:10.3.28-1.module_el8.3.0+757+d382997d @appstream +php.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-cli.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-common.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-fpm.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-mysqlnd.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-pdo.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-pear.noarch 1:1.10.5-9.module_el8.2.0+313+b04d0a66 @appstream +php-process.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +php-xml.x86_64 7.2.24-1.module_el8.2.0+313+b04d0a66 @appstream +``` + +{{< /tab >}} +{{< /tabs >}} \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/cloud-init/manage-users-with-cloud-init/index.md b/docs/guides/applications/configuration-management/cloud-init/manage-users-with-cloud-init/index.md new file mode 100644 index 00000000000..d639778b9db --- /dev/null +++ b/docs/guides/applications/configuration-management/cloud-init/manage-users-with-cloud-init/index.md @@ -0,0 +1,195 @@ +--- +slug: manage-users-with-cloud-init +title: "Use Cloud-Init to Manage Users on New Servers" +description: "Follow along with this guide to use cloud-init for managing users and user groups on new servers." +keywords: ['cloud-init','cloudinit','users','groups'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +authors: ["Nathaniel Stickman"] +published: 2023-11-15 +modified_by: + name: Nathaniel Stickman +external_resources: +- '[Cloud-init Documentation - Module Reference: Users and Groups](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#users-and-groups)' +--- + +[Cloud-init](https://cloudinit.readthedocs.io/en/latest/index.html) is an industry-standard solution for automating server deployments, with support across platforms and distributions. Combining platform metadata with custom user data scripts, cloud-init can vastly simplify the process of initializing new servers. + +With Akamai's [Metadata](/docs/products/compute/compute-instances/guides/metadata/) service, you can leverage cloud-init to deploy Compute Instances. A cloud-config user data script can define everything you need to initialize instances, from security and user set up to software installation and shell scripts. + +This guide details how to start working with users as part of your cloud-init deployments. Read on for cloud-config scripts to provision users, add SSH keys, and disable remote root access. + +Before getting started, you should review our guide on how to [Use Cloud-Init to Automatically Configure and Secure Your Servers](/docs/guides/configure-and-secure-servers-with-cloud-init/). There, you can see how to create a cloud-config file, which you need to follow the present guide. When you are ready to deploy your cloud-config, the guide linked above shows how. + +## Create User + +Within a cloud-config script, the `users` option handles user creation and most features for managing users. At its simplest, the option can work with as little as a `default` item, ensuring that cloud-init creates the default user. Whatever other users you add, it is recommended that you keep an entry for the `default` user as well. + +```file {title="cloud-config.yaml" lang="yaml"} +users: + - default +``` + +To create an additional user, add another item to the list with at least a `name` field defining the username. For instance, to create an `example-user`, you can use a configuration like this one. + +```file {title="cloud-config.yaml" lang="yaml" hl_lines="3"} +users: + - default + - name: example-user +``` + +The cloud-init process sets up the user with a range of defaults, like a home directory and user group. However, you typically want to take a bit more control of the user creation, especially if you intend to access the user over SSH. Further sections cover features like [assigning groups](#manage-and-assign-groups) (including `sudo` access) and [adding SSH keys](#add-an-ssh-key-to-a-user) to users. In the example below, and the accompanying breakdown, are some additional useful options for managing new users. + +```file {title="cloud-config.yaml" lang="yaml" hl_lines="4-7"} +users: + - default + - name: example-user + gecos: Example User,600-700-8090 + shell: /bin/bash + lock_passwd: false + passwd: +``` + +This creates a basic user, accessible by a username and password. Here is an explanation of what each part of the example does: + +- `name`: Defines the username for the user. This field is required. + +- `gecos`: Provides a comment on the user. This is where you can enter GECOS information for the user, such as real name and contact details Each piece of information should be separated by commas. + +- `shell`: Points to a shell for the user. While not required, the user's shell may behave unexpectedly if you do not explicitly provide this field. + +- `lock_passwd`: Whether to disable password logins for the user. The default is `true` as it is recommended to use SSH access instead. This is because, in addition to SSH keys generally being more secure, the password hash is included in the cloud-config, making it more difficult to secure. + +- `passwd`: Defines the password for the user as a password hash. To login with the user using this password, the `lock_passwd` option needs to be set to `false`. You can generate a password hash with the following command: + + ```command + mkpasswd --method=SHA-512 --rounds=4096 + ``` + + {{< note >}} + The configuration does support a `plain_text_passwd` option, where you can set a user password from plain text, rather than a hash. However, you should not use this option in a production environment, as the password becomes even more vulnerable to exposure. + {{< /note >}} + +For the full range of user configuration options, see cloud-init's [Users and Groups](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#users-and-groups) module reference documentation. + +## Manage and Assign Groups + +Your cloud-config script can manage user groups either independently using the `groups` option or within a `users` entry. The `groups` option gives you more control of groups themselves and allows you to add existing users, like the default `root` user, to new groups. + +Under `groups`, you have a list of groups to be added to the system. Just listing the name of a group, like `user-group` below, creates an empty group. Adding a list of usernames below a group name, as with `admin-group` below, initializes the system with those users belonging to the group. + +```file {title="cloud-config.yaml" lang="yaml"} +groups: + - admin-group: + - root + - user-group +``` + +Cloud-config also supports a `groups` option within each `users` entry. Using this `groups` option provides a more user-centered approach, allowing you to create and assign groups on a user-by-user basis. In the example below, a new `example-group` group is created along with the user, and the user is assigned to that group. + +```file {title="cloud-config.yaml" lang="yaml"} +users: + - name: example-user + groups: + - example-group +``` + +By default, cloud-init creates and assigns each user to a self-named user group. So the user above, `example-user`, actually belongs to two groups: `example-user` and `example-group`. You can set `no_user_group: true` on the user to not create the default `example-user` group. + +### Assigning Sudo Access + +Cloud-init controls `sudo` access on users primarily through the `sudo` option. This option takes a list of `sudo` rule strings, just they appear in the `sudoers` file. You can learn more about `sudo` access and `sudo` rules in the appropriate sections of our [Linux Users and Groups](/docs/guides/linux-users-and-groups/#understanding-the-sudo-linux-group-and-user) guide. + +In the example below, a new `example-user` is created and given `sudo` access. The one rule applied allows the user to run any command as `sudo` after entering the user's password. This example also adds the user to the `sudo` user group. + +```file {title="cloud-config.yaml" lang="yaml"} +users: + - name: example-user + groups: + - sudo + sudo: + - ALL=(ALL:ALL) ALL +``` + +Alternatively, you can use the following `sudoers` rule to permit `sudo` access without password entry. This is useful for users that you have set up SSH key access on, but have not provided a password for. + +```file {title="cloud-config.yaml" lang="yaml"} +... + sudo: + - ALL=(ALL) NOPASSWD:ALL +``` + +## Add an SSH Key to a User + +Using the `ssh_authorized_keys` option, you can authorize a list of SSH public keys for accessing a user remotely. Doing so provides a more secure authorization route than passwords, and so it is recommended over password configuration. + +If you do not have an SSH key pair yet, get one by following the relevant section of our guide on how to [Use SSH Public Key Authentication](/docs/guides/use-public-key-authentication-with-ssh/#generate-an-ssh-key-pair). + +Once you have the SSH key pair, you can add your SSH public key to the `ssh_authorized_keys` list in the user configuration. In this example, `example-user` has authorized access from two SSH keys: + +```file {title="cloud-config.yaml" lang="yaml"} +users: + - name: example-user + shell: /bin/bash + ssh_authorized_keys: + - + - +``` + +With this set up, a machine with the matching SSH private key (typically where you generated the key pair) can access `example-user` over SSH. The SSH key provides the authentication, and does so more securely than manual password entry. + +## Disable Root User + +From a security perspective, it is generally advisable to disable root login via SSH. This limits the exposure of your root user and the possibility of your system being accessed with full root privileges. + +To disable root access over SSH, you need to modify the SSH configuration file and restart your system's SSH service. All of this can be done with a series of shell commands, which cloud-config takes in the `runcmd` option. The example below uses three commands to modify the SSH service configuration: + +- The `sed` command removes any `PermitRootLogin` line already in the configuration file. Any existing setting is thus removed, and this bypasses more complicated commands that try to identify commented-out settings. + +- The `echo` command adds a new `PermitRootLogin` setting to the configuration file, with a value of `no` to disable root logins. + +- The `systemctl` command restarts the SSH service to ensure the setting takes effect immediately. + +```file {title="cloud-config.yaml" lang="yaml"} +runcmd: + - sed -i '/PermitRootLogin/d' /etc/ssh/sshd_config + - echo "PermitRootLogin no" >> /etc/ssh/sshd_config + - systemctl restart sshd +``` + +## Verify User Configuration + +Once cloud-init has completed the server's initialization, verify that your user and group configurations have deployed as intended. For several of the components configured throughout this tutorial, the simplest verification is often just connecting to the given user via SSH. + +For instance, if your cloud-config created an `example-user` with an SSH key, you should be able to connect to the server as that user via SSH. Replace `192.0.2.17` below with the deployed servers' actual IP address. + +```command +ssh example-user@192.0.2.17 +``` + +If you disabled remote root access, you should be able to verify that similarly, attempting to access the server as the `root` user: + +```command +ssh root@192.0.2.17 +``` + +To verify in more detail, you can use the `getent` and `groups` commands once you have logged into the server. The former, used with the `passwd` option and the username, provides a summary of a user's details on the system. + +In this example, you can see an entry for an `example-user` that has a GECOS comment, home directory, and an assigned shell program: + +```command +sudo getent passwd example-user +``` + +```output +example-user:x:1000:1002:Example User,600-700-8090:/home/example-user:/bin/bash +``` + +What is lacking above is verification of the user's group. You can get that with the `groups` command followed by the username. The example below does this for `example-user`, showing that the user belongs to a self-named user group along with `example-group`. + +```command +sudo groups example-user +``` + +```output +example-user : example-user sudo example-group +``` \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/cloud-init/run-shell-commands-with-cloud-init/index.md b/docs/guides/applications/configuration-management/cloud-init/run-shell-commands-with-cloud-init/index.md new file mode 100644 index 00000000000..452d2c233dd --- /dev/null +++ b/docs/guides/applications/configuration-management/cloud-init/run-shell-commands-with-cloud-init/index.md @@ -0,0 +1,113 @@ +--- +slug: run-shell-commands-with-cloud-init +title: "Use Cloud-Init to Run Commands and Bash Scripts on First Boot" +description: "In this tutorial, find out how you can use cloud-init to run shell commands and Bash scripts on first booting up a new server." +keywords: ['cloud-init','cloudinit','bash','shell script'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +authors: ["Nathaniel Stickman"] +published: 2023-11-15 +modified_by: + name: Nathaniel Stickman +external_resources: +- '[Cloud-init Documentation - Examples: Run Commands on First Boot](https://cloudinit.readthedocs.io/en/latest/reference/examples.html#run-commands-on-first-boot)' +--- + +[Cloud-init](https://cloudinit.readthedocs.io/en/latest/index.html) is an industry-standard, cross-platform tool that automates the process of initializing new servers. Cloud-init leverages metadata from your cloud platform to handle the deployment while taking custom user data to script the server setup to your needs. + +Akamai's [Metadata](/docs/products/compute/compute-instances/guides/metadata/) service lets you deploy Compute Instances using cloud-init. Applying a cloud-config script, you can define everything from security and user setup to software installation and shell script execution. + +This guide covers how to use cloud-init to run shell commands as part of server deployment. Whether you need to execute a single shell statement or a full Bash script, cloud-init can automatically run the necessary commands on your system's first boot. + +Before getting started, review our guide on how to [Use Cloud-Init to Automatically Configure and Secure Your Servers](/docs/guides/configure-and-secure-servers-with-cloud-init/). There, you can see how to create a cloud-config file, which you need to follow along with this guide. When you are ready to deploy your cloud-config, the guide linked above shows how. + +## Run Commands with `runcmd` Directive + +The `runcmd` option is the primary way to execute shell commands within cloud-config. The option takes a list of commands, which cloud-init then runs sequentially during the initialization process. + +Each command given to `runcmd` can be entered either as a list or as a string. The string directly gives the command, just as you would input the command into the shell. As a list, the first item is the command, and each subsequent item is an option to the command, in order. Execution follows the same approach as [execve(3)](https://linux.die.net/man/3/execve). + +To demonstrate, the example below provides a `runcmd` that uses both approaches. Each command runs a similar operation, appending a line to a `/run/test.txt`. + +```file {title="cloud-config.yaml" lang="yaml"} +runcmd: + - echo 'First command executed successfully!' >> /run/testing.txt + - [ sh, -c, "echo 'Second command executed successfully!' >> /run/testing.txt" ] +``` + +{{< note >}} +Cloud-init recommends against writing files to the `/tmp/` directory in your cloud-config as that directory is prone to being cleared during boot processes. Instead, cloud-init recommends using the `/run/` directory for temporary files, as in the example above and the examples to follow. +{{< /note >}} + +## Run Commands with `bootcmd` Directive + +Cloud-init has another option for running commands, `bootcmd`. Within the cloud-config file, `bootcmd` is set up just like `runcmd`, taking a list of commands (which can be either strings or lists themselves). + +There are two key features, however, that differentiate `bootcmd`. First, commands given in the `bootcmd` are executed early in the boot process. These commands run among the first tasks of system on boot. Second, they run on every system boot. Where `runcmd` commands only run once, during initialization, `bootcmd` commands become a part of your system's boot process, recurring with each boot. + +To use `bootcmd`, the setup within cloud-config only differs from `runcmd` in the option name. Take this example: + +```file {title="cloud-config.yaml" lang="yaml"} +bootcmd: + - echo 'Boot command executed successfully!' +``` + +Should you need to run a command early in the boot process, but do not want the command to run with each boot, you can still use `bootcmd`. To do so, run the command using cloud-init's `cloud-init-per` utility, which lets you specify execution frequency. + +In this next example, an `echo` command is run just as above, but `cloud-init-per` specifies that the command should be run only during instance initialization (`instance`). The `exampleinstanceecho` parameter names the command, and the actual command follows that. + +```file {title="cloud-config.yaml" lang="yaml"} +bootcmd: + - [ cloud-init-per, instance, example-instance-echo, echo, "Instance initialization command executed successfully!" ] +``` + +## Run a Bash Script + +More than just commands, cloud-init's `runcmd` can be used to execute shell scripts. Doing so requires that you deliver the script to the new server and use a shell command (via `runcmd`) to execute the script. + +If your script is hosted and accessible remotely, the most straightforward solution is to use a `wget` command to download it. From there, you can use a `runcmd` command to execute the script. [Object Storage](/docs/products/storage/object-storage/get-started/) can provide an effective way to host script files. + +However, most use cases favor adding the shell script directly as part of the cloud-init initialization, without hosting the script file elsewhere. In such cases, you can use cloud-init's `write_files` option to create the script file on initialization. To learn more about writing files with cloud-init, read our guide on how to [Use Cloud-Init to Write to a File](/docs/guides/write-files-with-cloud-init/). The guide includes explanations of all the `write_files` options used here. + +The example that follows demonstrates how `write_files` and `runcmd` can operate together in your cloud-config to create and execute a shell script. `write_files` creates a simple script file at `/run/scripts/test-script.sh` and gives the script executable permissions. The script `content` runs with Bash and appends a line to the `/run/testing.txt` file. `runcmd` executes the `test-script.sh` file as a command. This example uses the `sh` command to run the shell script: + +```file {title="cloud-config.yaml" lang="yaml"} +write_files: + - path: /run/scripts/test-script.sh + content: | + #!/bin/bash + + echo 'Script executed successfully!' >> /run/testing.txt + permissions: '0755' + +runcmd: + - [ sh, "/run/scripts/test-script.sh" ] +``` + +## Verify that Commands or Script has Run + +After your instance is deployed using cloud-init, verify the successful execution of commands. How you do so varies based on the nature of the commands. However, since many commands generate shell output by default, the most consistent way is to check the output in the cloud-init log file, located at `/var/log/cloud-init-output.log`. + +For instance, the example `bootcmd` commands above each use `echo` to output to the terminal. Thus, you can verify those commands by searching the cloud-init logs for the output. The example simplifies the search by using `grep` to filter the logs down to lines containing known output text from the commands. + +```command +sudo cat /var/log/cloud-init-output.log | grep 'executed successfully!' +``` + +```output +Boot command executed successfully! +Instance initialization command executed successfully! +``` + +Many command-line tools output information to the terminal, and this makes them straightforward to verify using the cloud-init logs. However, some commands output to their own log files. If not, you can always log actions with `echo` commands alongside your main commands. For example, the other commands above that run with `runcmd` output text to a `/run/testing.txt` file. + +This may be the case with other commands you run from cloud-init. The command you are using may automatically generate logs, or you may choose to manually incorporate logging, like with the example commands. In either case, reviewing the content of the appropriate log file can confirm cloud-init's successful execution of the commands: + +```command +sudo cat /run/testing.txt +``` + +```output +First command executed successfully! +Second command executed successfully! +Script executed successfully! +``` \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/cloud-init/write-files-with-cloud-init/index.md b/docs/guides/applications/configuration-management/cloud-init/write-files-with-cloud-init/index.md new file mode 100644 index 00000000000..7b1e533c5ed --- /dev/null +++ b/docs/guides/applications/configuration-management/cloud-init/write-files-with-cloud-init/index.md @@ -0,0 +1,147 @@ +--- +slug: write-files-with-cloud-init +title: "Use Cloud-Init to Write to a File" +description: "Find out how you can use cloud-init to automate writing and modifying files during your new servers' initialization." +keywords: ['cloud-init','cloudinit','write files','sed'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +authors: ["Nathaniel Stickman"] +published: 2023-11-15 +modified_by: + name: Nathaniel Stickman +external_resources: +- '[Cloud-init Documentation - Module Reference: Write Files](https://cloudinit.readthedocs.io/en/latest/reference/modules.html#write-files)' +- '[Cloud-init Documentation - Examples : Writing Out Arbitrary Files](https://cloudinit.readthedocs.io/en/latest/reference/examples.html#writing-out-arbitrary-files)' +--- + +[Cloud-init](https://cloudinit.readthedocs.io/en/latest/index.html) is an industry-standard tool that automates server initialization and has cross-distribution, cross-platform support. While the cloud platform provides cloud-init with metadata for server deployment, custom user data lets you script almost every aspect of server initialization. + +Akamai's [Metadata](/docs/products/compute/compute-instances/guides/metadata/) service allows you to leverage cloud-init to deploy Compute Instances. Using a cloud-config script, you can define everything you need, from security and user set up to software installation and shell script execution. + +In this guide, learn how to use a cloud-config script to write files to your server during initialization. Automate the process of creating and editing files so that your software and system configurations are precisely as you need them from the start. + +Before getting started, review our guide on how to [Use Cloud-Init to Automatically Configure and Secure Your Servers](/docs/guides/configure-and-secure-servers-with-cloud-init/). There, you can see how to create a cloud-config file, which you need to follow along with this guide. When you are ready to deploy your cloud-config, the guide linked above details how. + +## Write to a File + +Cloud-init includes a module for writing files, using the `write_files` option within your cloud-config script. This module provides the simplest setup when you want to create a new file or overwrite an existing file during initialization. + +By default, the `write_files` option takes a file path, creating a new file or overwriting an existing one at the given destination. Cloud-init also creates any parent directories that do not already exist. + +Here is an example that creates an HTML file: + +```file {title="cloud-config.yaml" lang="yaml"} +write_files: + - path: /var/www/html/example.com/index.html + content: | + + +

Hello, World!

+

Welcome to the example web page!

+ + + owner: 'root:root' + permissions: '0644' +``` + +The example defines a set of file contents as well as details like ownership and permissions. Here is a deeper breakdown of what this example `write_files` configuration does: + +- `path` points to the location for the created file. Any existing file at the location is overwritten, and parent directories are created as necessary. This is the one required option for `write_files`. + +- `content` defines the content for the file. This can be a single line, or, as above, you can use appropriate YAML formatting for multi-line file content. Leaving out the `content` option creates an empty file. + +- `owner` optionally lets you define a user and/or group to assign ownership of the file to. The default is `root:root`. To specify a user and/or group created within the cloud-config script, you should use the `defer: true` option, as described further below, to ensure the user/group is created before the file. + +- `permissions` optionally specifies the file's permissions. Use the format `0###` where `###` is an octal notation as used with `chmod`. You can learn more about permissions and octal notation in our guide on how to [Modify File Permissions with chmod](/docs/guides/modify-file-permissions-with-chmod/#using-octal-notation-syntax-with-chmod). + + Here, `permissions` gives the owner user read-write permission (`6--`), read permission for the user's group (`-4-`), and read permission for all other users (`--4`). + +An additional `defer` option can be useful when you want to delay creation of the file until the final stage of cloud-init's initialization. That way, you can ensure that a file is only created after all user creation and software installation. + +Here is a further example showing that feature off by creating an [Apache Web Server](/docs/guides/how-to-install-apache-web-server-ubuntu-18-04/) configuration. Using the `defer` option ensures that Apache is installed and the Apache user (typically `www-data` on Debian and Ubuntu) is created before the file. + +```file {title="cloud-config.yaml" lang="yaml"} +write_files: + - path: /etc/apache2/sites-available/example.com.conf + content: | + + ServerAdmin webmaster@example.com + ServerName example.com + ServerAlias www.example.com + DocumentRoot /var/www/example.com/html/ + ErrorLog /var/www/example.com/logs/error.log + CustomLog /var/www/example.com/logs/access.log combined + + owner: 'www-data:www-data' + permissions: '0640' + defer: true +``` + +Omitting the `defer: true` option above would result in an error since the `www-data` user would not yet exist at the time cloud-init attempts to create the file. + +## Modify a File + +When you need to modify a file, cloud-init has a couple of approaches to use: + +- The `write_files` option can achieve basic file modifications with its `append: true` option. For instance, the example below modifies the SSH service configuration by adding a `PermitRootLogin no` rule: + + ```file {title="cloud-config.yaml" lang="yaml"} + write_files: + - path: /etc/ssh/sshd_config + content: PermitRootLogin no + append: true + ``` + + Otherwise, `write_files` can only provide modifications by recreating the files. In that case, you would need to copy the whole configuration with your desired modifications into your cloud-config script. + +- The more approachable and maintainable solution is to use cloud-init's `runcmd` option to run `sed` commands on the server. `sed` provides text editing via shell commands, and `runcmd` lets you run shell commands from a cloud-init script. Learn more about using `runcmd` in our guide [Use Cloud-Init to Run Commands and Bash Scripts on First Boot](/docs/guides/run-shell-commands-with-cloud-init/) and more about `sed` in our guide [Manipulate Text from the Command Line with sed](/docs/guides/manipulate-text-from-the-command-line-with-sed/). + + The `runcmd` option takes a list of shell commands. In the example that follows, two shell commands run to change the SSH service configuration, similar to the example above. However, `sed` lets you replace existing settings, rather than just appending a new setting. + + ```file {title="cloud-config.yaml" lang="yaml"} + runcmd: + - sed -i -e 's/PermitRootLogin\s*yes/PermitRootLogin no/g' /etc/ssh/sshd_config + - sed -i -e 's/#*\s*PermitRootLogin/PermitRootLogin/g' /etc/ssh/sshd_config + ``` + + Each command in the `runcmd` list above makes an edit to the `/etc/ssh/sshd_config` file. Together, these two command effectively switch `PermitRootLogin` to `no` and ensure that the setting is not commented out. The first `sed` command replaces `PermitRootLogin[any number of spaces]yes` with `PermitRootLogin no`, while the second removes the comment marker (`#`) from the beginning of any occurrences of `PermitRootLogin`. + +## Verify File Contents + +Once your instance is up and running, use the `cat` command to verify that files are written as expected. Using `cat` on a file's location, you should see the contents of the file as dictated in the cloud-config. + +```command +cat /var/www/html/example.com/index.html +``` + +```output + + +

Hello, World!

+

Welcome to the example web page!

+ + +``` + +You can do the same for file modifications. However, you can make that check more efficient by limiting the output to matching terms. Here, the `sshd_config` contents are filtered to just the lines where cloud-init made changes (the ones containing `PermitRootLogin`). + +```command +cat /etc/ssh/sshd_config | grep PermitRootLogin +``` + +```output +PermitRootLogin no +``` + +To verify file permissions, use the `stat` command. In the case of the `example.com.conf` file from further above, look for the file to be owned by `www-data` and for the `0640` permission. + +```command +sudo stat /etc/apache2/sites-available/example.com.conf +``` + +```output + File: /etc/apache2/sites-available/example.com.conf + Size: 284 Blocks: 8 IO Block: 4096 regular file +Device: 800h/2048d Inode: 261541 Links: 1 +Access: (0640/-rw-r-----) Uid: ( 33/www-data) Gid: ( 33/www-data) +... +``` \ No newline at end of file diff --git a/docs/guides/applications/configuration-management/laravel-forge/_index.md b/docs/guides/applications/configuration-management/laravel-forge/_index.md index 9525f7e5778..84117b62114 100644 --- a/docs/guides/applications/configuration-management/laravel-forge/_index.md +++ b/docs/guides/applications/configuration-management/laravel-forge/_index.md @@ -1,6 +1,5 @@ --- description: 'Laravel Forge is a tool for deploying and configuring web applications developed by the makers of the Laravel framework, but can be used many other frameworks.' -og_description: 'Laravel Forge is a tool for deploying and configuring web applications. It was developed by the makers of the Laravel framework, but it can be used to automate the deployment of any web application that uses a PHP server.' keywords: ["laravel forge"] license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' published: 2020-07-15