From 9e2aaf3eceb730c6ebc3613506f92c9c5934c4ea Mon Sep 17 00:00:00 2001 From: David Legrand Date: Fri, 16 Aug 2024 14:35:23 +0200 Subject: [PATCH] docs: add Network Groups instructions Co-Authored-By: kannar --- docs/README.md | 1 + docs/networkgroups/commands.md | 167 ------------------ docs/networkgroups/tests.md | 310 --------------------------------- docs/ng.md | 131 ++++++++++++++ 4 files changed, 132 insertions(+), 477 deletions(-) delete mode 100644 docs/networkgroups/commands.md delete mode 100644 docs/networkgroups/tests.md create mode 100644 docs/ng.md diff --git a/docs/README.md b/docs/README.md index cc0284f..9048693 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,6 +11,7 @@ to ask for new features, enhancements or help us to provide them to our communit You'll find below the first commands to know to connect Clever Tools to your account, get its information and manage some options. Others are developed in dedicated pages: +- [Network Groups](/docs/ng.md) - [Applications: configuration](/docs/applications-config.md) - [Applications: management](/docs/applications-management.md) - [Applications: deployment and lifecycle](/docs/applications-deployment-lifecycle.md) diff --git a/docs/networkgroups/commands.md b/docs/networkgroups/commands.md deleted file mode 100644 index e60831e..0000000 --- a/docs/networkgroups/commands.md +++ /dev/null @@ -1,167 +0,0 @@ -# Network Groups CLI commands - -This document is a list of all Network Group-related commands. - -For each command, example call and output are commented right under. - -To improve readability, and to avoid errors, every option value is written inside quotes. - -> **Disclaimer:** This document isn't generated from code, and therefore might **not** be up-to-date. - -## Table Of Contents - -- [Table Of Contents](#table-of-contents) -- [`clever networkgroups` | `clever ng`](#clever-networkgroups--clever-ng) - - [`list`](#list) - - [`create`](#create) - - [`delete`](#delete) - - [`members`](#members) - - [`members list`](#members-list) - - [`members get`](#members-get) - - [`members add`](#members-add) - - [`members remove`](#members-remove) - - [`peers`](#peers) - - [`peers list`](#peers-list) - - [`peers get`](#peers-get) - - [`peers add-external`](#peers-add-external) - - [`peers remove-external`](#peers-remove-external) - -## `clever networkgroups` | `clever ng` - -List Network Groups commands - -### `list` - -List Network Groups with their labels - -| Param | Description | -| ----------------- | ------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `[--json, -j]` | Show result in JSON format (default: false) | - ---- - -### `create` - -Create a Network Group - -| Param | Description | -| ------------------------------ | ---------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--label NG_LABEL` | Network Group label, also used for dns context | -| `--description NG_DESCRIPTION` | Network Group description | -| `[--tags] TAGS` | List of tags separated by a comma | -| `[--json, -j]` | Show result in JSON format (default: false) | - ---- - -### `delete` - -Delete a Network Group - -| Param | Description | -| ----------------- | ------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | - ---- - -### `members` - -List commands for interacting with Network Groups members - -#### `members list` - -List members of a Network Group - -| Param | Description | -| ---------------------- | -------------------------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `[--natural-name, -n]` | Show application names or aliases if possible (default: false) | -| `[--json, -j]` | Show result in JSON format (default: false) | - -#### `members get` - -Get a Network Group member details - -| Param | Description | -| --------------------------- | ---------------------------------------------------------------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `--member-id, -m MEMBER_ID` | The member ID: an app ID (i.e. `app_xxx`), add-on ID (i.e. `addon_xxx`) or external node category ID | -| `[--natural-name, -n]` | Show application names or aliases if possible (default: false) | -| `[--json, -j]` | Show result in JSON format (default: false) | - -#### `members add` - -Add an app or addon as a Network Group member - -| Param | Description | -| --------------------------- | ---------------------------------------------------------------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `--member-id, -m MEMBER_ID` | The member ID: an app ID (i.e. `app_xxx`), add-on ID (i.e. `addon_xxx`) or external node category ID | -| `--type MEMBER_TYPE` | The member type ('application', 'addon' or 'external') | -| `--domain-name DOMAIN_NAME` | Member name used in the `.m..ng.clever-cloud.com` domain name alias | -| `[--label] MEMBER_LABEL` | The member label | - -#### `members remove` - -Remove an app or addon from a Network Group - -| Param | Description | -| --------------------------- | ---------------------------------------------------------------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `--member-id, -m MEMBER_ID` | The member ID: an app ID (i.e. `app_xxx`), add-on ID (i.e. `addon_xxx`) or external node category ID | - ---- - -### `peers` - -List commands for interacting with Network Groups peers - -#### `peers list` - -List peers of a Network Group - -| Param | Description | -| ----------------- | ------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| --ng NG | Network Group ID or label | -| [--json, -j] | Show result in JSON format (default: false) | - -#### `peers get` - -Get a Network Group peer details - -| Param | Description | -| ------------------- | ------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `--peer-id PEER_ID` | The peer ID | -| `[--json, -j]` | Show result in JSON format (default: false) | - -#### `peers add-external` - -Add an external node as a Network Group peer - -| Param | Description | -| ------------------------- | ------------------------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `--role PEER_ROLE` | The peer role, ('client' or 'server') | -| `--public-key PUBLIC_KEY` | A WireGuard® public key | -| `--label PEER_LABEL` | Network Group peer label | -| `--parent MEMBER_ID` | Network Group peer category ID (parent member ID) | - -#### `peers remove-external` - -Remove an external node from a Network Group - -| Param | Description | -| ------------------- | ------------------------------- | -| `[--verbose, -v]` | Verbose output (default: false) | -| `--ng NG` | Network Group ID or label | -| `--peer-id PEER_ID` | The peer ID | diff --git a/docs/networkgroups/tests.md b/docs/networkgroups/tests.md deleted file mode 100644 index 949aa2d..0000000 --- a/docs/networkgroups/tests.md +++ /dev/null @@ -1,310 +0,0 @@ -# Network Groups CLI tests - -This document is a list of manual integration tests for the Network Group-related commands. - -For each command, an example output is commented right under. - -To improve readability, and to avoid errors, every option value is written inside quotes. - -## Setup - -> In the following commands, `cleverr` refers to the local `clever-tools`. -> -> Tip: Add `alias cleverr=~/path/to/clever-tools/bin/clever.js` to your `.bashprofile`, `.zprofile` or whatever. - -First, let's define variables to facilitate command line calls. - -```sh -# Valid cases -ngLabel='temp-test' -testAppId='app_b888f06d-3adb-4cf1-b017-7eac4f096e90' -memberId1='my-member-1' -memberId2='my-member-2' -peerLabel1='my-peer-1' -peerLabel2='my-peer-2' -publicKey1=`wg genkey | wg pubkey` -publicKey2=`wg genkey | wg pubkey` - -# Invalid cases -ngLabelForInvalidCases1='test-1' -ngLabelForInvalidCases2='test-2' -``` - -## Tests - -### Valid cases - -#### Create a Network Group - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Create a Network Group' - # Network Group 'test-dev' was created with the id 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a'. - ``` - -2. Test - - ```sh - cleverr ng list - # Network Group ID Label Members Peers Description - # ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── - # ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a test-dev 0 0 [Test] Create a Network Group. - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a' was successfully deleted. - ``` - -#### Delete a Network Group - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Delete a Network Group' - # Network Group 'test-dev' was created with the id 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a'. - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a' was successfully deleted. - ``` - -2. Test - - ```sh - cleverr ng list - # No Network Group found. - ``` - -#### Add a member - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Add a member' - # Network Group 'test-dev' was created with the id 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a'. - ``` - -2. Test - - ```sh - cleverr ng --ng "$ngLabel" members list - # No member found - cleverr ng --ng "$ngLabel" members add --member-id "$testAppId" --type 'application' --domain-name 'api-tester' --label '[Test] API Tester' - # Successfully added member 'app_b888f06d-3adb-4cf1-b017-7eac4f096e90' to Network Group 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a'. - cleverr ng --ng "$ngLabel" members list - # Member ID Member Type Label Domain Name - # ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── - # app_b888f06d-3adb-4cf1-b017-7eac4f096e90 application [Test] API Tester api-tester - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a' was successfully deleted. - ``` - -#### Add an external peer - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Add an external peer' - # Network Group 'temp-test' was created with the id 'ng_84c65dce-4a48-4858-b327-83bddf5f0a79'. - cleverr ng --ng "$ngLabel" members add --member-id "$memberId1" --type 'external' --domain-name 'my-nodes-category' --label '[Test] My external nodes category' - # Successfully added member 'my-member-1' to Network Group 'ng_84c65dce-4a48-4858-b327-83bddf5f0a79'. - ``` - -2. Test - - ```sh - cleverr ng --ng "$ngLabel" peers list - # No peer found. You can add an external one with `clever networkgroups peers add-external`. - cleverr ng --ng "$ngLabel" peers add-external --role 'client' --public-key "$publicKey1" --label "$peerLabel1" --parent "$memberId1" - # External peer 'external_3b3e82e2-e656-450b-8cc7-b7498d0134f4' must have been added to Network Group 'ng_84c65dce-4a48-4858-b327-83bddf5f0a79'. - cleverr ng --ng "$ngLabel" peers list - # Peer ID Peer Type Endpoint Type Label Hostname IP Address - # ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── - # external_3b3e82e2-e656-450b-8cc7-b7498d0134f4 ExternalPeer ClientEndpoint my-peer-1 my-peer-1 10.105.0.5 - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_84c65dce-4a48-4858-b327-83bddf5f0a79' was successfully deleted. - ``` - -#### WireGuard® configuration updates when adding a peer - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] WireGuard® configuration updates when adding a peer' - # Network Group 'temp-test' was created with the id 'ng_620b3482-f286-4189-9931-a8910f2ea706'. - cleverr ng --ng "$ngLabel" members add --member-id "$memberId1" --type 'external' --domain-name 'my-nodes-category' --label '[Test] My external nodes category' - # Successfully added member 'my-member-1' to Network Group 'ng_620b3482-f286-4189-9931-a8910f2ea706'. - cleverr ng --ng "$ngLabel" peers add-external --role 'client' --public-key "$publicKey1" --label "$peerLabel1" --parent "$memberId1" - # External peer 'external_3056ea93-c10d-4175-a91d-b6ed2803ce7c' must have been added to Network Group 'ng_620b3482-f286-4189-9931-a8910f2ea706'. - cleverr ng --ng "$ngLabel" peers list - # Peer ID Peer Type Endpoint Type Label Hostname IP Address - # ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── - # external_3056ea93-c10d-4175-a91d-b6ed2803ce7c ExternalPeer ClientEndpoint my-peer-1 my-peer-1 10.105.0.5 - peerId='external_3056ea93-c10d-4175-a91d-b6ed2803ce7c' - ``` - -2. Test - - Call this endpoint: - - ```http - GET /organisations/:ownerId/networkgroups/:ngId/peers/:peerId/wireguard/configuration - ``` - - You should have something like: - - ```text - "CgpbSW50ZXJmYWNlXQpQcml2YXRlS2V5ID0gPCVQcml2YXRlS2V5JT4KQWRkcmVzcyA9IDEwLjEwNS4wLjUvMTYKCgoKCgo=" - ``` - - ```sh - cleverr ng --ng "$ngLabel" peers add-external --role 'client' --public-key "$publicKey2" --label "$peerLabel2" --parent "$memberId1" - # External peer 'external_0839394b-1ddf-49dc-a9a2-09b0437ed59e' must have been added to Network Group 'ng_620b3482-f286-4189-9931-a8910f2ea706'. - ``` - - Call this endpoint again: - - ```http - GET /organisations/:ownerId/networkgroups/:ngId/peers/:peerId/wireguard/configuration - ``` - - You should have something like: - - ```text - "CgpbSW50ZXJmYWNlXQpQcml2YXRlS2V5ID0gPCVQcml2YXRlS2V5JT4KQWRkcmVzcyA9IDEwLjEwNS4wLjUvMTYKCgoKCltQZWVyXQogICAgUHVibGljS2V5ID0gPHB1Yl9rZXlfMj4KICAgIEFsbG93ZWRJUHMgPSAxMC4xMDUuMC42LzMyCiAgICBQZXJzaXN0ZW50S2VlcGFsaXZlID0gMjUKCgoK" - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_620b3482-f286-4189-9931-a8910f2ea706' was successfully deleted. - ``` - -### Invalid cases - -#### Create two Network Groups with same label - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Create two Network Groups with same label' - # Network Group 'temp-test' was created with the id 'ng_ebee26cf-f1dc-464c-8359-d3a924a3fd97'. - ``` - -2. Test - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Should not be created' - # [ERROR] Error from API: 409 Conflict - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_ebee26cf-f1dc-464c-8359-d3a924a3fd97' was successfully deleted. - ``` - -#### Add invalid member - -> ⚠️ Does not work actually -> TODO: Create issue - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Add invalid member' - # Network Group 'temp-test' was created with the id 'ng_df116d7b-47f3-469b-bee7-ae5792ff92c4'. - ``` - -2. Test - - ```sh - cleverr ng --ng "$ngLabel" members add --member-id '' --type 'external' --domain-name 'invalid' - # [ERROR] Error from API: 404 Not Found - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_df116d7b-47f3-469b-bee7-ae5792ff92c4' was successfully deleted. - ``` - -#### Add peer with invalid parent - -1. Setup - - ```sh - cleverr ng create --label "$ngLabelForInvalidCases1" --description '[Test] Network Group 1' - # Network Group 'test-1' was created with the id 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a'. - cleverr ng create --label "$ngLabelForInvalidCases2" --description '[Test] Network Group 2' - # Network Group 'test-2' was created with the id 'ng_c977973a-42ce-4f67-b906-4ffc2dcb250c'. - cleverr ng --ng "$ngLabelForInvalidCases2" members add --member-id "$memberId1" --type 'external' --domain-name 'member-2' --label '[Test] Member in other Network Group' - # Successfully added member 'app_b888f06d-3adb-4cf1-b017-7eac4f096e90' to Network Group 'ng_c977973a-42ce-4f67-b906-4ffc2dcb250c'. - ``` - -2. Test - - - Invalid parent `id` - - ```sh - cleverr ng --ng "$ngLabelForInvalidCases1" peers add-external --role 'client' --public-key "$publicKey1" --label '[Test] Invalid parent' --parent 'invalid_id' - # [ERROR] Error from API: 404 Not Found - ``` - - - Parent in another Network Group - - ```sh - cleverr ng --ng "$ngLabelForInvalidCases1" peers add-external --role 'client' --public-key "$publicKey1" --label '[Test] Parent in other Network Group' --parent "$memberId1" - # [ERROR] Error from API: 404 Not Found - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabelForInvalidCases1" - # Network Group 'ng_cdf5cc11-4fdf-47cf-8d82-5b89f722450a' was successfully deleted. - cleverr ng delete --ng "$ngLabelForInvalidCases2" - # Network Group 'ng_c977973a-42ce-4f67-b906-4ffc2dcb250c' was successfully deleted. - ``` - -### Edge cases - -#### Create Network Group after delete - -1. Setup - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Should be deleted' - # Network Group 'temp-test' was created with the id 'ng_811d29f5-2d15-44a6-8f73-d16dee8f6316'. - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_811d29f5-2d15-44a6-8f73-d16dee8f6316' was successfully deleted. - ``` - -2. Test - - ```sh - cleverr ng create --label "$ngLabel" --description '[Test] Create Network Group after delete' - # Network Group 'temp-test' was created with the id 'ng_4e2d4e0e-9a47-4cb7-95a4-d85355b1ccb3'. - ``` - -3. Tear down - - ```sh - cleverr ng delete --ng "$ngLabel" - # Network Group 'ng_4e2d4e0e-9a47-4cb7-95a4-d85355b1ccb3' was successfully deleted. - ``` diff --git a/docs/ng.md b/docs/ng.md new file mode 100644 index 0000000..11776fc --- /dev/null +++ b/docs/ng.md @@ -0,0 +1,131 @@ +# Clever Cloud Network Groups + +Network Groups (NG) are a way to create a private secure network between resources inside Clever Cloud infrastructure, using [Wireguard](https://www.wireguard.com/). It's also possible to connect external resources to a Network Group. There are three components to this feature: + +* Network Group: A group of resources that can communicate with each through an encrypted tunnel +* Member: A resource that can be part of a Network Group (`application`, `addon` or `external`) +* Peer: Instance of a resource connected to a Network Group (can be `external`) + +A Network Group is defined by an ID (`ngId`) and a `label`. It can be completed by a `description` and `tags`. + +> [!NOTE] +> Network Groups are currently in public alpha testing phase. You only need a Clever Cloud account to use them. + +Tell us what you think of Network Groups and what features you need from it in [the dedicated section of our GitHub Community](https://github.com/CleverCloud/Community/discussions/categories/network-groups). + +## How it works + +When you create a Network Group, a Wireguard configuration is generated with a corresponding [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing). Then, you can, for example, add a Clever Cloud application and an associated add-on to the same Network Group. These are members, defined by an `id`, a `label`, a `type` and a `domain name`. + +When an application connects to a Network Group, you can reach it on any port inside a NG through its domain name. Any instance of this application is a peer, you can reach independently through an IP (from the attributed CIDR). It works the same way for add-ons and external resources. During alpha testing phase, only applications are supported. + +> [!TIP] +> A Network Group member domain name is composed this way: `.m..ng.clever-cloud.com` + +## Prerequisites + +To use Network Groups, you need [Clever Tools installed](/docs/setup-systems.md) in a test version or higher than `4.0.0`. You can check your version with the following command: + +``` +clever version +``` + +To activate the Network Groups feature, you need to create a `clever-tools-features.json` file in your `~/.config/clever-cloud/` directory with the following content: + +```json +{ + "ng": true +} +``` + +Then, check it works with the following command: + +``` +clever ng +``` + +In all the following examples, you can target a specific organization with the `--org` or `-o` option. + +## Create a Network Group + +A Network Group is simple to create: + +``` +clever ng create myNG +``` + +You can create it declaring its members: + +``` +clever ng create myNG --members-ids appId1,appId2 +``` + +You can add a description, tags and ask for a JSON output (`--format` or `-F`): + +``` +clever ng create myNG --description "My first NG" --tags test,ng -F json +``` + +## List Network Groups + +Once created, you can list your Network Groups: + +``` +clever ng list + +┌─────────┬───────-┬─────────-─┬───────────────┬─────────────────┬─────────┬───────┐ +| (index) │ id │ label │ networkIp │ lastAllocatedIp │ members │ peers │ +├─────────┼────────┼───────────┼───────────────┼─────────────────┼─────────┼───────┤ +│ 0 │ 'ngId' │ 'ngLabel' │ '10.x.y.z/16' │ '10.x.y.z' │ X │ Y │ +└─────────┴────────┴──────────-┴───────────────┴─────────────────┴─────────┴───────┘ +``` + +A `json` formatted output is available. + +## Delete Network Groups + +You can delete a Network Group through its ID or label: + +``` +clever ng delete ngId +clever ng delete ngLabel +``` + +## Manage members of a Network Group + +To add an application to a Network Group (a `label` is optional): + +``` +clever ng members add ngId appId +clever ng members add ngId appId --label 'member label' +``` + +To get information about members (a `json` formatted output is available): + +``` +clever ng members list ngId_or_ngLabel +clever ng members get ngId_or_ngLabel memberId +``` + +To delete a member from a Network Group: + +``` +clever ng members remove ngId_or_ngLabel memberId +``` + +## Manage peers of a Network Group + +To get information about peers (a `json` formatted output is available): + +``` +clever ng peers list ngId_or_ngLabel +clever ng peers get ngId_or_ngLabel peerId +``` + +## Demos & examples + +You can find ready to deploy projects using Network Groups in the following repositories: + +- XXX + +Create your own and let us know!