From ae06bd58cb24b593b1fdc00d37caca30ec9524c3 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Tue, 18 Oct 2022 15:03:55 +0200 Subject: [PATCH 01/15] init spec file --- text/0185-configuration-file.md | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 text/0185-configuration-file.md diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md new file mode 100644 index 00000000..ed0300ac --- /dev/null +++ b/text/0185-configuration-file.md @@ -0,0 +1,7 @@ +# Configuration File + +## 1. Summary +## 2. Motivation +## 3. Functional Specification +## 4. Technical Details +## 5. Future Possibilities \ No newline at end of file From 240aafd39c8b2cf9ed50dcdef799dd05c3fd269b Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 19 Oct 2022 11:07:25 +0200 Subject: [PATCH 02/15] Fill-in spec --- text/0119-instance-options.md | 11 +++++ text/0185-configuration-file.md | 77 ++++++++++++++++++++++++++++++++- 2 files changed, 87 insertions(+), 1 deletion(-) diff --git a/text/0119-instance-options.md b/text/0119-instance-options.md index 47e4ada0..1cf67fe6 100644 --- a/text/0119-instance-options.md +++ b/text/0119-instance-options.md @@ -120,6 +120,7 @@ The expected behavior of each flag is described in the list above. - [SSL require auth](#3327-ssl-require-auth) - [SSL resumption](#3328-ssl-resumption) - [SSL tickets](#3329-ssl-tickets) +- [Config file path](#3330-config-file-path) #### 3.3.1. Database path @@ -457,6 +458,16 @@ Activates SSL session resumption. Activates SSL tickets. +#### 3.3.30. Config file path + +**Environment variable**: `MEILI_CONFIG_FILE_PATH` +**CLI option**: `--config-file-path` +**Default**: `./config.toml` + +Define the config file to load at Meilisearch launch. + +See [Configuration File](0185-configuration-file.md) specification details. + ## 4. Technical Aspects N/A diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index ed0300ac..e8049f93 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -1,7 +1,82 @@ # Configuration File ## 1. Summary + +Meilisearch options can be defined in a `TOML` configuration file being loaded at launch. + ## 2. Motivation + +A configuration file can be versioned, tracked and shared more easily by teams. It makes it easier to learn about the different options available and to define at a glance how Meilisearch is configured. + ## 3. Functional Specification + +### 3.1. Supported Formats + +| Format | +|-----------| +| `.toml` | + +### 3.2. Priority Order + +`Configuration file < Env variables < Command-line options` + +It means the options in the configuration file are overwritten by the environment variable set (if they exist) and that can be themself overwritten by the command-line options (if they exist). + +### 3.3. Default Configuration File + +A default config file is defined as `config.toml` and is retrieved by Meilisearch by default in its current working directory. + +The default configuration file details all configurations keys, some of them are commented. + +### 3.4. `--config-file-path`/`MEILI_CONFIG_FILE_PATH` + +It is possible to override the default configuration file by setting the `MEILI_CONFIG_FILE_PATH` environment variable or by specifying the `--config-file-path` option. + +### 3.5. Configuration File Content Definition + +The current consistency rule is that all environment variables and launch options are configurable by a dedicated key in the configuration file. + +Configuration keys are named following the `snake_case` convention from the name of a CLI option. That is, `--import-dump` must be named `import_dump` within the configuration file. + +There is an exception rule. The `config_file_path` key is not taken into account in the configuration file and stop Meilisearch from being launch if specified. + +See [Instance Options](0119-instance-options.md) for more details. + +### 3.6. Indication of the loaded configuration file + +When Meilisearch is launched, the loaded configuration file is indicated by the `Config file path` key on the stdout. + +```bash +888b d888 d8b 888 d8b 888 +8888b d8888 Y8P 888 Y8P 888 +88888b.d88888 888 888 +888Y88888P888 .d88b. 888 888 888 .d8888b .d88b. 8888b. 888d888 .d8888b 88888b. +888 Y888P 888 d8P Y8b 888 888 888 88K d8P Y8b "88b 888P" d88P" 888 "88b +888 Y8P 888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888 888 888 888 +888 " 888 Y8b. 888 888 888 X88 Y8b. 888 888 888 Y88b. 888 888 +888 888 "Y8888 888 888 888 88888P' "Y8888 "Y888888 888 "Y8888P 888 888 + +Config file path: "./config.toml" +``` + +If Meilisearch can't find the default `config.toml` or if no other config file is specified with `MEILI_CONFIG_FILE_PATH` env var or `--config-file-path` options, `none` is indicated. + +### 3.7. Errors + +#### 3.7.1. Defining `config_file_path` in the configuration file + +Defining `config_file_path` in the configuration file is not valid and Meilisearch returns an error. + +```bash +`config_file_path` is not supported in config file +``` + ## 4. Technical Details -## 5. Future Possibilities \ No newline at end of file +N/A + +## 5. Future Possibilities + +- Better indicate a syntax error +- List all loaded configuration at launch on the stdout +- Introduce toml sections to replace comments to define blocks +- Introduce other formats \ No newline at end of file From 436e02d50113847f6335f928fb4ef2684275b99d Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 19 Oct 2022 11:12:51 +0200 Subject: [PATCH 03/15] Fix typing errors --- text/0185-configuration-file.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index e8049f93..93a33da8 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -2,11 +2,11 @@ ## 1. Summary -Meilisearch options can be defined in a `TOML` configuration file being loaded at launch. +Meilisearch options can be defined in a `.toml` configuration file being loaded at launch. ## 2. Motivation -A configuration file can be versioned, tracked and shared more easily by teams. It makes it easier to learn about the different options available and to define at a glance how Meilisearch is configured. +Teams can easily version, track, and share configuration files. It makes it easier to learn about the different options available and to define at a glance how Meilisearch is configured. ## 3. Functional Specification @@ -38,7 +38,7 @@ The current consistency rule is that all environment variables and launch option Configuration keys are named following the `snake_case` convention from the name of a CLI option. That is, `--import-dump` must be named `import_dump` within the configuration file. -There is an exception rule. The `config_file_path` key is not taken into account in the configuration file and stop Meilisearch from being launch if specified. +There is an exception rule. The `config_file_path` key is not accepted in the configuration file and stops Meilisearch from being launched if specified. See [Instance Options](0119-instance-options.md) for more details. From db7bb2a28ad6a3be54d848b617b659a371593684 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 19 Oct 2022 11:39:02 +0200 Subject: [PATCH 04/15] adds errors case --- text/0185-configuration-file.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 93a33da8..fb1a1191 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -71,6 +71,22 @@ Defining `config_file_path` in the configuration file is not valid and Meilisear `config_file_path` is not supported in config file ``` +#### 3.7.2. Defining a non-existent config file. + +Defining a configuration file other than the default file that does not exist returns an error when launching Meilisearch. + +```bash +unable to open or read the {:?} configuration file. +``` + +#### 3.7.3. Defining a config file with a syntax error. + +Defining a configuration file wrongly formatted or containing an invalid configuration key returns an error when launching Meilisearch. + +```bash +unable to open or read the {:?} configuration file. +``` + ## 4. Technical Details N/A From ebf9687b1e8c902eed484830069e6db467695ffe Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 26 Oct 2022 19:14:19 +0200 Subject: [PATCH 05/15] Update text/0185-configuration-file.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clémentine Urquizar - curqui --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index fb1a1191..7cee861f 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -68,7 +68,7 @@ If Meilisearch can't find the default `config.toml` or if no other config file i Defining `config_file_path` in the configuration file is not valid and Meilisearch returns an error. ```bash -`config_file_path` is not supported in config file +Error: `config_file_path` is not supported in config file ``` #### 3.7.2. Defining a non-existent config file. From 3aa56510c021b372fd04acf53ae547b13309555d Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 26 Oct 2022 19:16:05 +0200 Subject: [PATCH 06/15] Replace config file by configuration file in the error message --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 7cee861f..60e4da9f 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -68,7 +68,7 @@ If Meilisearch can't find the default `config.toml` or if no other config file i Defining `config_file_path` in the configuration file is not valid and Meilisearch returns an error. ```bash -Error: `config_file_path` is not supported in config file +Error: `config_file_path` is not supported in configuration file. ``` #### 3.7.2. Defining a non-existent config file. From 812ccb443ea58a946ea6a098f0d3846e73843e27 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 26 Oct 2022 19:16:30 +0200 Subject: [PATCH 07/15] Update text/0185-configuration-file.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clémentine Urquizar - curqui --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 60e4da9f..eb87e9d7 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -76,7 +76,7 @@ Error: `config_file_path` is not supported in configuration file. Defining a configuration file other than the default file that does not exist returns an error when launching Meilisearch. ```bash -unable to open or read the {:?} configuration file. +Error: unable to open or read the "XXX" configuration file: No such file or directory (os error 2). ``` #### 3.7.3. Defining a config file with a syntax error. From c204f0149dbca673275c1d651ef580a39ee976f9 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 26 Oct 2022 19:16:42 +0200 Subject: [PATCH 08/15] Update text/0185-configuration-file.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clémentine Urquizar - curqui --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index eb87e9d7..0642c6c9 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -84,7 +84,7 @@ Error: unable to open or read the "XXX" configuration file: No such file or dire Defining a configuration file wrongly formatted or containing an invalid configuration key returns an error when launching Meilisearch. ```bash -unable to open or read the {:?} configuration file. +Error: unknown field `XXX` at line Y column Z ``` ## 4. Technical Details From 8539b316afaa4e37fbfe2cfdaa5e43684939206a Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 26 Oct 2022 19:16:58 +0200 Subject: [PATCH 09/15] Update text/0185-configuration-file.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clémentine Urquizar - curqui --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 0642c6c9..08019205 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -24,7 +24,7 @@ It means the options in the configuration file are overwritten by the environmen ### 3.3. Default Configuration File -A default config file is defined as `config.toml` and is retrieved by Meilisearch by default in its current working directory. +By default, Meilisearch will try to read the following named configuration file `./config.toml`, so in its current working directory. The default configuration file details all configurations keys, some of them are commented. From 2793b75c26bc2c16355807d209a6273aca4dbceb Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 2 Nov 2022 18:57:42 +0100 Subject: [PATCH 10/15] Update text/0185-configuration-file.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clémentine Urquizar - curqui --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 08019205..5a419392 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -26,7 +26,7 @@ It means the options in the configuration file are overwritten by the environmen By default, Meilisearch will try to read the following named configuration file `./config.toml`, so in its current working directory. -The default configuration file details all configurations keys, some of them are commented. +We provide a default configuration file as an example [here](https://github.com/meilisearch/meilisearch/blob/main/config.toml). ### 3.4. `--config-file-path`/`MEILI_CONFIG_FILE_PATH` From 1135b15a73811cad93b39990e1771743a53cf38b Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 2 Nov 2022 19:08:35 +0100 Subject: [PATCH 11/15] Add infos.with_configuration_file to telemetry-policies --- text/0034-telemetry-policies.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0034-telemetry-policies.md b/text/0034-telemetry-policies.md index 27f27579..c2dfbe57 100644 --- a/text/0034-telemetry-policies.md +++ b/text/0034-telemetry-policies.md @@ -87,6 +87,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | `infos.log_level` | Value of `--log-level`/`MEILI_LOG_LEVEL` | debug | Every Hour | | `infos.max_indexing_memory` | Value of `--max-indexing-memory`/`MEILI_MAX_INDEXING_MEMORY` in bytes | 336042103 | Every Hour | | `infos.max_indexing_threads` | Value of `--max-indexing-threads`/`MEILI_MAX_INDEXING_THREADS` in integer | 4 | Every Hour | +| `infos.with_configuration_file` | `true` if the instance is launched with a configuration file, otherwise `false` | false | Every Hour | | `system.distribution` | Distribution on which MeiliSearch is launched | Arch Linux | Every hour | | `system.kernel_version` | Kernel version on which MeiliSearch is launched | 5.14.10 | Every hour | | `system.cores` | Number of cores | 24 | Every hour | @@ -179,6 +180,7 @@ This property allows us to gather essential information to better understand on | infos.log_level | Value of `--log-level`/`MEILI_LOG_LEVEL` | `debug` | | infos.max_indexing_memory | Value of `--max-indexing-memory`/`MEILI_MAX_INDEXING_MEMORY` in bytes | `336042103` | | infos.max_indexing_threads | Value of `--max-indexing-threads`/`MEILI_MAX_INDEXING_THREADS` in integer | `4` | +| infos.with_configuration_file | `true` if the instance is launched with a configuration file, otherwise `false` | `false` | ##### MeiliSearch Statistics `stats` From 8ad8b2f97bc174503f3822c54aef51ec0f1c03fb Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Mon, 7 Nov 2022 14:57:15 +0100 Subject: [PATCH 12/15] Update text/0185-configuration-file.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clémentine Urquizar - curqui --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 5a419392..6b86ea0f 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -20,7 +20,7 @@ Teams can easily version, track, and share configuration files. It makes it easi `Configuration file < Env variables < Command-line options` -It means the options in the configuration file are overwritten by the environment variable set (if they exist) and that can be themself overwritten by the command-line options (if they exist). +It means the options in the configuration file are overwritten by the environment variables set (if they exist) and that can be themself overwritten by the command-line options (if they exist). ### 3.3. Default Configuration File From 43cf41e14a917b06319b95951405767a40201b25 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 16 Nov 2022 11:23:17 +0100 Subject: [PATCH 13/15] Update text/0185-configuration-file.md Co-authored-by: Tamo --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 6b86ea0f..83411ef2 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -44,7 +44,7 @@ See [Instance Options](0119-instance-options.md) for more details. ### 3.6. Indication of the loaded configuration file -When Meilisearch is launched, the loaded configuration file is indicated by the `Config file path` key on the stdout. +When Meilisearch is launched, the loaded configuration file is indicated by the `Config file path` key on stdout. ```bash 888b d888 d8b 888 d8b 888 From 4ad6a9484092ff622b649db2a2a80052bd912dc5 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 16 Nov 2022 11:23:22 +0100 Subject: [PATCH 14/15] Update text/0185-configuration-file.md Co-authored-by: Tamo --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index 83411ef2..f6992ec0 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -24,7 +24,7 @@ It means the options in the configuration file are overwritten by the environmen ### 3.3. Default Configuration File -By default, Meilisearch will try to read the following named configuration file `./config.toml`, so in its current working directory. +By default, Meilisearch will try to read the `./config.toml` in the current working directory. We provide a default configuration file as an example [here](https://github.com/meilisearch/meilisearch/blob/main/config.toml). From 073e1d09a8edae9c7cd8399f67a52bf3b4184071 Mon Sep 17 00:00:00 2001 From: Guillaume Mourier Date: Wed, 16 Nov 2022 11:23:27 +0100 Subject: [PATCH 15/15] Update text/0185-configuration-file.md Co-authored-by: Tamo --- text/0185-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0185-configuration-file.md b/text/0185-configuration-file.md index f6992ec0..943adef3 100644 --- a/text/0185-configuration-file.md +++ b/text/0185-configuration-file.md @@ -94,5 +94,5 @@ N/A - Better indicate a syntax error - List all loaded configuration at launch on the stdout -- Introduce toml sections to replace comments to define blocks +- Introduce toml sections to group options under a common name. e.g. A dump section could contain a `schedule`, `dir`, `interval-sec` and `enable`. That could help navigate the toml. - Introduce other formats \ No newline at end of file