From fb59274bd3c159461c1d14ea63d888b1d5d9af35 Mon Sep 17 00:00:00 2001 From: Archer Date: Thu, 5 Sep 2024 18:24:16 -0500 Subject: [PATCH 1/8] Add Benchmark glossary Signed-off-by: Archer --- _benchmark/glossary.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 _benchmark/glossary.md diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md new file mode 100644 index 0000000000..e69de29bb2 From b7b9df6d24b04f3fed871d1b813cac90e1c78295 Mon Sep 17 00:00:00 2001 From: Archer Date: Thu, 5 Sep 2024 18:24:59 -0500 Subject: [PATCH 2/8] Add text Signed-off-by: Archer --- _benchmark/glossary.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index e69de29bb2..be8f9da35f 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -0,0 +1,21 @@ +--- +layout: default +title: Glossary +nav_order: 10 +--- + +# OpenSearch Benchmark glossary + +The following terms are commonly used in OpenSearch Benchmark: + +- **Corpora**: A collection of documents. +- **Latency**: Based on the `target-throughput` set by the user, the total amount of time that the request waits before receiving the response, in addition to any other delays that occur before the request is sent. +- **Metric keys**: The metrics that OpenSearch Benchmark stores, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). +- **Operations**: In workloads, a list of API requests performed by a workload. +- **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. +- **Schedule**: In workloads, a list of operations in a specific order. +- **Service time**: The amount of time that it takes for `opensearch-py` to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also _includes_ network latency, load balancer overhead, and deserialization/serialization. +- **Summary report**: A report output at the end a test based on the metric keys defined in the workload. +- **Test**: A single invocation of the OpenSearch Benchmark binary. +- **Throughput**: The number of operations completed in a given period of time. +- **Workload**: A collection of one or more benchmarking scenarios that use a specific document corpus to perform a benchmark against your cluster. The document corpus contains any indexes, data files, and operations invoked when the workload runs. \ No newline at end of file From 14b4dcd5434464296c221a910dd9370c8a1f5b06 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Thu, 12 Sep 2024 12:00:37 -0500 Subject: [PATCH 3/8] Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _benchmark/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index be8f9da35f..d57a9e6d25 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -9,7 +9,7 @@ nav_order: 10 The following terms are commonly used in OpenSearch Benchmark: - **Corpora**: A collection of documents. -- **Latency**: Based on the `target-throughput` set by the user, the total amount of time that the request waits before receiving the response, in addition to any other delays that occur before the request is sent. +- **Latency**: If `target-throughput` is disabled (has no value or a value of `0)`, latency is equivalent to service time. If `target-throughput` is enabled (has a value of 1 or greater), latency is the service time plus the time the request waits in the queue before being sent. - **Metric keys**: The metrics that OpenSearch Benchmark stores, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). - **Operations**: In workloads, a list of API requests performed by a workload. - **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. From aa9831e5d3ba1c59fa29f8ebfdfe17cd2f9f72ad Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Thu, 12 Sep 2024 12:02:08 -0500 Subject: [PATCH 4/8] Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _benchmark/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index d57a9e6d25..bb17a24693 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -14,7 +14,7 @@ The following terms are commonly used in OpenSearch Benchmark: - **Operations**: In workloads, a list of API requests performed by a workload. - **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. - **Schedule**: In workloads, a list of operations in a specific order. -- **Service time**: The amount of time that it takes for `opensearch-py` to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also _includes_ network latency, load balancer overhead, and deserialization/serialization. +- **Service time**: The amount of time that it takes for `opensearch-py`, the primary client for OpenSearch Benchmark, to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also _includes_ network latency, load balancer overhead, and deserialization/serialization. - **Summary report**: A report output at the end a test based on the metric keys defined in the workload. - **Test**: A single invocation of the OpenSearch Benchmark binary. - **Throughput**: The number of operations completed in a given period of time. From 1d9a3f1ebf24f9be5df2c0c3201c2e4e84ef1707 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Fri, 20 Sep 2024 09:23:10 -0500 Subject: [PATCH 5/8] Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _benchmark/glossary.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index bb17a24693..851295fd68 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -11,11 +11,11 @@ The following terms are commonly used in OpenSearch Benchmark: - **Corpora**: A collection of documents. - **Latency**: If `target-throughput` is disabled (has no value or a value of `0)`, latency is equivalent to service time. If `target-throughput` is enabled (has a value of 1 or greater), latency is the service time plus the time the request waits in the queue before being sent. - **Metric keys**: The metrics that OpenSearch Benchmark stores, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). -- **Operations**: In workloads, a list of API requests performed by a workload. +- **Operations**: In workloads, a list of API operations performed by a workload. - **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. - **Schedule**: In workloads, a list of operations in a specific order. -- **Service time**: The amount of time that it takes for `opensearch-py`, the primary client for OpenSearch Benchmark, to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also _includes_ network latency, load balancer overhead, and deserialization/serialization. +- **Service time**: The amount of time that it takes for `opensearch-py`, the primary client for OpenSearch Benchmark, to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also network latency, load balancer overhead, and deserialization/serialization. - **Summary report**: A report output at the end a test based on the metric keys defined in the workload. - **Test**: A single invocation of the OpenSearch Benchmark binary. - **Throughput**: The number of operations completed in a given period of time. -- **Workload**: A collection of one or more benchmarking scenarios that use a specific document corpus to perform a benchmark against your cluster. The document corpus contains any indexes, data files, and operations invoked when the workload runs. \ No newline at end of file +- **Workload**: A collection of one or more benchmarking tests that use a specific document corpus to perform a benchmark against your cluster. The document corpus contains any indexes, data files, and operations invoked when the workload runs. \ No newline at end of file From e606ba22c6ed913fdce7c06e86b2019cbc3c7c6e Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Fri, 20 Sep 2024 09:24:13 -0500 Subject: [PATCH 6/8] Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _benchmark/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index 851295fd68..ee4ca5a06b 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -13,7 +13,7 @@ The following terms are commonly used in OpenSearch Benchmark: - **Metric keys**: The metrics that OpenSearch Benchmark stores, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). - **Operations**: In workloads, a list of API operations performed by a workload. - **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. -- **Schedule**: In workloads, a list of operations in a specific order. +- **Schedule**: A list of two or more operations performed in the order they appear when a workload is run. - **Service time**: The amount of time that it takes for `opensearch-py`, the primary client for OpenSearch Benchmark, to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also network latency, load balancer overhead, and deserialization/serialization. - **Summary report**: A report output at the end a test based on the metric keys defined in the workload. - **Test**: A single invocation of the OpenSearch Benchmark binary. From 95f9450310541bc9471058de286820dcb85b7c73 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Fri, 20 Sep 2024 11:05:16 -0500 Subject: [PATCH 7/8] Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _benchmark/glossary.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index ee4ca5a06b..09a52df8e7 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -9,13 +9,13 @@ nav_order: 10 The following terms are commonly used in OpenSearch Benchmark: - **Corpora**: A collection of documents. -- **Latency**: If `target-throughput` is disabled (has no value or a value of `0)`, latency is equivalent to service time. If `target-throughput` is enabled (has a value of 1 or greater), latency is the service time plus the time the request waits in the queue before being sent. -- **Metric keys**: The metrics that OpenSearch Benchmark stores, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). +- **Latency**: If `target-throughput` is disabled (has no value or a value of `0)`, then latency is equivalent to service time. If `target-throughput` is enabled (has a value of 1 or greater), then latency is equal to the service time plus the amount of time the request waits in the queue before being sent. +- **Metric keys**: The metrics stored by OpenSearch Benchmark, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). - **Operations**: In workloads, a list of API operations performed by a workload. -- **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. +- **Pipeline**: A series of steps occurring both before and after running a workload that determines benchmark results. - **Schedule**: A list of two or more operations performed in the order they appear when a workload is run. -- **Service time**: The amount of time that it takes for `opensearch-py`, the primary client for OpenSearch Benchmark, to send a request and receive a response from the OpenSearch cluster. It includes the amount of time that it takes for the server to process a request and also network latency, load balancer overhead, and deserialization/serialization. -- **Summary report**: A report output at the end a test based on the metric keys defined in the workload. +- **Service time**: The amount of time taken for `opensearch-py`, the primary client for OpenSearch Benchmark, to send a request and receive a response from the OpenSearch cluster. It includes the amount of time taken for the server to process a request as well as for network latency, load balancer overhead, and deserialization/serialization. +- **Summary report**: A report generated at the end of a test based on the metric keys defined in the workload. - **Test**: A single invocation of the OpenSearch Benchmark binary. - **Throughput**: The number of operations completed in a given period of time. -- **Workload**: A collection of one or more benchmarking tests that use a specific document corpus to perform a benchmark against your cluster. The document corpus contains any indexes, data files, and operations invoked when the workload runs. \ No newline at end of file +- **Workload**: A collection of one or more benchmarking tests that use a specific document corpus to perform a benchmark against a cluster. The document corpus contains any indexes, data files, or operations invoked when the workload runs. \ No newline at end of file From 735660f4a51f6cc3e9451958c5f903feecd6aa9b Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Fri, 20 Sep 2024 11:05:40 -0500 Subject: [PATCH 8/8] Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _benchmark/glossary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_benchmark/glossary.md b/_benchmark/glossary.md index 09a52df8e7..f86591d3d9 100644 --- a/_benchmark/glossary.md +++ b/_benchmark/glossary.md @@ -9,7 +9,7 @@ nav_order: 10 The following terms are commonly used in OpenSearch Benchmark: - **Corpora**: A collection of documents. -- **Latency**: If `target-throughput` is disabled (has no value or a value of `0)`, then latency is equivalent to service time. If `target-throughput` is enabled (has a value of 1 or greater), then latency is equal to the service time plus the amount of time the request waits in the queue before being sent. +- **Latency**: If `target-throughput` is disabled (has no value or a value of `0)`, then latency is equal to service time. If `target-throughput` is enabled (has a value of 1 or greater), then latency is equal to the service time plus the amount of time the request waits in the queue before being sent. - **Metric keys**: The metrics stored by OpenSearch Benchmark, based on the configuration in the [metrics record]({{site.url}}{{site.baseurl}}/benchmark/metrics/metric-records/). - **Operations**: In workloads, a list of API operations performed by a workload. - **Pipeline**: A series of steps occurring both before and after running a workload that determines benchmark results.