Add Query Insights documentation #6261

ansjcy · 2024-01-25T22:23:55Z

Description

Initial documentation for query insights and top n queries.

Issues Resolved

Closes #6180

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chenyang Ji <cyji@amazon.com>

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

kolchfa-aws · 2024-01-29T23:13:52Z

@ansjcy I've reviewed the draft. A couple of comments/questions:

I don't think the diagram is necessary for the end user. It seems more appropriate for developers.
Is there any other possibility for exporter other than local index?
What is the repo of this plugin?
For settings, I provided API requests so users can update this on existing clusters. Please verify that the requests are technically accurate.
Thanks!

Signed-off-by: Chenyang Ji <cyji@amazon.com>

ansjcy · 2024-02-05T23:12:17Z

Hi @kolchfa-aws , thanks for reviewing this documentation PR for query insights!

I don't think the diagram is necessary for the end user. It seems more appropriate for developers.

Make sense, I removed the diagram in this PR.

Is there any other possibility for exporter other than local index?

Actually we decided to take out exporters in our initial release for query insights, thus I removed all exporter related document in this PR.

What is the repo of this plugin?

This plugin will live in the core OpenSearch repo.

For settings, I provided API requests so users can update this on existing clusters. Please verify that the requests are technically accurate.

Just reviewed, they all make sense to me. Thanks for the update.

kolchfa-aws · 2024-02-06T20:23:59Z

@ansjcy Thank you. Because this plugin will be part of core, will it come with default installation? We have steps to install the plugin so I want to make sure these steps are necessary.

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

ansjcy · 2024-02-07T20:24:58Z

it will not be bundled by default for now, we can add it in Additional plugins since that section mentions those plugins can be installed with bin/opensearch-plugin install <plugin-name>.

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

kolchfa-aws · 2024-02-07T21:07:24Z

Thanks, @ansjcy! I added installation steps.

…mentation-website into query-insights

natebower

@ansjcy @kolchfa-aws Please see my comments and changes and let me know if you have any questions. Of note, there are some naming/wording issues in this content that make it a little confusing and hard to follow in places. Thanks!

_install-and-configure/configuring-opensearch/plugin-settings.md

_observing-your-data/query-insights/index.md

natebower · 2024-02-08T12:29:37Z

_observing-your-data/query-insights/index.md

+has_toc: false
+---
+
+# Query insights


Global: It's not ideal to have one thing named the "Query Insights plugin" and to then name something within that "query insights" or "query insights features." This can result in reader confusion regarding which one we're referencing as well as awkward phrasing throughout.

_observing-your-data/query-insights/index.md

natebower · 2024-02-08T12:54:09Z

_observing-your-data/query-insights/top-n-queries.md

+
+## Configuring top n size
+
+You can configure the top n size using `search.insights.top_queries.latency.top_n_size`. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size:


Should the end of the first sentence read "the search.insights.top_queries.latency.top_n_size [noun]"? Replace "window size".

natebower · 2024-02-08T12:55:38Z

_observing-your-data/query-insights/top-n-queries.md

+```
+{% include copy-curl.html %}
+
+## Configuring top n size


This wording is also awkward and starts to become confusing when attempting to differentiate the content in this section from the preceding content. In other words, what is a "top n size"?

_observing-your-data/query-insights/top-n-queries.md

Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

kolchfa-aws

LGTM. Thank you, @ansjcy!

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

…mentation-website into query-insights

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Query Insights initial documentation Signed-off-by: Chenyang Ji <cyji@amazon.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * update endpoints and remove exporters Signed-off-by: Chenyang Ji <cyji@amazon.com> * Moved query insights to additional plugins section Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Added installation instructions Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * editorial comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Resolve merge conflicts Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * merge conflicts and link fix Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Fix link Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com>

Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Clarification of Customer Comment # 20230525 (opensearch-project#6374) * 20230525 clarificaton for customer comment on refernes to kibana Signed-off-by: leanne.laceybyrne@eliatra.com <leanne.laceybyrne@eliatra.com> * 20230525 clarificaton for customer comment on refernes to kibana - scentance clarification Signed-off-by: leanne.laceybyrne@eliatra.com <leanne.laceybyrne@eliatra.com> --------- Signed-off-by: leanne.laceybyrne@eliatra.com <leanne.laceybyrne@eliatra.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Add searchbp metrics to Performance Analyzer (opensearch-project#5390) * added searchbp metrics Signed-off-by: Heather Halter <hdhalter@amazon.com> * Update reference.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update reference.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Heather Halter <hdhalter@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Add documentation for new reranking feature in 2.12 (opensearch-project#6368) * Create reranking.md document new reranking feature in 2.12 Signed-off-by: HenryL27 <hmlindeman@yahoo.com> * Doc review and address comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _search-plugins/search-pipelines/rerank-processor.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _search-plugins/search-pipelines/rerank-processor.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: HenryL27 <hmlindeman@yahoo.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Add Query Insights documentation (opensearch-project#6261) * Query Insights initial documentation Signed-off-by: Chenyang Ji <cyji@amazon.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * update endpoints and remove exporters Signed-off-by: Chenyang Ji <cyji@amazon.com> * Moved query insights to additional plugins section Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Added installation instructions Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * editorial comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Resolve merge conflicts Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * merge conflicts and link fix Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Fix link Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Add Running a Workload (opensearch-project#6287) * Add Running a Workload draft Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update running-workloads.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _benchmark/user-guide/running-workloads.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Fix link Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add additional missing link Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add running workloads Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Add numbered steps Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com>

Clarification of Customer Comment # 20230525 (#6374) * 20230525 clarificaton for customer comment on refernes to kibana * 20230525 clarificaton for customer comment on refernes to kibana - scentance clarification --------- Add searchbp metrics to Performance Analyzer (#5390) * added searchbp metrics * Update reference.md * Update reference.md * Apply suggestions from code review --------- Add documentation for new reranking feature in 2.12 (#6368) * Create reranking.md document new reranking feature in 2.12 * Doc review and address comments * Apply suggestions from code review * Update _search-plugins/search-pipelines/rerank-processor.md * Update _search-plugins/search-pipelines/rerank-processor.md --------- Add Query Insights documentation (#6261) * Query Insights initial documentation * Doc review * update endpoints and remove exporters * Moved query insights to additional plugins section * Added installation instructions * Apply suggestions from code review * editorial comments * Resolve merge conflicts * merge conflicts and link fix * Fix link --------- Add Running a Workload (#6287) * Add Running a Workload draft * Apply suggestions from code review * Update running-workloads.md * Update _benchmark/user-guide/running-workloads.md * Fix link * Add additional missing link * Add running workloads * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Add numbered steps --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Query Insights initial documentation Signed-off-by: Chenyang Ji <cyji@amazon.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * update endpoints and remove exporters Signed-off-by: Chenyang Ji <cyji@amazon.com> * Moved query insights to additional plugins section Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Added installation instructions Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * editorial comments Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Resolve merge conflicts Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * merge conflicts and link fix Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Fix link Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Chenyang Ji <cyji@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>

Clarification of Customer Comment # 20230525 (opensearch-project#6374) * 20230525 clarificaton for customer comment on refernes to kibana * 20230525 clarificaton for customer comment on refernes to kibana - scentance clarification --------- Add searchbp metrics to Performance Analyzer (opensearch-project#5390) * added searchbp metrics * Update reference.md * Update reference.md * Apply suggestions from code review --------- Add documentation for new reranking feature in 2.12 (opensearch-project#6368) * Create reranking.md document new reranking feature in 2.12 * Doc review and address comments * Apply suggestions from code review * Update _search-plugins/search-pipelines/rerank-processor.md * Update _search-plugins/search-pipelines/rerank-processor.md --------- Add Query Insights documentation (opensearch-project#6261) * Query Insights initial documentation * Doc review * update endpoints and remove exporters * Moved query insights to additional plugins section * Added installation instructions * Apply suggestions from code review * editorial comments * Resolve merge conflicts * merge conflicts and link fix * Fix link --------- Add Running a Workload (opensearch-project#6287) * Add Running a Workload draft * Apply suggestions from code review * Update running-workloads.md * Update _benchmark/user-guide/running-workloads.md * Fix link * Add additional missing link * Add running workloads * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Add numbered steps --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Tianjing Li <tianjinglimail@gmail.com> Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Co-authored-by: Nathan Bower <nbower@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

Query Insights initial documentation

94f689e

Signed-off-by: Chenyang Ji <cyji@amazon.com>

ansjcy requested review from hdhalter, kolchfa-aws, Naarcha-AWS, vagimeli, AMoo-Miki, natebower and dlvenable as code owners January 25, 2024 22:23

ansjcy changed the title ~~Query Insights initial documentation~~ [DRAFT] Query Insights initial documentation Jan 25, 2024

ansjcy mentioned this pull request Jan 25, 2024

[META] Generic Query Insights Framework opensearch-project/OpenSearch#11522

Open

32 tasks

kolchfa-aws self-assigned this Jan 25, 2024

kolchfa-aws added release-notes PR: Include this PR in the automated release notes v2.12.0 labels Jan 25, 2024

hdhalter added the 4 - Doc review PR: Doc review in progress label Jan 29, 2024

Doc review

c466d4a

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

update endpoints and remove exporters

18c9959

Signed-off-by: Chenyang Ji <cyji@amazon.com>

hdhalter changed the title ~~[DRAFT] Query Insights initial documentation~~ Add Query Insights initial documentation Feb 7, 2024

kolchfa-aws and others added 2 commits February 7, 2024 13:47

Moved query insights to additional plugins section

28607d3

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

Merge branch 'main' into query-insights-documentation

b82330b

Added installation instructions

0fd4191

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

Merge branch 'query-insights-documentation' of github.com:ansjcy/docu…

f186bc7

…mentation-website into query-insights

natebower reviewed Feb 8, 2024

View reviewed changes

kolchfa-aws reviewed Feb 8, 2024

View reviewed changes

_observing-your-data/query-insights/top-n-queries.md Outdated Show resolved Hide resolved

Apply suggestions from code review

53c032c

Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

hdhalter removed the 4 - Doc review PR: Doc review in progress label Feb 8, 2024

hdhalter added the 5 - Editorial review PR: Editorial review in progress label Feb 8, 2024

natebower added 6 - Done but waiting to merge PR: The work is done and ready to merge and removed 5 - Editorial review PR: Editorial review in progress labels Feb 13, 2024

kolchfa-aws and others added 3 commits February 13, 2024 12:02

editorial comments

4f24d1e

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

Resolved merge conflicts

4a53088

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

Merge branch 'main' into query-insights-documentation

52b383b

kolchfa-aws approved these changes Feb 13, 2024

View reviewed changes

kolchfa-aws added 3 commits February 13, 2024 12:09

Resolve merge conflicts

42b9e34

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

Merge branch 'query-insights-documentation' of github.com:ansjcy/docu…

fd90983

…mentation-website into query-insights

merge conflicts and link fix

b3d2581

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

hdhalter changed the title ~~Add Query Insights initial documentation~~ Add Query Insights documentation Feb 13, 2024

Fix link

0a79b38

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

kolchfa-aws merged commit c87bd64 into opensearch-project:main Feb 13, 2024
2 of 3 checks passed

hdhalter added 3 - Done Issue is done/complete and removed 6 - Done but waiting to merge PR: The work is done and ready to merge labels Feb 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Query Insights documentation #6261

Add Query Insights documentation #6261

ansjcy commented Jan 25, 2024 •

edited by hdhalter

Loading

kolchfa-aws commented Jan 29, 2024

ansjcy commented Feb 5, 2024

kolchfa-aws commented Feb 6, 2024

ansjcy commented Feb 7, 2024

kolchfa-aws commented Feb 7, 2024

natebower left a comment

natebower Feb 8, 2024

natebower Feb 8, 2024

natebower Feb 8, 2024

kolchfa-aws Feb 13, 2024

kolchfa-aws left a comment


		## Configuring top n size

		You can configure the top n size using `search.insights.top_queries.latency.top_n_size`. For example, a cluster with the following configuration will collect the top 10 queries in the specified window size:

Add Query Insights documentation #6261

Add Query Insights documentation #6261

Conversation

ansjcy commented Jan 25, 2024 • edited by hdhalter Loading

Description

Issues Resolved

Checklist

kolchfa-aws commented Jan 29, 2024

ansjcy commented Feb 5, 2024

kolchfa-aws commented Feb 6, 2024

ansjcy commented Feb 7, 2024

kolchfa-aws commented Feb 7, 2024

natebower left a comment

Choose a reason for hiding this comment

natebower Feb 8, 2024

Choose a reason for hiding this comment

natebower Feb 8, 2024

Choose a reason for hiding this comment

natebower Feb 8, 2024

Choose a reason for hiding this comment

kolchfa-aws Feb 13, 2024

Choose a reason for hiding this comment

kolchfa-aws left a comment

Choose a reason for hiding this comment

ansjcy commented Jan 25, 2024 •

edited by hdhalter

Loading