Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[UnifiedSearch] ESQL support #137810

Closed
flash1293 opened this issue Aug 2, 2022 · 2 comments · Fixed by #140164 or #146971
Closed

[UnifiedSearch] ESQL support #137810

flash1293 opened this issue Aug 2, 2022 · 2 comments · Fixed by #140164 or #146971
Assignees
@stratoula
Labels
enhancement New value added to drive a business result Feature:Unified search Unified search related tasks impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@flash1293
Copy link
Contributor

Extend the existing text based query integration in unified search and discover by ESQL:

  • Add ESQL search strategy
  • Add ESQL expression function functioning similar to the SQL one
  • Add basic ESQL syntax highlighting support for Monaco
  • Add separate option in data view menu to switch to ESQL (similar to SQL mode)
@flash1293 flash1293 added enhancement New value added to drive a business result Team:Visualizations Visualization editors, elastic-charts and infrastructure Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Feature:Unified search Unified search related tasks labels Aug 2, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@flash1293 flash1293 mentioned this issue Sep 7, 2022
Merged
@ninoslavmiskovic ninoslavmiskovic added the impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. label Sep 15, 2022
@stratoula stratoula self-assigned this Jun 2, 2023
@stratoula stratoula mentioned this issue Jun 1, 2023
Merged
3 tasks
@kertal kertal mentioned this issue Jun 13, 2023
Closed
stratoula added a commit that referenced this issue Sep 1, 2023
@stratoula @alexwizp
@kibanamachine @abdonpijpelink @gchaps @ppisljar @dej611 @kertal @davismcphee
[ES|QL] Technical preview in Discover and Dashboards (#146971)
5d68129 
## Summary

Closes #137810
Part of #163248

This PR is the **first iteration** of the new ES|QL language in kibana.
The majority of the functionality is based on the existing functionality
for SQL (which has already been merged). This builds on top of it to
enable the functionality for ES|QL. We decided to remove SQL for now so
this PR enables ES|QL and hides SQL. We are not removing the code for
now.

### Important notes:
- This PR contains basic autocomplete functionality for the ES|QL
commands. We want to improve it in follow up PRs
- The majority of the tests for SQL were moved to work with ES|QL
instead
- The search strategy is a very simple endpoint for now as we don't have
async search nor pagination
- Now that we remove SQL, the ui for ES|QL selection has changed
- The documentation for ESQL has been handled by the docs team so it is
already reviewed and in sync with the official documentation
- ES|QL is disabled in serverless projects for now

### Changes from SQL:
- The Discover histogram now is being created with ES|QL (using the
date_trunc function). This gives it the ability to be saved on a
dashboard and also to be edited inline.
- ES|QL sometimes returns some warnings (on the search headers). For
example when we are trying to date parse a string that doesnt contain a
valid date). These warnings are also reported on the UI. There is a bug
in ES and the warning doesnt come always
elastic/elasticsearch-internal#1465)
- We have 2 types of commands:
- Transformational commands (keep, stats) -> when they exist in the
query Discover should render the Table view (selected columns)
- Non transformational commands (all the others) -> in that case
Discover renders the Document view
  - ESQL switch on advanced settings is now on by default


**Discover view with non transformational commands**
<img width="1678" alt="image"
src="https://github.com/elastic/kibana/assets/17003240/abe100e1-01e9-4fe0-9b89-6d8bdf6443fc">


**Discover view with transformational commands**
<img width="1679" alt="image"
src="https://github.com/elastic/kibana/assets/17003240/e46af422-daeb-4be5-88cf-522211674ff5">


### Missing
- ESQL autocomplete is not perfect, we are going to work on it on a
future PR. Specifically:
- There is a bug in autocomplete, sometimes writing a query overwrites
the existing one, will deal with it in a future PR
  - Further improvements

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Alexey Antonov <alexwizp@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Peter Pisljar <peter.pisljar@elastic.co>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stratoula stratoula

Labels
enhancement New value added to drive a business result Feature:Unified search Unified search related tasks impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet
Milestone
No milestone
4 participants
@flash1293 @elasticmachine @stratoula @ninoslavmiskovic