-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rollup support for TSVB #28762
Rollup support for TSVB #28762
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall code LGTM.
Please change all default exports to named exports as written in our styleguide.
x-pack/plugins/rollup/server/lib/search_strategies/rollup_search_strategy.js
Outdated
Show resolved
Hide resolved
x-pack/plugins/rollup/server/lib/search_strategies/rollup_search_strategy.js
Outdated
Show resolved
Hide resolved
src/legacy/core_plugins/metrics/server/lib/vis_data/series/get_request_params.js
Outdated
Show resolved
Hide resolved
src/legacy/core_plugins/metrics/server/lib/search_strategies/search_strategies_register.js
Outdated
Show resolved
Hide resolved
src/legacy/core_plugins/metrics/server/lib/search_strategies/searh_requests/search_request.js
Show resolved
Hide resolved
x-pack/plugins/rollup/server/lib/search_strategies/rollup_search_capabilities.js
Outdated
Show resolved
Hide resolved
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
@@ -66,62 +66,10 @@ export function registerFieldsForWildcardRoute(server) { | |||
// Keep meta fields | |||
metaFields.forEach(field => fieldsFromFieldCapsApi[field] && rollupFields.push(fieldsFromFieldCapsApi[field])); | |||
|
|||
// Merge rollup capabilities information with field information |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jen-huang Could you please review that? We want to reuse your code
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💔 Build Failed |
💚 Build Succeeded |
@cchaos could you please have a look? We need review from elastic/kibana-design |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Glanced at the SASS changes, just an addition of another selector. LGTM
This should not happen. It should show only the "original field name", because that's the field name we need to query against. So it should behave like the classical vis editor as you've pointed out. |
@timroes @AlonaNadler Let me explain. In example above Alona use the index pattern with wildcard (roll1*,kibana_sample_data_logs). From our requirements we cannot use Rollup search strategy for indexes with wildcard. See selected text below: For Rollup Search Strategy we reuse logic from classical vis editor. See x-pack/plugins/rollup/server/lib/merge_capabilities_with_fields.js file in this PR. It means that if we detect that we should use Rollup Search Strategy we show For Default Search Strategy all works like in the master branch. |
Initially, the team who implemented rollups in Kibana had a requirement to not support wildcard in rollup index pattern in Kibana. Apparently, there was a change and it is possible to use a wildcard with index pattern,
I didn't' know about this was changed and in the rollup requirements, I added it as a requirement to not support wildcard in rollup, which was implemented in this PR, apologies my fault. Is there any chance to revert this back? |
It's not a problem to remove 'if' condition but let me show you one case:
Actual behavior: user see data for all indexes with prefix "kibana_". In real world it can be e.g company name. TSVB works as expected. Let's go next:
Expected behaviour: I expect to see the aggregated data by kibana_* wildcard like in step 3 @AlonaNadler @timroes Are you ok with that? If not probably we don't need to revert it |
💚 Build Succeeded |
We should have a separate conversation and PR about wildcard support. It's not worth holding up this PR while we figure out what do. |
* Added a feature of rollup search on the UI side Signed-off-by: Alexey Antonov <alexwizp@gmail.com> * Rollup Feature - initial commit * Revert "Added a feature of rollup search on the UI side" This reverts commit 9568b0970b16f5102f50b748bb4d691a8612c2c2. # Conflicts: # src/legacy/core_plugins/metrics/public/components/index_pattern.js * Remove the 'label' property from the search strategies * Changed search by strategy from the last * add single search request * rollup_search_strategy add base implementation of isViable method * rollup_search_strategy add base implementation of isViable method -fix * Changed requests due to search request type * refactoring of import Base classes / remove '../../../../../../ * remove extra await * rollup_search_strategy. Refactoring of isRollupJobExists method * remove question * Add support of annotations and table data * skeleton for adding Search Strategy restrictions * Add rollup search capabilities * apply search strategy for annotations request * set fields capabilities for rollup strategy * add timezone, interval into SearchCapabilities * Add fields from capabilities * add timezone, interval into SearchCapabilities * fix default timezone * Merging of two Rollup Jobs was removed * move getFieldsForWildcard to searchStrategy * Fix TSVB search requests should have a timeout # Conflicts: # src/legacy/core_plugins/metrics/server/lib/vis_data/get_annotations.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/get_request_params.js * Add unit test * apply getEsShardTimeout for annorations/get_request_params, series/get_request_params * rename metrics -> tsvb * search_strategies_register refactoring: move 'add' method from class * Add merge rollup capabilities with fields * Add merge rollup capabilities with fields - small fixes * Add support of 'Everything' aggregation for Rollup Search * Return back metrics plugin * remove 'metrics' from the X-pack\rollup require * Fix test cases * fix broken test: fail: "apis InfraOps GraphQL Endpoints metrics should basically work" * rollup search - split by terms is not working * Add count metric * /get_bucket_size.js. Add support of 'auto' interval, Add support of gte intervals e.g.: >=1m * fix build_request_body test * [Rollup] [Phase 1] Error handling - rollup search errors should be more user friendly * [Rollup] [Phase 1] Table View - research the query to ES - sorting is not wokring * Merge elastic#26006 into rollup # Conflicts: # src/legacy/core_plugins/metrics/server/lib/vis_data/annorations/build_request_body.js # src/legacy/core_plugins/metrics/server/lib/vis_data/get_annotations.js # src/legacy/core_plugins/metrics/server/lib/vis_data/get_series_data.js # src/legacy/core_plugins/metrics/server/lib/vis_data/get_table_data.js # src/legacy/core_plugins/metrics/server/lib/vis_data/request_processors/annotations/query.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/__tests__/build_request_body.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/build_request_body.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/get_request_params.js * Add table view support * fix broken build * fix broken build * [Rollup] [Phase 1] - write new tests (rollup_search_request, rollup_search_strategy) * [Rollup] [Phase 1] - write tests for rollup_search_capabilities * Add test on default_search_capabilities, abstract_search_strategy, search_strategies_register * Add test cases for search_requests folder * [Rollup] [Phase 1] - write tests for rollup_search_strategy * FIx broken build * remove todo * fix calculation of interval value for rollup search * add unit tests * Remove default exports * fix PR comments * fix calendar intervals
* Added a feature of rollup search on the UI side Signed-off-by: Alexey Antonov <alexwizp@gmail.com> * Rollup Feature - initial commit * Revert "Added a feature of rollup search on the UI side" This reverts commit 9568b0970b16f5102f50b748bb4d691a8612c2c2. # Conflicts: # src/legacy/core_plugins/metrics/public/components/index_pattern.js * Remove the 'label' property from the search strategies * Changed search by strategy from the last * add single search request * rollup_search_strategy add base implementation of isViable method * rollup_search_strategy add base implementation of isViable method -fix * Changed requests due to search request type * refactoring of import Base classes / remove '../../../../../../ * remove extra await * rollup_search_strategy. Refactoring of isRollupJobExists method * remove question * Add support of annotations and table data * skeleton for adding Search Strategy restrictions * Add rollup search capabilities * apply search strategy for annotations request * set fields capabilities for rollup strategy * add timezone, interval into SearchCapabilities * Add fields from capabilities * add timezone, interval into SearchCapabilities * fix default timezone * Merging of two Rollup Jobs was removed * move getFieldsForWildcard to searchStrategy * Fix TSVB search requests should have a timeout # Conflicts: # src/legacy/core_plugins/metrics/server/lib/vis_data/get_annotations.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/get_request_params.js * Add unit test * apply getEsShardTimeout for annorations/get_request_params, series/get_request_params * rename metrics -> tsvb * search_strategies_register refactoring: move 'add' method from class * Add merge rollup capabilities with fields * Add merge rollup capabilities with fields - small fixes * Add support of 'Everything' aggregation for Rollup Search * Return back metrics plugin * remove 'metrics' from the X-pack\rollup require * Fix test cases * fix broken test: fail: "apis InfraOps GraphQL Endpoints metrics should basically work" * rollup search - split by terms is not working * Add count metric * /get_bucket_size.js. Add support of 'auto' interval, Add support of gte intervals e.g.: >=1m * fix build_request_body test * [Rollup] [Phase 1] Error handling - rollup search errors should be more user friendly * [Rollup] [Phase 1] Table View - research the query to ES - sorting is not wokring * Merge #26006 into rollup # Conflicts: # src/legacy/core_plugins/metrics/server/lib/vis_data/annorations/build_request_body.js # src/legacy/core_plugins/metrics/server/lib/vis_data/get_annotations.js # src/legacy/core_plugins/metrics/server/lib/vis_data/get_series_data.js # src/legacy/core_plugins/metrics/server/lib/vis_data/get_table_data.js # src/legacy/core_plugins/metrics/server/lib/vis_data/request_processors/annotations/query.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/__tests__/build_request_body.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/build_request_body.js # src/legacy/core_plugins/metrics/server/lib/vis_data/series/get_request_params.js * Add table view support * fix broken build * fix broken build * [Rollup] [Phase 1] - write new tests (rollup_search_request, rollup_search_strategy) * [Rollup] [Phase 1] - write tests for rollup_search_capabilities * Add test on default_search_capabilities, abstract_search_strategy, search_strategies_register * Add test cases for search_requests folder * [Rollup] [Phase 1] - write tests for rollup_search_strategy * FIx broken build * remove todo * fix calculation of interval value for rollup search * add unit tests * Remove default exports * fix PR comments * fix calendar intervals
Pinging @elastic/kibana-app |
Rollup fields were were merged twice with other fields, this caused duplicates. Fix of a bug introduced by #28762
Rollup fields were were merged twice with other fields, this caused duplicates. Fix of a bug introduced by elastic#28762
Rollup fields were were merged twice with other fields, this caused duplicates. Fix of a bug introduced by elastic#28762
Rollup fields were were merged twice with other fields, this caused duplicates. Fix of a bug introduced by elastic#28762
Summary
Roll-up functionality support for TSVB.
Index pattern and rollups
Aggregations
term
oreverything
Time intervals
Time zones
Checklist