Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Saved Queries] Rework saved query privileges #202863

Merged
merged 98 commits into from
Jan 29, 2025

Conversation

davismcphee
Copy link
Contributor

@davismcphee davismcphee commented Dec 4, 2024

Summary

This PR reworks saved query privileges to rely solely on a single global savedQueryManagement privilege, and eliminates app-specific overrides. This change simplifies the security model for users, fixes bugginess in the saved query management UI, and reduces code complexity associated with maintaining two separate security mechanisms (app-specific overrides and global saved query management privileges).

Background

Saved queries allow users to store a combination of KQL or Lucene queries, filters, and time filters to use across various applications in Kibana. Access to saved query saved objects are currently granted by the following feature privileges:

[
  "feature_discover.all",
  "feature_dashboard.all",
  "feature_savedQueryManagement.all",
  "feature_maps.all",
  "feature_savedObjectsManagement.all",
  "feature_visualize.all"
]

There is also a saved query management UI within the Unified Search bar shared by applications across Kibana:

The way access to this UI is managed in Kibana is currently confusing and buggy:

  • If a user has feature_discover.all and feature_dashboard.all they will be able to load and save queries in Discover and Dashboard.
  • If a user has feature_discover.all and feature_dashboard.read they will be able to load queries in both Discover and Dashboard, but only save queries in Discover (even though they have write access to the SO, and API access). Instead they have to navigate to Discover to save a query before navigating back to Dashboard to load it, making for a confusing and frustrating UX.
  • Access to the UI is even more confusing in apps not listed in the above feature privileges (e.g. alerting, SLOs). Some of them chose to check one of the above feature privileges, meaning users who otherwise should have saved query access won't see the management UI if they don't also have the exact feature privilege being checked. Other apps just always show the management UI, leading to bugs and failures when users without one of the above feature privileges attempt to save queries.

Existing improvements

In v8.11.0, we introduced a new "Saved Query Management" privilege, allowing users to access saved queries across all of Kibana with a single global privilege:

When this privilege is added to a role, it solves the feature_discover.all and feature_dashboard.read issue mentioned above. However, it does not fix any of the mentioned issues for roles without the new privilege. We have so far postponed further improvements to avoid a breaking change.

Approach

To fully resolve these issues and migrate to a single global privilege, these changes have been made:

  • Remove saved query SO access from all application feature privileges and instead only allow access through the global saved query management privilege.
  • Stop relying on application feature privileges for toggling the saved query management UI, and instead rely on the global privilege.

To implement this with minimal breaking changes, we've used the Kibana privilege migration framework. This allows us to seamlessly migrate existing roles containing feature privileges that currently provide access to saved queries, ensuring they are assigned the global saved query management privilege on upgrade.

As a result, we had to deprecate the following feature privileges, replacing them with V2 privileges without saved query SO access:

[
  "feature_discover.all",
  "feature_dashboard.all",
  "feature_maps.all",
  "feature_visualize.all"
]

Each area of code that currently relies on any of these feature privileges had to be updated to instead access feature_X_V2 instead (as well as future code).

This PR still introduces a minor breaking change, since users who have feature_discover.all and feature_dashboard.read are now able to save queries in Dashboard after upgrade, but we believe this is a better UX (and likely the expected one) and worth a small breaking change.

Testing

  • All existing privileges should continue to work as they do now, including deprecated V1 feature privileges and customized serverless privileges. There should be no changes for existing user roles apart from the minor breaking change outlined above.
  • Check that code changes in your area don't introduce breaking changes to existing behaviour. Many of the changes are just updating client UI capabilities code from feature.privilege to feature_v2.privilege, which is backward compatible.
  • The savedQueryManagement feature should now globally control access to saved query management in Unified Search for all new user roles. Regardless of privileges for Discover, Dashboard, Maps, or Visualize, new user roles should follow this behaviour:
    • If savedQueryManagement is none, the user cannot see or access the saved query management UI or APIs.
    • If savedQueryManagement is read, the user can load queries from the UI and access read APIs, but cannot save queries from the UI or make changes to queries through APIs.
    • If savedQueryManagement is all, the user can both load and save queries from the UI and through APIs.

Checklist

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

This PR risks introducing unintended breaking changes to user privileges related to saved queries if the deprecated features have not been properly migrated, and users could gain or lose access to saved query management on upgrade. This would be bad if it happened, but not overly severe since it wouldn't grant them access to any ES data they couldn't previously access (only query saved objects). We have automated testing in place to help ensure features have been migrated correctly, but the scope of these changes are broad and touch many places in the codebase.

Additionally, the UI capabilities types are not very strict, and are referenced with string paths in many places, which makes changing them riskier than changing strictly typed code. A combination of regex searches and temporarily modifying the Capabilities type to cause type errors for deprecated privileges was used to identify references in code. Reviewers should consider if there are any other ways that UI capabilities can be referenced which were not addressed in this PR.

Our automated tests already help mitigate the risk, but it's important that code owners thoroughly review the changes in their area and consider if they could have unintended consequences. The Platform Security team should also review this PR thoroughly, especially since some changes were made to platform code around privilege handling. The Data Discovery team will also manually test the behaviour when upgrading existing user roles with deprecated feature privileges as part of 9.0 upgrade testing.

@davismcphee davismcphee added release_note:breaking backport:skip This commit does not require backporting Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. labels Dec 4, 2024
@davismcphee davismcphee self-assigned this Dec 4, 2024
viduni94 pushed a commit to viduni94/kibana that referenced this pull request Jan 23, 2025
…re replacements (elastic#206660)

## Summary

Today, when a developer deprecates a feature and replaces its privileges
with those of another feature, we reasonably assume that the new feature
fully replaces the old one in all possible contexts - whether in role
management UIs or in the Spaces feature toggles visibility UI. However,
when deprecated privileges are replaced by the privileges of multiple
features, such as in [this
case](elastic#202863 (comment))
where the Discover/Dashboard/Maps feature privileges are replaced by the
privileges of Discover_v2/Dashboard_v2/Maps_v2, respectively, **and**
the privileges of the Saved Query Management feature, the choice is
ambiguous.

Which of these features should be treated as the replacement for the
deprecated feature in contexts that deal with entire features (like the
Spaces feature toggles visibility UI) rather than individual privileges
(like in role management UIs)? Should all referenced features be
considered replacements? Or just a subset - or even a single feature? If
so, which one? Currently, we treat all referenced features as
replacements for the deprecated feature, which creates problems, as
described in detail in [this
discussion](elastic#202863 (comment)).

This PR allows developers to customize this behavior by specifying which
features Kibana should treat as direct successors to deprecated features
in contexts that deal with whole features rather than individual
privileges:

```ts
deps.features.registerKibanaFeature({
  deprecated: {
    notice: 'The feature is deprecated because … well, there’s a reason.',
    --> replacedBy: ['feature_id_v2'], <--
  },
  id: 'feature_id'
  name: `Case elastic#4 feature ${suffix} (DEPRECATED)`,
  …
});
```

## How to test

1. Run test server
```bash
node scripts/functional_tests_server.js --config x-pack/test/security_api_integration/features.config.ts
```

2. Execute the following request from the Dev Tools (`case_4_feature_a`
is a deprecated feature that is replaced by multiple features and
**doesn't use** `deprecated.replacedBy`)
```http
PUT kbn:/api/spaces/space/default?overwrite=true
{
  "id":"default",
  "name":"Default",
  "description":"This is your default space!",
  "color":"#00bfb3",
  "disabledFeatures":["case_4_feature_a"],
  "_reserved":true,
  "imageUrl":"",
  "initials":"D"
}
```

3. Observe that in response deprecated `case_4_feature_a` is replaced by
two features (you can also check
http://localhost:5620/app/management/kibana/spaces/edit/default to see
how it's reflected in UI)
```http
{
  "id": "default",
  "name": "Default",
  "description": "This is your default space!",
  "color": "#00bfb3",
  "initials": "D",
  "imageUrl": "",
  "disabledFeatures": [
    "case_4_feature_a_v2",
    "case_4_feature_c"
  ],
  "_reserved": true
}
```

4. Execute the following request from the Dev Tools (`case_4_feature_b`
is a deprecated feature that is replaced by multiple features, but
**uses** `deprecated.replacedBy` to set the conceptual
feature-successor)
```http
PUT kbn:/api/spaces/space/default?overwrite=true
{
  "id":"default",
  "name":"Default",
  "description":"This is your default space!",
  "color":"#00bfb3",
  "disabledFeatures":["case_4_feature_b"],
  "_reserved":true,
  "imageUrl":"",
  "initials":"D"
}
```

5. Observe that in response deprecated `case_4_feature_b` is replaced by
a single feature (you can also check
http://localhost:5620/app/management/kibana/spaces/edit/default to see
how it's reflected in UI)
```http
{
  "id": "default",
  "name": "Default",
  "description": "This is your default space!",
  "color": "#00bfb3",
  "initials": "D",
  "imageUrl": "",
  "disabledFeatures": [
    "case_4_feature_b_v2"
  ],
  "_reserved": true
}
```

__Required by:__
elastic#202863 (comment)

//cc @davismcphee
@davismcphee
Copy link
Contributor Author

Thanks for the heads up @rylnd, and I left a comment on the issue with some details. I just want to confirm that the plan is to verify this soon, right (since these changes are targeting 9.00 FF)?

Copy link
Contributor

@rylnd rylnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my previous review: I just received a confirmation of the expected behavior, so this is good on our end.

Copy link
Contributor

@jughosta jughosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍
Great to have a single explicit Saved Query privilege rather than multiple implicit ones!

@elasticmachine
Copy link
Contributor

elasticmachine commented Jan 29, 2025

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
dashboard 106 105 -1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
aiops 605.5KB 605.5KB +15.0B
alerting 98.7KB 98.7KB -29.0B
apm 4.0MB 4.0MB +6.0B
cloudSecurityPosture 523.7KB 523.7KB -33.0B
dashboard 602.0KB 601.9KB -123.0B
dataVisualizer 598.7KB 598.7KB +12.0B
discover 834.1KB 834.1KB -82.0B
eventAnnotationListing 241.0KB 241.0KB +3.0B
infra 1.2MB 1.2MB -163.0B
inventory 228.1KB 228.1KB +6.0B
lens 1.6MB 1.6MB -26.0B
maps 3.0MB 3.0MB -69.0B
ml 4.7MB 4.7MB +30.0B
monitoring 515.2KB 515.2KB +6.0B
observability 1.3MB 1.3MB -29.0B
osquery 1.0MB 1.0MB +6.0B
presentationUtil 84.1KB 84.1KB +12.0B
securitySolution 21.4MB 21.4MB -80.0B
slo 885.8KB 885.8KB -42.0B
stackAlerts 75.2KB 75.2KB -29.0B
transform 470.1KB 470.1KB +6.0B
triggersActionsUi 1.7MB 1.7MB -91.0B
unifiedHistogram 71.0KB 71.0KB +3.0B
unifiedSearch 375.3KB 375.4KB +108.0B
visualizations 368.2KB 368.1KB -88.0B
total -671.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
unifiedSearch 21 20 -1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
cases 162.1KB 162.2KB +24.0B
discover 44.7KB 44.7KB +9.0B
discoverEnhanced 7.1KB 7.2KB +6.0B
lens 50.2KB 50.2KB +15.0B
maps 51.9KB 51.9KB +12.0B
reporting 47.7KB 47.8KB +12.0B
triggersActionsUi 130.2KB 130.2KB -29.0B
unifiedSearch 44.7KB 44.6KB -95.0B
visualizations 38.8KB 38.8KB +6.0B
total -40.0B
Unknown metric groups

API count

id before after diff
@kbn/core-capabilities-common 5 1 -4
dashboard 110 109 -1
total -5

History

cc @kertal @davismcphee

@davismcphee davismcphee merged commit b53d399 into elastic:main Jan 29, 2025
9 checks passed
@davismcphee davismcphee deleted the rework-saved-query-privileges branch January 29, 2025 21:35
semd added a commit that referenced this pull request Jan 31, 2025
…verless (#208911)

## Summary

Remove the implicit grant of the `savedQueryManagement` feature with the
Security Solution basic feature (ID: `siemV2`) in Serverless.


This is a follow-up of #202863

### Feature `siemV2`
This change only affects new roles created with the `siemV2` feature,
introduced recently
[here](#201780).
This change will align the Roles UI in Serverless and ESS, both
requiring the `savedQueryManagement` feature to be explicitly granted to
be able to manage saved queries.

### Feature `siem`
Roles using the deprecated `siem` feature will still implicitly receive
the `savedQueryManagement` feature (via an implicit grant of `discover`,
`dashboard`, `visualize`, and `maps`) + migration to their `*v2`
features which include `savedQueryManagement`. So there's no behavior
change for existing roles using the old `siem` feature (no breaking
change).

## Screenshots

The siem/siemV2 feature toggle:
<img width="774" alt="siem feature"
src="https://github.com/user-attachments/assets/2759988a-3cf8-4e1f-9431-16c09cf9d95c"
/>

The savedQueryManagement feature toggle:
<img width="774" alt="Saved query feature"
src="https://github.com/user-attachments/assets/d0145244-f4b8-4577-b91f-93f4dd1f758b"
/>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 31, 2025
…verless (elastic#208911)

## Summary

Remove the implicit grant of the `savedQueryManagement` feature with the
Security Solution basic feature (ID: `siemV2`) in Serverless.

This is a follow-up of elastic#202863

### Feature `siemV2`
This change only affects new roles created with the `siemV2` feature,
introduced recently
[here](elastic#201780).
This change will align the Roles UI in Serverless and ESS, both
requiring the `savedQueryManagement` feature to be explicitly granted to
be able to manage saved queries.

### Feature `siem`
Roles using the deprecated `siem` feature will still implicitly receive
the `savedQueryManagement` feature (via an implicit grant of `discover`,
`dashboard`, `visualize`, and `maps`) + migration to their `*v2`
features which include `savedQueryManagement`. So there's no behavior
change for existing roles using the old `siem` feature (no breaking
change).

## Screenshots

The siem/siemV2 feature toggle:
<img width="774" alt="siem feature"
src="https://github.com/user-attachments/assets/2759988a-3cf8-4e1f-9431-16c09cf9d95c"
/>

The savedQueryManagement feature toggle:
<img width="774" alt="Saved query feature"
src="https://github.com/user-attachments/assets/d0145244-f4b8-4577-b91f-93f4dd1f758b"
/>

(cherry picked from commit 3d5972a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:project-deploy-observability Create an Observability project release_note:breaking Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team Team:obs-ux-management Observability Management User Experience Team v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.