[Fleet] Allow to install Kibana assets in multiple spaces #186620

nchaulet · 2024-06-21T12:28:01Z

Summary

The package saved object is agnostic, but some of the installed assets (Kibana one are space aware), because of this we currently only allow to install kibana assets in one space.

That PR allow to install kibana assets in multiple spaces by introducing a new API

POST kbn:/api/fleet/epm/packages/{pkgName}/{pkgVersion}/kibana_assets

I also introduce an API to be able to delete those newly created assets.

DELETE kbn:/api/fleet/epm/packages/{pkgName}/{pkgVersion}/kibana_assets

Instead of introducing a new property to the package saved object, we could eventually introduce a new saved object space aware installation_kibana_assets that contains just the kibana assets, but this introduce more complexity.

On package reinstall/upgrade it will upgrade assets through all spaces

The feature is behind a feature flag useSpaceAwareness

Implementation details

Saved object change

I added a new property additional_spaces_installed_kibana on the installation saved object to keep track of the installed assets in different spaces:

It looks like this

{
  additional_spaces_installed_kibana?: {
   [spaceId: string]: KibanaAssetsReference[]
   }
}

I added a new

UI Changes

With nginx installed in the default space:

In the test space:

Assets not installed

Assets installed

Open questions/Issues found during implementation

Delete API it is needed? I think it could help for future supportability.
IDs in markdown visualization are not working => should we rewrite them?

Todo

Put those new APIs/UIs behind a feature flag
API integration tests
Create a follow issue for markdown in dashboard IDs rewrite POC.

obltmachine · 2024-06-21T12:28:14Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

nchaulet · 2024-06-21T14:11:07Z

@kpollich Before I start writing test and polishing that one I would love to get your initial thought on the implemented behavior

kpollich · 2024-06-21T14:43:45Z

Delete API it is needed? I think it could help for future supportability.

+1 I think we should have an API for deleting these assets. It will almost certainly come up in SDH's as a troubleshooting step.

IDs in markdown visualization are not working => should we rewrite them?

If it's possible to rewrite the ID's on the fly while imported the visualization, that could be an intriguing option. This would also help with mitigating #176552, and could be a workaround for that issue specific to Fleet-managed assets. @drewdaemon I wonder if you have thoughts on this. Does it make sense for Fleet to try and rewrite ID's within these markdown panels on-the-fly during the package installation process, or does that sound like a concerning workflow? I could see it being brittle or expensive to parse large dashboard JSON objects and running a regex or something to do string replacement for the proper ID's.

nchaulet · 2024-06-21T18:14:10Z

/ci

…te --fix'

nchaulet · 2024-06-21T19:15:49Z

/ci

nchaulet · 2024-06-21T19:17:19Z

x-pack/plugins/fleet/common/experimental_features.ts

+ * A list of allowed values that can be used in `xpack.fleet.enableExperimental`.
+ * This object is then used to validate and parse the value entered.
+ */
+export const allowedExperimentalValues = Object.freeze<


Not really related to that feature, but it allow to add some type safety to allowedExperimentalValues , and fix the autocompletion that was bothering me

… src/core/server/integration_tests/ci_checks'

…nchaulet/kibana into feature-kibana-install-differents-space

nchaulet · 2024-06-21T21:20:32Z

/ci

drewdaemon · 2024-06-24T17:41:53Z

@kpollich

Does it make sense for Fleet to try and rewrite ID's within these markdown panels on-the-fly during the package installation process, or does that sound like a concerning workflow?

It could be worth a PoC. I have performed this operation manually before to help particular customers. It could be safe actually because we're talking about find-and-replace on UUIDs.

…install-differents-space

elasticmachine · 2024-06-25T13:46:38Z

Pinging @elastic/fleet (Team:Fleet)

afharo · 2024-06-25T15:17:16Z

packages/kbn-check-mappings-update-cli/current_fields.json

@@ -291,6 +291,7 @@
  ],
  "enterprise_search_telemetry": [],
  "epm-packages": [
+    "additionnal_spaces_installed_kibana",


nit: additional has one n 😇

afharo

Additive mappings LGTM

…te --fix'

… src/core/server/integration_tests/ci_checks'

jen-huang · 2024-06-25T23:32:19Z

UX suggestion:

For the case where assets are not yet installed in this space, why don't we offer the Install Kibana assets button directly in this callout?:

This way, we can remove it from the Settings tab. The current addition to the settings tab is confusing IMO with the existing reinstall option.

Btw, what happens if the user chooses to Uninstall or Reinstall while in the secondary space? Will all assets across all spaces be un/re-installed?

nchaulet · 2024-06-26T01:20:54Z

For the case where assets are not yet installed in this space, why don't we offer the Install Kibana assets button directly in this callout?:

Yes that's a great idea, and will simplify the settings page

Btw, what happens if the user chooses to Uninstall or Reinstall while in the secondary space? Will all assets across all spaces be un/re-installed?

Yes uninstall will uninstall the package and his assets in all space (if no integration policy use it), and reinstall will reinstall assets across all spaces (similar to an update)

nchaulet · 2024-06-26T12:37:02Z

@jen-huang Just updated my PR to have the install button in the assets page, makes a lot more sense

…install-differents-space

jen-huang

Small copy fixes, otherwise LGTM. Tested locally.

...pplications/integrations/sections/epm/screens/detail/assets/install_kibana_assets_button.tsx

jen-huang · 2024-06-26T16:49:05Z

...plugins/fleet/public/applications/integrations/sections/epm/screens/detail/assets/assets.tsx

@@ -231,6 +241,13 @@ export const AssetsPage = ({ packageInfo, refetchPackageInfo }: AssetsPanelProps
                }}


is there already a docs issue to update https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html once this feature is enabled by default? if not, let's go ahead and create a placeholder issue

No there is not, I will create one for the whole space awareness project as there is probably more than this to document.

Co-authored-by: Jen Huang <its.jenetic@gmail.com>

kibana-ci · 2024-06-26T18:55:43Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: a6b5c18

Failed CI Steps

FTR Configs #42

Test Failures

[job] [logs] FTR Configs #42 / Actions and Triggers app Stack alerts page Loads the page Disables the SIEM solution filter when any other is applied

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`fleet`	1036	1037	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`fleet`	1219	1226	+7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`fleet`	1.4MB	1.4MB	+1.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`fleet`	165.9KB	166.5KB	+623.0B

Unknown metric groups

API count

id	before	after	diff
`fleet`	1341	1348	+7

History

💛 Build #217903 was flaky 50632d7
💔 Build #217769 failed b7fcb9b
💔 Build #217742 failed 09c6abe
💔 Build #217700 failed 587c9f9
💛 Build #217266 was flaky 9eb9135
💔 Build #217237 failed 7dfbc24

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[Fleet] Allow to install Kibana assets in multiple spaces

00d685b

nchaulet added release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team labels Jun 21, 2024

Add UI

901424e

UI fixes

80320de

nchaulet force-pushed the feature-kibana-install-differents-space branch from 60adb25 to 80320de Compare June 21, 2024 18:13

[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…

7dfbc24

…te --fix'

nchaulet commented Jun 21, 2024

View reviewed changes

kibanamachine and others added 3 commits June 21, 2024 19:57

[CI] Auto-commit changed files from 'node scripts/jest_integration -u…

168704c

… src/core/server/integration_tests/ci_checks'

fix types

8ff55eb

Merge branch 'feature-kibana-install-differents-space' of github.com:…

9eb9135

…nchaulet/kibana into feature-kibana-install-differents-space

nchaulet added 2 commits June 25, 2024 09:16

Merge branch 'main' of github.com:elastic/kibana into feature-kibana-…

a30de0a

…install-differents-space

Move tests

8223489

nchaulet marked this pull request as ready for review June 25, 2024 13:46

nchaulet requested review from a team as code owners June 25, 2024 13:46

nchaulet added 2 commits June 25, 2024 10:47

add api integration tests

e944f52

add delete API integrations tests

b74f13e

afharo reviewed Jun 25, 2024

View reviewed changes

afharo approved these changes Jun 25, 2024

View reviewed changes

rename additionnal -> additional

587c9f9

nchaulet and others added 4 commits June 25, 2024 13:33

fix renaming

3ac768f

[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…

2cd5196

…te --fix'

[CI] Auto-commit changed files from 'node scripts/jest_integration -u…

09c6abe

… src/core/server/integration_tests/ci_checks'

refacto

b7fcb9b

Move the install assets button to the assets page

e86f532

Merge branch 'main' of github.com:elastic/kibana into feature-kibana-…

50632d7

…install-differents-space

jen-huang self-requested a review June 26, 2024 16:33

jen-huang approved these changes Jun 26, 2024

View reviewed changes

Apply suggestions from code review

a6b5c18

Co-authored-by: Jen Huang <its.jenetic@gmail.com>

nchaulet merged commit ace2996 into elastic:main Jun 26, 2024
24 checks passed

nchaulet deleted the feature-kibana-install-differents-space branch June 26, 2024 20:12

kibanamachine added v8.15.0 backport:skip This commit does not require backporting labels Jun 26, 2024

nchaulet mentioned this pull request Jun 28, 2024

[SLOs][Fleet] Install integration SLO assets #186974

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fleet] Allow to install Kibana assets in multiple spaces #186620

[Fleet] Allow to install Kibana assets in multiple spaces #186620

nchaulet commented Jun 21, 2024 •

edited

Loading

obltmachine commented Jun 21, 2024

nchaulet commented Jun 21, 2024

kpollich commented Jun 21, 2024

nchaulet commented Jun 21, 2024

nchaulet commented Jun 21, 2024

nchaulet Jun 21, 2024 •

edited

Loading

nchaulet commented Jun 21, 2024

drewdaemon commented Jun 24, 2024

elasticmachine commented Jun 25, 2024

afharo Jun 25, 2024

afharo left a comment

jen-huang commented Jun 25, 2024

nchaulet commented Jun 26, 2024

nchaulet commented Jun 26, 2024

jen-huang left a comment

jen-huang Jun 26, 2024

nchaulet Jun 26, 2024

kibana-ci commented Jun 26, 2024

API count

		@@ -231,6 +241,13 @@ export const AssetsPage = ({ packageInfo, refetchPackageInfo }: AssetsPanelProps
		}}

[Fleet] Allow to install Kibana assets in multiple spaces #186620

[Fleet] Allow to install Kibana assets in multiple spaces #186620

Conversation

nchaulet commented Jun 21, 2024 • edited Loading

Summary

Implementation details

UI Changes

Open questions/Issues found during implementation

Todo

obltmachine commented Jun 21, 2024

🤖 GitHub comments

nchaulet commented Jun 21, 2024

kpollich commented Jun 21, 2024

nchaulet commented Jun 21, 2024

nchaulet commented Jun 21, 2024

nchaulet Jun 21, 2024 • edited Loading

Choose a reason for hiding this comment

nchaulet commented Jun 21, 2024

drewdaemon commented Jun 24, 2024

elasticmachine commented Jun 25, 2024

afharo Jun 25, 2024

Choose a reason for hiding this comment

afharo left a comment

Choose a reason for hiding this comment

jen-huang commented Jun 25, 2024

nchaulet commented Jun 26, 2024

nchaulet commented Jun 26, 2024

jen-huang left a comment

Choose a reason for hiding this comment

jen-huang Jun 26, 2024

Choose a reason for hiding this comment

nchaulet Jun 26, 2024

Choose a reason for hiding this comment

kibana-ci commented Jun 26, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

History

nchaulet commented Jun 21, 2024 •

edited

Loading

nchaulet Jun 21, 2024 •

edited

Loading