requiredPlugins. |
-| [requiredPlugins](./kibana-plugin-core-server.discoveredplugin.requiredplugins.md) | readonly PluginName\[\] | An optional list of the other plugins that \*\*must be\*\* installed and enabled for this plugin to function properly. |
-| [type](./kibana-plugin-core-server.discoveredplugin.type.md) | PluginType | Type of the plugin, defaults to standard. |
-
diff --git a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.optionalplugins.md b/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.optionalplugins.md
deleted file mode 100644
index 0fc42048be90c..0000000000000
--- a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.optionalplugins.md
+++ /dev/null
@@ -1,13 +0,0 @@
-
-
-[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [DiscoveredPlugin](./kibana-plugin-core-server.discoveredplugin.md) > [optionalPlugins](./kibana-plugin-core-server.discoveredplugin.optionalplugins.md)
-
-## DiscoveredPlugin.optionalPlugins property
-
-An optional list of the other plugins that if installed and enabled \*\*may be\*\* leveraged by this plugin for some additional functionality but otherwise are not required for this plugin to work properly.
-
-Signature:
-
-```typescript
-readonly optionalPlugins: readonly PluginName[];
-```
diff --git a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.requiredbundles.md b/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.requiredbundles.md
deleted file mode 100644
index 6d54adb5236ea..0000000000000
--- a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.requiredbundles.md
+++ /dev/null
@@ -1,18 +0,0 @@
-
-
-[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [DiscoveredPlugin](./kibana-plugin-core-server.discoveredplugin.md) > [requiredBundles](./kibana-plugin-core-server.discoveredplugin.requiredbundles.md)
-
-## DiscoveredPlugin.requiredBundles property
-
-List of plugin ids that this plugin's UI code imports modules from that are not in `requiredPlugins`.
-
-Signature:
-
-```typescript
-readonly requiredBundles: readonly PluginName[];
-```
-
-## Remarks
-
-The plugins listed here will be loaded in the browser, even if the plugin is disabled. Required by `@kbn/optimizer` to support cross-plugin imports. "core" and plugins already listed in `requiredPlugins` do not need to be duplicated here.
-
diff --git a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.requiredplugins.md b/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.requiredplugins.md
deleted file mode 100644
index b039891904669..0000000000000
--- a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.requiredplugins.md
+++ /dev/null
@@ -1,13 +0,0 @@
-
-
-[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [DiscoveredPlugin](./kibana-plugin-core-server.discoveredplugin.md) > [requiredPlugins](./kibana-plugin-core-server.discoveredplugin.requiredplugins.md)
-
-## DiscoveredPlugin.requiredPlugins property
-
-An optional list of the other plugins that \*\*must be\*\* installed and enabled for this plugin to function properly.
-
-Signature:
-
-```typescript
-readonly requiredPlugins: readonly PluginName[];
-```
diff --git a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.type.md b/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.type.md
deleted file mode 100644
index 0a33be0d63f5c..0000000000000
--- a/docs/development/core/server/kibana-plugin-core-server.discoveredplugin.type.md
+++ /dev/null
@@ -1,13 +0,0 @@
-
-
-[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [DiscoveredPlugin](./kibana-plugin-core-server.discoveredplugin.md) > [type](./kibana-plugin-core-server.discoveredplugin.type.md)
-
-## DiscoveredPlugin.type property
-
-Type of the plugin, defaults to `standard`.
-
-Signature:
-
-```typescript
-readonly type: PluginType;
-```
diff --git a/docs/development/core/server/kibana-plugin-core-server.kibanaresponsefactory.md b/docs/development/core/server/kibana-plugin-core-server.kibanaresponsefactory.md
index be28cd76b16ee..e02f208ae86ac 100644
--- a/docs/development/core/server/kibana-plugin-core-server.kibanaresponsefactory.md
+++ b/docs/development/core/server/kibana-plugin-core-server.kibanaresponsefactory.md
@@ -10,7 +10,7 @@ Set of helpers used to create `KibanaResponse` to form HTTP response on an incom
```typescript
kibanaResponseFactory: {
- custom: preboot plugins at setup. |
| [ElasticsearchErrorDetails](./kibana-plugin-core-server.elasticsearcherrordetails.md) | |
diff --git a/docs/development/core/server/kibana-plugin-core-server.plugintype.md b/docs/development/core/server/kibana-plugin-core-server.plugintype.md
deleted file mode 100644
index e4a252a392949..0000000000000
--- a/docs/development/core/server/kibana-plugin-core-server.plugintype.md
+++ /dev/null
@@ -1,20 +0,0 @@
-
-
-[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [PluginType](./kibana-plugin-core-server.plugintype.md)
-
-## PluginType enum
-
-
-Signature:
-
-```typescript
-export declare enum PluginType
-```
-
-## Enumeration Members
-
-| Member | Value | Description |
-| --- | --- | --- |
-| preboot | "preboot" | Preboot plugins are special-purpose plugins that only function during preboot stage. |
-| standard | "standard" | Standard plugins are plugins that start to function as soon as Kibana is fully booted and are active until it shuts down. |
-
diff --git a/docs/management/cases/images/cases-visualization.png b/docs/management/cases/images/cases-visualization.png
index 77f249f26d091..53c4893bcc598 100644
Binary files a/docs/management/cases/images/cases-visualization.png and b/docs/management/cases/images/cases-visualization.png differ
diff --git a/docs/management/cases/images/cases.png b/docs/management/cases/images/cases.png
index b244b3df16a20..a69f21c2ec9f8 100644
Binary files a/docs/management/cases/images/cases.png and b/docs/management/cases/images/cases.png differ
diff --git a/docs/settings/apm-settings.asciidoc b/docs/settings/apm-settings.asciidoc
index 2cfd3169b45a3..65f291a1c11cb 100644
--- a/docs/settings/apm-settings.asciidoc
+++ b/docs/settings/apm-settings.asciidoc
@@ -18,9 +18,14 @@ It is enabled by default.
// Any changes made in this file will be seen there as well.
// tag::apm-indices-settings[]
-Index defaults can be changed in the APM app. Select **Settings** > **Indices**.
+The APM app uses data views to query APM indices.
+To change the default APM indices that the APM app queries, open the APM app and select **Settings** > **Indices**.
Index settings in the APM app take precedence over those set in `kibana.yml`.
+Starting in version 8.2.0, APM indices are {kib} Spaces-aware;
+Changes to APM index settings will only apply to the currently enabled space.
+See <
- Card title
+
+ Card title
+
- Card title
+
+ Card title
+
- Card title
+
+ Card title
+
- Card title
+
+ Card title
+
- Card title
+
+ Card title
+
= ({
canAccessFleet,
title = elasticAgentCardTitle,
+ description,
...cardRest
}) => {
const props = canAccessFleet
? {
title,
- description: elasticAgentCardDescription,
+ description: description || elasticAgentCardDescription,
}
: {
title: {noPermissionTitle} ,
diff --git a/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.stories.tsx b/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.stories.tsx
index 84cbfb1c73a94..a87da6ff9ca0e 100644
--- a/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.stories.tsx
+++ b/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.stories.tsx
@@ -31,6 +31,10 @@ PureComponent.argTypes = {
control: 'boolean',
defaultValue: true,
},
+ description: {
+ control: 'text',
+ defaultValue: '',
+ },
};
export const ConnectedComponent = () => {
diff --git a/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.test.tsx b/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.test.tsx
index 2eda3d4c3491a..ed0b4471fa32c 100644
--- a/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.test.tsx
+++ b/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/elastic_agent_card.test.tsx
@@ -58,6 +58,17 @@ describe('ElasticAgentCard', () => {
});
});
+ describe('description', () => {
+ test('renders custom description if provided', () => {
+ const component = mount(
+
-
);
};
diff --git a/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/no_data_card.tsx b/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/no_data_card.tsx
index 705eb0444bb8d..508d03d5028b8 100644
--- a/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/no_data_card.tsx
+++ b/packages/kbn-shared-ux-components/src/page_template/no_data_page/no_data_card/no_data_card.tsx
@@ -8,7 +8,7 @@
import { i18n } from '@kbn/i18n';
import React, { FunctionComponent } from 'react';
-import { EuiButton, EuiCard } from '@elastic/eui';
+import { EuiButton, EuiCard, EuiScreenReaderOnly } from '@elastic/eui';
import type { NoDataCardProps } from './types';
import { NoDataCardStyles } from './no_data_card.styles';
@@ -21,7 +21,7 @@ const defaultDescription = i18n.translate(
);
export const NoDataCard: FunctionComponent = ({
- title,
+ title: titleProp,
button,
description,
isDisabled,
@@ -39,10 +39,18 @@ export const NoDataCard: FunctionComponent = ({
return button;
}
// Default footer action is a button with the provided or default string
- return {button || title} ;
+ return {button || titleProp} ;
};
+
const cardDescription = description || defaultDescription;
+ // Fix the need for an a11y title even though the button exists by setting to screen reader only
+ const title = titleProp ? (
+
+ {titleProp}
+
+ ) : null;
+
return (
;
+export type NoDataPageActions = ElasticAgentCardProps;
export interface NoDataPageProps extends CommonProps {
/**
diff --git a/packages/kbn-utility-types-jest/BUILD.bazel b/packages/kbn-utility-types-jest/BUILD.bazel
new file mode 100644
index 0000000000000..beb01908abe3e
--- /dev/null
+++ b/packages/kbn-utility-types-jest/BUILD.bazel
@@ -0,0 +1,95 @@
+load("@npm//@bazel/typescript:index.bzl", "ts_config")
+load("@build_bazel_rules_nodejs//:index.bzl", "js_library")
+load("//src/dev/bazel:index.bzl", "jsts_transpiler", "pkg_npm", "pkg_npm_types", "ts_project")
+
+PKG_BASE_NAME = "kbn-utility-types-jest"
+PKG_REQUIRE_NAME = "@kbn/utility-types-jest"
+
+SOURCE_FILES = glob([
+ "src/index.ts",
+])
+
+SRCS = SOURCE_FILES
+
+filegroup(
+ name = "srcs",
+ srcs = SRCS,
+)
+
+NPM_MODULE_EXTRA_FILES = [
+ "package.json",
+]
+
+RUNTIME_DEPS = []
+
+TYPES_DEPS = [
+ "@npm//@types/node",
+ "@npm//@types/jest",
+]
+
+jsts_transpiler(
+ name = "target_node",
+ srcs = SRCS,
+ build_pkg_name = package_name(),
+)
+
+ts_config(
+ name = "tsconfig",
+ src = "tsconfig.json",
+ deps = [
+ "//:tsconfig.base.json",
+ "//:tsconfig.bazel.json",
+ ],
+)
+
+ts_project(
+ name = "tsc_types",
+ args = ['--pretty'],
+ srcs = SRCS,
+ deps = TYPES_DEPS,
+ declaration = True,
+ emit_declaration_only = True,
+ out_dir = "target_types",
+ root_dir = "src",
+ tsconfig = ":tsconfig",
+)
+
+js_library(
+ name = PKG_BASE_NAME,
+ srcs = NPM_MODULE_EXTRA_FILES,
+ deps = RUNTIME_DEPS + [":target_node"],
+ package_name = PKG_REQUIRE_NAME,
+ visibility = ["//visibility:public"],
+)
+
+pkg_npm(
+ name = "npm_module",
+ deps = [
+ ":%s" % PKG_BASE_NAME,
+ ]
+)
+
+filegroup(
+ name = "build",
+ srcs = [
+ ":npm_module",
+ ],
+ visibility = ["//visibility:public"],
+)
+
+pkg_npm_types(
+ name = "npm_module_types",
+ srcs = SRCS,
+ deps = [":tsc_types"],
+ package_name = PKG_REQUIRE_NAME,
+ tsconfig = ":tsconfig",
+ visibility = ["//visibility:public"],
+)
+
+filegroup(
+ name = "build_types",
+ srcs = [
+ ":npm_module_types",
+ ],
+ visibility = ["//visibility:public"],
+)
diff --git a/packages/kbn-utility-types-jest/README.md b/packages/kbn-utility-types-jest/README.md
new file mode 100644
index 0000000000000..496630464e862
--- /dev/null
+++ b/packages/kbn-utility-types-jest/README.md
@@ -0,0 +1,20 @@
+# `@kbn/utility-types-jest`
+
+TypeScript Jest utility types for usage in Kibana.
+You can add as much as any other types you think that makes sense to add here.
+
+
+## Usage
+
+```ts
+import type { MockedKeys } from '@kbn/utility-types-jest';
+
+type A = MockedKeys;
+```
+
+
+## Reference
+
+- `DeeplyMockedKeys`
+- `MockedKeys`
+
diff --git a/packages/kbn-utility-types-jest/package.json b/packages/kbn-utility-types-jest/package.json
new file mode 100644
index 0000000000000..808dd51dec793
--- /dev/null
+++ b/packages/kbn-utility-types-jest/package.json
@@ -0,0 +1,10 @@
+{
+ "name": "@kbn/utility-types-jest",
+ "version": "1.0.0",
+ "private": true,
+ "license": "SSPL-1.0 OR Elastic License 2.0",
+ "main": "target_node/index.js",
+ "kibana": {
+ "devOnly": false
+ }
+}
\ No newline at end of file
diff --git a/packages/kbn-utility-types/src/jest/index.ts b/packages/kbn-utility-types-jest/src/index.ts
similarity index 100%
rename from packages/kbn-utility-types/src/jest/index.ts
rename to packages/kbn-utility-types-jest/src/index.ts
diff --git a/packages/kbn-utility-types-jest/tsconfig.json b/packages/kbn-utility-types-jest/tsconfig.json
new file mode 100644
index 0000000000000..1d7104a6fc254
--- /dev/null
+++ b/packages/kbn-utility-types-jest/tsconfig.json
@@ -0,0 +1,17 @@
+{
+ "extends": "../../tsconfig.bazel.json",
+ "compilerOptions": {
+ "declaration": true,
+ "emitDeclarationOnly": true,
+ "outDir": "./target_types",
+ "rootDir": "./src",
+ "stripInternal": true,
+ "types": [
+ "jest",
+ "node"
+ ]
+ },
+ "include": [
+ "src/**/*",
+ ]
+}
diff --git a/packages/kbn-utility-types/BUILD.bazel b/packages/kbn-utility-types/BUILD.bazel
index c556751d7550e..a990350cc4afc 100644
--- a/packages/kbn-utility-types/BUILD.bazel
+++ b/packages/kbn-utility-types/BUILD.bazel
@@ -6,7 +6,6 @@ PKG_BASE_NAME = "kbn-utility-types"
PKG_REQUIRE_NAME = "@kbn/utility-types"
SOURCE_FILES = glob([
- "src/jest/index.ts",
"src/serializable/**",
"src/index.ts"
])
@@ -19,9 +18,7 @@ filegroup(
)
NPM_MODULE_EXTRA_FILES = [
- "jest/package.json",
"package.json",
- "README.md",
]
RUNTIME_DEPS = [
@@ -30,8 +27,7 @@ RUNTIME_DEPS = [
TYPES_DEPS = [
"@npm//utility-types",
- "@npm//@types/node",
- "@npm//@types/jest",
+ "@npm//@types/node"
]
jsts_transpiler(
@@ -65,7 +61,7 @@ js_library(
name = PKG_BASE_NAME,
srcs = NPM_MODULE_EXTRA_FILES,
deps = RUNTIME_DEPS + [":target_node"],
- package_name = "@kbn/utility-types",
+ package_name = PKG_REQUIRE_NAME,
visibility = ["//visibility:public"],
)
diff --git a/packages/kbn-utility-types/jest/package.json b/packages/kbn-utility-types/jest/package.json
deleted file mode 100644
index a5d6c9ef48887..0000000000000
--- a/packages/kbn-utility-types/jest/package.json
+++ /dev/null
@@ -1,3 +0,0 @@
-{
- "types": "../target_types/jest/index.d.ts"
-}
diff --git a/packages/kbn-utility-types/tsconfig.json b/packages/kbn-utility-types/tsconfig.json
index 1d7104a6fc254..db0a3994f6ff7 100644
--- a/packages/kbn-utility-types/tsconfig.json
+++ b/packages/kbn-utility-types/tsconfig.json
@@ -7,7 +7,6 @@
"rootDir": "./src",
"stripInternal": true,
"types": [
- "jest",
"node"
]
},
diff --git a/packages/shared-ux/prompt/no_data_views/BUILD.bazel b/packages/shared-ux/prompt/no_data_views/BUILD.bazel
index fbfd4dcb4dfea..99e752358a000 100644
--- a/packages/shared-ux/prompt/no_data_views/BUILD.bazel
+++ b/packages/shared-ux/prompt/no_data_views/BUILD.bazel
@@ -10,6 +10,7 @@ SOURCE_FILES = glob(
"src/**/*.ts",
"src/**/*.tsx",
"src/**/*.mdx",
+ "src/**/*.svg",
],
exclude = [
"**/*.test.*",
@@ -86,6 +87,10 @@ jsts_transpiler(
srcs = SRCS,
build_pkg_name = package_name(),
web = True,
+ additional_args = [
+ "--copy-files",
+ "--quiet"
+ ],
)
ts_config(
diff --git a/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.svg b/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.svg
new file mode 100644
index 0000000000000..4f499534a5ec2
--- /dev/null
+++ b/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.svg
@@ -0,0 +1 @@
+
diff --git a/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.tsx b/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.tsx
index 8a889a9267dee..55781fb13b444 100644
--- a/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.tsx
+++ b/packages/shared-ux/prompt/no_data_views/src/data_view_illustration.tsx
@@ -8,545 +8,22 @@
import React from 'react';
import { useEuiTheme } from '@elastic/eui';
-import { css } from '@emotion/css';
+import { css as emotion } from '@emotion/css';
+
+import svg from './data_view_illustration.svg';
export const DataViewIllustration = () => {
const { euiTheme } = useEuiTheme();
const { colors } = euiTheme;
- const dataViewIllustrationVerticalStripes = css`
- fill: ${colors.fullShade};
- `;
-
- const dataViewIllustrationDots = css`
- fill: ${colors.lightShade};
+ const css = emotion`
+ .dataViewIllustrationVerticalStripes {
+ fill: ${colors.fullShade};
+ };
+ .dataViewIllustrationDots {
+ fill: ${colors.lightShade};
+ }
`;
- return (
-
- );
+ return 
;
};
diff --git a/src/core/public/analytics/analytics_service.ts b/src/core/public/analytics/analytics_service.ts
index b55dc85e5296b..b774109e79d61 100644
--- a/src/core/public/analytics/analytics_service.ts
+++ b/src/core/public/analytics/analytics_service.ts
@@ -6,12 +6,12 @@
* Side Public License, v 1.
*/
+import { of } from 'rxjs';
import type { AnalyticsClient } from '@kbn/analytics-client';
import { createAnalytics } from '@kbn/analytics-client';
import type { CoreContext } from '@kbn/core-base-browser-internal';
-import { of } from 'rxjs';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import { trackClicks } from './track_clicks';
-import { InjectedMetadataSetup } from '../injected_metadata';
import { getSessionId } from './get_session_id';
import { createLogger } from './logger';
@@ -33,7 +33,7 @@ export type AnalyticsServiceStart = Pick<
/** @internal */
export interface AnalyticsServiceSetupDeps {
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
export class AnalyticsService {
@@ -173,7 +173,7 @@ export class AnalyticsService {
* @param injectedMetadata The injected metadata service.
* @private
*/
- private registerElasticsearchInfoContext(injectedMetadata: InjectedMetadataSetup) {
+ private registerElasticsearchInfoContext(injectedMetadata: InternalInjectedMetadataSetup) {
this.analyticsClient.registerContextProvider({
name: 'elasticsearch info',
context$: of(injectedMetadata.getElasticsearchInfo()),
diff --git a/src/core/public/apm_system.test.ts b/src/core/public/apm_system.test.ts
index 0a3a1dee63e57..68ea7b8dfe5db 100644
--- a/src/core/public/apm_system.test.ts
+++ b/src/core/public/apm_system.test.ts
@@ -7,7 +7,7 @@
*/
jest.mock('@elastic/apm-rum');
-import type { DeeplyMockedKeys, MockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys, MockedKeys } from '@kbn/utility-types-jest';
import { init, apm } from '@elastic/apm-rum';
import type { Transaction } from '@elastic/apm-rum';
import { ApmSystem } from './apm_system';
diff --git a/src/core/public/chrome/chrome_service.mock.ts b/src/core/public/chrome/chrome_service.mock.ts
index 08c14e955bcbe..ec1cfca7d65d1 100644
--- a/src/core/public/chrome/chrome_service.mock.ts
+++ b/src/core/public/chrome/chrome_service.mock.ts
@@ -8,7 +8,7 @@
import { BehaviorSubject } from 'rxjs';
import type { PublicMethodsOf } from '@kbn/utility-types';
-import type { DeeplyMockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
import { ChromeBadge, ChromeBreadcrumb, ChromeService, InternalChromeStart } from '.';
const createStartContractMock = () => {
diff --git a/src/core/public/chrome/chrome_service.test.ts b/src/core/public/chrome/chrome_service.test.ts
index dbdc867ea33b3..664b6a3ee1617 100644
--- a/src/core/public/chrome/chrome_service.test.ts
+++ b/src/core/public/chrome/chrome_service.test.ts
@@ -10,11 +10,11 @@ import { shallow } from 'enzyme';
import React from 'react';
import * as Rx from 'rxjs';
import { toArray } from 'rxjs/operators';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { App, PublicAppInfo } from '../application';
import { applicationServiceMock } from '../application/application_service.mock';
import { docLinksServiceMock } from '../doc_links/doc_links_service.mock';
import { httpServiceMock } from '../http/http_service.mock';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
import { notificationServiceMock } from '../notifications/notifications_service.mock';
import { uiSettingsServiceMock } from '../ui_settings/ui_settings_service.mock';
import { ChromeService } from './chrome_service';
diff --git a/src/core/public/chrome/chrome_service.tsx b/src/core/public/chrome/chrome_service.tsx
index a9678613b5fd0..f631819b26e4e 100644
--- a/src/core/public/chrome/chrome_service.tsx
+++ b/src/core/public/chrome/chrome_service.tsx
@@ -12,11 +12,11 @@ import { BehaviorSubject, combineLatest, merge, Observable, of, ReplaySubject }
import { flatMap, map, takeUntil } from 'rxjs/operators';
import { parse } from 'url';
import { EuiLink } from '@elastic/eui';
+import type { InternalInjectedMetadataStart } from '@kbn/core-injected-metadata-browser-internal';
import { mountReactNode } from '../utils/mount';
import { InternalApplicationStart } from '../application';
import { DocLinksStart } from '../doc_links';
import { HttpStart } from '../http';
-import { InjectedMetadataStart } from '../injected_metadata';
import { NotificationsStart } from '../notifications';
import { KIBANA_ASK_ELASTIC_LINK } from './constants';
import { ChromeDocTitle, DocTitleService } from './doc_title';
@@ -47,7 +47,7 @@ export interface StartDeps {
application: InternalApplicationStart;
docLinks: DocLinksStart;
http: HttpStart;
- injectedMetadata: InjectedMetadataStart;
+ injectedMetadata: InternalInjectedMetadataStart;
notifications: NotificationsStart;
}
diff --git a/src/core/public/core_app/core_app.ts b/src/core/public/core_app/core_app.ts
index 30a9515a2709b..5c328096c36f2 100644
--- a/src/core/public/core_app/core_app.ts
+++ b/src/core/public/core_app/core_app.ts
@@ -8,6 +8,7 @@
import { UnregisterCallback } from 'history';
import type { CoreContext } from '@kbn/core-base-browser-internal';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import {
InternalApplicationSetup,
InternalApplicationStart,
@@ -17,7 +18,6 @@ import {
import type { HttpSetup, HttpStart } from '../http';
import type { NotificationsSetup, NotificationsStart } from '../notifications';
import type { IUiSettingsClient } from '../ui_settings';
-import type { InjectedMetadataSetup } from '../injected_metadata';
import {
renderApp as renderErrorApp,
setupPublicBaseUrlConfigWarning,
@@ -29,7 +29,7 @@ import { DocLinksStart } from '../doc_links';
export interface SetupDeps {
application: InternalApplicationSetup;
http: HttpSetup;
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
notifications: NotificationsSetup;
}
diff --git a/src/core/public/core_system.test.mocks.ts b/src/core/public/core_system.test.mocks.ts
index ff24cc8839794..b0accbcd3df70 100644
--- a/src/core/public/core_system.test.mocks.ts
+++ b/src/core/public/core_system.test.mocks.ts
@@ -6,12 +6,12 @@
* Side Public License, v 1.
*/
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { applicationServiceMock } from './application/application_service.mock';
import { chromeServiceMock } from './chrome/chrome_service.mock';
import { fatalErrorsServiceMock } from './fatal_errors/fatal_errors_service.mock';
import { httpServiceMock } from './http/http_service.mock';
import { i18nServiceMock } from './i18n/i18n_service.mock';
-import { injectedMetadataServiceMock } from './injected_metadata/injected_metadata_service.mock';
import { notificationServiceMock } from './notifications/notifications_service.mock';
import { overlayServiceMock } from './overlays/overlay_service.mock';
import { pluginsServiceMock } from './plugins/plugins_service.mock';
@@ -40,7 +40,7 @@ export const MockInjectedMetadataService = injectedMetadataServiceMock.create();
export const InjectedMetadataServiceConstructor = jest
.fn()
.mockImplementation(() => MockInjectedMetadataService);
-jest.doMock('./injected_metadata', () => ({
+jest.doMock('@kbn/core-injected-metadata-browser-internal', () => ({
InjectedMetadataService: InjectedMetadataServiceConstructor,
}));
diff --git a/src/core/public/core_system.ts b/src/core/public/core_system.ts
index 0ad746f32e490..cf2840eb4f0ad 100644
--- a/src/core/public/core_system.ts
+++ b/src/core/public/core_system.ts
@@ -7,17 +7,17 @@
*/
import type { CoreContext } from '@kbn/core-base-browser-internal';
+import {
+ InjectedMetadataService,
+ InjectedMetadataParams,
+ InternalInjectedMetadataSetup,
+ InternalInjectedMetadataStart,
+} from '@kbn/core-injected-metadata-browser-internal';
import { CoreSetup, CoreStart } from '.';
import { ChromeService } from './chrome';
import { FatalErrorsService, FatalErrorsSetup } from './fatal_errors';
import { HttpService } from './http';
import { I18nService } from './i18n';
-import {
- InjectedMetadataParams,
- InjectedMetadataService,
- InjectedMetadataSetup,
- InjectedMetadataStart,
-} from './injected_metadata';
import { NotificationsService } from './notifications';
import { OverlayService } from './overlays';
import { PluginsService } from './plugins';
@@ -45,13 +45,13 @@ interface Params {
/** @internal */
export interface InternalCoreSetup extends Omit {
application: InternalApplicationSetup;
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
/** @internal */
export interface InternalCoreStart extends Omit {
application: InternalApplicationStart;
- injectedMetadata: InjectedMetadataStart;
+ injectedMetadata: InternalInjectedMetadataStart;
}
/**
diff --git a/src/core/public/doc_links/doc_links_service.mock.ts b/src/core/public/doc_links/doc_links_service.mock.ts
index a4dd00eec84cc..b88007d68ba17 100644
--- a/src/core/public/doc_links/doc_links_service.mock.ts
+++ b/src/core/public/doc_links/doc_links_service.mock.ts
@@ -7,7 +7,7 @@
*/
import type { PublicMethodsOf } from '@kbn/utility-types';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { DocLinksService, DocLinksStart } from './doc_links_service';
const createStartContractMock = (): DocLinksStart => {
diff --git a/src/core/public/doc_links/doc_links_service.test.ts b/src/core/public/doc_links/doc_links_service.test.ts
index 9ddb0fc884b9b..b30082e62c653 100644
--- a/src/core/public/doc_links/doc_links_service.test.ts
+++ b/src/core/public/doc_links/doc_links_service.test.ts
@@ -8,7 +8,7 @@
import { getDocLinksMock, getDocLinksMetaMock } from './doc_links_service.test.mocks';
import { DocLinksService } from './doc_links_service';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
describe('DocLinksService', () => {
let injectedMetadata: ReturnType;
diff --git a/src/core/public/doc_links/doc_links_service.ts b/src/core/public/doc_links/doc_links_service.ts
index 09f997028e6d8..7da4737c0d5e1 100644
--- a/src/core/public/doc_links/doc_links_service.ts
+++ b/src/core/public/doc_links/doc_links_service.ts
@@ -8,10 +8,10 @@
import { getDocLinks, getDocLinksMeta } from '@kbn/doc-links';
import type { DocLinks } from '@kbn/doc-links';
-import { InjectedMetadataSetup } from '../injected_metadata';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
export interface StartDeps {
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
/** @internal */
diff --git a/src/core/public/fatal_errors/fatal_errors_service.test.ts b/src/core/public/fatal_errors/fatal_errors_service.test.ts
index 4b243979c8e4d..f95b036c36df3 100644
--- a/src/core/public/fatal_errors/fatal_errors_service.test.ts
+++ b/src/core/public/fatal_errors/fatal_errors_service.test.ts
@@ -14,7 +14,7 @@ expect.addSnapshotSerializer({
});
import { mockRender } from './fatal_errors_service.test.mocks';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { themeServiceMock } from '../theme/theme_service.mock';
import { FatalErrorsService } from './fatal_errors_service';
diff --git a/src/core/public/fatal_errors/fatal_errors_service.tsx b/src/core/public/fatal_errors/fatal_errors_service.tsx
index 0e72b99bc6b92..36d5ebf5c02e9 100644
--- a/src/core/public/fatal_errors/fatal_errors_service.tsx
+++ b/src/core/public/fatal_errors/fatal_errors_service.tsx
@@ -11,8 +11,8 @@ import { render } from 'react-dom';
import * as Rx from 'rxjs';
import { first, tap } from 'rxjs/operators';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import { I18nStart } from '../i18n';
-import { InjectedMetadataSetup } from '../injected_metadata';
import { ThemeServiceSetup } from '../theme';
import { CoreContextProvider } from '../utils';
import { FatalErrorsScreen } from './fatal_errors_screen';
@@ -21,7 +21,7 @@ import { FatalErrorInfo, getErrorInfo } from './get_error_info';
export interface Deps {
i18n: I18nStart;
theme: ThemeServiceSetup;
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
/**
diff --git a/src/core/public/http/external_url_service.ts b/src/core/public/http/external_url_service.ts
index e478e49552444..0e5994a253ace 100644
--- a/src/core/public/http/external_url_service.ts
+++ b/src/core/public/http/external_url_service.ts
@@ -7,14 +7,14 @@
*/
import type { CoreService } from '@kbn/core-base-browser-internal';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import { IExternalUrlPolicy } from '../../server/types';
import { IExternalUrl } from './types';
-import { InjectedMetadataSetup } from '../injected_metadata';
import { Sha256 } from '../utils';
interface SetupDeps {
location: Pick;
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
function* getHostHashes(actualHost: string) {
diff --git a/src/core/public/http/http_service.test.ts b/src/core/public/http/http_service.test.ts
index 698fa876433d4..af6e2343d5f8a 100644
--- a/src/core/public/http/http_service.test.ts
+++ b/src/core/public/http/http_service.test.ts
@@ -11,7 +11,7 @@ import fetchMock from 'fetch-mock/es5/client';
import { loadingServiceMock } from './http_service.test.mocks';
import { fatalErrorsServiceMock } from '../fatal_errors/fatal_errors_service.mock';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { HttpService } from './http_service';
import { Observable } from 'rxjs';
import { executionContextServiceMock } from '../execution_context/execution_context_service.mock';
diff --git a/src/core/public/http/http_service.ts b/src/core/public/http/http_service.ts
index 5e0c2dd6def0a..4507b808e5a4a 100644
--- a/src/core/public/http/http_service.ts
+++ b/src/core/public/http/http_service.ts
@@ -7,8 +7,8 @@
*/
import type { CoreService } from '@kbn/core-base-browser-internal';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import { HttpSetup, HttpStart } from './types';
-import { InjectedMetadataSetup } from '../injected_metadata';
import { FatalErrorsSetup } from '../fatal_errors';
import { BasePath } from './base_path';
import { AnonymousPathsService } from './anonymous_paths_service';
@@ -18,7 +18,7 @@ import { ExternalUrlService } from './external_url_service';
import { ExecutionContextSetup } from '../execution_context';
interface HttpDeps {
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
fatalErrors: FatalErrorsSetup;
executionContext: ExecutionContextSetup;
}
diff --git a/src/core/public/index.ts b/src/core/public/index.ts
index 378427bbd7cd3..21003b17928dd 100644
--- a/src/core/public/index.ts
+++ b/src/core/public/index.ts
@@ -26,6 +26,10 @@
import './index.scss';
+import type {
+ InjectedMetadataSetup,
+ InjectedMetadataStart,
+} from '@kbn/core-injected-metadata-browser';
import {
ChromeBadge,
ChromeBreadcrumb,
@@ -243,16 +247,8 @@ export interface CoreSetup unknown;
- };
+ /** {@link InjectedMetadataSetup} */
+ injectedMetadata: InjectedMetadataSetup;
/** {@link ThemeServiceSetup} */
theme: ThemeServiceSetup;
/** {@link StartServicesAccessor} */
@@ -309,16 +305,8 @@ export interface CoreStart {
deprecations: DeprecationsServiceStart;
/** {@link ThemeServiceStart} */
theme: ThemeServiceStart;
- /**
- * exposed temporarily until https://github.com/elastic/kibana/issues/41990 done
- * use *only* to retrieve config values. There is no way to set injected values
- * in the new platform.
- * @deprecated
- * @removeBy 8.8.0
- * */
- injectedMetadata: {
- getInjectedVar: (name: string, defaultValue?: any) => unknown;
- };
+ /** {@link InjectedMetadataStart} */
+ injectedMetadata: InjectedMetadataStart;
}
export type {
diff --git a/src/core/public/injected_metadata/injected_metadata_service.ts b/src/core/public/injected_metadata/injected_metadata_service.ts
deleted file mode 100644
index 2aa5bbc01de89..0000000000000
--- a/src/core/public/injected_metadata/injected_metadata_service.ts
+++ /dev/null
@@ -1,197 +0,0 @@
-/*
- * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
- * or more contributor license agreements. Licensed under the Elastic License
- * 2.0 and the Server Side Public License, v 1; you may not use this file except
- * in compliance with, at your election, the Elastic License 2.0 or the Server
- * Side Public License, v 1.
- */
-
-import { get } from 'lodash';
-import { deepFreeze } from '@kbn/std';
-import type { ThemeVersion } from '@kbn/ui-shared-deps-npm';
-import type { EnvironmentMode, PackageInfo } from '@kbn/config';
-import { DiscoveredPlugin, PluginName } from '../../server';
-import { IExternalUrlPolicy, UiSettingsParams, UserProvidedValues } from '../../server/types';
-import { AppCategory } from '..';
-
-export interface InjectedPluginMetadata {
- id: PluginName;
- plugin: DiscoveredPlugin;
- config?: {
- [key: string]: unknown;
- };
-}
-
-export interface InjectedMetadataClusterInfo {
- cluster_uuid?: string;
- cluster_name?: string;
- cluster_version?: string;
-}
-
-/** @internal */
-export interface InjectedMetadataParams {
- injectedMetadata: {
- version: string;
- buildNumber: number;
- branch: string;
- basePath: string;
- serverBasePath: string;
- publicBaseUrl: string;
- clusterInfo: InjectedMetadataClusterInfo;
- category?: AppCategory;
- csp: {
- warnLegacyBrowsers: boolean;
- };
- externalUrl: {
- policy: IExternalUrlPolicy[];
- };
- vars: {
- [key: string]: unknown;
- };
- theme: {
- darkMode: boolean;
- version: ThemeVersion;
- };
- env: {
- mode: Readonly;
- packageInfo: Readonly;
- };
- uiPlugins: InjectedPluginMetadata[];
- anonymousStatusPage: boolean;
- legacyMetadata: {
- uiSettings: {
- defaults: Record;
- user?: Record;
- };
- };
- };
-}
-
-/**
- * Provides access to the metadata that is injected by the
- * server into the page. The metadata is actually defined
- * in the entry file for the bundle containing the new platform
- * and is read from the DOM in most cases.
- *
- * @internal
- */
-export class InjectedMetadataService {
- private state: InjectedMetadataParams['injectedMetadata'];
-
- constructor(private readonly params: InjectedMetadataParams) {
- this.state = deepFreeze(
- this.params.injectedMetadata
- ) as InjectedMetadataParams['injectedMetadata'];
- }
-
- public start(): InjectedMetadataStart {
- return this.setup();
- }
-
- public setup(): InjectedMetadataSetup {
- return {
- getBasePath: () => {
- return this.state.basePath;
- },
-
- getServerBasePath: () => {
- return this.state.serverBasePath;
- },
-
- getPublicBaseUrl: () => {
- return this.state.publicBaseUrl;
- },
-
- getAnonymousStatusPage: () => {
- return this.state.anonymousStatusPage;
- },
-
- getKibanaVersion: () => {
- return this.state.version;
- },
-
- getCspConfig: () => {
- return this.state.csp;
- },
-
- getExternalUrlConfig: () => {
- return this.state.externalUrl;
- },
-
- getPlugins: () => {
- return this.state.uiPlugins;
- },
-
- getLegacyMetadata: () => {
- return this.state.legacyMetadata;
- },
-
- getInjectedVar: (name: string, defaultValue?: any): unknown => {
- return get(this.state.vars, name, defaultValue);
- },
-
- getInjectedVars: () => {
- return this.state.vars;
- },
-
- getKibanaBuildNumber: () => {
- return this.state.buildNumber;
- },
-
- getKibanaBranch: () => {
- return this.state.branch;
- },
-
- getTheme: () => {
- return this.state.theme;
- },
-
- getElasticsearchInfo: () => {
- return this.state.clusterInfo;
- },
- };
- }
-}
-
-/**
- * Provides access to the metadata injected by the server into the page
- *
- * @internal
- */
-export interface InjectedMetadataSetup {
- getBasePath: () => string;
- getServerBasePath: () => string;
- getPublicBaseUrl: () => string;
- getKibanaBuildNumber: () => number;
- getKibanaBranch: () => string;
- getKibanaVersion: () => string;
- getCspConfig: () => {
- warnLegacyBrowsers: boolean;
- };
- getExternalUrlConfig: () => {
- policy: IExternalUrlPolicy[];
- };
- getTheme: () => {
- darkMode: boolean;
- version: ThemeVersion;
- };
- getElasticsearchInfo: () => InjectedMetadataClusterInfo;
- /**
- * An array of frontend plugins in topological order.
- */
- getPlugins: () => InjectedPluginMetadata[];
- getAnonymousStatusPage: () => boolean;
- getLegacyMetadata: () => {
- uiSettings: {
- defaults: Record;
- user?: Record | undefined;
- };
- };
- getInjectedVar: (name: string, defaultValue?: any) => unknown;
- getInjectedVars: () => {
- [key: string]: unknown;
- };
-}
-
-/** @internal */
-export type InjectedMetadataStart = InjectedMetadataSetup;
diff --git a/src/core/public/mocks.ts b/src/core/public/mocks.ts
index 4dbb00967822f..38ae84c59c095 100644
--- a/src/core/public/mocks.ts
+++ b/src/core/public/mocks.ts
@@ -8,6 +8,7 @@
import { createMemoryHistory } from 'history';
import type { CoreContext } from '@kbn/core-base-browser-internal';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
// Only import types from '.' to avoid triggering default Jest mocks.
import { PluginInitializerContext, AppMountParameters } from '.';
@@ -25,11 +26,11 @@ import { notificationServiceMock } from './notifications/notifications_service.m
import { overlayServiceMock } from './overlays/overlay_service.mock';
import { uiSettingsServiceMock } from './ui_settings/ui_settings_service.mock';
import { savedObjectsServiceMock } from './saved_objects/saved_objects_service.mock';
-import { injectedMetadataServiceMock } from './injected_metadata/injected_metadata_service.mock';
import { deprecationsServiceMock } from './deprecations/deprecations_service.mock';
import { themeServiceMock } from './theme/theme_service.mock';
import { executionContextServiceMock } from './execution_context/execution_context_service.mock';
+export { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
export { chromeServiceMock } from './chrome/chrome_service.mock';
export { docLinksServiceMock } from './doc_links/doc_links_service.mock';
export { executionContextServiceMock } from './execution_context/execution_context_service.mock';
@@ -37,7 +38,6 @@ export { analyticsServiceMock } from './analytics/analytics_service.mock';
export { fatalErrorsServiceMock } from './fatal_errors/fatal_errors_service.mock';
export { httpServiceMock } from './http/http_service.mock';
export { i18nServiceMock } from './i18n/i18n_service.mock';
-export { injectedMetadataServiceMock } from './injected_metadata/injected_metadata_service.mock';
export { notificationServiceMock } from './notifications/notifications_service.mock';
export { overlayServiceMock } from './overlays/overlay_service.mock';
export { uiSettingsServiceMock } from './ui_settings/ui_settings_service.mock';
diff --git a/src/core/public/notifications/notifications_service.mock.ts b/src/core/public/notifications/notifications_service.mock.ts
index ddb5e74c96c77..01a2a4a1d5ac9 100644
--- a/src/core/public/notifications/notifications_service.mock.ts
+++ b/src/core/public/notifications/notifications_service.mock.ts
@@ -7,7 +7,7 @@
*/
import type { PublicMethodsOf } from '@kbn/utility-types';
-import type { MockedKeys } from '@kbn/utility-types/jest';
+import type { MockedKeys } from '@kbn/utility-types-jest';
import {
NotificationsService,
NotificationsSetup,
diff --git a/src/core/public/overlays/overlay_service.mock.ts b/src/core/public/overlays/overlay_service.mock.ts
index 192ea46ab7466..1769dd8265f9a 100644
--- a/src/core/public/overlays/overlay_service.mock.ts
+++ b/src/core/public/overlays/overlay_service.mock.ts
@@ -7,7 +7,7 @@
*/
import type { PublicMethodsOf } from '@kbn/utility-types';
-import type { DeeplyMockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
import { OverlayService, OverlayStart } from './overlay_service';
import { overlayBannersServiceMock } from './banners/banners_service.mock';
import { overlayFlyoutServiceMock } from './flyout/flyout_service.mock';
diff --git a/src/core/public/plugins/plugins_service.test.ts b/src/core/public/plugins/plugins_service.test.ts
index c8438a03fcec7..6531e38e3196a 100644
--- a/src/core/public/plugins/plugins_service.test.ts
+++ b/src/core/public/plugins/plugins_service.test.ts
@@ -21,7 +21,7 @@ import {
PluginsServiceSetupDeps,
} from './plugins_service';
-import { InjectedPluginMetadata } from '../injected_metadata';
+import type { InjectedMetadataPlugin } from '@kbn/core-injected-metadata-common-internal';
import { notificationServiceMock } from '../notifications/notifications_service.mock';
import { applicationServiceMock } from '../application/application_service.mock';
import { i18nServiceMock } from '../i18n/i18n_service.mock';
@@ -29,7 +29,7 @@ import { overlayServiceMock } from '../overlays/overlay_service.mock';
import { chromeServiceMock } from '../chrome/chrome_service.mock';
import { fatalErrorsServiceMock } from '../fatal_errors/fatal_errors_service.mock';
import { uiSettingsServiceMock } from '../ui_settings/ui_settings_service.mock';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { httpServiceMock } from '../http/http_service.mock';
import { CoreSetup, CoreStart, PluginInitializerContext } from '..';
import { docLinksServiceMock } from '../doc_links/doc_links_service.mock';
@@ -45,7 +45,7 @@ mockPluginInitializerProvider.mockImplementation(
(pluginName) => mockPluginInitializers.get(pluginName)!
);
-let plugins: InjectedPluginMetadata[];
+let plugins: InjectedMetadataPlugin[];
type DeeplyMocked = { [P in keyof T]: jest.Mocked };
diff --git a/src/core/public/plugins/plugins_service.ts b/src/core/public/plugins/plugins_service.ts
index 9c2ebe9e85640..dc0a77d096ff4 100644
--- a/src/core/public/plugins/plugins_service.ts
+++ b/src/core/public/plugins/plugins_service.ts
@@ -8,6 +8,7 @@
import type { CoreService, CoreContext } from '@kbn/core-base-browser-internal';
import type { PluginName, PluginOpaqueId } from '@kbn/core-base-common';
+import type { InjectedMetadataPlugin } from '@kbn/core-injected-metadata-common-internal';
import { PluginWrapper } from './plugin';
import {
createPluginInitializerContext,
@@ -15,7 +16,6 @@ import {
createPluginStartContext,
} from './plugin_context';
import { InternalCoreSetup, InternalCoreStart } from '../core_system';
-import { InjectedPluginMetadata } from '../injected_metadata';
/** @internal */
export type PluginsServiceSetupDeps = InternalCoreSetup;
@@ -26,6 +26,7 @@ export type PluginsServiceStartDeps = InternalCoreStart;
export interface PluginsServiceSetup {
contracts: ReadonlyMap;
}
+
/** @internal */
export interface PluginsServiceStart {
contracts: ReadonlyMap;
@@ -44,7 +45,7 @@ export class PluginsService implements CoreService(plugins.map((p) => [p.id, Symbol(p.id)]));
diff --git a/src/core/public/public.api.md b/src/core/public/public.api.md
index 132fd586b74e7..e655ccc210303 100644
--- a/src/core/public/public.api.md
+++ b/src/core/public/public.api.md
@@ -9,7 +9,6 @@
import { Action } from 'history';
import { AnalyticsClient } from '@kbn/analytics-client';
import type { ButtonColor } from '@elastic/eui';
-import type { ConfigPath } from '@kbn/config';
import { ContextProviderOpts } from '@kbn/analytics-client';
import { CoreContext } from '@kbn/core-base-browser-internal';
import type { DocLinks } from '@kbn/doc-links';
@@ -28,6 +27,9 @@ import { EventTypeOpts } from '@kbn/analytics-client';
import { History as History_2 } from 'history';
import { Href } from 'history';
import { IconType } from '@elastic/eui';
+import { InjectedMetadataParams } from '@kbn/core-injected-metadata-browser-internal';
+import type { InjectedMetadataSetup } from '@kbn/core-injected-metadata-browser';
+import type { InjectedMetadataStart } from '@kbn/core-injected-metadata-browser';
import { IShipper } from '@kbn/analytics-client';
import { Location as Location_2 } from 'history';
import { LocationDescriptorObject } from 'history';
@@ -36,7 +38,6 @@ import { Observable } from 'rxjs';
import { OptInConfig } from '@kbn/analytics-client';
import { PackageInfo } from '@kbn/config';
import { Path } from 'history';
-import { PluginName } from '@kbn/core-base-common';
import { PluginOpaqueId } from '@kbn/core-base-common';
import { default as React_2 } from 'react';
import { RecursiveReadonly } from '@kbn/utility-types';
@@ -44,7 +45,6 @@ import * as Rx from 'rxjs';
import { ShipperClassConstructor } from '@kbn/analytics-client';
import { TelemetryCounter } from '@kbn/analytics-client';
import { TelemetryCounterType } from '@kbn/analytics-client';
-import type { ThemeVersion } from '@kbn/ui-shared-deps-npm';
import { TransitionPromptHook } from 'history';
import { Type } from '@kbn/config-schema';
import { UiCounterMetricType } from '@kbn/analytics';
@@ -407,10 +407,10 @@ export interface CoreSetup;
// (undocumented)
http: HttpSetup;
- // @deprecated
- injectedMetadata: {
- getInjectedVar: (name: string, defaultValue?: any) => unknown;
- };
+ // Warning: (ae-unresolved-link) The @link reference could not be resolved: The package "kibana" does not have an export "InjectedMetadataSetup"
+ //
+ // (undocumented)
+ injectedMetadata: InjectedMetadataSetup;
// (undocumented)
notifications: NotificationsSetup;
// (undocumented)
@@ -439,10 +439,10 @@ export interface CoreStart {
http: HttpStart;
// (undocumented)
i18n: I18nStart;
- // @deprecated
- injectedMetadata: {
- getInjectedVar: (name: string, defaultValue?: any) => unknown;
- };
+ // Warning: (ae-unresolved-link) The @link reference could not be resolved: The package "kibana" does not have an export "InjectedMetadataStart"
+ //
+ // (undocumented)
+ injectedMetadata: InjectedMetadataStart;
// (undocumented)
notifications: NotificationsStart;
// (undocumented)
diff --git a/src/core/public/theme/theme_service.test.ts b/src/core/public/theme/theme_service.test.ts
index d38ef98735a3d..9bc01dc3af26e 100644
--- a/src/core/public/theme/theme_service.test.ts
+++ b/src/core/public/theme/theme_service.test.ts
@@ -7,7 +7,7 @@
*/
import { take } from 'rxjs/operators';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { ThemeService } from './theme_service';
describe('ThemeService', () => {
diff --git a/src/core/public/theme/theme_service.ts b/src/core/public/theme/theme_service.ts
index e70bd901cde80..f648727789580 100644
--- a/src/core/public/theme/theme_service.ts
+++ b/src/core/public/theme/theme_service.ts
@@ -8,11 +8,11 @@
import { Subject, Observable, of } from 'rxjs';
import { shareReplay, takeUntil } from 'rxjs/operators';
-import { InjectedMetadataSetup } from '../injected_metadata';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import type { CoreTheme, ThemeServiceSetup, ThemeServiceStart } from './types';
export interface SetupDeps {
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
export class ThemeService {
diff --git a/src/core/public/ui_settings/ui_settings_service.test.ts b/src/core/public/ui_settings/ui_settings_service.test.ts
index 83b2c26091bfd..b4af250e5b69f 100644
--- a/src/core/public/ui_settings/ui_settings_service.test.ts
+++ b/src/core/public/ui_settings/ui_settings_service.test.ts
@@ -9,7 +9,7 @@
import * as Rx from 'rxjs';
import { httpServiceMock } from '../http/http_service.mock';
-import { injectedMetadataServiceMock } from '../injected_metadata/injected_metadata_service.mock';
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { UiSettingsService } from './ui_settings_service';
const httpSetup = httpServiceMock.createSetupContract();
diff --git a/src/core/public/ui_settings/ui_settings_service.ts b/src/core/public/ui_settings/ui_settings_service.ts
index 1a3f275aa31ed..17a5f189146fb 100644
--- a/src/core/public/ui_settings/ui_settings_service.ts
+++ b/src/core/public/ui_settings/ui_settings_service.ts
@@ -8,8 +8,8 @@
import { Subject } from 'rxjs';
+import type { InternalInjectedMetadataSetup } from '@kbn/core-injected-metadata-browser-internal';
import { HttpSetup } from '../http';
-import { InjectedMetadataSetup } from '../injected_metadata';
import { UiSettingsApi } from './ui_settings_api';
import { UiSettingsClient } from './ui_settings_client';
@@ -17,7 +17,7 @@ import { IUiSettingsClient } from './types';
export interface UiSettingsServiceDeps {
http: HttpSetup;
- injectedMetadata: InjectedMetadataSetup;
+ injectedMetadata: InternalInjectedMetadataSetup;
}
/** @internal */
diff --git a/src/core/server/core_context.mock.ts b/src/core/server/core_context.mock.ts
index 872054837f73d..cf48a05bc82fb 100644
--- a/src/core/server/core_context.mock.ts
+++ b/src/core/server/core_context.mock.ts
@@ -7,7 +7,7 @@
*/
import { REPO_ROOT } from '@kbn/utils';
-import type { DeeplyMockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
import { Env, IConfigService } from '@kbn/config';
import { configServiceMock, getEnvOptions } from '@kbn/config-mocks';
import type { CoreContext } from '@kbn/core-base-server-internal';
diff --git a/src/core/server/index.ts b/src/core/server/index.ts
index b11c571c44880..9ad6612f979c6 100644
--- a/src/core/server/index.ts
+++ b/src/core/server/index.ts
@@ -261,7 +261,7 @@ export type {
LogLevel,
} from '@kbn/logging';
-export { PluginType } from './plugins';
+export { PluginType } from '@kbn/core-base-common';
export type {
DiscoveredPlugin,
diff --git a/src/core/server/logging/appenders/rewrite/rewrite_appender.test.ts b/src/core/server/logging/appenders/rewrite/rewrite_appender.test.ts
index f4ce64ee65075..47042f5646a09 100644
--- a/src/core/server/logging/appenders/rewrite/rewrite_appender.test.ts
+++ b/src/core/server/logging/appenders/rewrite/rewrite_appender.test.ts
@@ -6,7 +6,7 @@
* Side Public License, v 1.
*/
-import type { MockedKeys } from '@kbn/utility-types/jest';
+import type { MockedKeys } from '@kbn/utility-types-jest';
import { createRewritePolicyMock, resetAllMocks } from './rewrite_appender.test.mocks';
import { rewriteAppenderMocks } from './mocks';
import { LogLevel, LogRecord, LogMeta, DisposableAppender } from '@kbn/logging';
diff --git a/src/core/server/mocks.ts b/src/core/server/mocks.ts
index 45fed1fd54dbb..aedf80ca552bd 100644
--- a/src/core/server/mocks.ts
+++ b/src/core/server/mocks.ts
@@ -10,7 +10,7 @@ import { of } from 'rxjs';
import { duration } from 'moment';
import { ByteSizeValue } from '@kbn/config-schema';
import { isPromise } from '@kbn/std';
-import type { MockedKeys } from '@kbn/utility-types/jest';
+import type { MockedKeys } from '@kbn/utility-types-jest';
import type {
PluginInitializerContext,
CoreSetup,
diff --git a/src/core/server/plugins/discovery/plugin_manifest_parser.ts b/src/core/server/plugins/discovery/plugin_manifest_parser.ts
index bf78842173136..0a54899856ac1 100644
--- a/src/core/server/plugins/discovery/plugin_manifest_parser.ts
+++ b/src/core/server/plugins/discovery/plugin_manifest_parser.ts
@@ -12,7 +12,8 @@ import { coerce } from 'semver';
import { promisify } from 'util';
import { snakeCase } from 'lodash';
import { isConfigPath, PackageInfo } from '@kbn/config';
-import { PluginManifest, PluginType } from '../types';
+import { PluginType } from '@kbn/core-base-common';
+import { PluginManifest } from '../types';
import { PluginDiscoveryError } from './plugin_discovery_error';
import { isCamelCase } from './is_camel_case';
diff --git a/src/core/server/plugins/plugin.ts b/src/core/server/plugins/plugin.ts
index 2dbb82accc14e..9ddab175d313a 100644
--- a/src/core/server/plugins/plugin.ts
+++ b/src/core/server/plugins/plugin.ts
@@ -12,7 +12,7 @@ import { firstValueFrom, Subject } from 'rxjs';
import { isPromise } from '@kbn/std';
import { isConfigSchema } from '@kbn/config-schema';
import type { Logger } from '@kbn/logging';
-
+import { PluginType } from '@kbn/core-base-common';
import {
AsyncPlugin,
Plugin,
@@ -21,7 +21,6 @@ import {
PluginInitializerContext,
PluginManifest,
PluginOpaqueId,
- PluginType,
PrebootPlugin,
} from './types';
import { CorePreboot, CoreSetup, CoreStart } from '..';
diff --git a/src/core/server/plugins/types.ts b/src/core/server/plugins/types.ts
index 474c7cf08b2c8..c6088e129de73 100644
--- a/src/core/server/plugins/types.ts
+++ b/src/core/server/plugins/types.ts
@@ -17,7 +17,7 @@ import type {
PackageInfo,
ConfigDeprecationProvider,
} from '@kbn/config';
-import type { PluginName, PluginOpaqueId } from '@kbn/core-base-common';
+import type { PluginName, PluginOpaqueId, PluginType } from '@kbn/core-base-common';
import { ElasticsearchConfigType } from '../elasticsearch/elasticsearch_config';
import { SavedObjectsConfigType } from '../saved_objects/saved_objects_config';
@@ -26,7 +26,8 @@ import { CorePreboot, CoreSetup, CoreStart } from '..';
type Maybe = T | undefined;
// re-exporting for now to avoid adapting all imports, will be removed later on in the migration process
-export type { PluginName, PluginOpaqueId } from '@kbn/core-base-common';
+export type { PluginName, PluginOpaqueId, DiscoveredPlugin } from '@kbn/core-base-common';
+export { PluginType } from '@kbn/core-base-common';
/**
* Dedicated type for plugin configuration schema.
@@ -130,18 +131,6 @@ export type MakeUsageFromSchema = {
: boolean;
};
-/** @public */
-export enum PluginType {
- /**
- * Preboot plugins are special-purpose plugins that only function during preboot stage.
- */
- preboot = 'preboot',
- /**
- * Standard plugins are plugins that start to function as soon as Kibana is fully booted and are active until it shuts down.
- */
- standard = 'standard',
-}
-
/** @internal */
export interface PluginDependencies {
asNames: ReadonlyMap;
@@ -265,59 +254,6 @@ export interface PluginManifest {
readonly enabledOnAnonymousPages?: boolean;
}
-/**
- * Small container object used to expose information about discovered plugins that may
- * or may not have been started.
- * @public
- */
-export interface DiscoveredPlugin {
- /**
- * Identifier of the plugin.
- */
- readonly id: PluginName;
-
- /**
- * Root configuration path used by the plugin, defaults to "id" in snake_case format.
- */
- readonly configPath: ConfigPath;
-
- /**
- * Type of the plugin, defaults to `standard`.
- */
- readonly type: PluginType;
-
- /**
- * An optional list of the other plugins that **must be** installed and enabled
- * for this plugin to function properly.
- */
- readonly requiredPlugins: readonly PluginName[];
-
- /**
- * An optional list of the other plugins that if installed and enabled **may be**
- * leveraged by this plugin for some additional functionality but otherwise are
- * not required for this plugin to work properly.
- */
- readonly optionalPlugins: readonly PluginName[];
-
- /**
- * List of plugin ids that this plugin's UI code imports modules from that are
- * not in `requiredPlugins`.
- *
- * @remarks
- * The plugins listed here will be loaded in the browser, even if the plugin is
- * disabled. Required by `@kbn/optimizer` to support cross-plugin imports.
- * "core" and plugins already listed in `requiredPlugins` do not need to be
- * duplicated here.
- */
- readonly requiredBundles: readonly PluginName[];
-
- /**
- * Specifies whether this plugin - and its required dependencies - will be enabled for anonymous pages (login page, status page when
- * configured, etc.) Default is false.
- */
- readonly enabledOnAnonymousPages?: boolean;
-}
-
/**
* @internal
*/
diff --git a/src/core/server/rendering/index.ts b/src/core/server/rendering/index.ts
index ce38cfab16de0..6cf0e2a74aa1f 100644
--- a/src/core/server/rendering/index.ts
+++ b/src/core/server/rendering/index.ts
@@ -8,7 +8,6 @@
export { RenderingService } from './rendering_service';
export type {
- InjectedMetadata,
InternalRenderingServicePreboot,
InternalRenderingServiceSetup,
IRenderOptions,
diff --git a/src/core/server/rendering/types.ts b/src/core/server/rendering/types.ts
index 09e54c362f4f1..c665df5d84d73 100644
--- a/src/core/server/rendering/types.ts
+++ b/src/core/server/rendering/types.ts
@@ -8,15 +8,13 @@
import { i18n } from '@kbn/i18n';
import type { ThemeVersion } from '@kbn/ui-shared-deps-npm';
-
-import type { EnvironmentMode, PackageInfo } from '@kbn/config';
+import type { InjectedMetadata } from '@kbn/core-injected-metadata-common-internal';
import { InternalElasticsearchServiceSetup } from '../elasticsearch';
import { ICspConfig } from '../csp';
import { InternalHttpServicePreboot, InternalHttpServiceSetup, KibanaRequest } from '../http';
-import { UiPlugins, DiscoveredPlugin } from '../plugins';
-import { IUiSettingsClient, UserProvidedValues } from '../ui_settings';
+import { UiPlugins } from '../plugins';
+import { IUiSettingsClient } from '../ui_settings';
import type { InternalStatusServiceSetup } from '../status';
-import { IExternalUrlPolicy } from '../external_url';
/** @internal */
export interface RenderingMetadata {
@@ -31,47 +29,6 @@ export interface RenderingMetadata {
injectedMetadata: InjectedMetadata;
}
-/** @internal */
-export interface InjectedMetadata {
- version: string;
- buildNumber: number;
- branch: string;
- basePath: string;
- serverBasePath: string;
- publicBaseUrl?: string;
- clusterInfo: {
- cluster_uuid?: string;
- cluster_name?: string;
- cluster_version?: string;
- };
- env: {
- mode: EnvironmentMode;
- packageInfo: PackageInfo;
- };
- anonymousStatusPage: boolean;
- i18n: {
- translationsUrl: string;
- };
- theme: {
- darkMode: boolean;
- version: ThemeVersion;
- };
- csp: Pick;
- externalUrl: { policy: IExternalUrlPolicy[] };
- vars: Record;
- uiPlugins: Array<{
- id: string;
- plugin: DiscoveredPlugin;
- config?: Record;
- }>;
- legacyMetadata: {
- uiSettings: {
- defaults: Record;
- user: Record>;
- };
- };
-}
-
/** @internal */
export interface RenderingPrebootDeps {
http: InternalHttpServicePreboot;
diff --git a/src/core/server/saved_objects/deprecations/unknown_object_types.test.ts b/src/core/server/saved_objects/deprecations/unknown_object_types.test.ts
index 533ada6078caa..84a8d94f8bf2c 100644
--- a/src/core/server/saved_objects/deprecations/unknown_object_types.test.ts
+++ b/src/core/server/saved_objects/deprecations/unknown_object_types.test.ts
@@ -8,21 +8,11 @@
import { getIndexForTypeMock } from './unknown_object_types.test.mocks';
-import type * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { deleteUnknownTypeObjects, getUnknownTypesDeprecations } from './unknown_object_types';
import { typeRegistryMock } from '../saved_objects_type_registry.mock';
import { elasticsearchClientMock } from '../../elasticsearch/client/mocks';
import { SavedObjectsType } from '../..';
-
-const createSearchResponse = (count: number): estypes.SearchResponse => {
- return {
- hits: {
- total: count,
- max_score: 0,
- hits: new Array(count).fill({}),
- },
- } as estypes.SearchResponse;
-};
+import { createAggregateTypesSearchResponse } from '../migrations/actions/check_for_unknown_docs.mocks';
describe('unknown saved object types deprecation', () => {
const kibanaVersion = '8.0.0';
@@ -48,7 +38,7 @@ describe('unknown saved object types deprecation', () => {
describe('getUnknownTypesDeprecations', () => {
beforeEach(() => {
- esClient.asInternalUser.search.mockResponse(createSearchResponse(0));
+ esClient.asInternalUser.search.mockResponse(createAggregateTypesSearchResponse());
});
it('calls `esClient.asInternalUser.search` with the correct parameters', async () => {
@@ -62,19 +52,36 @@ describe('unknown saved object types deprecation', () => {
expect(esClient.asInternalUser.search).toHaveBeenCalledTimes(1);
expect(esClient.asInternalUser.search).toHaveBeenCalledWith({
index: ['foo-index', 'bar-index'],
- body: {
- size: 10000,
- query: {
- bool: {
- must_not: [{ term: { type: 'foo' } }, { term: { type: 'bar' } }],
+ size: 0,
+ aggs: {
+ typesAggregation: {
+ terms: {
+ missing: '__UNKNOWN__',
+ field: 'type',
+ size: 1000,
},
+ aggs: {
+ docs: {
+ top_hits: {
+ size: 100,
+ _source: {
+ excludes: ['*'],
+ },
+ },
+ },
+ },
+ },
+ },
+ query: {
+ bool: {
+ must_not: [{ term: { type: 'foo' } }, { term: { type: 'bar' } }],
},
},
});
});
it('returns no deprecation if no unknown type docs are found', async () => {
- esClient.asInternalUser.search.mockResponse(createSearchResponse(0));
+ esClient.asInternalUser.search.mockResponse(createAggregateTypesSearchResponse());
const deprecations = await getUnknownTypesDeprecations({
esClient,
@@ -87,7 +94,13 @@ describe('unknown saved object types deprecation', () => {
});
it('returns a deprecation if any unknown type docs are found', async () => {
- esClient.asInternalUser.search.mockResponse(createSearchResponse(1));
+ esClient.asInternalUser.search.mockResponse(
+ createAggregateTypesSearchResponse({
+ someType: ['id1', 'id2'],
+ anotherType: ['id3'],
+ __UNKNOWN__: ['id4'],
+ })
+ );
const deprecations = await getUnknownTypesDeprecations({
esClient,
@@ -125,6 +138,7 @@ describe('unknown saved object types deprecation', () => {
});
expect(esClient.asInternalUser.deleteByQuery).toHaveBeenCalledTimes(1);
+
expect(esClient.asInternalUser.deleteByQuery).toHaveBeenCalledWith({
index: ['foo-index', 'bar-index'],
wait_for_completion: false,
diff --git a/src/core/server/saved_objects/deprecations/unknown_object_types.ts b/src/core/server/saved_objects/deprecations/unknown_object_types.ts
index 28dbc4dcd41fa..9e48c84860b48 100644
--- a/src/core/server/saved_objects/deprecations/unknown_object_types.ts
+++ b/src/core/server/saved_objects/deprecations/unknown_object_types.ts
@@ -6,12 +6,12 @@
* Side Public License, v 1.
*/
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { i18n } from '@kbn/i18n';
import type { DeprecationsDetails } from '../../deprecations';
import { IScopedClusterClient } from '../../elasticsearch';
+import { getAggregatedTypesDocuments } from '../migrations/actions/check_for_unknown_docs';
+import { addExcludedTypesToBoolQuery } from '../migrations/model/helpers';
import { ISavedObjectTypeRegistry } from '../saved_objects_type_registry';
-import { SavedObjectsRawDocSource } from '../serialization';
import { getIndexForType } from '../service/lib';
interface UnknownTypesDeprecationOptions {
@@ -49,16 +49,6 @@ const getTargetIndices = ({
];
};
-const getUnknownTypesQuery = (knownTypes: string[]): estypes.QueryDslQueryContainer => {
- return {
- bool: {
- must_not: knownTypes.map((type) => ({
- term: { type },
- })),
- },
- };
-};
-
const getUnknownSavedObjects = async ({
typeRegistry,
esClient,
@@ -72,18 +62,12 @@ const getUnknownSavedObjects = async ({
kibanaIndex,
kibanaVersion,
});
- const query = getUnknownTypesQuery(knownTypes);
-
- const body = await esClient.asInternalUser.search({
- index: targetIndices,
- body: {
- size: 10000,
- query,
- },
- });
- const { hits: unknownDocs } = body.hits;
-
- return unknownDocs.map((doc) => ({ id: doc._id, type: doc._source?.type ?? 'unknown' }));
+ const excludeRegisteredTypes = addExcludedTypesToBoolQuery(knownTypes);
+ return await getAggregatedTypesDocuments(
+ esClient.asInternalUser,
+ targetIndices,
+ excludeRegisteredTypes
+ );
};
export const getUnknownTypesDeprecations = async (
@@ -149,13 +133,13 @@ export const deleteUnknownTypeObjects = async ({
kibanaIndex,
kibanaVersion,
});
- const query = getUnknownTypesQuery(knownTypes);
+ const nonRegisteredTypesQuery = addExcludedTypesToBoolQuery(knownTypes);
await esClient.asInternalUser.deleteByQuery({
index: targetIndices,
wait_for_completion: false,
body: {
- query,
+ query: nonRegisteredTypesQuery,
},
});
};
diff --git a/src/core/server/saved_objects/migrations/__snapshots__/migrations_state_action_machine.test.ts.snap b/src/core/server/saved_objects/migrations/__snapshots__/migrations_state_action_machine.test.ts.snap
index 971e2d9129d47..a812c07ba3786 100644
--- a/src/core/server/saved_objects/migrations/__snapshots__/migrations_state_action_machine.test.ts.snap
+++ b/src/core/server/saved_objects/migrations/__snapshots__/migrations_state_action_machine.test.ts.snap
@@ -20,54 +20,9 @@ Object {
"batchSize": 1000,
"controlState": "LEGACY_REINDEX",
"currentAlias": ".my-so-index",
+ "discardUnknownObjects": false,
"excludeFromUpgradeFilterHooks": Object {},
- "indexPrefix": ".my-so-index",
- "kibanaVersion": "7.11.0",
- "knownTypes": Array [],
- "legacyIndex": ".my-so-index",
- "logs": Array [
- Object {
- "level": "info",
- "message": "Log from LEGACY_REINDEX control state",
- },
- ],
- "maxBatchSizeBytes": 100000000,
- "migrationDocLinks": Object {
- "clusterShardLimitExceeded": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#cluster-shard-limit-exceeded",
- "repeatedTimeoutRequests": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#_repeated_time_out_requests_that_eventually_fail",
- "resolveMigrationFailures": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html",
- "routingAllocationDisabled": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#routing-allocation-disabled",
- },
- "outdatedDocuments": Array [],
- "outdatedDocumentsQuery": Object {
- "bool": Object {
- "should": Array [],
- },
- },
- "preMigrationScript": Object {
- "_tag": "None",
- },
- "retryAttempts": 5,
- "retryCount": 0,
- "retryDelay": 0,
- "targetIndexMappings": Object {
- "properties": Object {},
- },
- "tempIndex": ".my-so-index_7.11.0_reindex_temp",
- "tempIndexMappings": Object {
- "dynamic": false,
- "properties": Object {
- "migrationVersion": Object {
- "dynamic": "true",
- "type": "object",
- },
- "type": Object {
- "type": "keyword",
- },
- },
- },
- "transformedDocBatches": Array [],
- "unusedTypesQuery": Object {
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -164,31 +119,6 @@ Object {
],
},
},
- "versionAlias": ".my-so-index_7.11.0",
- "versionIndex": ".my-so-index_7.11.0_001",
- },
- },
- },
- },
- ],
- Array [
- "[.my-so-index] LEGACY_REINDEX RESPONSE",
- Object {
- "_tag": "Right",
- "right": "response",
- },
- ],
- Array [
- "[.my-so-index] LEGACY_REINDEX -> LEGACY_DELETE. took: 0ms.",
- Object {
- "kibana": Object {
- "migrations": Object {
- "duration": 0,
- "state": Object {
- "batchSize": 1000,
- "controlState": "LEGACY_DELETE",
- "currentAlias": ".my-so-index",
- "excludeFromUpgradeFilterHooks": Object {},
"indexPrefix": ".my-so-index",
"kibanaVersion": "7.11.0",
"knownTypes": Array [],
@@ -198,10 +128,6 @@ Object {
"level": "info",
"message": "Log from LEGACY_REINDEX control state",
},
- Object {
- "level": "info",
- "message": "Log from LEGACY_DELETE control state",
- },
],
"maxBatchSizeBytes": 100000000,
"migrationDocLinks": Object {
@@ -239,7 +165,33 @@ Object {
},
},
"transformedDocBatches": Array [],
- "unusedTypesQuery": Object {
+ "versionAlias": ".my-so-index_7.11.0",
+ "versionIndex": ".my-so-index_7.11.0_001",
+ },
+ },
+ },
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_REINDEX RESPONSE",
+ Object {
+ "_tag": "Right",
+ "right": "response",
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_REINDEX -> LEGACY_DELETE. took: 0ms.",
+ Object {
+ "kibana": Object {
+ "migrations": Object {
+ "duration": 0,
+ "state": Object {
+ "batchSize": 1000,
+ "controlState": "LEGACY_DELETE",
+ "currentAlias": ".my-so-index",
+ "discardUnknownObjects": false,
+ "excludeFromUpgradeFilterHooks": Object {},
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -336,31 +288,6 @@ Object {
],
},
},
- "versionAlias": ".my-so-index_7.11.0",
- "versionIndex": ".my-so-index_7.11.0_001",
- },
- },
- },
- },
- ],
- Array [
- "[.my-so-index] LEGACY_DELETE RESPONSE",
- Object {
- "_tag": "Right",
- "right": "response",
- },
- ],
- Array [
- "[.my-so-index] LEGACY_DELETE -> LEGACY_DELETE. took: 0ms.",
- Object {
- "kibana": Object {
- "migrations": Object {
- "duration": 0,
- "state": Object {
- "batchSize": 1000,
- "controlState": "LEGACY_DELETE",
- "currentAlias": ".my-so-index",
- "excludeFromUpgradeFilterHooks": Object {},
"indexPrefix": ".my-so-index",
"kibanaVersion": "7.11.0",
"knownTypes": Array [],
@@ -374,10 +301,6 @@ Object {
"level": "info",
"message": "Log from LEGACY_DELETE control state",
},
- Object {
- "level": "info",
- "message": "Log from LEGACY_DELETE control state",
- },
],
"maxBatchSizeBytes": 100000000,
"migrationDocLinks": Object {
@@ -415,7 +338,33 @@ Object {
},
},
"transformedDocBatches": Array [],
- "unusedTypesQuery": Object {
+ "versionAlias": ".my-so-index_7.11.0",
+ "versionIndex": ".my-so-index_7.11.0_001",
+ },
+ },
+ },
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_DELETE RESPONSE",
+ Object {
+ "_tag": "Right",
+ "right": "response",
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_DELETE -> LEGACY_DELETE. took: 0ms.",
+ Object {
+ "kibana": Object {
+ "migrations": Object {
+ "duration": 0,
+ "state": Object {
+ "batchSize": 1000,
+ "controlState": "LEGACY_DELETE",
+ "currentAlias": ".my-so-index",
+ "discardUnknownObjects": false,
+ "excludeFromUpgradeFilterHooks": Object {},
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -512,31 +461,6 @@ Object {
],
},
},
- "versionAlias": ".my-so-index_7.11.0",
- "versionIndex": ".my-so-index_7.11.0_001",
- },
- },
- },
- },
- ],
- Array [
- "[.my-so-index] LEGACY_DELETE RESPONSE",
- Object {
- "_tag": "Right",
- "right": "response",
- },
- ],
- Array [
- "[.my-so-index] LEGACY_DELETE -> DONE. took: 0ms.",
- Object {
- "kibana": Object {
- "migrations": Object {
- "duration": 0,
- "state": Object {
- "batchSize": 1000,
- "controlState": "DONE",
- "currentAlias": ".my-so-index",
- "excludeFromUpgradeFilterHooks": Object {},
"indexPrefix": ".my-so-index",
"kibanaVersion": "7.11.0",
"knownTypes": Array [],
@@ -554,10 +478,6 @@ Object {
"level": "info",
"message": "Log from LEGACY_DELETE control state",
},
- Object {
- "level": "info",
- "message": "Log from DONE control state",
- },
],
"maxBatchSizeBytes": 100000000,
"migrationDocLinks": Object {
@@ -595,7 +515,33 @@ Object {
},
},
"transformedDocBatches": Array [],
- "unusedTypesQuery": Object {
+ "versionAlias": ".my-so-index_7.11.0",
+ "versionIndex": ".my-so-index_7.11.0_001",
+ },
+ },
+ },
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_DELETE RESPONSE",
+ Object {
+ "_tag": "Right",
+ "right": "response",
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_DELETE -> DONE. took: 0ms.",
+ Object {
+ "kibana": Object {
+ "migrations": Object {
+ "duration": 0,
+ "state": Object {
+ "batchSize": 1000,
+ "controlState": "DONE",
+ "currentAlias": ".my-so-index",
+ "discardUnknownObjects": false,
+ "excludeFromUpgradeFilterHooks": Object {},
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -692,6 +638,64 @@ Object {
],
},
},
+ "indexPrefix": ".my-so-index",
+ "kibanaVersion": "7.11.0",
+ "knownTypes": Array [],
+ "legacyIndex": ".my-so-index",
+ "logs": Array [
+ Object {
+ "level": "info",
+ "message": "Log from LEGACY_REINDEX control state",
+ },
+ Object {
+ "level": "info",
+ "message": "Log from LEGACY_DELETE control state",
+ },
+ Object {
+ "level": "info",
+ "message": "Log from LEGACY_DELETE control state",
+ },
+ Object {
+ "level": "info",
+ "message": "Log from DONE control state",
+ },
+ ],
+ "maxBatchSizeBytes": 100000000,
+ "migrationDocLinks": Object {
+ "clusterShardLimitExceeded": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#cluster-shard-limit-exceeded",
+ "repeatedTimeoutRequests": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#_repeated_time_out_requests_that_eventually_fail",
+ "resolveMigrationFailures": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html",
+ "routingAllocationDisabled": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#routing-allocation-disabled",
+ },
+ "outdatedDocuments": Array [],
+ "outdatedDocumentsQuery": Object {
+ "bool": Object {
+ "should": Array [],
+ },
+ },
+ "preMigrationScript": Object {
+ "_tag": "None",
+ },
+ "retryAttempts": 5,
+ "retryCount": 0,
+ "retryDelay": 0,
+ "targetIndexMappings": Object {
+ "properties": Object {},
+ },
+ "tempIndex": ".my-so-index_7.11.0_reindex_temp",
+ "tempIndexMappings": Object {
+ "dynamic": false,
+ "properties": Object {
+ "migrationVersion": Object {
+ "dynamic": "true",
+ "type": "object",
+ },
+ "type": Object {
+ "type": "keyword",
+ },
+ },
+ },
+ "transformedDocBatches": Array [],
"versionAlias": ".my-so-index_7.11.0",
"versionIndex": ".my-so-index_7.11.0_001",
},
@@ -754,65 +758,9 @@ Object {
"batchSize": 1000,
"controlState": "LEGACY_DELETE",
"currentAlias": ".my-so-index",
+ "discardUnknownObjects": false,
"excludeFromUpgradeFilterHooks": Object {},
- "indexPrefix": ".my-so-index",
- "kibanaVersion": "7.11.0",
- "knownTypes": Array [],
- "legacyIndex": ".my-so-index",
- "logs": Array [
- Object {
- "level": "info",
- "message": "Log from LEGACY_DELETE control state",
- },
- ],
- "maxBatchSizeBytes": 100000000,
- "migrationDocLinks": Object {
- "clusterShardLimitExceeded": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#cluster-shard-limit-exceeded",
- "repeatedTimeoutRequests": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#_repeated_time_out_requests_that_eventually_fail",
- "resolveMigrationFailures": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html",
- "routingAllocationDisabled": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#routing-allocation-disabled",
- },
- "outdatedDocuments": Array [
- Object {
- "_id": "1234",
- },
- ],
- "outdatedDocumentsQuery": Object {
- "bool": Object {
- "should": Array [],
- },
- },
- "preMigrationScript": Object {
- "_tag": "None",
- },
- "reason": "the fatal reason",
- "retryAttempts": 5,
- "retryCount": 0,
- "retryDelay": 0,
- "targetIndexMappings": Object {
- "properties": Object {},
- },
- "tempIndex": ".my-so-index_7.11.0_reindex_temp",
- "tempIndexMappings": Object {
- "dynamic": false,
- "properties": Object {
- "migrationVersion": Object {
- "dynamic": "true",
- "type": "object",
- },
- "type": Object {
- "type": "keyword",
- },
- },
- },
- "transformedDocBatches": Array [
- Array [
- Object {
- "_id": "1234",
- },
- ],
- ],
- "unusedTypesQuery": Object {
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -909,31 +857,6 @@ Object {
],
},
},
- "versionAlias": ".my-so-index_7.11.0",
- "versionIndex": ".my-so-index_7.11.0_001",
- },
- },
- },
- },
- ],
- Array [
- "[.my-so-index] LEGACY_DELETE RESPONSE",
- Object {
- "_tag": "Right",
- "right": "response",
- },
- ],
- Array [
- "[.my-so-index] LEGACY_DELETE -> FATAL. took: 0ms.",
- Object {
- "kibana": Object {
- "migrations": Object {
- "duration": 0,
- "state": Object {
- "batchSize": 1000,
- "controlState": "FATAL",
- "currentAlias": ".my-so-index",
- "excludeFromUpgradeFilterHooks": Object {},
"indexPrefix": ".my-so-index",
"kibanaVersion": "7.11.0",
"knownTypes": Array [],
@@ -943,10 +866,6 @@ Object {
"level": "info",
"message": "Log from LEGACY_DELETE control state",
},
- Object {
- "level": "info",
- "message": "Log from FATAL control state",
- },
],
"maxBatchSizeBytes": 100000000,
"migrationDocLinks": Object {
@@ -995,7 +914,33 @@ Object {
},
],
],
- "unusedTypesQuery": Object {
+ "versionAlias": ".my-so-index_7.11.0",
+ "versionIndex": ".my-so-index_7.11.0_001",
+ },
+ },
+ },
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_DELETE RESPONSE",
+ Object {
+ "_tag": "Right",
+ "right": "response",
+ },
+ ],
+ Array [
+ "[.my-so-index] LEGACY_DELETE -> FATAL. took: 0ms.",
+ Object {
+ "kibana": Object {
+ "migrations": Object {
+ "duration": 0,
+ "state": Object {
+ "batchSize": 1000,
+ "controlState": "FATAL",
+ "currentAlias": ".my-so-index",
+ "discardUnknownObjects": false,
+ "excludeFromUpgradeFilterHooks": Object {},
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -1092,6 +1037,67 @@ Object {
],
},
},
+ "indexPrefix": ".my-so-index",
+ "kibanaVersion": "7.11.0",
+ "knownTypes": Array [],
+ "legacyIndex": ".my-so-index",
+ "logs": Array [
+ Object {
+ "level": "info",
+ "message": "Log from LEGACY_DELETE control state",
+ },
+ Object {
+ "level": "info",
+ "message": "Log from FATAL control state",
+ },
+ ],
+ "maxBatchSizeBytes": 100000000,
+ "migrationDocLinks": Object {
+ "clusterShardLimitExceeded": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#cluster-shard-limit-exceeded",
+ "repeatedTimeoutRequests": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#_repeated_time_out_requests_that_eventually_fail",
+ "resolveMigrationFailures": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html",
+ "routingAllocationDisabled": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#routing-allocation-disabled",
+ },
+ "outdatedDocuments": Array [
+ Object {
+ "_id": "1234",
+ },
+ ],
+ "outdatedDocumentsQuery": Object {
+ "bool": Object {
+ "should": Array [],
+ },
+ },
+ "preMigrationScript": Object {
+ "_tag": "None",
+ },
+ "reason": "the fatal reason",
+ "retryAttempts": 5,
+ "retryCount": 0,
+ "retryDelay": 0,
+ "targetIndexMappings": Object {
+ "properties": Object {},
+ },
+ "tempIndex": ".my-so-index_7.11.0_reindex_temp",
+ "tempIndexMappings": Object {
+ "dynamic": false,
+ "properties": Object {
+ "migrationVersion": Object {
+ "dynamic": "true",
+ "type": "object",
+ },
+ "type": Object {
+ "type": "keyword",
+ },
+ },
+ },
+ "transformedDocBatches": Array [
+ Array [
+ Object {
+ "_id": "1234",
+ },
+ ],
+ ],
"versionAlias": ".my-so-index_7.11.0",
"versionIndex": ".my-so-index_7.11.0_001",
},
diff --git a/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.test.ts b/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.test.ts
index 0f41233fd58f2..68bd6c934a80f 100644
--- a/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.test.ts
+++ b/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.test.ts
@@ -28,14 +28,10 @@ describe('calculateExcludeFilters', () => {
expect(hook2).toHaveBeenCalledWith({ readonlyEsClient: { search: expect.any(Function) } });
expect(Either.isRight(result)).toBe(true);
expect((result as Either.Right).right).toEqual({
- excludeFilter: {
- bool: {
- must_not: [
- { bool: { must: { term: { fieldA: '123' } } } },
- { bool: { must: { term: { fieldB: 'abc' } } } },
- ],
- },
- },
+ mustNotClauses: [
+ { bool: { must: { term: { fieldA: '123' } } } },
+ { bool: { must: { term: { fieldB: 'abc' } } } },
+ ],
errorsByType: {},
});
});
@@ -53,11 +49,7 @@ describe('calculateExcludeFilters', () => {
expect(Either.isRight(result)).toBe(true);
expect((result as Either.Right).right).toEqual({
- excludeFilter: {
- bool: {
- must_not: [{ bool: { must: { term: { fieldB: 'abc' } } } }],
- },
- },
+ mustNotClauses: [{ bool: { must: { term: { fieldB: 'abc' } } } }],
errorsByType: { type1: error },
});
});
@@ -99,11 +91,7 @@ describe('calculateExcludeFilters', () => {
expect(Either.isRight(result)).toBe(true);
expect((result as Either.Right).right).toEqual({
- excludeFilter: {
- bool: {
- must_not: [{ bool: { must: { term: { fieldB: 'abc' } } } }],
- },
- },
+ mustNotClauses: [{ bool: { must: { term: { fieldB: 'abc' } } } }],
errorsByType: expect.any(Object),
});
expect((result as Either.Right).right.errorsByType.type1.toString()).toMatchInlineSnapshot(
diff --git a/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.ts b/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.ts
index 27ce7bd4c404b..a2cea776792ad 100644
--- a/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.ts
+++ b/src/core/server/saved_objects/migrations/actions/calculate_exclude_filters.ts
@@ -6,13 +6,13 @@
* Side Public License, v 1.
*/
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
+import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import { withTimeout } from '@kbn/std';
import * as Either from 'fp-ts/lib/Either';
import * as TaskEither from 'fp-ts/lib/TaskEither';
-import { RetryableEsClientError } from '.';
-import { ElasticsearchClient } from '../../../elasticsearch';
-import { SavedObjectTypeExcludeFromUpgradeFilterHook } from '../../types';
+import type { RetryableEsClientError } from '.';
+import type { ElasticsearchClient } from '../../../elasticsearch';
+import type { SavedObjectTypeExcludeFromUpgradeFilterHook } from '../../types';
import { catchRetryableEsClientErrors } from './catch_retryable_es_client_errors';
export interface CalculateExcludeFiltersParams {
@@ -22,8 +22,8 @@ export interface CalculateExcludeFiltersParams {
}
export interface CalculatedExcludeFilter {
- /** Composite filter of all calculated filters */
- excludeFilter: estypes.QueryDslQueryContainer;
+ /** Array with all the clauses that must be bool.must_not'ed */
+ mustNotClauses: QueryDslQueryContainer[];
/** Any errors that were encountered during filter calculation, keyed by the type name */
errorsByType: Record;
}
@@ -39,7 +39,7 @@ export const calculateExcludeFilters =
> =>
() => {
return Promise.all<
- | Either.Right
+ | Either.Right
| Either.Left<{ soType: string; error: Error | RetryableEsClientError }>
>(
Object.entries(excludeFromUpgradeFilterHooks).map(([soType, hook]) =>
@@ -91,22 +91,17 @@ export const calculateExcludeFilters =
}
const errorsByType: Array<[string, Error]> = [];
- const filters: estypes.QueryDslQueryContainer[] = [];
+ const mustNotClauses: QueryDslQueryContainer[] = [];
// Loop through all results and collect successes and errors
results.forEach((r) =>
Either.isRight(r)
- ? filters.push(r.right)
+ ? mustNotClauses.push(r.right)
: Either.isLeft(r) && errorsByType.push([r.left.soType, r.left.error as Error])
);
- // Composite filter from all calculated filters that successfully executed
- const excludeFilter: estypes.QueryDslQueryContainer = {
- bool: { must_not: filters },
- };
-
return Either.right({
- excludeFilter,
+ mustNotClauses,
errorsByType: Object.fromEntries(errorsByType),
});
});
diff --git a/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.mocks.ts b/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.mocks.ts
new file mode 100644
index 0000000000000..a762a9f18e0b6
--- /dev/null
+++ b/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.mocks.ts
@@ -0,0 +1,42 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
+
+interface DocIdsByType {
+ [type: string]: string[];
+}
+
+export const createAggregateTypesSearchResponse = (typesIds: DocIdsByType = {}): SearchResponse => {
+ return {
+ took: 0,
+ timed_out: false,
+ _shards: {
+ total: 1,
+ successful: 1,
+ skipped: 0,
+ failed: 0,
+ },
+ hits: {
+ total: {
+ value: Object.keys(typesIds).length,
+ relation: 'eq',
+ },
+ max_score: null,
+ hits: [],
+ },
+ aggregations: {
+ typesAggregation: {
+ buckets: Object.entries(typesIds).map(([type, ids]) => ({
+ key: type,
+ docs: { hits: { hits: ids.map((_id) => ({ _id })) } },
+ })),
+ },
+ },
+ };
+};
diff --git a/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.test.ts b/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.test.ts
index 4254c152a1fa8..d5c99f01ada33 100644
--- a/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.test.ts
+++ b/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.test.ts
@@ -9,14 +9,15 @@
import * as Either from 'fp-ts/lib/Either';
import { catchRetryableEsClientErrors } from './catch_retryable_es_client_errors';
import { errors as EsErrors } from '@elastic/elasticsearch';
-import type * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
+import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import { elasticsearchClientMock } from '../../../elasticsearch/client/mocks';
import { checkForUnknownDocs } from './check_for_unknown_docs';
+import { createAggregateTypesSearchResponse } from './check_for_unknown_docs.mocks';
jest.mock('./catch_retryable_es_client_errors');
describe('checkForUnknownDocs', () => {
- const unusedTypesQuery: estypes.QueryDslQueryContainer = {
+ const excludeOnUpgradeQuery: QueryDslQueryContainer = {
bool: { must: [{ term: { hello: 'dolly' } }] },
};
const knownTypes = ['foo', 'bar'];
@@ -41,7 +42,7 @@ describe('checkForUnknownDocs', () => {
client,
indexName: '.kibana_8.0.0',
knownTypes,
- unusedTypesQuery,
+ excludeOnUpgradeQuery,
});
try {
await task();
@@ -60,7 +61,7 @@ describe('checkForUnknownDocs', () => {
client,
indexName: '.kibana_8.0.0',
knownTypes,
- unusedTypesQuery,
+ excludeOnUpgradeQuery,
});
await task();
@@ -68,18 +69,37 @@ describe('checkForUnknownDocs', () => {
expect(client.search).toHaveBeenCalledTimes(1);
expect(client.search).toHaveBeenCalledWith({
index: '.kibana_8.0.0',
- body: {
- query: {
- bool: {
- must: unusedTypesQuery,
- must_not: knownTypes.map((type) => ({
- term: {
- type,
+ size: 0,
+ aggs: {
+ typesAggregation: {
+ terms: {
+ // assign type __UNKNOWN__ to those documents that don't define one
+ missing: '__UNKNOWN__',
+ field: 'type',
+ size: 1000, // collect up to 1000 non-registered types
+ },
+ aggs: {
+ docs: {
+ top_hits: {
+ size: 100, // collect up to 100 docs for each non-registered type
+ _source: {
+ excludes: ['*'],
+ },
},
- })),
+ },
},
},
},
+ query: {
+ bool: {
+ ...excludeOnUpgradeQuery.bool,
+ must_not: knownTypes.map((type) => ({
+ term: {
+ type,
+ },
+ })),
+ },
+ },
});
});
@@ -92,7 +112,7 @@ describe('checkForUnknownDocs', () => {
client,
indexName: '.kibana_8.0.0',
knownTypes,
- unusedTypesQuery,
+ excludeOnUpgradeQuery,
});
const result = await task();
@@ -101,59 +121,36 @@ describe('checkForUnknownDocs', () => {
expect((result as Either.Right).right).toEqual({});
});
- it('resolves with `Either.left` when unknown docs are found', async () => {
- const client = elasticsearchClientMock.createInternalClient(
- Promise.resolve({
- hits: {
- hits: [
- { _id: '12', _source: { type: 'foo' } },
- { _id: '14', _source: { type: 'bar' } },
- ],
- },
- })
- );
-
- const task = checkForUnknownDocs({
- client,
- indexName: '.kibana_8.0.0',
- knownTypes,
- unusedTypesQuery,
- });
-
- const result = await task();
-
- expect(Either.isLeft(result)).toBe(true);
- expect((result as Either.Left).left).toEqual({
- type: 'unknown_docs_found',
- unknownDocs: [
- { id: '12', type: 'foo' },
- { id: '14', type: 'bar' },
- ],
- });
- });
-
- it('uses `unknown` as the type when the document does not contain a type field', async () => {
- const client = elasticsearchClientMock.createInternalClient(
- Promise.resolve({
- hits: {
- hits: [{ _id: '12', _source: {} }],
- },
- })
- );
-
- const task = checkForUnknownDocs({
- client,
- indexName: '.kibana_8.0.0',
- knownTypes,
- unusedTypesQuery,
- });
-
- const result = await task();
-
- expect(Either.isLeft(result)).toBe(true);
- expect((result as Either.Left).left).toEqual({
- type: 'unknown_docs_found',
- unknownDocs: [{ id: '12', type: 'unknown' }],
+ describe('when unknown doc types are found', () => {
+ it('resolves with `Either.right`, returning the unknown doc types', async () => {
+ const client = elasticsearchClientMock.createInternalClient(
+ Promise.resolve(
+ createAggregateTypesSearchResponse({
+ foo: ['12'],
+ bar: ['14'],
+ __UNKNOWN__: ['16'],
+ })
+ )
+ );
+
+ const task = checkForUnknownDocs({
+ client,
+ indexName: '.kibana_8.0.0',
+ knownTypes,
+ excludeOnUpgradeQuery,
+ });
+
+ const result = await task();
+
+ expect(Either.isRight(result)).toBe(true);
+ expect((result as Either.Right).right).toEqual({
+ type: 'unknown_docs_found',
+ unknownDocs: [
+ { id: '12', type: 'foo' },
+ { id: '14', type: 'bar' },
+ { id: '16', type: '__UNKNOWN__' },
+ ],
+ });
});
});
});
diff --git a/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.ts b/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.ts
index f9e3df9a0443a..b475f47e3d7f2 100644
--- a/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.ts
+++ b/src/core/server/saved_objects/migrations/actions/check_for_unknown_docs.ts
@@ -8,24 +8,31 @@
import * as Either from 'fp-ts/lib/Either';
import * as TaskEither from 'fp-ts/lib/TaskEither';
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
+import { flatten } from 'lodash';
+import type {
+ AggregationsMultiBucketAggregateBase,
+ Indices,
+ QueryDslQueryContainer,
+ SearchRequest,
+} from '@elastic/elasticsearch/lib/api/types';
import type { SavedObjectsRawDocSource } from '../../serialization';
-import { ElasticsearchClient } from '../../../elasticsearch';
+import type { ElasticsearchClient } from '../../../elasticsearch';
import {
catchRetryableEsClientErrors,
- RetryableEsClientError,
+ type RetryableEsClientError,
} from './catch_retryable_es_client_errors';
+import { addExcludedTypesToBoolQuery } from '../model/helpers';
/** @internal */
export interface CheckForUnknownDocsParams {
client: ElasticsearchClient;
indexName: string;
- unusedTypesQuery: estypes.QueryDslQueryContainer;
+ excludeOnUpgradeQuery: QueryDslQueryContainer;
knownTypes: string[];
}
/** @internal */
-export interface CheckForUnknownDocsFoundDoc {
+export interface DocumentIdAndType {
id: string;
type: string;
}
@@ -33,55 +40,90 @@ export interface CheckForUnknownDocsFoundDoc {
/** @internal */
export interface UnknownDocsFound {
type: 'unknown_docs_found';
- unknownDocs: CheckForUnknownDocsFoundDoc[];
+ unknownDocs: DocumentIdAndType[];
+}
+
+/**
+ * Performs a search in ES, aggregating documents by type,
+ * retrieving a bunch of documents for each type.
+ * @internal
+ * @param esClient The ES client to perform the search query
+ * @param targetIndices The ES indices to target
+ * @param query An optional query that can be used to filter
+ * @returns A list of documents with their types
+ */
+export async function getAggregatedTypesDocuments(
+ esClient: ElasticsearchClient,
+ targetIndices: Indices,
+ query?: QueryDslQueryContainer
+): Promise {
+ const params: SearchRequest = {
+ index: targetIndices,
+ size: 0,
+ // apply the desired filters (e.g. filter out registered types)
+ query,
+ // aggregate docs by type, so that we have a sneak peak of all types
+ aggs: {
+ typesAggregation: {
+ terms: {
+ // assign type __UNKNOWN__ to those documents that don't define one
+ missing: '__UNKNOWN__',
+ field: 'type',
+ size: 1000, // collect up to 1000 types
+ },
+ aggs: {
+ docs: {
+ top_hits: {
+ size: 100, // collect up to 100 docs for each type
+ _source: {
+ excludes: ['*'],
+ },
+ },
+ },
+ },
+ },
+ },
+ };
+
+ const body = await esClient.search(params);
+
+ if (!body.aggregations) return [];
+
+ const { typesAggregation } = body.aggregations;
+ const buckets = (typesAggregation as AggregationsMultiBucketAggregateBase).buckets;
+
+ const bucketsArray = Array.isArray(buckets) ? buckets : Object.values(buckets);
+
+ return flatten(
+ bucketsArray.map(
+ (bucket: any) =>
+ bucket.docs?.hits?.hits?.map((doc: any) => ({
+ id: doc._id,
+ type: bucket.key,
+ })) || []
+ )
+ );
}
export const checkForUnknownDocs =
({
client,
indexName,
- unusedTypesQuery,
+ excludeOnUpgradeQuery,
knownTypes,
}: CheckForUnknownDocsParams): TaskEither.TaskEither<
- RetryableEsClientError | UnknownDocsFound,
- {}
+ RetryableEsClientError,
+ UnknownDocsFound | {}
> =>
- () => {
- const query = createUnknownDocQuery(unusedTypesQuery, knownTypes);
-
- return client
- .search({
- index: indexName,
- body: {
- query,
- },
- })
- .then((body) => {
- const { hits } = body.hits;
- if (hits.length) {
- return Either.left({
- type: 'unknown_docs_found' as const,
- unknownDocs: hits.map((hit) => ({ id: hit._id, type: hit._source?.type ?? 'unknown' })),
- });
- } else {
- return Either.right({});
+ async () => {
+ const excludeQuery = addExcludedTypesToBoolQuery(knownTypes, excludeOnUpgradeQuery.bool);
+ return getAggregatedTypesDocuments(client, indexName, excludeQuery)
+ .then((unknownDocs) => {
+ if (unknownDocs.length) {
+ return Either.right({ type: 'unknown_docs_found' as const, unknownDocs });
}
+
+ return Either.right({});
})
.catch(catchRetryableEsClientErrors);
};
-
-const createUnknownDocQuery = (
- unusedTypesQuery: estypes.QueryDslQueryContainer,
- knownTypes: string[]
-): estypes.QueryDslQueryContainer => {
- return {
- bool: {
- must: unusedTypesQuery,
- must_not: knownTypes.map((type) => ({
- term: {
- type,
- },
- })),
- },
- };
-};
diff --git a/src/core/server/saved_objects/migrations/actions/index.ts b/src/core/server/saved_objects/migrations/actions/index.ts
index 3a387d764fa4c..4ac6bfa24fee6 100644
--- a/src/core/server/saved_objects/migrations/actions/index.ts
+++ b/src/core/server/saved_objects/migrations/actions/index.ts
@@ -93,7 +93,7 @@ import { ClusterShardLimitExceeded } from './create_index';
export type {
CheckForUnknownDocsParams,
UnknownDocsFound,
- CheckForUnknownDocsFoundDoc,
+ DocumentIdAndType,
} from './check_for_unknown_docs';
export { checkForUnknownDocs } from './check_for_unknown_docs';
@@ -160,7 +160,7 @@ export interface ActionErrorTypeMap {
/**
* Type guard for narrowing the type of a left
*/
-export function isLeftTypeof(
+export function isTypeof(
res: any,
typeString: T
): res is ActionErrorTypeMap[T] {
diff --git a/src/core/server/saved_objects/migrations/actions/integration_tests/actions.test.ts b/src/core/server/saved_objects/migrations/actions/integration_tests/actions.test.ts
index d47d53aa367e7..a799ced0ffe39 100644
--- a/src/core/server/saved_objects/migrations/actions/integration_tests/actions.test.ts
+++ b/src/core/server/saved_objects/migrations/actions/integration_tests/actions.test.ts
@@ -6,42 +6,42 @@
* Side Public License, v 1.
*/
-import { ElasticsearchClient } from '../../../..';
+import Path from 'path';
+import * as Either from 'fp-ts/lib/Either';
+import * as Option from 'fp-ts/lib/Option';
+import { errors } from '@elastic/elasticsearch';
+import type { TaskEither } from 'fp-ts/lib/TaskEither';
+import type { ElasticsearchClient } from '../../../..';
import * as kbnTestServer from '../../../../../test_helpers/kbn_server';
-import { SavedObjectsRawDoc } from '../../../serialization';
+import type { SavedObjectsRawDoc } from '../../../serialization';
import {
bulkOverwriteTransformedDocuments,
cloneIndex,
closePit,
createIndex,
openPit,
- OpenPitResponse,
+ type OpenPitResponse,
reindex,
readWithPit,
- ReadWithPit,
+ type ReadWithPit,
searchForOutdatedDocuments,
SearchResponse,
setWriteBlock,
updateAliases,
waitForReindexTask,
- ReindexResponse,
+ type ReindexResponse,
waitForPickupUpdatedMappingsTask,
pickupUpdatedMappings,
- UpdateByQueryResponse,
+ type UpdateByQueryResponse,
updateAndPickupMappings,
- UpdateAndPickupMappingsResponse,
+ type UpdateAndPickupMappingsResponse,
verifyReindex,
removeWriteBlock,
transformDocs,
waitForIndexStatusYellow,
initAction,
} from '..';
-import * as Either from 'fp-ts/lib/Either';
-import * as Option from 'fp-ts/lib/Option';
-import { errors } from '@elastic/elasticsearch';
-import { DocumentsTransformFailed, DocumentsTransformSuccess } from '../../core';
-import { TaskEither } from 'fp-ts/lib/TaskEither';
-import Path from 'path';
+import type { DocumentsTransformFailed, DocumentsTransformSuccess } from '../../core';
const { startES } = kbnTestServer.createTestServers({
adjustTimeout: (t: number) => jest.setTimeout(t),
@@ -610,7 +610,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
await expect(task()).resolves.toMatchInlineSnapshot(`
@@ -637,14 +637,14 @@ describe('migration actions', () => {
]
`);
});
- it('resolves right and excludes all documents not matching the unusedTypesQuery', async () => {
+ it('resolves right and excludes all documents not matching the excludeOnUpgradeQuery', async () => {
const res = (await reindex({
client,
sourceIndex: 'existing_index_with_docs',
targetIndex: 'reindex_target_excluded_docs',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: {
+ excludeOnUpgradeQuery: {
bool: {
must_not: ['f_agent_event', 'another_unused_type'].map((type) => ({
term: { type },
@@ -683,7 +683,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_2',
reindexScript: Option.some(`ctx._source.title = ctx._source.title + '_updated'`),
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
await expect(task()).resolves.toMatchInlineSnapshot(`
@@ -718,7 +718,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_3',
reindexScript: Option.some(`ctx._source.title = ctx._source.title + '_updated'`),
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
let task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
await expect(task()).resolves.toMatchInlineSnapshot(`
@@ -735,7 +735,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_3',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
await expect(task()).resolves.toMatchInlineSnapshot(`
@@ -794,7 +794,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_4',
reindexScript: Option.some(`ctx._source.title = ctx._source.title + '_updated'`),
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
await expect(task()).resolves.toMatchInlineSnapshot(`
@@ -850,7 +850,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_5',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: reindexTaskId, timeout: '10s' });
@@ -889,7 +889,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_6',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: reindexTaskId, timeout: '10s' });
@@ -910,7 +910,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: {
+ excludeOnUpgradeQuery: {
match_all: {},
},
})()) as Either.Right;
@@ -933,7 +933,7 @@ describe('migration actions', () => {
targetIndex: 'existing_index_with_write_block',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
@@ -955,7 +955,7 @@ describe('migration actions', () => {
targetIndex: 'existing_index_with_write_block',
reindexScript: Option.none,
requireAlias: true,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' });
@@ -983,7 +983,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
const task = waitForReindexTask({ client, taskId: res.right.taskId, timeout: '0s' });
@@ -1010,7 +1010,7 @@ describe('migration actions', () => {
targetIndex: 'reindex_target_7',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: { match_all: {} },
+ excludeOnUpgradeQuery: { match_all: {} },
})()) as Either.Right;
await waitForReindexTask({ client, taskId: res.right.taskId, timeout: '10s' })();
diff --git a/src/core/server/saved_objects/migrations/actions/reindex.test.ts b/src/core/server/saved_objects/migrations/actions/reindex.test.ts
index f53368bd9321b..3352e4eebadca 100644
--- a/src/core/server/saved_objects/migrations/actions/reindex.test.ts
+++ b/src/core/server/saved_objects/migrations/actions/reindex.test.ts
@@ -36,7 +36,7 @@ describe('reindex', () => {
targetIndex: 'my_target_index',
reindexScript: Option.none,
requireAlias: false,
- unusedTypesQuery: {},
+ excludeOnUpgradeQuery: {},
});
try {
await task();
diff --git a/src/core/server/saved_objects/migrations/actions/reindex.ts b/src/core/server/saved_objects/migrations/actions/reindex.ts
index cfd7449971b7f..46f20928e1bef 100644
--- a/src/core/server/saved_objects/migrations/actions/reindex.ts
+++ b/src/core/server/saved_objects/migrations/actions/reindex.ts
@@ -9,11 +9,11 @@
import * as Either from 'fp-ts/lib/Either';
import * as TaskEither from 'fp-ts/lib/TaskEither';
import * as Option from 'fp-ts/lib/Option';
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
-import { ElasticsearchClient } from '../../../elasticsearch';
+import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
+import type { ElasticsearchClient } from '../../../elasticsearch';
import {
catchRetryableEsClientErrors,
- RetryableEsClientError,
+ type RetryableEsClientError,
} from './catch_retryable_es_client_errors';
import { BATCH_SIZE } from './constants';
@@ -33,7 +33,7 @@ export interface ReindexParams {
* are no longer used. These saved objects will still be kept in the outdated
* index for backup purposes, but won't be available in the upgraded index.
*/
- unusedTypesQuery: estypes.QueryDslQueryContainer;
+ excludeOnUpgradeQuery: QueryDslQueryContainer;
}
/**
@@ -51,7 +51,7 @@ export const reindex =
targetIndex,
reindexScript,
requireAlias,
- unusedTypesQuery,
+ excludeOnUpgradeQuery,
}: ReindexParams): TaskEither.TaskEither =>
() => {
return client
@@ -67,7 +67,7 @@ export const reindex =
// Set reindex batch size
size: BATCH_SIZE,
// Exclude saved object types
- query: unusedTypesQuery,
+ query: excludeOnUpgradeQuery,
},
dest: {
index: targetIndex,
diff --git a/src/core/server/saved_objects/migrations/core/unused_types.ts b/src/core/server/saved_objects/migrations/core/unused_types.ts
index 076bdb489cf49..d80dcb23b7189 100644
--- a/src/core/server/saved_objects/migrations/core/unused_types.ts
+++ b/src/core/server/saved_objects/migrations/core/unused_types.ts
@@ -6,7 +6,7 @@
* Side Public License, v 1.
*/
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
+import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
/**
* Types that are no longer registered and need to be removed
@@ -41,7 +41,7 @@ export const REMOVED_TYPES: string[] = [
// saved objects which are no longer used. These saved objects will still be
// kept in the outdated index for backup purposes, but won't be available in
// the upgraded index.
-export const excludeUnusedTypesQuery: estypes.QueryDslQueryContainer = {
+export const excludeUnusedTypesQuery: QueryDslQueryContainer = {
bool: {
must_not: [
...REMOVED_TYPES.map((typeName) => ({
diff --git a/src/core/server/saved_objects/migrations/initial_state.test.ts b/src/core/server/saved_objects/migrations/initial_state.test.ts
index c2a2736541208..cfa9c9f3a2435 100644
--- a/src/core/server/saved_objects/migrations/initial_state.test.ts
+++ b/src/core/server/saved_objects/migrations/initial_state.test.ts
@@ -8,12 +8,14 @@
import { ByteSizeValue } from '@kbn/config-schema';
import * as Option from 'fp-ts/Option';
-import { DocLinksServiceSetup } from '../../doc_links';
-import { docLinksServiceMock } from '../../mocks';
-import { SavedObjectsMigrationConfigType } from '../saved_objects_config';
+import type { DocLinksServiceSetup } from '../../doc_links';
+import { docLinksServiceMock, loggingSystemMock } from '../../mocks';
+import type { SavedObjectsMigrationConfigType } from '../saved_objects_config';
import { SavedObjectTypeRegistry } from '../saved_objects_type_registry';
import { createInitialState } from './initial_state';
+const mockLogger = loggingSystemMock.create();
+
describe('createInitialState', () => {
let typeRegistry: SavedObjectTypeRegistry;
let docLinks: DocLinksServiceSetup;
@@ -41,62 +43,16 @@ describe('createInitialState', () => {
migrationsConfig,
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
})
).toMatchInlineSnapshot(`
Object {
"batchSize": 1000,
"controlState": "INIT",
"currentAlias": ".kibana_task_manager",
+ "discardUnknownObjects": false,
"excludeFromUpgradeFilterHooks": Object {},
- "indexPrefix": ".kibana_task_manager",
- "kibanaVersion": "8.1.0",
- "knownTypes": Array [],
- "legacyIndex": ".kibana_task_manager",
- "logs": Array [],
- "maxBatchSizeBytes": 104857600,
- "migrationDocLinks": Object {
- "clusterShardLimitExceeded": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#cluster-shard-limit-exceeded",
- "repeatedTimeoutRequests": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#_repeated_time_out_requests_that_eventually_fail",
- "resolveMigrationFailures": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html",
- "routingAllocationDisabled": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#routing-allocation-disabled",
- },
- "outdatedDocumentsQuery": Object {
- "bool": Object {
- "should": Array [],
- },
- },
- "preMigrationScript": Object {
- "_tag": "None",
- },
- "retryAttempts": 15,
- "retryCount": 0,
- "retryDelay": 0,
- "targetIndexMappings": Object {
- "dynamic": "strict",
- "properties": Object {
- "my_type": Object {
- "properties": Object {
- "title": Object {
- "type": "text",
- },
- },
- },
- },
- },
- "tempIndex": ".kibana_task_manager_8.1.0_reindex_temp",
- "tempIndexMappings": Object {
- "dynamic": false,
- "properties": Object {
- "migrationVersion": Object {
- "dynamic": "true",
- "type": "object",
- },
- "type": Object {
- "type": "keyword",
- },
- },
- },
- "unusedTypesQuery": Object {
+ "excludeOnUpgradeQuery": Object {
"bool": Object {
"must_not": Array [
Object {
@@ -193,6 +149,54 @@ describe('createInitialState', () => {
],
},
},
+ "indexPrefix": ".kibana_task_manager",
+ "kibanaVersion": "8.1.0",
+ "knownTypes": Array [],
+ "legacyIndex": ".kibana_task_manager",
+ "logs": Array [],
+ "maxBatchSizeBytes": 104857600,
+ "migrationDocLinks": Object {
+ "clusterShardLimitExceeded": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#cluster-shard-limit-exceeded",
+ "repeatedTimeoutRequests": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#_repeated_time_out_requests_that_eventually_fail",
+ "resolveMigrationFailures": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html",
+ "routingAllocationDisabled": "https://www.elastic.co/guide/en/kibana/test-branch/resolve-migrations-failures.html#routing-allocation-disabled",
+ },
+ "outdatedDocumentsQuery": Object {
+ "bool": Object {
+ "should": Array [],
+ },
+ },
+ "preMigrationScript": Object {
+ "_tag": "None",
+ },
+ "retryAttempts": 15,
+ "retryCount": 0,
+ "retryDelay": 0,
+ "targetIndexMappings": Object {
+ "dynamic": "strict",
+ "properties": Object {
+ "my_type": Object {
+ "properties": Object {
+ "title": Object {
+ "type": "text",
+ },
+ },
+ },
+ },
+ },
+ "tempIndex": ".kibana_task_manager_8.1.0_reindex_temp",
+ "tempIndexMappings": Object {
+ "dynamic": false,
+ "properties": Object {
+ "migrationVersion": Object {
+ "dynamic": "true",
+ "type": "object",
+ },
+ "type": Object {
+ "type": "keyword",
+ },
+ },
+ },
"versionAlias": ".kibana_task_manager_8.1.0",
"versionIndex": ".kibana_task_manager_8.1.0_001",
}
@@ -224,6 +228,7 @@ describe('createInitialState', () => {
migrationsConfig,
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
});
expect(initialState.knownTypes).toEqual(['foo', 'bar']);
@@ -250,6 +255,7 @@ describe('createInitialState', () => {
migrationsConfig,
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
});
expect(initialState.excludeFromUpgradeFilterHooks).toEqual({ foo: fooExcludeOnUpgradeHook });
@@ -269,6 +275,7 @@ describe('createInitialState', () => {
migrationsConfig,
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
});
expect(Option.isSome(initialState.preMigrationScript)).toEqual(true);
@@ -291,6 +298,7 @@ describe('createInitialState', () => {
migrationsConfig,
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
}).preMigrationScript
)
).toEqual(true);
@@ -309,6 +317,7 @@ describe('createInitialState', () => {
migrationsConfig,
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
}).outdatedDocumentsQuery
).toMatchInlineSnapshot(`
Object {
@@ -347,4 +356,71 @@ describe('createInitialState', () => {
}
`);
});
+
+ it('initializes the `discardUnknownObjects` flag to false if the flag is not provided in the config', () => {
+ const logger = mockLogger.get();
+ const initialState = createInitialState({
+ kibanaVersion: '8.1.0',
+ targetMappings: {
+ dynamic: 'strict',
+ properties: { my_type: { properties: { title: { type: 'text' } } } },
+ },
+ migrationVersionPerType: {},
+ indexPrefix: '.kibana_task_manager',
+ migrationsConfig,
+ typeRegistry,
+ docLinks,
+ logger,
+ });
+
+ expect(logger.warn).not.toBeCalled();
+ expect(initialState.discardUnknownObjects).toEqual(false);
+ });
+
+ it('initializes the `discardUnknownObjects` flag to false if the value provided in the config does not match the current kibana version', () => {
+ const logger = mockLogger.get();
+ const initialState = createInitialState({
+ kibanaVersion: '8.1.0',
+ targetMappings: {
+ dynamic: 'strict',
+ properties: { my_type: { properties: { title: { type: 'text' } } } },
+ },
+ migrationVersionPerType: {},
+ indexPrefix: '.kibana_task_manager',
+ migrationsConfig: {
+ ...migrationsConfig,
+ discardUnknownObjects: '8.0.0',
+ },
+ typeRegistry,
+ docLinks,
+ logger,
+ });
+
+ expect(initialState.discardUnknownObjects).toEqual(false);
+ expect(logger.warn).toBeCalledTimes(1);
+ expect(logger.warn).toBeCalledWith(
+ 'The flag `migrations.discardUnknownObjects` is defined but does not match the current kibana version; unknown objects will NOT be ignored.'
+ );
+ });
+
+ it('initializes the `discardUnknownObjects` flag to true if the value provided in the config matches the current kibana version', () => {
+ const initialState = createInitialState({
+ kibanaVersion: '8.1.0',
+ targetMappings: {
+ dynamic: 'strict',
+ properties: { my_type: { properties: { title: { type: 'text' } } } },
+ },
+ migrationVersionPerType: {},
+ indexPrefix: '.kibana_task_manager',
+ migrationsConfig: {
+ ...migrationsConfig,
+ discardUnknownObjects: '8.1.0',
+ },
+ typeRegistry,
+ docLinks,
+ logger: mockLogger.get(),
+ });
+
+ expect(initialState.discardUnknownObjects).toEqual(true);
+ });
});
diff --git a/src/core/server/saved_objects/migrations/initial_state.ts b/src/core/server/saved_objects/migrations/initial_state.ts
index ae0fe54049505..f576a5a93429a 100644
--- a/src/core/server/saved_objects/migrations/initial_state.ts
+++ b/src/core/server/saved_objects/migrations/initial_state.ts
@@ -7,13 +7,14 @@
*/
import * as Option from 'fp-ts/Option';
-import { IndexMapping } from '../mappings';
-import { SavedObjectsMigrationVersion } from '../../../types';
-import { SavedObjectsMigrationConfigType } from '../saved_objects_config';
+import type { Logger } from '@kbn/logging';
+import type { IndexMapping } from '../mappings';
+import type { SavedObjectsMigrationVersion } from '../../../types';
+import type { SavedObjectsMigrationConfigType } from '../saved_objects_config';
import type { ISavedObjectTypeRegistry } from '../saved_objects_type_registry';
-import { InitState } from './state';
+import type { InitState } from './state';
import { excludeUnusedTypesQuery } from './core';
-import { DocLinksServiceStart } from '../../doc_links';
+import type { DocLinksServiceStart } from '../../doc_links';
/**
* Construct the initial state for the model
@@ -27,6 +28,7 @@ export const createInitialState = ({
migrationsConfig,
typeRegistry,
docLinks,
+ logger,
}: {
kibanaVersion: string;
targetMappings: IndexMapping;
@@ -36,6 +38,7 @@ export const createInitialState = ({
migrationsConfig: SavedObjectsMigrationConfigType;
typeRegistry: ISavedObjectTypeRegistry;
docLinks: DocLinksServiceStart;
+ logger: Logger;
}): InitState => {
const outdatedDocumentsQuery = {
bool: {
@@ -70,6 +73,15 @@ export const createInitialState = ({
// short key to access savedObjects entries directly from docLinks
const migrationDocLinks = docLinks.links.kibanaUpgradeSavedObjects;
+ if (
+ migrationsConfig.discardUnknownObjects &&
+ migrationsConfig.discardUnknownObjects !== kibanaVersion
+ ) {
+ logger.warn(
+ 'The flag `migrations.discardUnknownObjects` is defined but does not match the current kibana version; unknown objects will NOT be ignored.'
+ );
+ }
+
return {
controlState: 'INIT',
indexPrefix,
@@ -88,8 +100,9 @@ export const createInitialState = ({
retryAttempts: migrationsConfig.retryAttempts,
batchSize: migrationsConfig.batchSize,
maxBatchSizeBytes: migrationsConfig.maxBatchSizeBytes.getValueInBytes(),
+ discardUnknownObjects: migrationsConfig.discardUnknownObjects === kibanaVersion,
logs: [],
- unusedTypesQuery: excludeUnusedTypesQuery,
+ excludeOnUpgradeQuery: excludeUnusedTypesQuery,
knownTypes,
excludeFromUpgradeFilterHooks: excludeFilterHooks,
migrationDocLinks,
diff --git a/src/core/server/saved_objects/migrations/integration_tests/7_13_0_unknown_types.test.ts b/src/core/server/saved_objects/migrations/integration_tests/7_13_0_unknown_types.test.ts
index dc521baad6335..6ccb6497516f9 100644
--- a/src/core/server/saved_objects/migrations/integration_tests/7_13_0_unknown_types.test.ts
+++ b/src/core/server/saved_objects/migrations/integration_tests/7_13_0_unknown_types.test.ts
@@ -8,9 +8,13 @@
import Path from 'path';
import fs from 'fs/promises';
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
+import type { IndicesIndexSettings } from '@elastic/elasticsearch/lib/api/types';
+import { Env } from '@kbn/config';
+import { REPO_ROOT } from '@kbn/utils';
+import { getEnvOptions } from '@kbn/config-mocks';
import * as kbnTestServer from '../../../../test_helpers/kbn_server';
-import { Root } from '../../../root';
+import type { Root } from '../../../root';
+import type { ElasticsearchClient } from '../../../elasticsearch';
const logFilePath = Path.join(__dirname, '7_13_unknown_types.log');
@@ -54,62 +58,83 @@ describe('migration v2', () => {
await new Promise((resolve) => setTimeout(resolve, 10000));
});
- it('fails the migration if unknown types are found in the source index', async () => {
- // Start kibana with foo and space types disabled
- root = createRoot();
- esServer = await startES();
- await root.preboot();
- await root.setup();
-
- try {
- await root.start();
- expect('should have thrown').toEqual('but it did not');
- } catch (err) {
- const errorMessage = err.message;
-
- expect(
- errorMessage.startsWith(
- 'Unable to complete saved object migrations for the [.kibana] index: Migration failed because documents ' +
- 'were found for unknown saved object types. To proceed with the migration, please delete these documents from the ' +
- '".kibana_7.13.0_001" index.'
- )
- ).toBeTruthy();
-
- const unknownDocs = [
- { type: 'space', id: 'space:default' },
- { type: 'space', id: 'space:first' },
- { type: 'space', id: 'space:second' },
- { type: 'space', id: 'space:third' },
- { type: 'space', id: 'space:forth' },
- { type: 'space', id: 'space:fifth' },
- { type: 'space', id: 'space:sixth' },
- { type: 'foo', id: 'P2SQfHkBs3dBRGh--No5' },
- { type: 'foo', id: 'QGSZfHkBs3dBRGh-ANoD' },
- { type: 'foo', id: 'QWSZfHkBs3dBRGh-hNob' },
- ];
-
- unknownDocs.forEach(({ id, type }) => {
- expect(errorMessage).toEqual(expect.stringContaining(`- "${id}" (type: "${type}")`));
- });
-
- const client = esServer.es.getClient();
- const { body: response } = await client.indices.getSettings(
- { index: '.kibana_7.13.0_001' },
- { meta: true }
- );
- const settings = response['.kibana_7.13.0_001'].settings as estypes.IndicesIndexSettings;
- expect(settings.index).not.toBeUndefined();
- expect(settings.index!.blocks?.write).not.toEqual('true');
- }
+ describe('when `discardUnknownObjects` does not match current kibana version', () => {
+ it('fails the migration if unknown types are found in the source index', async () => {
+ // Start kibana with foo and space types disabled
+ root = createRoot('7.13.0');
+ esServer = await startES();
+ await root.preboot();
+ await root.setup();
+
+ try {
+ await root.start();
+ expect('should have thrown').toEqual('but it did not');
+ } catch (err) {
+ const errorMessage = err.message;
+
+ expect(
+ errorMessage.startsWith(
+ 'Unable to complete saved object migrations for the [.kibana] index: Migration failed because some documents ' +
+ 'were found which use unknown saved object types:'
+ )
+ ).toBeTruthy();
+
+ const unknownDocs = [
+ { type: 'space', id: 'space:default' },
+ { type: 'space', id: 'space:first' },
+ { type: 'space', id: 'space:second' },
+ { type: 'space', id: 'space:third' },
+ { type: 'space', id: 'space:forth' },
+ { type: 'space', id: 'space:fifth' },
+ { type: 'space', id: 'space:sixth' },
+ { type: 'foo', id: 'P2SQfHkBs3dBRGh--No5' },
+ { type: 'foo', id: 'QGSZfHkBs3dBRGh-ANoD' },
+ { type: 'foo', id: 'QWSZfHkBs3dBRGh-hNob' },
+ ];
+
+ unknownDocs.forEach(({ id, type }) => {
+ expect(errorMessage).toEqual(expect.stringContaining(`- "${id}" (type: "${type}")`));
+ });
+
+ const client = esServer.es.getClient();
+ const { body: response } = await client.indices.getSettings(
+ { index: '.kibana_7.13.0_001' },
+ { meta: true }
+ );
+ const settings = response['.kibana_7.13.0_001'].settings as IndicesIndexSettings;
+ expect(settings.index).not.toBeUndefined();
+ expect(settings.index!.blocks?.write).not.toEqual('true');
+ }
+ });
+ });
+
+ describe('when `discardUnknownObjects` matches current kibana version', () => {
+ const currentVersion = Env.createDefault(REPO_ROOT, getEnvOptions()).packageInfo.version;
+
+ it('discards the documents with unknown types and finishes the migration successfully', async () => {
+ // Start kibana with foo and space types disabled
+ root = createRoot(currentVersion);
+ esServer = await startES();
+ await root.preboot();
+ await root.setup();
+
+ // the migration process should finish successfully
+ await expect(root.start()).resolves.not.toThrowError();
+
+ const esClient: ElasticsearchClient = esServer.es.getClient();
+ const body = await esClient.count({ q: 'type:foo|space' });
+ expect(body.count).toEqual(0);
+ });
});
});
-function createRoot() {
+function createRoot(discardUnknownObjects?: string) {
return kbnTestServer.createRootWithCorePlugins(
{
migrations: {
skip: false,
batchSize: 5,
+ discardUnknownObjects,
},
logging: {
appenders: {
diff --git a/src/core/server/saved_objects/migrations/integration_tests/type_registrations.test.ts b/src/core/server/saved_objects/migrations/integration_tests/type_registrations.test.ts
index 79a76d0a9f4fc..c50d29ba521e7 100644
--- a/src/core/server/saved_objects/migrations/integration_tests/type_registrations.test.ts
+++ b/src/core/server/saved_objects/migrations/integration_tests/type_registrations.test.ts
@@ -114,7 +114,7 @@ const previouslyRegisteredTypes = [
].sort();
describe('SO type registrations', () => {
- it('does not remove types from registrations without updating unusedTypesQuery', async () => {
+ it('does not remove types from registrations without updating excludeOnUpgradeQuery', async () => {
const root = kbnTestServer.createRoot({}, { oss: false });
await root.preboot();
const setup = await root.setup();
diff --git a/src/core/server/saved_objects/migrations/kibana_migrator.ts b/src/core/server/saved_objects/migrations/kibana_migrator.ts
index 54feeccb9573f..90413d70c99bb 100644
--- a/src/core/server/saved_objects/migrations/kibana_migrator.ts
+++ b/src/core/server/saved_objects/migrations/kibana_migrator.ts
@@ -118,9 +118,7 @@ export class KibanaMigrator {
* The promise resolves with an array of migration statuses, one for each
* elasticsearch index which was migrated.
*/
- public runMigrations({ rerun = false }: { rerun?: boolean } = {}): Promise<
- Array<{ status: string }>
- > {
+ public runMigrations({ rerun = false }: { rerun?: boolean } = {}): Promise {
if (this.migrationResult === undefined || rerun) {
// Reruns are only used by CI / EsArchiver. Publishing status updates on reruns results in slowing down CI
// unnecessarily, so we skip it in this case.
@@ -147,7 +145,7 @@ export class KibanaMigrator {
return this.status$.asObservable();
}
- private runMigrationsInternal() {
+ private runMigrationsInternal(): Promise {
const indexMap = createIndexMap({
kibanaIndexName: this.kibanaIndex,
indexMap: this.mappingProperties,
diff --git a/src/core/server/saved_objects/migrations/migrations_state_action_machine.test.ts b/src/core/server/saved_objects/migrations/migrations_state_action_machine.test.ts
index 93e6476f8e78c..f99f336a9ac74 100644
--- a/src/core/server/saved_objects/migrations/migrations_state_action_machine.test.ts
+++ b/src/core/server/saved_objects/migrations/migrations_state_action_machine.test.ts
@@ -50,6 +50,7 @@ describe('migrationsStateActionMachine', () => {
},
typeRegistry,
docLinks,
+ logger: mockLogger.get(),
});
const next = jest.fn((s: State) => {
diff --git a/src/core/server/saved_objects/migrations/model/extract_errors.test.ts b/src/core/server/saved_objects/migrations/model/extract_errors.test.ts
index ea6b312c2053d..036ead853cf55 100644
--- a/src/core/server/saved_objects/migrations/model/extract_errors.test.ts
+++ b/src/core/server/saved_objects/migrations/model/extract_errors.test.ts
@@ -8,35 +8,53 @@
import {
extractUnknownDocFailureReason,
+ extractDiscardedUnknownDocs,
fatalReasonDocumentExceedsMaxBatchSizeBytes,
} from './extract_errors';
describe('extractUnknownDocFailureReason', () => {
it('generates the correct error message', () => {
expect(
- extractUnknownDocFailureReason(
- [
- {
- id: 'unknownType:12',
- type: 'unknownType',
- },
- {
- id: 'anotherUnknownType:42',
- type: 'anotherUnknownType',
- },
- ],
- '.kibana_15'
- )
+ extractUnknownDocFailureReason('some-url.co', [
+ {
+ id: 'unknownType:12',
+ type: 'unknownType',
+ },
+ {
+ id: 'anotherUnknownType:42',
+ type: 'anotherUnknownType',
+ },
+ ])
).toMatchInlineSnapshot(`
- "Migration failed because documents were found for unknown saved object types. To proceed with the migration, please delete these documents from the \\".kibana_15\\" index.
- The documents with unknown types are:
+ "Migration failed because some documents were found which use unknown saved object types:
- \\"unknownType:12\\" (type: \\"unknownType\\")
- \\"anotherUnknownType:42\\" (type: \\"anotherUnknownType\\")
- You can delete them using the following command:
- curl -X POST \\"{elasticsearch}/.kibana_15/_bulk?pretty\\" -H 'Content-Type: application/json' -d'
- { \\"delete\\" : { \\"_id\\" : \\"unknownType:12\\" } }
- { \\"delete\\" : { \\"_id\\" : \\"anotherUnknownType:42\\" } }
- '"
+
+ To proceed with the migration you can configure Kibana to discard unknown saved objects for this migration.
+ Please refer to some-url.co for more information."
+ `);
+ });
+});
+
+describe('extractDiscardedUnknownDocs', () => {
+ it('generates the correct error message', () => {
+ expect(
+ extractDiscardedUnknownDocs([
+ {
+ id: 'unknownType:12',
+ type: 'unknownType',
+ },
+ {
+ id: 'anotherUnknownType:42',
+ type: 'anotherUnknownType',
+ },
+ ])
+ ).toMatchInlineSnapshot(`
+ "Kibana has been configured to discard unknown documents for this migration.
+ Therefore, the following documents with unknown types will not be taken into account and they will not be available after the migration:
+ - \\"unknownType:12\\" (type: \\"unknownType\\")
+ - \\"anotherUnknownType:42\\" (type: \\"anotherUnknownType\\")
+ "
`);
});
});
diff --git a/src/core/server/saved_objects/migrations/model/extract_errors.ts b/src/core/server/saved_objects/migrations/model/extract_errors.ts
index c529e1b1da269..1d4d697b74774 100644
--- a/src/core/server/saved_objects/migrations/model/extract_errors.ts
+++ b/src/core/server/saved_objects/migrations/model/extract_errors.ts
@@ -6,8 +6,8 @@
* Side Public License, v 1.
*/
-import { TransformErrorObjects } from '../core';
-import { CheckForUnknownDocsFoundDoc } from '../actions';
+import type { TransformErrorObjects } from '../core';
+import type { DocumentIdAndType } from '../actions';
/**
* Constructs migration failure message strings from corrupt document ids and document transformation errors
@@ -36,19 +36,23 @@ export function extractTransformFailuresReason(
);
}
+export function extractDiscardedUnknownDocs(unknownDocs: DocumentIdAndType[]): string {
+ return (
+ `Kibana has been configured to discard unknown documents for this migration.\n` +
+ `Therefore, the following documents with unknown types will not be taken into account and they will not be available after the migration:\n` +
+ unknownDocs.map((doc) => `- "${doc.id}" (type: "${doc.type}")\n`).join('')
+ );
+}
+
export function extractUnknownDocFailureReason(
- unknownDocs: CheckForUnknownDocsFoundDoc[],
- sourceIndex: string
+ resolveMigrationFailuresUrl: string,
+ unknownDocs: DocumentIdAndType[]
): string {
return (
- `Migration failed because documents were found for unknown saved object types. ` +
- `To proceed with the migration, please delete these documents from the "${sourceIndex}" index.\n` +
- `The documents with unknown types are:\n` +
+ `Migration failed because some documents were found which use unknown saved object types:\n` +
unknownDocs.map((doc) => `- "${doc.id}" (type: "${doc.type}")\n`).join('') +
- `You can delete them using the following command:\n` +
- `curl -X POST "{elasticsearch}/${sourceIndex}/_bulk?pretty" -H 'Content-Type: application/json' -d'\n` +
- unknownDocs.map((doc) => `{ "delete" : { "_id" : "${doc.id}" } }\n`).join('') +
- `'`
+ `\nTo proceed with the migration you can configure Kibana to discard unknown saved objects for this migration.\n` +
+ `Please refer to ${resolveMigrationFailuresUrl} for more information.`
);
}
diff --git a/src/core/server/saved_objects/migrations/model/helpers.test.ts b/src/core/server/saved_objects/migrations/model/helpers.test.ts
new file mode 100644
index 0000000000000..0e2056eacac66
--- /dev/null
+++ b/src/core/server/saved_objects/migrations/model/helpers.test.ts
@@ -0,0 +1,176 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+import {
+ addExcludedTypesToBoolQuery,
+ addMustClausesToBoolQuery,
+ addMustNotClausesToBoolQuery,
+} from './helpers';
+
+describe('addExcludedTypesToBoolQuery', () => {
+ it('generates a bool query which filters out the specified types', () => {
+ const boolQuery = { must_not: [] };
+ const types = ['type1', 'type2'];
+ const result = addExcludedTypesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must_not: [{ term: { type: 'type1' } }, { term: { type: 'type2' } }],
+ },
+ });
+ });
+});
+
+describe('addMustClausesToBoolQuery', () => {
+ it('generates a new bool query when no query is provided', () => {
+ const boolQuery = undefined;
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must: [{ term: { type: 'type1' } }, { term: { type: 'type2' } }],
+ },
+ });
+ });
+
+ it('adds a new must clause to the provided bool query, if it did exist', () => {
+ const boolQuery = {
+ should: [
+ { match: { 'name.first': { query: 'shay', _name: 'first' } } },
+ { match: { 'name.last': { query: 'banon', _name: 'last' } } },
+ ],
+ };
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ should: [
+ { match: { 'name.first': { query: 'shay', _name: 'first' } } },
+ { match: { 'name.last': { query: 'banon', _name: 'last' } } },
+ ],
+ must: [{ term: { type: 'type1' } }, { term: { type: 'type2' } }],
+ },
+ });
+ });
+
+ it('appends the given clauses to the existing must', () => {
+ const boolQuery = {
+ must: [
+ { match: { type: 'search-session' } },
+ { match: { 'search-session.persisted': false } },
+ ],
+ };
+
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must: [
+ { match: { type: 'search-session' } },
+ { match: { 'search-session.persisted': false } },
+ { term: { type: 'type1' } },
+ { term: { type: 'type2' } },
+ ],
+ },
+ });
+ });
+
+ it('arrayifys the existing must clause if needed', () => {
+ const boolQuery = {
+ must: {
+ term: { type: 'type0' },
+ },
+ };
+
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must: [
+ { term: { type: 'type0' } },
+ { term: { type: 'type1' } },
+ { term: { type: 'type2' } },
+ ],
+ },
+ });
+ });
+});
+
+describe('addMustNotClausesToBoolQuery', () => {
+ it('generates a new bool query when no query is provided', () => {
+ const boolQuery = undefined;
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustNotClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must_not: [{ term: { type: 'type1' } }, { term: { type: 'type2' } }],
+ },
+ });
+ });
+
+ it('adds a new must_not clause to the provided bool query, if it did not exist', () => {
+ const boolQuery = {
+ should: [
+ { match: { 'name.first': { query: 'shay', _name: 'first' } } },
+ { match: { 'name.last': { query: 'banon', _name: 'last' } } },
+ ],
+ };
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustNotClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ should: [
+ { match: { 'name.first': { query: 'shay', _name: 'first' } } },
+ { match: { 'name.last': { query: 'banon', _name: 'last' } } },
+ ],
+ must_not: [{ term: { type: 'type1' } }, { term: { type: 'type2' } }],
+ },
+ });
+ });
+
+ it('appends the given clauses to the existing must_not', () => {
+ const boolQuery = {
+ must_not: [
+ { match: { type: 'search-session' } },
+ { match: { 'search-session.persisted': false } },
+ ],
+ };
+
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustNotClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must_not: [
+ { match: { type: 'search-session' } },
+ { match: { 'search-session.persisted': false } },
+ { term: { type: 'type1' } },
+ { term: { type: 'type2' } },
+ ],
+ },
+ });
+ });
+
+ it('arrayifys the existing must_not clause if needed', () => {
+ const boolQuery = {
+ must_not: {
+ term: { type: 'type0' },
+ },
+ };
+
+ const types = [{ term: { type: 'type1' } }, { term: { type: 'type2' } }];
+ const result = addMustNotClausesToBoolQuery(types, boolQuery);
+ expect(result).toEqual({
+ bool: {
+ must_not: [
+ { term: { type: 'type0' } },
+ { term: { type: 'type1' } },
+ { term: { type: 'type2' } },
+ ],
+ },
+ });
+ });
+});
diff --git a/src/core/server/saved_objects/migrations/model/helpers.ts b/src/core/server/saved_objects/migrations/model/helpers.ts
index c3a4c85679680..a139089bd3a2d 100644
--- a/src/core/server/saved_objects/migrations/model/helpers.ts
+++ b/src/core/server/saved_objects/migrations/model/helpers.ts
@@ -7,9 +7,13 @@
*/
import { gt, valid } from 'semver';
-import { State } from '../state';
-import { IndexMapping } from '../../mappings';
-import { FetchIndexResponse } from '../actions';
+import type {
+ QueryDslBoolQuery,
+ QueryDslQueryContainer,
+} from '@elastic/elasticsearch/lib/api/types';
+import type { State } from '../state';
+import type { IndexMapping } from '../../mappings';
+import type { FetchIndexResponse } from '../actions';
/**
* A helper function/type for ensuring that all control state's are handled.
@@ -68,6 +72,75 @@ export function indexBelongsToLaterVersion(indexName: string, kibanaVersion: str
return version != null ? gt(version, kibanaVersion) : false;
}
+/**
+ * Add new must_not clauses to the given query
+ * in order to filter out the specified types
+ * @param boolQuery the bool query to be enriched
+ * @param types the types to be filtered out
+ * @returns a new query container with the enriched query
+ */
+export function addExcludedTypesToBoolQuery(
+ types: string[],
+ boolQuery?: QueryDslBoolQuery
+): QueryDslQueryContainer {
+ return addMustNotClausesToBoolQuery(
+ types.map((type) => ({ term: { type } })),
+ boolQuery
+ );
+}
+
+/**
+ * Add the given clauses to the 'must' of the given query
+ * @param boolQuery the bool query to be enriched
+ * @param mustClauses the clauses to be added to a 'must'
+ * @returns a new query container with the enriched query
+ */
+export function addMustClausesToBoolQuery(
+ mustClauses: QueryDslQueryContainer[],
+ boolQuery?: QueryDslBoolQuery
+): QueryDslQueryContainer {
+ let must: QueryDslQueryContainer[] = [];
+
+ if (boolQuery?.must) {
+ must = must.concat(boolQuery.must);
+ }
+
+ must.push(...mustClauses);
+
+ return {
+ bool: {
+ ...boolQuery,
+ must,
+ },
+ };
+}
+
+/**
+ * Add the given clauses to the 'must_not' of the given query
+ * @param boolQuery the bool query to be enriched
+ * @param mustNotClauses the clauses to be added to a 'must_not'
+ * @returns a new query container with the enriched query
+ */
+export function addMustNotClausesToBoolQuery(
+ mustNotClauses: QueryDslQueryContainer[],
+ boolQuery?: QueryDslBoolQuery
+): QueryDslQueryContainer {
+ let mustNot: QueryDslQueryContainer[] = [];
+
+ if (boolQuery?.must_not) {
+ mustNot = mustNot.concat(boolQuery.must_not);
+ }
+
+ mustNot.push(...mustNotClauses);
+
+ return {
+ bool: {
+ ...boolQuery,
+ must_not: mustNot,
+ },
+ };
+}
+
/**
* Extracts the version number from a >= 7.11 index
* @param indexName A >= v7.11 index name
diff --git a/src/core/server/saved_objects/migrations/model/model.test.ts b/src/core/server/saved_objects/migrations/model/model.test.ts
index 1782ece9b4827..5a63e180e7aa9 100644
--- a/src/core/server/saved_objects/migrations/model/model.test.ts
+++ b/src/core/server/saved_objects/migrations/model/model.test.ts
@@ -59,6 +59,7 @@ describe('migrations v2 model', () => {
retryAttempts: 15,
batchSize: 1000,
maxBatchSizeBytes: 1e8,
+ discardUnknownObjects: false,
indexPrefix: '.kibana',
outdatedDocumentsQuery: {},
targetIndexMappings: {
@@ -81,7 +82,7 @@ describe('migrations v2 model', () => {
versionAlias: '.kibana_7.11.0',
versionIndex: '.kibana_7.11.0_001',
tempIndex: '.kibana_7.11.0_reindex_temp',
- unusedTypesQuery: {
+ excludeOnUpgradeQuery: {
bool: {
must_not: [
{
@@ -805,7 +806,7 @@ describe('migrations v2 model', () => {
},
} as const;
- test('CHECK_UNKNOWN_DOCUMENTS -> SET_SOURCE_WRITE_BLOCK if action succeeds', () => {
+ test('CHECK_UNKNOWN_DOCUMENTS -> SET_SOURCE_WRITE_BLOCK if action succeeds and no unknown docs are found', () => {
const checkUnknownDocumentsSourceState: CheckUnknownDocumentsState = {
...baseState,
controlState: 'CHECK_UNKNOWN_DOCUMENTS',
@@ -853,29 +854,76 @@ describe('migrations v2 model', () => {
expect(newState.logs).toEqual([]);
});
- test('CHECK_UNKNOWN_DOCUMENTS -> FATAL if action fails and unknown docs were found', () => {
- const checkUnknownDocumentsSourceState: CheckUnknownDocumentsState = {
- ...baseState,
- controlState: 'CHECK_UNKNOWN_DOCUMENTS',
- sourceIndex: Option.some('.kibana_3') as Option.Some,
- sourceIndexMappings: mappingsWithUnknownType,
- };
+ describe('when unknown docs are found', () => {
+ test('CHECK_UNKNOWN_DOCUMENTS -> SET_SOURCE_WRITE_BLOCK if discardUnknownObjects=true', () => {
+ const checkUnknownDocumentsSourceState: CheckUnknownDocumentsState = {
+ ...baseState,
+ discardUnknownObjects: true,
+ controlState: 'CHECK_UNKNOWN_DOCUMENTS',
+ sourceIndex: Option.some('.kibana_3') as Option.Some,
+ sourceIndexMappings: mappingsWithUnknownType,
+ };
+
+ const res: ResponseType<'CHECK_UNKNOWN_DOCUMENTS'> = Either.right({
+ type: 'unknown_docs_found',
+ unknownDocs: [
+ { id: 'dashboard:12', type: 'dashboard' },
+ { id: 'foo:17', type: 'foo' },
+ ],
+ });
+ const newState = model(checkUnknownDocumentsSourceState, res);
- const res: ResponseType<'CHECK_UNKNOWN_DOCUMENTS'> = Either.left({
- type: 'unknown_docs_found',
- unknownDocs: [
- { id: 'dashboard:12', type: 'dashboard' },
- { id: 'foo:17', type: 'foo' },
- ],
+ expect(newState).toMatchObject({
+ controlState: 'SET_SOURCE_WRITE_BLOCK',
+ sourceIndex: Option.some('.kibana_3'),
+ targetIndex: '.kibana_7.11.0_001',
+ });
+
+ expect(newState.excludeOnUpgradeQuery).toEqual({
+ bool: {
+ must_not: [
+ { term: { type: 'unused-fleet-agent-events' } },
+ { term: { type: 'dashboard' } },
+ { term: { type: 'foo' } },
+ ],
+ must: [{ exists: { field: 'type' } }],
+ },
+ });
+
+ // we should have a warning in the logs about the ignored types
+ expect(
+ newState.logs.find(({ level, message }) => {
+ return (
+ level === 'warning' && message.includes('dashboard') && message.includes('foo')
+ );
+ })
+ ).toBeDefined();
});
- const newState = model(checkUnknownDocumentsSourceState, res);
- expect(newState.controlState).toEqual('FATAL');
- expect(newState).toMatchObject({
- controlState: 'FATAL',
- reason: expect.stringContaining(
- 'Migration failed because documents were found for unknown saved object types'
- ),
+ test('CHECK_UNKNOWN_DOCUMENTS -> FATAL if discardUnknownObjects=false', () => {
+ const checkUnknownDocumentsSourceState: CheckUnknownDocumentsState = {
+ ...baseState,
+ controlState: 'CHECK_UNKNOWN_DOCUMENTS',
+ sourceIndex: Option.some('.kibana_3') as Option.Some,
+ sourceIndexMappings: mappingsWithUnknownType,
+ };
+
+ const res: ResponseType<'CHECK_UNKNOWN_DOCUMENTS'> = Either.right({
+ type: 'unknown_docs_found',
+ unknownDocs: [
+ { id: 'dashboard:12', type: 'dashboard' },
+ { id: 'foo:17', type: 'foo' },
+ ],
+ });
+ const newState = model(checkUnknownDocumentsSourceState, res);
+ expect(newState.controlState).toEqual('FATAL');
+
+ expect(newState).toMatchObject({
+ controlState: 'FATAL',
+ reason: expect.stringContaining(
+ 'Migration failed because some documents were found which use unknown saved object types'
+ ),
+ });
});
});
});
@@ -926,30 +974,27 @@ describe('migrations v2 model', () => {
const newState = model(state, res);
expect(newState.controlState).toEqual('CALCULATE_EXCLUDE_FILTERS');
});
- test('CALCULATE_EXCLUDE_FILTERS -> CREATE_REINDEX_TEMP if action succeeds with filters', () => {
+ it('CALCULATE_EXCLUDE_FILTERS -> CREATE_REINDEX_TEMP if action succeeds with filters', () => {
const res: ResponseType<'CALCULATE_EXCLUDE_FILTERS'> = Either.right({
- excludeFilter: { bool: { must: { term: { fieldA: 'abc' } } } },
+ mustNotClauses: [{ term: { fieldA: 'abc' } }],
errorsByType: { type1: new Error('an error!') },
});
const newState = model(state, res);
expect(newState.controlState).toEqual('CREATE_REINDEX_TEMP');
- expect(newState.unusedTypesQuery).toEqual({
- // New filter should be combined unused type query and filter from response
+
+ expect(newState.excludeOnUpgradeQuery).toEqual({
+ // new filters should be added inside a must_not clause, enriching excludeOnUpgradeQuery
bool: {
- filter: [
+ must_not: [
{
- bool: {
- must_not: [
- {
- term: {
- type: 'unused-fleet-agent-events',
- },
- },
- ],
+ term: {
+ type: 'unused-fleet-agent-events',
},
},
{
- bool: { must: { term: { fieldA: 'abc' } } },
+ term: {
+ fieldA: 'abc',
+ },
},
],
},
diff --git a/src/core/server/saved_objects/migrations/model/model.ts b/src/core/server/saved_objects/migrations/model/model.ts
index accff9553c808..b552b9cbb049a 100644
--- a/src/core/server/saved_objects/migrations/model/model.ts
+++ b/src/core/server/saved_objects/migrations/model/model.ts
@@ -8,10 +8,9 @@
import * as Either from 'fp-ts/lib/Either';
import * as Option from 'fp-ts/lib/Option';
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
-import { AliasAction, isLeftTypeof } from '../actions';
-import { AllActionStates, State } from '../state';
+import { type AliasAction, isTypeof } from '../actions';
+import type { AllActionStates, State } from '../state';
import type { ResponseType } from '../next';
import { disableUnknownTypeMappingFields } from '../core';
import {
@@ -25,9 +24,13 @@ import {
extractTransformFailuresReason,
extractUnknownDocFailureReason,
fatalReasonDocumentExceedsMaxBatchSizeBytes,
+ extractDiscardedUnknownDocs,
} from './extract_errors';
import type { ExcludeRetryableEsError } from './types';
import {
+ addExcludedTypesToBoolQuery,
+ addMustClausesToBoolQuery,
+ addMustNotClausesToBoolQuery,
getAliases,
indexBelongsToLaterVersion,
indexVersion,
@@ -36,6 +39,7 @@ import {
throwBadResponse,
} from './helpers';
import { createBatches } from './create_batches';
+import type { MigrationLog } from '../types';
export const FATAL_REASON_REQUEST_ENTITY_TOO_LARGE = `While indexing a batch of saved objects, Elasticsearch returned a 413 Request Entity Too Large exception. Ensure that the Kibana configuration option 'migrations.maxBatchSizeBytes' is set to a value that is lower than or equal to the Elasticsearch 'http.max_content_length' configuration option.`;
const CLUSTER_SHARD_LIMIT_EXCEEDED_REASON = `[cluster_shard_limit_exceeded] Upgrading Kibana requires adding a small number of new shards. Ensure that Kibana is able to add 10 more shards by increasing the cluster.max_shards_per_node setting, or removing indices to clear up resources.`;
@@ -53,7 +57,7 @@ export const model = (currentState: State, resW: ResponseType):
// Handle retryable_es_client_errors. Other left values need to be handled
// by the control state specific code below.
if (Either.isLeft(resW)) {
- if (isLeftTypeof(resW.left, 'retryable_es_client_error')) {
+ if (isTypeof(resW.left, 'retryable_es_client_error')) {
// Retry the same step after an exponentially increasing delay.
return delayRetryState(stateP, resW.left.message, stateP.retryAttempts);
}
@@ -67,7 +71,7 @@ export const model = (currentState: State, resW: ResponseType):
if (Either.isLeft(res)) {
const left = res.left;
- if (isLeftTypeof(left, 'incompatible_cluster_routing_allocation')) {
+ if (isTypeof(left, 'incompatible_cluster_routing_allocation')) {
const retryErrorMessage = `[${left.type}] Incompatible Elasticsearch cluster settings detected. Remove the persistent and transient Elasticsearch cluster setting 'cluster.routing.allocation.enable' or set it to a value of 'all' to allow migrations to proceed. Refer to ${stateP.migrationDocLinks.routingAllocationDisabled} for more information on how to resolve the issue.`;
return delayRetryState(stateP, retryErrorMessage, stateP.retryAttempts);
} else {
@@ -209,7 +213,7 @@ export const model = (currentState: State, resW: ResponseType):
// If the write block failed because the index doesn't exist, it means
// another instance already completed the legacy pre-migration. Proceed
// to the next step.
- if (isLeftTypeof(res.left, 'index_not_found_exception')) {
+ if (isTypeof(res.left, 'index_not_found_exception')) {
return { ...stateP, controlState: 'LEGACY_CREATE_REINDEX_TARGET' };
} else {
// @ts-expect-error TS doesn't correctly narrow this type to never
@@ -222,7 +226,7 @@ export const model = (currentState: State, resW: ResponseType):
const res = resW as ExcludeRetryableEsError>;
if (Either.isLeft(res)) {
const left = res.left;
- if (isLeftTypeof(left, 'index_not_yellow_timeout')) {
+ if (isTypeof(left, 'index_not_yellow_timeout')) {
// `index_not_yellow_timeout` for the LEGACY_CREATE_REINDEX_TARGET source index:
// A yellow status timeout could theoretically be temporary for a busy cluster
// that takes a long time to allocate the primary and we retry the action to see if
@@ -231,7 +235,7 @@ export const model = (currentState: State, resW: ResponseType):
// continue to timeout and eventually lead to a failed migration.
const retryErrorMessage = `${left.message} Refer to ${stateP.migrationDocLinks.repeatedTimeoutRequests} for information on how to resolve the issue.`;
return delayRetryState(stateP, retryErrorMessage, stateP.retryAttempts);
- } else if (isLeftTypeof(left, 'cluster_shard_limit_exceeded')) {
+ } else if (isTypeof(left, 'cluster_shard_limit_exceeded')) {
return {
...stateP,
controlState: 'FATAL',
@@ -272,8 +276,8 @@ export const model = (currentState: State, resW: ResponseType):
} else {
const left = res.left;
if (
- (isLeftTypeof(left, 'index_not_found_exception') && left.index === stateP.legacyIndex) ||
- isLeftTypeof(left, 'target_index_had_write_block')
+ (isTypeof(left, 'index_not_found_exception') && left.index === stateP.legacyIndex) ||
+ isTypeof(left, 'target_index_had_write_block')
) {
// index_not_found_exception for the LEGACY_REINDEX source index:
// another instance already complete the LEGACY_DELETE step.
@@ -286,15 +290,15 @@ export const model = (currentState: State, resW: ResponseType):
// step. However, by not skipping ahead we limit branches in the
// control state progression and simplify the implementation.
return { ...stateP, controlState: 'LEGACY_DELETE' };
- } else if (isLeftTypeof(left, 'wait_for_task_completion_timeout')) {
+ } else if (isTypeof(left, 'wait_for_task_completion_timeout')) {
// After waiting for the specified timeout, the task has not yet
// completed. Retry this step to see if the task has completed after an
// exponential delay. We will basically keep polling forever until the
// Elasticsearch task succeeds or fails.
return delayRetryState(stateP, left.message, Number.MAX_SAFE_INTEGER);
} else if (
- isLeftTypeof(left, 'index_not_found_exception') ||
- isLeftTypeof(left, 'incompatible_mapping_exception')
+ isTypeof(left, 'index_not_found_exception') ||
+ isTypeof(left, 'incompatible_mapping_exception')
) {
// We don't handle the following errors as the algorithm will never
// run into these during the LEGACY_REINDEX_WAIT_FOR_TASK step:
@@ -312,8 +316,8 @@ export const model = (currentState: State, resW: ResponseType):
} else if (Either.isLeft(res)) {
const left = res.left;
if (
- isLeftTypeof(left, 'remove_index_not_a_concrete_index') ||
- (isLeftTypeof(left, 'index_not_found_exception') && left.index === stateP.legacyIndex)
+ isTypeof(left, 'remove_index_not_a_concrete_index') ||
+ (isTypeof(left, 'index_not_found_exception') && left.index === stateP.legacyIndex)
) {
// index_not_found_exception, another Kibana instance already
// deleted the legacy index
@@ -327,8 +331,8 @@ export const model = (currentState: State, resW: ResponseType):
// control state progression and simplify the implementation.
return { ...stateP, controlState: 'SET_SOURCE_WRITE_BLOCK' };
} else if (
- isLeftTypeof(left, 'index_not_found_exception') ||
- isLeftTypeof(left, 'alias_not_found_exception')
+ isTypeof(left, 'index_not_found_exception') ||
+ isTypeof(left, 'alias_not_found_exception')
) {
// We don't handle the following errors as the migration algorithm
// will never cause them to occur:
@@ -351,7 +355,7 @@ export const model = (currentState: State, resW: ResponseType):
};
} else if (Either.isLeft(res)) {
const left = res.left;
- if (isLeftTypeof(left, 'index_not_yellow_timeout')) {
+ if (isTypeof(left, 'index_not_yellow_timeout')) {
// A yellow status timeout could theoretically be temporary for a busy cluster
// that takes a long time to allocate the primary and we retry the action to see if
// we get a response.
@@ -368,36 +372,61 @@ export const model = (currentState: State, resW: ResponseType):
} else if (stateP.controlState === 'CHECK_UNKNOWN_DOCUMENTS') {
const res = resW as ExcludeRetryableEsError>;
- if (Either.isRight(res)) {
- const source = stateP.sourceIndex;
- const target = stateP.versionIndex;
- return {
- ...stateP,
- controlState: 'SET_SOURCE_WRITE_BLOCK',
- sourceIndex: source,
- targetIndex: target,
- targetIndexMappings: disableUnknownTypeMappingFields(
- stateP.targetIndexMappings,
- stateP.sourceIndexMappings
- ),
- versionIndexReadyActions: Option.some([
- { remove: { index: source.value, alias: stateP.currentAlias, must_exist: true } },
- { add: { index: target, alias: stateP.currentAlias } },
- { add: { index: target, alias: stateP.versionAlias } },
- { remove_index: { index: stateP.tempIndex } },
- ]),
- };
- } else {
- if (isLeftTypeof(res.left, 'unknown_docs_found')) {
+ let logs: MigrationLog[] = stateP.logs;
+ let excludeOnUpgradeQuery = stateP.excludeOnUpgradeQuery;
+
+ if (isTypeof(res.right, 'unknown_docs_found')) {
+ if (!stateP.discardUnknownObjects) {
return {
...stateP,
controlState: 'FATAL',
- reason: extractUnknownDocFailureReason(res.left.unknownDocs, stateP.sourceIndex.value),
+ reason: extractUnknownDocFailureReason(
+ stateP.migrationDocLinks.resolveMigrationFailures,
+ res.right.unknownDocs
+ ),
};
- } else {
- return throwBadResponse(stateP, res.left);
}
+
+ // at this point, users have configured kibana to discard unknown objects
+ // thus, we can ignore unknown documents and proceed with the migration
+ logs = [
+ ...stateP.logs,
+ { level: 'warning', message: extractDiscardedUnknownDocs(res.right.unknownDocs) },
+ ];
+
+ const unknownTypes = [...new Set(res.right.unknownDocs.map(({ type }) => type))];
+
+ excludeOnUpgradeQuery = addExcludedTypesToBoolQuery(
+ unknownTypes,
+ stateP.excludeOnUpgradeQuery?.bool
+ );
+
+ excludeOnUpgradeQuery = addMustClausesToBoolQuery(
+ [{ exists: { field: 'type' } }],
+ excludeOnUpgradeQuery.bool
+ );
}
+
+ const source = stateP.sourceIndex;
+ const target = stateP.versionIndex;
+ return {
+ ...stateP,
+ controlState: 'SET_SOURCE_WRITE_BLOCK',
+ logs,
+ excludeOnUpgradeQuery,
+ sourceIndex: source,
+ targetIndex: target,
+ targetIndexMappings: disableUnknownTypeMappingFields(
+ stateP.targetIndexMappings,
+ stateP.sourceIndexMappings
+ ),
+ versionIndexReadyActions: Option.some([
+ { remove: { index: source.value, alias: stateP.currentAlias, must_exist: true } },
+ { add: { index: target, alias: stateP.currentAlias } },
+ { add: { index: target, alias: stateP.versionAlias } },
+ { remove_index: { index: stateP.tempIndex } },
+ ]),
+ };
} else if (stateP.controlState === 'SET_SOURCE_WRITE_BLOCK') {
const res = resW as ExcludeRetryableEsError>;
if (Either.isRight(res)) {
@@ -406,7 +435,7 @@ export const model = (currentState: State, resW: ResponseType):
...stateP,
controlState: 'CALCULATE_EXCLUDE_FILTERS',
};
- } else if (isLeftTypeof(res.left, 'index_not_found_exception')) {
+ } else if (isTypeof(res.left, 'index_not_found_exception')) {
// We don't handle the following errors as the migration algorithm
// will never cause them to occur:
// - index_not_found_exception
@@ -418,16 +447,15 @@ export const model = (currentState: State, resW: ResponseType):
const res = resW as ExcludeRetryableEsError>;
if (Either.isRight(res)) {
- const unusedTypesQuery: estypes.QueryDslQueryContainer = {
- bool: {
- filter: [stateP.unusedTypesQuery, res.right.excludeFilter],
- },
- };
+ const excludeOnUpgradeQuery = addMustNotClausesToBoolQuery(
+ res.right.mustNotClauses,
+ stateP.excludeOnUpgradeQuery?.bool
+ );
return {
...stateP,
controlState: 'CREATE_REINDEX_TEMP',
- unusedTypesQuery,
+ excludeOnUpgradeQuery,
logs: [
...stateP.logs,
...Object.entries(res.right.errorsByType).map(([soType, error]) => ({
@@ -445,7 +473,7 @@ export const model = (currentState: State, resW: ResponseType):
return { ...stateP, controlState: 'REINDEX_SOURCE_TO_TEMP_OPEN_PIT' };
} else if (Either.isLeft(res)) {
const left = res.left;
- if (isLeftTypeof(left, 'index_not_yellow_timeout')) {
+ if (isTypeof(left, 'index_not_yellow_timeout')) {
// `index_not_yellow_timeout` for the CREATE_REINDEX_TEMP target temp index:
// The index status did not go yellow within the specified timeout period.
// A yellow status timeout could theoretically be temporary for a busy cluster.
@@ -454,7 +482,7 @@ export const model = (currentState: State, resW: ResponseType):
// continue to timeout and eventually lead to a failed migration.
const retryErrorMessage = `${left.message} Refer to ${stateP.migrationDocLinks.repeatedTimeoutRequests} for information on how to resolve the issue.`;
return delayRetryState(stateP, retryErrorMessage, stateP.retryAttempts);
- } else if (isLeftTypeof(left, 'cluster_shard_limit_exceeded')) {
+ } else if (isTypeof(left, 'cluster_shard_limit_exceeded')) {
return {
...stateP,
controlState: 'FATAL',
@@ -588,7 +616,7 @@ export const model = (currentState: State, resW: ResponseType):
} else {
// we have failures from the current batch of documents and add them to the lists
const left = res.left;
- if (isLeftTypeof(left, 'documents_transform_failed')) {
+ if (isTypeof(left, 'documents_transform_failed')) {
return {
...stateP,
controlState: 'REINDEX_SOURCE_TO_TEMP_READ',
@@ -621,8 +649,8 @@ export const model = (currentState: State, resW: ResponseType):
}
} else {
if (
- isLeftTypeof(res.left, 'target_index_had_write_block') ||
- isLeftTypeof(res.left, 'index_not_found_exception')
+ isTypeof(res.left, 'target_index_had_write_block') ||
+ isTypeof(res.left, 'index_not_found_exception')
) {
// When the temp index has a write block or has been deleted another
// instance already completed this step. Close the PIT search and carry
@@ -631,7 +659,7 @@ export const model = (currentState: State, resW: ResponseType):
...stateP,
controlState: 'REINDEX_SOURCE_TO_TEMP_CLOSE_PIT',
};
- } else if (isLeftTypeof(res.left, 'request_entity_too_large_exception')) {
+ } else if (isTypeof(res.left, 'request_entity_too_large_exception')) {
return {
...stateP,
controlState: 'FATAL',
@@ -649,7 +677,7 @@ export const model = (currentState: State, resW: ResponseType):
};
} else {
const left = res.left;
- if (isLeftTypeof(left, 'index_not_found_exception')) {
+ if (isTypeof(left, 'index_not_found_exception')) {
// index_not_found_exception:
// another instance completed the MARK_VERSION_INDEX_READY and
// removed the temp index.
@@ -673,7 +701,7 @@ export const model = (currentState: State, resW: ResponseType):
};
} else {
const left = res.left;
- if (isLeftTypeof(left, 'index_not_found_exception')) {
+ if (isTypeof(left, 'index_not_found_exception')) {
// index_not_found_exception means another instance already completed
// the MARK_VERSION_INDEX_READY step and removed the temp index
// We still perform the REFRESH_TARGET, OUTDATED_DOCUMENTS_* and
@@ -683,7 +711,7 @@ export const model = (currentState: State, resW: ResponseType):
...stateP,
controlState: 'REFRESH_TARGET',
};
- } else if (isLeftTypeof(left, 'index_not_yellow_timeout')) {
+ } else if (isTypeof(left, 'index_not_yellow_timeout')) {
// `index_not_yellow_timeout` for the CLONE_TEMP_TO_TARGET source -> target index:
// The target index status did not go yellow within the specified timeout period.
// The cluster could just be busy and we retry the action.
@@ -695,7 +723,7 @@ export const model = (currentState: State, resW: ResponseType):
// continue to timeout and eventually lead to a failed migration.
const retryErrorMessage = `${left.message} Refer to ${stateP.migrationDocLinks.repeatedTimeoutRequests} for information on how to resolve the issue.`;
return delayRetryState(stateP, retryErrorMessage, stateP.retryAttempts);
- } else if (isLeftTypeof(left, 'cluster_shard_limit_exceeded')) {
+ } else if (isTypeof(left, 'cluster_shard_limit_exceeded')) {
return {
...stateP,
controlState: 'FATAL',
@@ -817,7 +845,7 @@ export const model = (currentState: State, resW: ResponseType):
};
}
} else {
- if (isLeftTypeof(res.left, 'documents_transform_failed')) {
+ if (isTypeof(res.left, 'documents_transform_failed')) {
// continue to build up any more transformation errors before failing the migration.
return {
...stateP,
@@ -849,15 +877,15 @@ export const model = (currentState: State, resW: ResponseType):
hasTransformedDocs: true,
};
} else {
- if (isLeftTypeof(res.left, 'request_entity_too_large_exception')) {
+ if (isTypeof(res.left, 'request_entity_too_large_exception')) {
return {
...stateP,
controlState: 'FATAL',
reason: FATAL_REASON_REQUEST_ENTITY_TOO_LARGE,
};
} else if (
- isLeftTypeof(res.left, 'target_index_had_write_block') ||
- isLeftTypeof(res.left, 'index_not_found_exception')
+ isTypeof(res.left, 'target_index_had_write_block') ||
+ isTypeof(res.left, 'index_not_found_exception')
) {
// we fail on these errors since the target index will never get
// deleted and should only have a write block if a newer version of
@@ -929,7 +957,7 @@ export const model = (currentState: State, resW: ResponseType):
}
} else {
const left = res.left;
- if (isLeftTypeof(left, 'wait_for_task_completion_timeout')) {
+ if (isTypeof(left, 'wait_for_task_completion_timeout')) {
// After waiting for the specified timeout, the task has not yet
// completed. Retry this step to see if the task has completed after an
// exponential delay. We will basically keep polling forever until the
@@ -948,7 +976,7 @@ export const model = (currentState: State, resW: ResponseType):
};
} else if (Either.isLeft(res)) {
const left = res.left;
- if (isLeftTypeof(left, 'index_not_yellow_timeout')) {
+ if (isTypeof(left, 'index_not_yellow_timeout')) {
// `index_not_yellow_timeout` for the CREATE_NEW_TARGET target index:
// The cluster might just be busy and we retry the action for a set number of times.
// If the cluster hit the low watermark for disk usage the action will continue to timeout.
@@ -956,7 +984,7 @@ export const model = (currentState: State, resW: ResponseType):
// continue to timeout and eventually lead to a failed migration.
const retryErrorMessage = `${left.message} Refer to ${stateP.migrationDocLinks.repeatedTimeoutRequests} for information on how to resolve the issue.`;
return delayRetryState(stateP, retryErrorMessage, stateP.retryAttempts);
- } else if (isLeftTypeof(left, 'cluster_shard_limit_exceeded')) {
+ } else if (isTypeof(left, 'cluster_shard_limit_exceeded')) {
return {
...stateP,
controlState: 'FATAL',
@@ -977,13 +1005,13 @@ export const model = (currentState: State, resW: ResponseType):
return { ...stateP, controlState: 'DONE' };
} else {
const left = res.left;
- if (isLeftTypeof(left, 'alias_not_found_exception')) {
+ if (isTypeof(left, 'alias_not_found_exception')) {
// the versionIndexReadyActions checks that the currentAlias is still
// pointing to the source index. If this fails with an
// alias_not_found_exception another instance has completed a
// migration from the same source.
return { ...stateP, controlState: 'MARK_VERSION_INDEX_READY_CONFLICT' };
- } else if (isLeftTypeof(left, 'index_not_found_exception')) {
+ } else if (isTypeof(left, 'index_not_found_exception')) {
if (left.index === stateP.tempIndex) {
// another instance has already completed the migration and deleted
// the temporary index
@@ -994,7 +1022,7 @@ export const model = (currentState: State, resW: ResponseType):
// index handled above.
throwBadResponse(stateP, left as never);
}
- } else if (isLeftTypeof(left, 'remove_index_not_a_concrete_index')) {
+ } else if (isTypeof(left, 'remove_index_not_a_concrete_index')) {
// We don't handle this error as the migration algorithm will never
// cause it to occur (this error is only relevant to the LEGACY_DELETE
// step).
diff --git a/src/core/server/saved_objects/migrations/next.ts b/src/core/server/saved_objects/migrations/next.ts
index 24a4204c3009e..e50331d3b7658 100644
--- a/src/core/server/saved_objects/migrations/next.ts
+++ b/src/core/server/saved_objects/migrations/next.ts
@@ -40,9 +40,9 @@ import type {
CheckUnknownDocumentsState,
CalculateExcludeFiltersState,
} from './state';
-import { TransformRawDocs } from './types';
+import type { TransformRawDocs } from './types';
import * as Actions from './actions';
-import { ElasticsearchClient } from '../../elasticsearch';
+import type { ElasticsearchClient } from '../../elasticsearch';
type ActionMap = ReturnType;
@@ -66,7 +66,7 @@ export const nextActionMap = (client: ElasticsearchClient, transformRawDocs: Tra
Actions.checkForUnknownDocs({
client,
indexName: state.sourceIndex.value,
- unusedTypesQuery: state.unusedTypesQuery,
+ excludeOnUpgradeQuery: state.excludeOnUpgradeQuery,
knownTypes: state.knownTypes,
}),
SET_SOURCE_WRITE_BLOCK: (state: SetSourceWriteBlockState) =>
@@ -98,7 +98,7 @@ export const nextActionMap = (client: ElasticsearchClient, transformRawDocs: Tra
* are no longer used. These saved objects will still be kept in the outdated
* index for backup purposes, but won't be available in the upgraded index.
*/
- query: state.unusedTypesQuery,
+ query: state.excludeOnUpgradeQuery,
batchSize: state.batchSize,
searchAfter: state.lastHitSortValue,
}),
@@ -187,7 +187,7 @@ export const nextActionMap = (client: ElasticsearchClient, transformRawDocs: Tra
targetIndex: state.sourceIndex.value,
reindexScript: state.preMigrationScript,
requireAlias: false,
- unusedTypesQuery: state.unusedTypesQuery,
+ excludeOnUpgradeQuery: state.excludeOnUpgradeQuery,
}),
LEGACY_REINDEX_WAIT_FOR_TASK: (state: LegacyReindexWaitForTaskState) =>
Actions.waitForReindexTask({ client, taskId: state.legacyReindexTaskId, timeout: '60s' }),
diff --git a/src/core/server/saved_objects/migrations/run_resilient_migrator.ts b/src/core/server/saved_objects/migrations/run_resilient_migrator.ts
index 6e5766c9f5dcf..194496b826b0e 100644
--- a/src/core/server/saved_objects/migrations/run_resilient_migrator.ts
+++ b/src/core/server/saved_objects/migrations/run_resilient_migrator.ts
@@ -59,6 +59,7 @@ export async function runResilientMigrator({
migrationsConfig,
typeRegistry,
docLinks,
+ logger,
});
return migrationStateActionMachine({
initialState,
diff --git a/src/core/server/saved_objects/migrations/state.ts b/src/core/server/saved_objects/migrations/state.ts
index dee6839d6b902..3dac96987eb92 100644
--- a/src/core/server/saved_objects/migrations/state.ts
+++ b/src/core/server/saved_objects/migrations/state.ts
@@ -7,15 +7,15 @@
*/
import * as Option from 'fp-ts/lib/Option';
-import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
+import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import type { DocLinks } from '@kbn/doc-links';
-import { ControlState } from './state_action_machine';
-import { AliasAction } from './actions';
-import { IndexMapping } from '../mappings';
-import { SavedObjectsRawDoc } from '..';
-import { TransformErrorObjects } from './core';
-import { SavedObjectTypeExcludeFromUpgradeFilterHook } from '../types';
-import { MigrationLog, Progress } from './types';
+import type { ControlState } from './state_action_machine';
+import type { AliasAction } from './actions';
+import type { IndexMapping } from '../mappings';
+import type { SavedObjectsRawDoc } from '..';
+import type { TransformErrorObjects } from './core';
+import type { SavedObjectTypeExcludeFromUpgradeFilterHook } from '../types';
+import type { MigrationLog, Progress } from './types';
export interface BaseState extends ControlState {
/** The first part of the index name such as `.kibana` or `.kibana_task_manager` */
@@ -39,7 +39,7 @@ export interface BaseState extends ControlState {
readonly tempIndexMappings: IndexMapping;
/** Script to apply to a legacy index before it can be used as a migration source */
readonly preMigrationScript: Option.Option;
- readonly outdatedDocumentsQuery: estypes.QueryDslQueryContainer;
+ readonly outdatedDocumentsQuery: QueryDslQueryContainer;
readonly retryCount: number;
readonly retryDelay: number;
/**
@@ -87,6 +87,14 @@ export interface BaseState extends ControlState {
*/
readonly maxBatchSizeBytes: number;
readonly logs: MigrationLog[];
+ /**
+ * If saved objects exist which have an unknown type they will cause
+ * the migration to fail. If this flag is set to `true`, kibana will
+ * discard the unknown objects and proceed with the migration.
+ * This can happen, for instance, if a plugin that had registered some
+ * saved objects is disabled.
+ */
+ readonly discardUnknownObjects: boolean;
/**
* The current alias e.g. `.kibana` which always points to the latest
* version index
@@ -107,11 +115,16 @@ export interface BaseState extends ControlState {
*/
readonly tempIndex: string;
/**
- * When reindexing we use a source query to exclude saved objects types which
- * are no longer used. These saved objects will still be kept in the outdated
+ * When upgrading to a more recent kibana version, some saved object types
+ * might be conflicting or no longer used.
+ * When reindexing, we use a source query to exclude types which are:
+ * - no longer used
+ * - unknown (e.g. belonging to plugins that have been disabled)
+ * - explicitly excluded from upgrades by plugin developers
+ * These saved objects will still be kept in the outdated
* index for backup purposes, but won't be available in the upgraded index.
*/
- readonly unusedTypesQuery: estypes.QueryDslQueryContainer;
+ readonly excludeOnUpgradeQuery: QueryDslQueryContainer;
/**
* The list of known SO types that are registered.
*/
@@ -146,7 +159,7 @@ export interface PostInitState extends BaseState {
/** The target index is the index to which the migration writes */
readonly targetIndex: string;
readonly versionIndexReadyActions: Option.Option;
- readonly outdatedDocumentsQuery: estypes.QueryDslQueryContainer;
+ readonly outdatedDocumentsQuery: QueryDslQueryContainer;
}
export interface DoneState extends PostInitState {
diff --git a/src/core/server/saved_objects/saved_objects_config.ts b/src/core/server/saved_objects/saved_objects_config.ts
index 4a35ea116e8ee..f3b73897fbf1b 100644
--- a/src/core/server/saved_objects/saved_objects_config.ts
+++ b/src/core/server/saved_objects/saved_objects_config.ts
@@ -6,12 +6,19 @@
* Side Public License, v 1.
*/
+import { valid } from 'semver';
import { schema, TypeOf } from '@kbn/config-schema';
import type { ServiceConfigDescriptor } from '@kbn/core-base-server-internal';
const migrationSchema = schema.object({
batchSize: schema.number({ defaultValue: 1_000 }),
maxBatchSizeBytes: schema.byteSize({ defaultValue: '100mb' }), // 100mb is the default http.max_content_length Elasticsearch config value
+ discardUnknownObjects: schema.maybe(
+ schema.string({
+ validate: (value: string) =>
+ valid(value) ? undefined : 'The value is not a valid semantic version',
+ })
+ ),
scrollDuration: schema.string({ defaultValue: '15m' }),
pollInterval: schema.number({ defaultValue: 1_500 }),
skip: schema.boolean({ defaultValue: false }),
diff --git a/src/core/server/saved_objects/service/lib/find_shared_origin_objects.test.ts b/src/core/server/saved_objects/service/lib/find_shared_origin_objects.test.ts
index c8e0796dea18e..ed6ecf7b33d61 100644
--- a/src/core/server/saved_objects/service/lib/find_shared_origin_objects.test.ts
+++ b/src/core/server/saved_objects/service/lib/find_shared_origin_objects.test.ts
@@ -6,7 +6,7 @@
* Side Public License, v 1.
*/
-import type { DeeplyMockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
import type { CreatePointInTimeFinderFn, PointInTimeFinder } from './point_in_time_finder';
import { savedObjectsPointInTimeFinderMock } from './point_in_time_finder.mock';
diff --git a/src/core/server/saved_objects/service/lib/legacy_url_aliases/find_legacy_url_aliases.test.ts b/src/core/server/saved_objects/service/lib/legacy_url_aliases/find_legacy_url_aliases.test.ts
index f0399f4b54aa0..f171b0651ffb4 100644
--- a/src/core/server/saved_objects/service/lib/legacy_url_aliases/find_legacy_url_aliases.test.ts
+++ b/src/core/server/saved_objects/service/lib/legacy_url_aliases/find_legacy_url_aliases.test.ts
@@ -6,7 +6,7 @@
* Side Public License, v 1.
*/
-import type { DeeplyMockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
import { LegacyUrlAlias, LEGACY_URL_ALIAS_TYPE } from '../../../object_types';
import type { CreatePointInTimeFinderFn, PointInTimeFinder } from '../point_in_time_finder';
diff --git a/src/core/server/saved_objects/service/lib/preflight_check_for_create.test.ts b/src/core/server/saved_objects/service/lib/preflight_check_for_create.test.ts
index 8d7cfd25ac885..b7c61bc6f7711 100644
--- a/src/core/server/saved_objects/service/lib/preflight_check_for_create.test.ts
+++ b/src/core/server/saved_objects/service/lib/preflight_check_for_create.test.ts
@@ -11,7 +11,7 @@ import {
mockRawDocExistsInNamespaces,
} from './preflight_check_for_create.test.mock';
-import type { DeeplyMockedKeys } from '@kbn/utility-types/jest';
+import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
import type { ElasticsearchClient } from '../../../elasticsearch';
import { elasticsearchClientMock } from '../../../elasticsearch/client/mocks';
diff --git a/src/core/server/server.api.md b/src/core/server/server.api.md
index a05d25bb37ecb..b65d335783d43 100644
--- a/src/core/server/server.api.md
+++ b/src/core/server/server.api.md
@@ -25,6 +25,7 @@ import { ConfigService } from '@kbn/config';
import { ContextProviderOpts } from '@kbn/analytics-client';
import { CoreId } from '@kbn/core-base-common-internal';
import { DetailedPeerCertificate } from 'tls';
+import { DiscoveredPlugin } from '@kbn/core-base-common';
import type { DocLinks } from '@kbn/doc-links';
import { Duration } from 'moment';
import { Duration as Duration_2 } from 'moment-timezone';
@@ -56,6 +57,7 @@ import { PathConfigType } from '@kbn/utils';
import { PeerCertificate } from 'tls';
import { PluginName } from '@kbn/core-base-common';
import { PluginOpaqueId } from '@kbn/core-base-common';
+import { PluginType } from '@kbn/core-base-common';
import { Readable } from 'stream';
import { RecursiveReadonly } from '@kbn/utility-types';
import { Request as Request_2 } from '@hapi/hapi';
@@ -922,16 +924,7 @@ export interface DeprecationsServiceSetup {
// @public
export type DestructiveRouteMethod = 'post' | 'put' | 'delete' | 'patch';
-// @public
-export interface DiscoveredPlugin {
- readonly configPath: ConfigPath;
- readonly enabledOnAnonymousPages?: boolean;
- readonly id: PluginName;
- readonly optionalPlugins: readonly PluginName[];
- readonly requiredBundles: readonly PluginName[];
- readonly requiredPlugins: readonly PluginName[];
- readonly type: PluginType;
-}
+export { DiscoveredPlugin }
// @public (undocumented)
export interface DocLinksServiceSetup {
@@ -1816,11 +1809,7 @@ export interface PluginsServiceStart {
contracts: Map;
}
-// @public (undocumented)
-export enum PluginType {
- preboot = "preboot",
- standard = "standard"
-}
+export { PluginType }
// @public (undocumented)
export const pollEsNodesVersion: ({ internalClient, log, kibanaVersion, ignoreVersionMismatch, esVersionCheckInterval: healthCheckInterval, }: PollEsNodesVersionOptions) => Observable;
@@ -3256,8 +3245,8 @@ export const validBodyOutput: readonly ["data", "stream"];
//
// src/core/server/elasticsearch/client/types.ts:81:7 - (ae-forgotten-export) The symbol "Explanation" needs to be exported by the entry point index.d.ts
// src/core/server/http/router/response.ts:302:3 - (ae-forgotten-export) The symbol "KibanaResponse" needs to be exported by the entry point index.d.ts
-// src/core/server/plugins/types.ts:403:3 - (ae-forgotten-export) The symbol "SharedGlobalConfigKeys" needs to be exported by the entry point index.d.ts
-// src/core/server/plugins/types.ts:405:3 - (ae-forgotten-export) The symbol "SavedObjectsConfigType" needs to be exported by the entry point index.d.ts
-// src/core/server/plugins/types.ts:512:5 - (ae-unresolved-link) The @link reference could not be resolved: The package "kibana" does not have an export "create"
+// src/core/server/plugins/types.ts:339:3 - (ae-forgotten-export) The symbol "SharedGlobalConfigKeys" needs to be exported by the entry point index.d.ts
+// src/core/server/plugins/types.ts:341:3 - (ae-forgotten-export) The symbol "SavedObjectsConfigType" needs to be exported by the entry point index.d.ts
+// src/core/server/plugins/types.ts:448:5 - (ae-unresolved-link) The @link reference could not be resolved: The package "kibana" does not have an export "create"
```
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.mock.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.mock.ts
index 11dabaca71e13..11ff892b89bf1 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.mock.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.mock.ts
@@ -6,7 +6,17 @@
* Side Public License, v 1.
*/
-export const createOrUpgradeSavedConfigMock = jest.fn();
-jest.doMock('./create_or_upgrade_saved_config', () => ({
- createOrUpgradeSavedConfig: createOrUpgradeSavedConfigMock,
+import type { TransformConfigFn } from '../saved_objects';
+import type { getUpgradeableConfig } from './get_upgradeable_config';
+
+export const mockTransform = jest.fn() as jest.MockedFunction;
+jest.mock('../saved_objects', () => ({
+ transforms: [mockTransform],
+}));
+
+export const mockGetUpgradeableConfig = jest.fn() as jest.MockedFunction<
+ typeof getUpgradeableConfig
+>;
+jest.mock('./get_upgradeable_config', () => ({
+ getUpgradeableConfig: mockGetUpgradeableConfig,
}));
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.ts
index 669849dcd8d9b..fe6aa83bb0296 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.ts
@@ -6,29 +6,28 @@
* Side Public License, v 1.
*/
-import Chance from 'chance';
-
-import { getUpgradeableConfigMock } from './get_upgradeable_config.test.mock';
+import {
+ mockTransform,
+ mockGetUpgradeableConfig,
+} from './create_or_upgrade_saved_config.test.mock';
import { SavedObjectsErrorHelpers } from '../../saved_objects';
import { savedObjectsClientMock } from '../../saved_objects/service/saved_objects_client.mock';
import { loggingSystemMock } from '../../logging/logging_system.mock';
import { createOrUpgradeSavedConfig } from './create_or_upgrade_saved_config';
-const chance = new Chance();
describe('uiSettings/createOrUpgradeSavedConfig', function () {
afterEach(() => jest.resetAllMocks());
const version = '4.0.1';
const prevVersion = '4.0.0';
- const buildNum = chance.integer({ min: 1000, max: 5000 });
+ const buildNum = 1337;
function setup() {
const logger = loggingSystemMock.create();
- const getUpgradeableConfig = getUpgradeableConfigMock;
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.create.mockImplementation(
- async (type, attributes, options = {}) =>
+ async (type, _, options = {}) =>
({
type,
id: options.id,
@@ -46,8 +45,8 @@ describe('uiSettings/createOrUpgradeSavedConfig', function () {
...options,
});
- expect(getUpgradeableConfigMock).toHaveBeenCalledTimes(1);
- expect(getUpgradeableConfig).toHaveBeenCalledWith({ savedObjectsClient, version });
+ expect(mockGetUpgradeableConfig).toHaveBeenCalledTimes(1);
+ expect(mockGetUpgradeableConfig).toHaveBeenCalledWith({ savedObjectsClient, version });
return resp;
}
@@ -58,7 +57,6 @@ describe('uiSettings/createOrUpgradeSavedConfig', function () {
run,
version,
savedObjectsClient,
- getUpgradeableConfig,
};
}
@@ -83,25 +81,21 @@ describe('uiSettings/createOrUpgradeSavedConfig', function () {
describe('something is upgradeable', () => {
it('should merge upgraded attributes with current build number in new config', async () => {
- const { run, getUpgradeableConfig, savedObjectsClient } = setup();
+ const { run, savedObjectsClient } = setup();
const savedAttributes = {
buildNum: buildNum - 100,
- [chance.word()]: chance.sentence(),
- [chance.word()]: chance.sentence(),
- [chance.word()]: chance.sentence(),
+ defaultIndex: 'some-index',
};
- getUpgradeableConfig.mockResolvedValue({
+ mockGetUpgradeableConfig.mockResolvedValue({
id: prevVersion,
attributes: savedAttributes,
- type: '',
- references: [],
});
await run();
- expect(getUpgradeableConfig).toHaveBeenCalledTimes(1);
+ expect(mockGetUpgradeableConfig).toHaveBeenCalledTimes(1);
expect(savedObjectsClient.create).toHaveBeenCalledTimes(1);
expect(savedObjectsClient.create).toHaveBeenCalledWith(
'config',
@@ -115,14 +109,42 @@ describe('uiSettings/createOrUpgradeSavedConfig', function () {
);
});
+ it('should prefer transformed attributes when merging', async () => {
+ const { run, savedObjectsClient } = setup();
+ mockGetUpgradeableConfig.mockResolvedValue({
+ id: prevVersion,
+ attributes: {
+ buildNum: buildNum - 100,
+ defaultIndex: 'some-index',
+ },
+ });
+ mockTransform.mockResolvedValue({
+ defaultIndex: 'another-index',
+ isDefaultIndexMigrated: true,
+ });
+
+ await run();
+
+ expect(mockGetUpgradeableConfig).toHaveBeenCalledTimes(1);
+ expect(mockTransform).toHaveBeenCalledTimes(1);
+ expect(savedObjectsClient.create).toHaveBeenCalledTimes(1);
+ expect(savedObjectsClient.create).toHaveBeenCalledWith(
+ 'config',
+ {
+ buildNum,
+ defaultIndex: 'another-index',
+ isDefaultIndexMigrated: true,
+ },
+ { id: version }
+ );
+ });
+
it('should log a message for upgrades', async () => {
- const { getUpgradeableConfig, logger, run } = setup();
+ const { logger, run } = setup();
- getUpgradeableConfig.mockResolvedValue({
+ mockGetUpgradeableConfig.mockResolvedValue({
id: prevVersion,
attributes: { buildNum: buildNum - 100 },
- type: '',
- references: [],
});
await run();
@@ -144,13 +166,11 @@ describe('uiSettings/createOrUpgradeSavedConfig', function () {
});
it('does not log when upgrade fails', async () => {
- const { getUpgradeableConfig, logger, run, savedObjectsClient } = setup();
+ const { logger, run, savedObjectsClient } = setup();
- getUpgradeableConfig.mockResolvedValue({
+ mockGetUpgradeableConfig.mockResolvedValue({
id: prevVersion,
attributes: { buildNum: buildNum - 100 },
- type: '',
- references: [],
});
savedObjectsClient.create.mockRejectedValue(new Error('foo'));
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.ts
index d55a6537957cb..247a3cce974f4 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/create_or_upgrade_saved_config.ts
@@ -9,10 +9,12 @@
import { defaults } from 'lodash';
import type { Logger, LogMeta } from '@kbn/logging';
+import { asyncForEach } from '@kbn/std';
import { SavedObjectsClientContract } from '../../saved_objects/types';
import { SavedObjectsErrorHelpers } from '../../saved_objects';
import { getUpgradeableConfig } from './get_upgradeable_config';
+import { transforms } from '../saved_objects';
interface ConfigLogMeta extends LogMeta {
kibana: {
@@ -39,10 +41,22 @@ export async function createOrUpgradeSavedConfig(
version,
});
+ let transformDefaults = {};
+ await asyncForEach(transforms, async (transformFn) => {
+ const result = await transformFn({
+ savedObjectsClient,
+ configAttributes: upgradeableConfig?.attributes,
+ });
+ transformDefaults = { ...transformDefaults, ...result };
+ });
+
// default to the attributes of the upgradeableConfig if available
const attributes = defaults(
- { buildNum },
- upgradeableConfig ? (upgradeableConfig.attributes as any) : {}
+ {
+ buildNum,
+ ...transformDefaults, // Any defaults that should be applied from transforms
+ },
+ upgradeableConfig?.attributes
);
try {
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.test.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.test.ts
index 320fa9056fcaf..34ef0c5b4f2e1 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.test.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.test.ts
@@ -8,69 +8,70 @@
import { getUpgradeableConfig } from './get_upgradeable_config';
import { savedObjectsClientMock } from '../../saved_objects/service/saved_objects_client.mock';
+import { SavedObjectsFindResponse } from '../../saved_objects';
describe('getUpgradeableConfig', () => {
it('finds saved objects with type "config"', async () => {
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.find.mockResolvedValue({
- saved_objects: [{ id: '7.5.0' }],
- } as any);
+ saved_objects: [{ id: '7.5.0', attributes: 'foo' }],
+ } as SavedObjectsFindResponse);
await getUpgradeableConfig({ savedObjectsClient, version: '7.5.0' });
expect(savedObjectsClient.find.mock.calls[0][0].type).toBe('config');
});
it('finds saved config with version < than Kibana version', async () => {
- const savedConfig = { id: '7.4.0' };
+ const savedConfig = { id: '7.4.0', attributes: 'foo' };
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.find.mockResolvedValue({
saved_objects: [savedConfig],
- } as any);
+ } as SavedObjectsFindResponse);
const result = await getUpgradeableConfig({ savedObjectsClient, version: '7.5.0' });
- expect(result).toBe(savedConfig);
+ expect(result).toEqual(savedConfig);
});
it('finds saved config with RC version === Kibana version', async () => {
- const savedConfig = { id: '7.5.0-rc1' };
+ const savedConfig = { id: '7.5.0-rc1', attributes: 'foo' };
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.find.mockResolvedValue({
saved_objects: [savedConfig],
- } as any);
+ } as SavedObjectsFindResponse);
const result = await getUpgradeableConfig({ savedObjectsClient, version: '7.5.0' });
- expect(result).toBe(savedConfig);
+ expect(result).toEqual(savedConfig);
});
it('does not find saved config with version === Kibana version', async () => {
- const savedConfig = { id: '7.5.0' };
+ const savedConfig = { id: '7.5.0', attributes: 'foo' };
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.find.mockResolvedValue({
saved_objects: [savedConfig],
- } as any);
+ } as SavedObjectsFindResponse);
const result = await getUpgradeableConfig({ savedObjectsClient, version: '7.5.0' });
- expect(result).toBe(undefined);
+ expect(result).toBe(null);
});
it('does not find saved config with version > Kibana version', async () => {
- const savedConfig = { id: '7.6.0' };
+ const savedConfig = { id: '7.6.0', attributes: 'foo' };
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.find.mockResolvedValue({
saved_objects: [savedConfig],
- } as any);
+ } as SavedObjectsFindResponse);
const result = await getUpgradeableConfig({ savedObjectsClient, version: '7.5.0' });
- expect(result).toBe(undefined);
+ expect(result).toBe(null);
});
it('handles empty config', async () => {
const savedObjectsClient = savedObjectsClientMock.create();
savedObjectsClient.find.mockResolvedValue({
saved_objects: [],
- } as any);
+ } as unknown as SavedObjectsFindResponse);
const result = await getUpgradeableConfig({ savedObjectsClient, version: '7.5.0' });
- expect(result).toBe(undefined);
+ expect(result).toBe(null);
});
});
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.ts
index 493efa518f22b..6c8e8cbe9ac3a 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/get_upgradeable_config.ts
@@ -7,8 +7,18 @@
*/
import { SavedObjectsClientContract } from '../../saved_objects/types';
+import type { ConfigAttributes } from '../saved_objects';
import { isConfigVersionUpgradeable } from './is_config_version_upgradeable';
+/**
+ * This contains a subset of `config` object attributes that are relevant for upgrading it using transform functions.
+ * It is a superset of all the attributes needed for all of the transform functions defined in `transforms.ts`.
+ */
+export interface UpgradeableConfigAttributes extends ConfigAttributes {
+ defaultIndex?: string;
+ isDefaultIndexMigrated?: boolean;
+}
+
/**
* Find the most recent SavedConfig that is upgradeable to the specified version
* @param {Object} options
@@ -24,14 +34,21 @@ export async function getUpgradeableConfig({
version: string;
}) {
// attempt to find a config we can upgrade
- const { saved_objects: savedConfigs } = await savedObjectsClient.find({
- type: 'config',
- page: 1,
- perPage: 1000,
- sortField: 'buildNum',
- sortOrder: 'desc',
- });
+ const { saved_objects: savedConfigs } =
+ await savedObjectsClient.find({
+ type: 'config',
+ page: 1,
+ perPage: 1000,
+ sortField: 'buildNum',
+ sortOrder: 'desc',
+ });
// try to find a config that we can upgrade
- return savedConfigs.find((savedConfig) => isConfigVersionUpgradeable(savedConfig.id, version));
+ const findResult = savedConfigs.find((savedConfig) =>
+ isConfigVersionUpgradeable(savedConfig.id, version)
+ );
+ if (findResult) {
+ return { id: findResult.id, attributes: findResult.attributes };
+ }
+ return null;
}
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/index.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/index.ts
index 3559c8555803a..e5f8f895e8631 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/index.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/index.ts
@@ -7,3 +7,4 @@
*/
export { createOrUpgradeSavedConfig } from './create_or_upgrade_saved_config';
+export type { UpgradeableConfigAttributes } from './get_upgradeable_config';
diff --git a/src/core/server/ui_settings/create_or_upgrade_saved_config/integration_tests/create_or_upgrade.test.ts b/src/core/server/ui_settings/create_or_upgrade_saved_config/integration_tests/create_or_upgrade.test.ts
index 1da8bad330c31..44efc558e1029 100644
--- a/src/core/server/ui_settings/create_or_upgrade_saved_config/integration_tests/create_or_upgrade.test.ts
+++ b/src/core/server/ui_settings/create_or_upgrade_saved_config/integration_tests/create_or_upgrade.test.ts
@@ -90,6 +90,9 @@ describe('createOrUpgradeSavedConfig()', () => {
// 5.4.0-SNAPSHOT and @@version were ignored so we only have the
// attributes from 5.4.0-rc1, even though the other build nums are greater
'5.4.0-rc1': true,
+
+ // Should have the transform(s) applied
+ isDefaultIndexMigrated: true,
});
// add the 5.4.0 flag to the 5.4.0 savedConfig
@@ -115,6 +118,9 @@ describe('createOrUpgradeSavedConfig()', () => {
// should also include properties from 5.4.0 and 5.4.0-rc1
'5.4.0': true,
'5.4.0-rc1': true,
+
+ // Should have the transform(s) applied
+ isDefaultIndexMigrated: true,
});
// add the 5.4.1 flag to the 5.4.1 savedConfig
@@ -141,6 +147,9 @@ describe('createOrUpgradeSavedConfig()', () => {
'5.4.1': true,
'5.4.0': true,
'5.4.0-rc1': true,
+
+ // Should have the transform(s) applied
+ isDefaultIndexMigrated: true,
});
// tag the 7.0.0-rc1 doc
@@ -168,6 +177,9 @@ describe('createOrUpgradeSavedConfig()', () => {
'5.4.1': true,
'5.4.0': true,
'5.4.0-rc1': true,
+
+ // Should have the transform(s) applied
+ isDefaultIndexMigrated: true,
});
// tag the 7.0.0 doc
@@ -194,6 +206,9 @@ describe('createOrUpgradeSavedConfig()', () => {
'5.4.1': true,
'5.4.0': true,
'5.4.0-rc1': true,
+
+ // Should have the transform(s) applied
+ isDefaultIndexMigrated: true,
});
}, 30000);
});
diff --git a/src/core/server/ui_settings/saved_objects/index.ts b/src/core/server/ui_settings/saved_objects/index.ts
index eb8e7cfcc2a1b..92ef9961bb908 100644
--- a/src/core/server/ui_settings/saved_objects/index.ts
+++ b/src/core/server/ui_settings/saved_objects/index.ts
@@ -6,4 +6,7 @@
* Side Public License, v 1.
*/
+export type { ConfigAttributes } from './ui_settings';
export { uiSettingsType } from './ui_settings';
+export type { TransformConfigFn } from './transforms';
+export { transforms } from './transforms';
diff --git a/src/core/server/ui_settings/saved_objects/transforms.test.ts b/src/core/server/ui_settings/saved_objects/transforms.test.ts
new file mode 100644
index 0000000000000..afcd525673aa0
--- /dev/null
+++ b/src/core/server/ui_settings/saved_objects/transforms.test.ts
@@ -0,0 +1,115 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+import { savedObjectsClientMock } from '../../mocks';
+import { SavedObjectsErrorHelpers } from '../../saved_objects';
+import { SavedObject } from '../../types';
+import type { UpgradeableConfigAttributes } from '../create_or_upgrade_saved_config';
+import { transformDefaultIndex } from './transforms';
+
+/**
+ * Test each transform function individually, not the entire exported `transforms` array.
+ */
+describe('#transformDefaultIndex', () => {
+ const savedObjectsClient = savedObjectsClientMock.create();
+
+ beforeEach(() => {
+ jest.resetAllMocks();
+ });
+
+ it('should return early if the config object has already been transformed', async () => {
+ const result = await transformDefaultIndex({
+ savedObjectsClient,
+ configAttributes: { isDefaultIndexMigrated: true } as UpgradeableConfigAttributes, // We don't care about the other attributes
+ });
+
+ expect(savedObjectsClient.resolve).not.toHaveBeenCalled();
+ expect(result).toEqual(null); // This is the only time we expect a null result
+ });
+
+ it('should return early if configAttributes is undefined', async () => {
+ const result = await transformDefaultIndex({ savedObjectsClient, configAttributes: undefined });
+
+ expect(savedObjectsClient.resolve).not.toHaveBeenCalled();
+ expect(result).toEqual({ isDefaultIndexMigrated: true });
+ });
+
+ it('should return early if the defaultIndex attribute is undefined', async () => {
+ const result = await transformDefaultIndex({
+ savedObjectsClient,
+ configAttributes: { defaultIndex: undefined } as UpgradeableConfigAttributes, // We don't care about the other attributes
+ });
+
+ expect(savedObjectsClient.resolve).not.toHaveBeenCalled();
+ expect(result).toEqual({ isDefaultIndexMigrated: true });
+ });
+
+ describe('should resolve the data view for the defaultIndex and return the result according to the outcome', () => {
+ it('outcome: exactMatch', async () => {
+ savedObjectsClient.resolve.mockResolvedValue({
+ outcome: 'exactMatch',
+ alias_target_id: 'another-index', // This wouldn't realistically be set if the outcome is exactMatch, but we're including it in this test to assert that the returned defaultIndex will be 'some-index'
+ saved_object: {} as SavedObject, // Doesn't matter
+ });
+ const result = await transformDefaultIndex({
+ savedObjectsClient,
+ configAttributes: { defaultIndex: 'some-index' } as UpgradeableConfigAttributes, // We don't care about the other attributes
+ });
+
+ expect(savedObjectsClient.resolve).toHaveBeenCalledTimes(1);
+ expect(savedObjectsClient.resolve).toHaveBeenCalledWith('index-pattern', 'some-index');
+ expect(result).toEqual({ isDefaultIndexMigrated: true, defaultIndex: 'some-index' });
+ });
+
+ for (const outcome of ['aliasMatch' as const, 'conflict' as const]) {
+ it(`outcome: ${outcome}`, async () => {
+ savedObjectsClient.resolve.mockResolvedValue({
+ outcome,
+ alias_target_id: 'another-index',
+ saved_object: {} as SavedObject, // Doesn't matter
+ });
+ const result = await transformDefaultIndex({
+ savedObjectsClient,
+ configAttributes: { defaultIndex: 'some-index' } as UpgradeableConfigAttributes, // We don't care about the other attributes
+ });
+
+ expect(savedObjectsClient.resolve).toHaveBeenCalledTimes(1);
+ expect(savedObjectsClient.resolve).toHaveBeenCalledWith('index-pattern', 'some-index');
+ expect(result).toEqual({ isDefaultIndexMigrated: true, defaultIndex: 'another-index' });
+ });
+ }
+
+ it('returns the expected result if resolve fails with a Not Found error', async () => {
+ savedObjectsClient.resolve.mockRejectedValue(
+ SavedObjectsErrorHelpers.createGenericNotFoundError('Oh no!')
+ );
+ const result = await transformDefaultIndex({
+ savedObjectsClient,
+ configAttributes: { defaultIndex: 'some-index' } as UpgradeableConfigAttributes, // We don't care about the other attributes
+ });
+
+ expect(savedObjectsClient.resolve).toHaveBeenCalledTimes(1);
+ expect(savedObjectsClient.resolve).toHaveBeenCalledWith('index-pattern', 'some-index');
+ expect(result).toEqual({ isDefaultIndexMigrated: true, defaultIndex: 'some-index' });
+ });
+
+ it('returns the expected result if resolve fails with another error', async () => {
+ savedObjectsClient.resolve.mockRejectedValue(
+ SavedObjectsErrorHelpers.createIndexAliasNotFoundError('Oh no!')
+ );
+ const result = await transformDefaultIndex({
+ savedObjectsClient,
+ configAttributes: { defaultIndex: 'some-index' } as UpgradeableConfigAttributes, // We don't care about the other attributes
+ });
+
+ expect(savedObjectsClient.resolve).toHaveBeenCalledTimes(1);
+ expect(savedObjectsClient.resolve).toHaveBeenCalledWith('index-pattern', 'some-index');
+ expect(result).toEqual({ isDefaultIndexMigrated: false, defaultIndex: 'some-index' });
+ });
+ });
+});
diff --git a/src/core/server/ui_settings/saved_objects/transforms.ts b/src/core/server/ui_settings/saved_objects/transforms.ts
new file mode 100644
index 0000000000000..cabf14a2e6469
--- /dev/null
+++ b/src/core/server/ui_settings/saved_objects/transforms.ts
@@ -0,0 +1,96 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+import { SavedObjectsErrorHelpers } from '../../saved_objects';
+import type { SavedObjectsClientContract } from '../../types';
+import type { UpgradeableConfigAttributes } from '../create_or_upgrade_saved_config';
+
+/**
+ * The params needed to execute each transform function.
+ */
+interface TransformParams {
+ savedObjectsClient: SavedObjectsClientContract;
+ configAttributes: UpgradeableConfigAttributes | undefined;
+}
+
+/**
+ * The resulting attributes that should be used when upgrading the config object.
+ * This should be a union of all transform function return types (A | B | C | ...).
+ */
+type TransformReturnType = TransformDefaultIndexReturnType;
+
+/**
+ * The return type for `transformDefaultIndex`.
+ * If this config object has already been upgraded, it returns `null` because it doesn't need to set different default attributes.
+ * Otherwise, it always sets a default for the `isDefaultIndexMigrated` attribute, and it optionally sets the `defaultIndex` attribute
+ * depending on the outcome.
+ */
+type TransformDefaultIndexReturnType = {
+ isDefaultIndexMigrated: boolean;
+ defaultIndex?: string;
+} | null;
+
+export type TransformConfigFn = (params: TransformParams) => Promise;
+
+/**
+ * Any transforms that should be applied during `createOrUpgradeSavedConfig` need to be included in this array.
+ */
+export const transforms: TransformConfigFn[] = [transformDefaultIndex];
+
+/**
+ * This optionally transforms the `defaultIndex` attribute of a config saved object. The `defaultIndex` attribute points to a data view ID,
+ * but those saved object IDs were regenerated in the 8.0 upgrade. That resulted in a bug where the `defaultIndex` would be broken in custom
+ * spaces.
+ *
+ * We are fixing this bug after the fact in 8.3, and we can't retroactively change a saved object that's already been migrated, so we use
+ * this transformation instead to ensure that the `defaultIndex` attribute is not broken.
+ *
+ * Note that what used to be called "index patterns" prior to 8.0 have been renamed to "data views", but the object type cannot be changed,
+ * so that type remains `index-pattern`.
+ *
+ * Note also that this function is only exported for unit testing. It is also included in the `transforms` export above, which is how it is
+ * applied during `createOrUpgradeSavedConfig`.
+ */
+export async function transformDefaultIndex(
+ params: TransformParams
+): Promise {
+ const { savedObjectsClient, configAttributes } = params;
+ if (configAttributes?.isDefaultIndexMigrated) {
+ // This config object has already been migrated, return null because we don't need to set different defaults for the new config object.
+ return null;
+ }
+ if (!configAttributes?.defaultIndex) {
+ // If configAttributes is undefined (there's no config object being upgraded), OR if configAttributes is defined but the defaultIndex
+ // attribute is not set, set isDefaultIndexMigrated to true and return. This means there was no defaultIndex to upgrade, so we will just
+ // avoid attempting to transform this again in the future.
+ return { isDefaultIndexMigrated: true };
+ }
+
+ let defaultIndex = configAttributes.defaultIndex; // Retain the existing defaultIndex attribute in case we run into a resolve error
+ let isDefaultIndexMigrated: boolean;
+ try {
+ // The defaultIndex for this config object was created prior to 8.3, and it might refer to a data view ID that is no longer valid.
+ // We should try to resolve the data view and change the defaultIndex to the new ID, if necessary.
+ const resolvedDataView = await savedObjectsClient.resolve('index-pattern', defaultIndex);
+ if (resolvedDataView.outcome === 'aliasMatch' || resolvedDataView.outcome === 'conflict') {
+ // This resolved to an aliasMatch or conflict outcome; that means we should change the defaultIndex to the data view's new ID.
+ // Note, the alias_target_id field is guaranteed to exist iff the resolve outcome is aliasMatch or conflict.
+ defaultIndex = resolvedDataView.alias_target_id!;
+ }
+ isDefaultIndexMigrated = true; // Regardless of the resolve outcome, we now consider this defaultIndex attribute to be migrated
+ } catch (err) {
+ // If the defaultIndex is not found at all, it will throw a Not Found error and we should mark the defaultIndex attribute as upgraded.
+ if (SavedObjectsErrorHelpers.isNotFoundError(err)) {
+ isDefaultIndexMigrated = true;
+ } else {
+ // For any other error, explicitly set isDefaultIndexMigrated to false so we can try this upgrade again in the future.
+ isDefaultIndexMigrated = false;
+ }
+ }
+ return { isDefaultIndexMigrated, defaultIndex };
+}
diff --git a/src/core/server/ui_settings/saved_objects/ui_settings.ts b/src/core/server/ui_settings/saved_objects/ui_settings.ts
index 9db1402dc2d26..184ec5d41987a 100644
--- a/src/core/server/ui_settings/saved_objects/ui_settings.ts
+++ b/src/core/server/ui_settings/saved_objects/ui_settings.ts
@@ -9,6 +9,14 @@
import { SavedObjectsType } from '../../saved_objects';
import { migrations } from './migrations';
+/**
+ * The `config` object type contains many attributes that are defined by consumers.
+ */
+export interface ConfigAttributes {
+ buildNum: number;
+ [key: string]: unknown;
+}
+
export const uiSettingsType: SavedObjectsType = {
name: 'config',
hidden: false,
diff --git a/src/core/server/ui_settings/ui_settings_client.test.mock.ts b/src/core/server/ui_settings/ui_settings_client.test.mock.ts
new file mode 100644
index 0000000000000..d2a96352b36ea
--- /dev/null
+++ b/src/core/server/ui_settings/ui_settings_client.test.mock.ts
@@ -0,0 +1,16 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+import type { createOrUpgradeSavedConfig } from './create_or_upgrade_saved_config';
+
+export const mockCreateOrUpgradeSavedConfig = jest.fn() as jest.MockedFunction<
+ typeof createOrUpgradeSavedConfig
+>;
+jest.mock('./create_or_upgrade_saved_config', () => ({
+ createOrUpgradeSavedConfig: mockCreateOrUpgradeSavedConfig,
+}));
diff --git a/src/core/server/ui_settings/ui_settings_client.test.ts b/src/core/server/ui_settings/ui_settings_client.test.ts
index 99ab87610db90..844e17e53ab89 100644
--- a/src/core/server/ui_settings/ui_settings_client.test.ts
+++ b/src/core/server/ui_settings/ui_settings_client.test.ts
@@ -10,7 +10,7 @@ import Chance from 'chance';
import { schema } from '@kbn/config-schema';
import { loggingSystemMock } from '../logging/logging_system.mock';
-import { createOrUpgradeSavedConfigMock } from './create_or_upgrade_saved_config/create_or_upgrade_saved_config.test.mock';
+import { mockCreateOrUpgradeSavedConfig } from './ui_settings_client.test.mock';
import { SavedObjectsClient } from '../saved_objects';
import { savedObjectsClientMock } from '../saved_objects/service/saved_objects_client.mock';
@@ -47,12 +47,9 @@ describe('ui settings', () => {
log: logger,
});
- const createOrUpgradeSavedConfig = createOrUpgradeSavedConfigMock;
-
return {
uiSettings,
savedObjectsClient,
- createOrUpgradeSavedConfig,
};
}
@@ -84,7 +81,7 @@ describe('ui settings', () => {
});
it('automatically creates the savedConfig if it is missing', async () => {
- const { uiSettings, savedObjectsClient, createOrUpgradeSavedConfig } = setup();
+ const { uiSettings, savedObjectsClient } = setup();
savedObjectsClient.update
.mockRejectedValueOnce(SavedObjectsClient.errors.createGenericNotFoundError())
.mockResolvedValueOnce({} as any);
@@ -92,14 +89,14 @@ describe('ui settings', () => {
await uiSettings.setMany({ foo: 'bar' });
expect(savedObjectsClient.update).toHaveBeenCalledTimes(2);
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledTimes(1);
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledWith(
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledTimes(1);
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledWith(
expect.objectContaining({ handleWriteErrors: false })
);
});
it('only tried to auto create once and throws NotFound', async () => {
- const { uiSettings, savedObjectsClient, createOrUpgradeSavedConfig } = setup();
+ const { uiSettings, savedObjectsClient } = setup();
savedObjectsClient.update.mockRejectedValue(
SavedObjectsClient.errors.createGenericNotFoundError()
);
@@ -112,8 +109,8 @@ describe('ui settings', () => {
}
expect(savedObjectsClient.update).toHaveBeenCalledTimes(2);
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledTimes(1);
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledWith(
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledTimes(1);
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledWith(
expect.objectContaining({ handleWriteErrors: false })
);
});
@@ -374,7 +371,7 @@ describe('ui settings', () => {
});
it('automatically creates the savedConfig if it is missing and returns empty object', async () => {
- const { uiSettings, savedObjectsClient, createOrUpgradeSavedConfig } = setup();
+ const { uiSettings, savedObjectsClient } = setup();
savedObjectsClient.get = jest
.fn()
.mockRejectedValueOnce(SavedObjectsClient.errors.createGenericNotFoundError())
@@ -384,15 +381,15 @@ describe('ui settings', () => {
expect(savedObjectsClient.get).toHaveBeenCalledTimes(2);
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledTimes(1);
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledWith(
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledTimes(1);
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledWith(
expect.objectContaining({ handleWriteErrors: true })
);
});
it('returns result of savedConfig creation in case of notFound error', async () => {
- const { uiSettings, savedObjectsClient, createOrUpgradeSavedConfig } = setup();
- createOrUpgradeSavedConfig.mockResolvedValue({ foo: 'bar ' });
+ const { uiSettings, savedObjectsClient } = setup();
+ mockCreateOrUpgradeSavedConfig.mockResolvedValue({ foo: 'bar ' });
savedObjectsClient.get.mockRejectedValue(
SavedObjectsClient.errors.createGenericNotFoundError()
);
@@ -401,23 +398,23 @@ describe('ui settings', () => {
});
it('returns an empty object on Forbidden responses', async () => {
- const { uiSettings, savedObjectsClient, createOrUpgradeSavedConfig } = setup();
+ const { uiSettings, savedObjectsClient } = setup();
const error = SavedObjectsClient.errors.decorateForbiddenError(new Error());
savedObjectsClient.get.mockRejectedValue(error);
expect(await uiSettings.getUserProvided()).toStrictEqual({});
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledTimes(0);
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledTimes(0);
});
it('returns an empty object on EsUnavailable responses', async () => {
- const { uiSettings, savedObjectsClient, createOrUpgradeSavedConfig } = setup();
+ const { uiSettings, savedObjectsClient } = setup();
const error = SavedObjectsClient.errors.decorateEsUnavailableError(new Error());
savedObjectsClient.get.mockRejectedValue(error);
expect(await uiSettings.getUserProvided()).toStrictEqual({});
- expect(createOrUpgradeSavedConfig).toHaveBeenCalledTimes(0);
+ expect(mockCreateOrUpgradeSavedConfig).toHaveBeenCalledTimes(0);
});
it('throws Unauthorized errors', async () => {
diff --git a/src/core/test_helpers/http_test_setup.ts b/src/core/test_helpers/http_test_setup.ts
index 67b340898aab4..2a7d6451319cb 100644
--- a/src/core/test_helpers/http_test_setup.ts
+++ b/src/core/test_helpers/http_test_setup.ts
@@ -6,9 +6,9 @@
* Side Public License, v 1.
*/
+import { injectedMetadataServiceMock } from '@kbn/core-injected-metadata-browser-mocks';
import { HttpService } from '../public/http';
import { fatalErrorsServiceMock } from '../public/fatal_errors/fatal_errors_service.mock';
-import { injectedMetadataServiceMock } from '../public/injected_metadata/injected_metadata_service.mock';
import { executionContextServiceMock } from '../public/execution_context/execution_context_service.mock';
export type SetupTap = (
diff --git a/src/dev/license_checker/config.ts b/src/dev/license_checker/config.ts
index 2b3bccc164e6d..00b1260b1f90f 100644
--- a/src/dev/license_checker/config.ts
+++ b/src/dev/license_checker/config.ts
@@ -76,7 +76,12 @@ export const DEV_ONLY_LICENSE_ALLOWED = ['MPL-2.0'];
export const LICENSE_OVERRIDES = {
'jsts@1.6.2': ['Eclipse Distribution License - v 1.0'], // cf. https://github.com/bjornharrtell/jsts
'@mapbox/jsonlint-lines-primitives@2.0.2': ['MIT'], // license in readme https://github.com/tmcw/jsonlint
+<<<<<<< HEAD
'@elastic/ems-client@8.3.2': ['Elastic License 2.0'],
'@elastic/eui@58.0.0': ['SSPL-1.0 OR Elastic License 2.0'],
+=======
+ '@elastic/ems-client@8.3.3': ['Elastic License 2.0'],
+ '@elastic/eui@55.1.4': ['SSPL-1.0 OR Elastic License 2.0'],
+>>>>>>> 7695aad7a16a3816fb343252581f9ca8c02d65a2
'language-subtag-registry@0.3.21': ['CC-BY-4.0'], // retired ODC‑By license https://github.com/mattcg/language-subtag-registry
};
diff --git a/src/dev/storybook/aliases.ts b/src/dev/storybook/aliases.ts
index 45b8aad7df8cf..702b63fe48fc0 100644
--- a/src/dev/storybook/aliases.ts
+++ b/src/dev/storybook/aliases.ts
@@ -38,5 +38,5 @@ export const storybookAliases = {
presentation: 'src/plugins/presentation_util/storybook',
security_solution: 'x-pack/plugins/security_solution/.storybook',
shared_ux: 'packages/kbn-shared-ux-storybook/src/config',
- ui_actions_enhanced: 'x-pack/plugins/ui_actions_enhanced/.storybook',
+ ui_actions_enhanced: 'src/plugins/ui_actions_enhanced/.storybook',
};
diff --git a/src/plugins/chart_expressions/expression_partition_vis/common/types/expression_renderers.ts b/src/plugins/chart_expressions/expression_partition_vis/common/types/expression_renderers.ts
index 89a242fe26de1..93f9365987553 100644
--- a/src/plugins/chart_expressions/expression_partition_vis/common/types/expression_renderers.ts
+++ b/src/plugins/chart_expressions/expression_partition_vis/common/types/expression_renderers.ts
@@ -24,7 +24,7 @@ export interface Dimension {
accessor: number;
format: {
id?: string;
- params?: SerializedFieldFormat