Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet] Update logic for "Keep policies up to date" defaults in 8.0 #119126

Merged
merged 7 commits into from
Nov 21, 2021
Merged

[Fleet] Update logic for "Keep policies up to date" defaults in 8.0 #119126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5d27402
Auto-upgrade policies for all managed packages by default
kpollich Nov 18, 2021
040bf25
Fixing unused import in test
kpollich Nov 18, 2021
8f7aa92
Add migration to installation list
kpollich Nov 18, 2021
7a444a4
Update naming + disable input for APM + Synthetics
kpollich Nov 19, 2021
c25851a
Remove unused import + add message for disabled policy update settingg
kpollich Nov 19, 2021
4a8ff98
Merge branch 'main' into upgrade-managed-policies
kibanamachine Nov 19, 2021
47590a2
Fix failing tests
kpollich Nov 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions x-pack/plugins/fleet/common/constants/epm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,13 @@ export const STANDALONE_RUN_INSTRUCTIONS = './elastic-agent install';
removable, but it doesn't install by default. Following the table, it needs to
be in `unremovablePackages` and in `autoUpdatePackages`, but not in
`defaultPackages`.
We also define "auto upgrade policies" packages below. These are packages that are considered "stack-aligned"
and require policies to be auto-upgraded in order to properly function. Commonly, packages that ship custom policy
editor UI's in the Kibana codebase will be included in this set of packages to avoid backwards-compatibility concerns
in their custom policy editor implementations.
*/

export const unremovablePackages = [
Expand All @@ -49,6 +56,8 @@ export const autoUpdatePackages = [
FLEET_SYNTHETICS_PACKAGE,
];

export const autoUpgradePoliciesPackages = [FLEET_APM_PACKAGE, FLEET_SYNTHETICS_PACKAGE];

export const agentAssetTypes = {
Input: 'input',
} as const;
Expand Down
15 changes: 15 additions & 0 deletions x-pack/plugins/fleet/common/constants/preconfiguration.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
* 2.0.
*/

import { uniqBy } from 'lodash';

import type { PreconfiguredAgentPolicy } from '../types';

import {
Expand All @@ -13,6 +15,7 @@ import {
FLEET_SERVER_PACKAGE,
autoUpdatePackages,
monitoringTypes,
autoUpgradePoliciesPackages,
} from './epm';

export const PRECONFIGURATION_DELETION_RECORD_SAVED_OBJECT_TYPE =
Expand Down Expand Up @@ -72,6 +75,18 @@ export const AUTO_UPDATE_PACKAGES = autoUpdatePackages.map((name) => ({
version: PRECONFIGURATION_LATEST_KEYWORD,
}));

// These packages default to `keep_policies_up_to_date: true` and don't allow users to opt out
export const AUTO_UPGRADE_POLICIES_PACKAGES = autoUpgradePoliciesPackages.map((name) => ({
name,
version: PRECONFIGURATION_LATEST_KEYWORD,
}));

// Controls whether the `Keep Policies up to date` setting is exposed to the user
export const KEEP_POLICIES_UP_TO_DATE_PACKAGES = uniqBy(
[...AUTO_UPGRADE_POLICIES_PACKAGES, ...DEFAULT_PACKAGES, ...AUTO_UPDATE_PACKAGES],
({ name }) => name
);

export interface PreconfigurationError {
package?: { name: string; version: string };
agentPolicy?: { name: string };
Expand Down
2 changes: 1 addition & 1 deletion x-pack/plugins/fleet/common/types/models/epm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,7 @@ export interface Installation extends SavedObjectAttributes {
install_version: string;
install_started_at: string;
install_source: InstallSource;
keep_policies_up_to_date: boolean;
keep_policies_up_to_date?: boolean;
}

export interface PackageUsageStats {
Expand Down
18 changes: 14 additions & 4 deletions ...s/integrations/sections/epm/screens/detail/components/keep_policies_up_to_date_switch.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@ import { EuiSwitch, EuiSpacer, EuiText, EuiFlexGroup, EuiFlexItem, EuiIcon } fro

interface Props {
checked: boolean;
disabled?: boolean;
onChange: () => void;
}

export const KeepPoliciesUpToDateSwitch: React.FunctionComponent<Props> = ({
checked,
disabled = false,
onChange,
}) => (
<>
Expand All @@ -27,6 +29,7 @@ export const KeepPoliciesUpToDateSwitch: React.FunctionComponent<Props> = ({
)}
checked={checked}
onChange={onChange}
disabled={disabled}
/>
<EuiSpacer size="s" />
<EuiText color="subdued" size="xs">
Expand All @@ -35,10 +38,17 @@ export const KeepPoliciesUpToDateSwitch: React.FunctionComponent<Props> = ({
<EuiIcon type="iInCircle" />
</EuiFlexItem>
<EuiFlexItem grow={false}>
<FormattedMessage
id="xpack.fleet.integrations.settings.keepIntegrationPoliciesUpToDateDescription"
defaultMessage="When enabled, Fleet will attempt to upgrade and deploy integration policies automatically"
/>
{disabled ? (
<FormattedMessage
id="xpack.fleet.integrations.settings.keepIntegrationPoliciesUpToDateDisabledDescription"
defaultMessage="This integration requires Fleet to automatically upgrade its integration policies"
/>
) : (
<FormattedMessage
id="xpack.fleet.integrations.settings.keepIntegrationPoliciesUpToDateDescription"
defaultMessage="When enabled, Fleet will attempt to upgrade and deploy integration policies automatically"
/>
)}
</EuiFlexItem>
</EuiFlexGroup>
</EuiText>
Expand Down
14 changes: 7 additions & 7 deletions .../fleet/public/applications/integrations/sections/epm/screens/detail/settings/settings.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import React, { memo, useCallback, useEffect, useMemo, useState } from 'react';
import styled from 'styled-components';
import { FormattedMessage } from '@kbn/i18n/react';
import semverLt from 'semver/functions/lt';
import { uniq } from 'lodash';

import {
EuiCallOut,
Expand All @@ -35,8 +34,8 @@ import {
} from '../../../../../hooks';
import {
PACKAGE_POLICY_SAVED_OBJECT_TYPE,
AUTO_UPDATE_PACKAGES,
DEFAULT_PACKAGES,
KEEP_POLICIES_UP_TO_DATE_PACKAGES,
AUTO_UPGRADE_POLICIES_PACKAGES,
} from '../../../../../constants';

import { KeepPoliciesUpToDateSwitch } from '../components';
Expand Down Expand Up @@ -107,11 +106,11 @@ export const SettingsPage: React.FC<Props> = memo(({ packageInfo }: Props) => {
const { notifications } = useStartServices();

const shouldShowKeepPoliciesUpToDateSwitch = useMemo(() => {
const packages = [...DEFAULT_PACKAGES, ...AUTO_UPDATE_PACKAGES];

const packageNames = uniq(packages.map((pkg) => pkg.name));
return KEEP_POLICIES_UP_TO_DATE_PACKAGES.some((pkg) => pkg.name === name);
}, [name]);

return packageNames.includes(name);
const isShowKeepPoliciesUpToDateSwitchDisabled = useMemo(() => {
return AUTO_UPGRADE_POLICIES_PACKAGES.some((pkg) => pkg.name === name);
}, [name]);

const [keepPoliciesUpToDateSwitchValue, setKeepPoliciesUpToDateSwitchValue] = useState<boolean>(
Expand Down Expand Up @@ -274,6 +273,7 @@ export const SettingsPage: React.FC<Props> = memo(({ packageInfo }: Props) => {
<KeepPoliciesUpToDateSwitch
checked={keepPoliciesUpToDateSwitchValue}
onChange={handleKeepPoliciesUpToDateSwitchChange}
disabled={isShowKeepPoliciesUpToDateSwitchDisabled}
/>
<EuiSpacer size="l" />
</>
Expand Down
2 changes: 2 additions & 0 deletions x-pack/plugins/fleet/public/constants/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ export {
// Preconfiguration
AUTO_UPDATE_PACKAGES,
DEFAULT_PACKAGES,
KEEP_POLICIES_UP_TO_DATE_PACKAGES,
AUTO_UPGRADE_POLICIES_PACKAGES,
} from '../../common/constants';

export * from './page_paths';
Expand Down
3 changes: 2 additions & 1 deletion x-pack/plugins/fleet/server/saved_objects/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ import {
import { migratePackagePolicyToV7140, migrateInstallationToV7140 } from './migrations/to_v7_14_0';
import { migratePackagePolicyToV7150 } from './migrations/to_v7_15_0';
import { migrateInstallationToV7160, migratePackagePolicyToV7160 } from './migrations/to_v7_16_0';
import { migrateOutputToV800 } from './migrations/to_v8_0_0';
import { migrateInstallationToV800, migrateOutputToV800 } from './migrations/to_v8_0_0';

/*
* Saved object types and mappings
Expand Down Expand Up @@ -255,6 +255,7 @@ const getSavedObjectTypes = (
'7.14.0': migrateInstallationToV7140,
'7.14.1': migrateInstallationToV7140,
'7.16.0': migrateInstallationToV7160,
'8.0.0': migrateInstallationToV800,
},
},
[ASSETS_SAVED_OBJECT_TYPE]: {
Expand Down
21 changes: 19 additions & 2 deletions x-pack/plugins/fleet/server/saved_objects/migrations/to_v8_0_0.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@

import type { SavedObjectMigrationFn } from 'kibana/server';

import type { Output } from '../../../common';
import {} from '../../../common';
import type { Installation, Output } from '../../../common';
import { AUTO_UPGRADE_POLICIES_PACKAGES } from '../../../common';

export const migrateOutputToV800: SavedObjectMigrationFn<Output, Output> = (
outputDoc,
Expand All @@ -20,3 +20,20 @@ export const migrateOutputToV800: SavedObjectMigrationFn<Output, Output> = (

return outputDoc;
};

export const migrateInstallationToV800: SavedObjectMigrationFn<Installation, Installation> = (
installationDoc,
migrationContext
) => {
const updatedInstallationDoc = installationDoc;

const shouldKeepPoliciesUpToDate = AUTO_UPGRADE_POLICIES_PACKAGES.some(
(pkg) => pkg.name === updatedInstallationDoc.attributes.name
);

if (shouldKeepPoliciesUpToDate) {
updatedInstallationDoc.attributes.keep_policies_up_to_date = true;
}

return updatedInstallationDoc;
};
13 changes: 8 additions & 5 deletions x-pack/plugins/fleet/server/services/epm/packages/install.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import type {
InstallablePackage,
InstallSource,
} from '../../../../common';
import { DEFAULT_PACKAGES } from '../../../../common';
import { AUTO_UPGRADE_POLICIES_PACKAGES } from '../../../../common';
import {
IngestManagerError,
PackageOperationNotSupportedError,
Expand Down Expand Up @@ -534,11 +534,14 @@ export async function createInstallation(options: {
const removable = !isUnremovablePackage(pkgName);
const toSaveESIndexPatterns = generateESIndexPatterns(packageInfo.data_streams);

// For default packages, default the `keep_policies_up_to_date` setting to true. For all other
// package, default it to false.
const defaultKeepPoliciesUpToDate = DEFAULT_PACKAGES.some(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original intent here should have been to leave keep_policies_up_to_date set to undefined unless it was explicitly set by the user, but this code introduced in #116993 sets the default value to false instead for any non-default package that's installed.

Because we explicitly set this to false during installation for any non-default package, we can't reliably honor user settings during our migration.

For example, if a user installed apm on their 7.16 Kibana instance, the installation SO would have keep_policies_up_to_date: false. So, we have to make a choice in this PR whether we should discard any user-provided value for keep_policies_up_to_date on managed policies and simply set them all globally to true.

Copy link
Member Author

@kpollich kpollich Nov 19, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To close the loop here, I spoke with Josh offline and got a few pieces of direction here.

  • The only packages that must have their policies upgraded are APM and Synthetics
  • We shouldn't automatically opt system into keeping policies up to date, because it's widely used and could cause rollout/performance issues in environments with many agents

With those two things in mind, I elected to make the changes detailed in my previous comment.

// For "stack-aligned" packages, default the `keep_policies_up_to_date` setting to true. For all other
// packages, default it to undefined. Use undefined rather than false to allow us to differentiate
// between "unset" and "user explicitly disabled".
const defaultKeepPoliciesUpToDate = AUTO_UPGRADE_POLICIES_PACKAGES.some(
({ name }) => name === packageInfo.name
);
)
? true
: undefined;

const created = await savedObjectsClient.create<Installation>(
PACKAGES_SAVED_OBJECT_TYPE,
Expand Down
45 changes: 0 additions & 45 deletions x-pack/plugins/fleet/server/services/managed_package_policies.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
import { elasticsearchServiceMock, savedObjectsClientMock } from 'src/core/server/mocks';

import type { Installation, PackageInfo } from '../../common';
import { AUTO_UPDATE_PACKAGES } from '../../common';

import { shouldUpgradePolicies, upgradeManagedPackagePolicies } from './managed_package_policies';
import { packagePolicyService } from './package_policy';
Expand Down Expand Up @@ -227,50 +226,6 @@ describe('upgradeManagedPackagePolicies', () => {
});

describe('shouldUpgradePolicies', () => {
describe('package is marked as AUTO_UPDATE', () => {
describe('keep_policies_up_to_date is true', () => {
it('returns false', () => {
const packageInfo = {
version: '1.0.0',
keepPoliciesUpToDate: true,
name: AUTO_UPDATE_PACKAGES[0].name,
};

const installedPackage = {
version: '1.0.0',
};

const result = shouldUpgradePolicies(
packageInfo as PackageInfo,
installedPackage as Installation
);

expect(result).toBe(false);
});
});

describe('keep_policies_up_to_date is false', () => {
it('returns false', () => {
const packageInfo = {
version: '1.0.0',
keepPoliciesUpToDate: false,
name: AUTO_UPDATE_PACKAGES[0].name,
};

const installedPackage = {
version: '1.0.0',
};

const result = shouldUpgradePolicies(
packageInfo as PackageInfo,
installedPackage as Installation
);

expect(result).toBe(false);
});
});
});

describe('package policy is up-to-date', () => {
describe('keep_policies_up_to_date is true', () => {
it('returns false', () => {
Expand Down
1 change: 0 additions & 1 deletion x-pack/test/fleet_api_integration/apis/epm/__snapshots__/install_by_upload.snap
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion x-pack/test/fleet_api_integration/apis/epm/install_remove_assets.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -673,7 +673,6 @@ const expectAssetsInstalled = ({
install_status: 'installed',
install_started_at: res.attributes.install_started_at,
install_source: 'registry',
keep_policies_up_to_date: false,
});
});
};
1 change: 0 additions & 1 deletion x-pack/test/fleet_api_integration/apis/epm/update_assets.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,6 @@ export default function (providerContext: FtrProviderContext) {
install_status: 'installed',
install_started_at: res.attributes.install_started_at,
install_source: 'registry',
keep_policies_up_to_date: false,
});
});
});
Expand Down