Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Archive Migrations] security_solution-timelines #142363

Merged
merged 10 commits into from
Nov 4, 2022
Merged

[Archive Migrations] security_solution-timelines #142363

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2b6e527
[Archive Migrations] security_solution-timelines
wayneseymour Sep 26, 2022
332f8b1
Drop exclusive
wayneseymour Sep 30, 2022
dab78a4
Drop unneeded fn.
wayneseymour Oct 13, 2022
3b69c2b
Lint in my pocket.
wayneseymour Oct 13, 2022
d7c854d
Thanks to J. Buttner, "re-drop" non needed objects from the
wayneseymour Oct 14, 2022
4369b74
Merge branch 'main' into migrations/security_solution-timelines
kibanamachine Oct 17, 2022
8430dc8
Merge branch 'main' into migrations/security_solution-timelines
wayneseymour Oct 18, 2022
d895c84
Merge branch 'main' into migrations/security_solution-timelines
wayneseymour Nov 3, 2022
a41bd94
Merge branch 'main' into migrations/security_solution-timelines
wayneseymour Nov 3, 2022
b8abedf
Merge branch 'main' into migrations/security_solution-timelines
wayneseymour Nov 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 23 additions & 6 deletions scripts/archive_migration_functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,30 @@
standard_list="url,index-pattern,query,graph-workspace,tag,visualization,canvas-element,canvas-workpad,dashboard,search,lens,map,cases,uptime-dynamic-settings,osquery-saved-query,osquery-pack,infrastructure-ui-source,metrics-explorer-view,inventory-view,infrastructure-monitoring-log-view,apm-indices"
#!/bin/bash

# ??? Should we migrate
# x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space
# ### Yes, it needs migration
# ### Saved Object type(s) that we care about:
# index-pattern
# ### Test file(s) that use it:
# x-pack/test/api_integration/apis/security_solution/timeline_migrations.ts
# ### Config(s) that govern the test file(s):
# x-pack/test/api_integration/config.ts
# The other types it contains:
# config
# index-pattern
# siem-ui-timeline
# siem-ui-timeline-note
# siem-ui-timeline-pinned-event
# space

orig_archive="x-pack/test/functional/es_archives/banners/multispace"
new_archive="x-pack/test/functional/fixtures/kbn_archiver/banners/multi_space"
standard_list="url,index-pattern,query,graph-workspace,tag,visualization,canvas-element,canvas-workpad,dashboard,search,lens,map,cases,uptime-dynamic-settings,osquery-saved-query,osquery-pack,infrastructure-ui-source,metrics-explorer-view,inventory-view,infrastructure-monitoring-log-view,apm-indices"

# newArchives=("x-pack/test/functional/fixtures/kbn_archiver/dashboard/session_in_space")
orig_archive="x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space"
new_archive="x-pack/test/functional/fixtures/kbn_archiver/security_solution/timelines/7.15.0_space"

# testFiles=("x-pack/test/functional/apps/discover/preserve_url.ts")
testFiles=("x-pack/test/api_integration/apis/security_solution/timeline_migrations.ts")

test_config="x-pack/test/banners_functional/config.ts"
test_config="x-pack/test/api_integration/config.ts"

list_stragglers() {

Expand Down
12 changes: 7 additions & 5 deletions x-pack/test/api_integration/apis/security_solution/timeline_migrations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ export default function ({ getService }: FtrProviderContext) {
describe('Timeline migrations', () => {
const esArchiver = getService('esArchiver');
const es = getService('es');
const kibanaServer = getService('kibanaServer');
const spacesService = getService('spaces');

describe('8.0 id migration', () => {
const resolveWithSpaceApi = '/s/awesome-space/api/timeline/resolve';
Expand All @@ -44,14 +46,14 @@ export default function ({ getService }: FtrProviderContext) {
await esArchiver.load(
'x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space'
);
});

after(async () => {
await esArchiver.unload(
wayneseymour marked this conversation as resolved.
Show resolved Hide resolved
'x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space'
await kibanaServer.importExport.load(
'x-pack/test/functional/fixtures/kbn_archiver/security_solution/timelines/7.15.0_space',
{ space: 'awesome-space' }
);
});

after(async () => await spacesService.delete('awesome-space'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add the esArchiver.unload() here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrmm, well if they archives are loading into space then no....
Lemme triple check as you may be onto something.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, good thing you had me check!

I noticed two objects that should not have been within the archive. (awesome!)
Also, yes we should still be good only dropping the space.
This is because all the archives are loaded into that space && deleting a space drops all of the objects.

TLDR

Since it's on disk as a gzipped archive, I ran the following and then visually inspected it
node scripts/es_archiver.js edit x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space --config x-pack/test/api_integration/config.ts

jq '[.value.id]| .[]' x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space/data.json
"awesome-space:siem-ui-timeline:1211b430-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline:120f9150-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline:12205a30-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline:12208140-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline:1e2e9850-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline-note:2d230670-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline-pinned-event:2694b600-25f8-11ec-a981-b77847c6ef30"
"awesome-space:siem-ui-timeline-note:323df0c0-25f8-11ec-a981-b77847c6ef30"

Unless I missed my guess (totally possible lol), I think we are good as they are "namespaced" to awesome-space.


describe('resolve', () => {
it('should return an aliasMatch outcome', async () => {
const resp = await supertest
Expand Down
Binary file modified x-pack/test/functional/es_archives/security_solution/timelines/7.15.0_space/data.json.gz
View file
Open in desktop
Binary file not shown.
20 changes: 20 additions & 0 deletions x-pack/test/functional/fixtures/kbn_archiver/security_solution/timelines/7.15.0_space
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"attributes": {
"fieldAttrs": "{}",
"fields": "[]",
"runtimeFieldMap": "{}",
"timeFieldName": "",
"title": ".kibana_7.15.0",
"typeMeta": "{}"
},
"coreMigrationVersion": "8.6.0",
"id": "cf0a7895-d367-5eff-9d71-8d6b58b0b5d9",
"migrationVersion": {
"index-pattern": "8.0.0"
},
"originId": "4f57d680-25f3-11ec-a981-b77847c6ef30",
"references": [],
"type": "index-pattern",
"updated_at": "2021-10-05T15:45:48.019Z",
"version": "WzIyLDJd"
}