Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update UI for integrations setup #1052

Merged
merged 82 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
c5cea8d
Stub catalog reader interface
Swiddis Aug 17, 2023
77ad89b
Add basic catalog functionality to new catalog reader
Swiddis Aug 17, 2023
676591d
Refactor validation logic with a deeper interface
Swiddis Aug 21, 2023
12c4bcf
Refactor validation logic with a deeper interface
Swiddis Aug 21, 2023
412480e
Remove redundant test.
Swiddis Aug 21, 2023
6d2bad1
Add tests for new validators
Swiddis Aug 21, 2023
c3dd111
Merge branch 'better-validation' into prototype-network-catalog
Swiddis Aug 21, 2023
55d6490
Make better failure mode for invalid objects
Swiddis Aug 21, 2023
c62d28b
Merge branch 'main' into prototype-network-catalog
Swiddis Aug 22, 2023
ba56fb8
Generalize Result type
Swiddis Aug 22, 2023
cb29208
Convert backend to use catalog reader (unstable)
Swiddis Aug 22, 2023
c9de36e
Repair tests for integrations class (unstable)
Swiddis Aug 22, 2023
37c7980
Refactor repository for new integration interface
Swiddis Aug 22, 2023
b331090
Fix outer repository and backend tests
Swiddis Aug 23, 2023
4b29e17
Add tests for sample data
Swiddis Aug 23, 2023
b35dfa8
Merge remote-tracking branch 'upstream/main' into prototype-network-c…
Swiddis Aug 23, 2023
1bef29e
Add CatalogReader JavaDocs
Swiddis Aug 23, 2023
a18cf47
Repair integrations builder
Swiddis Aug 23, 2023
047b14a
Remove extra commented test
Swiddis Aug 23, 2023
e65fe8b
Remove unnecessary log statement
Swiddis Aug 23, 2023
00ddb5b
Repair getSchemas behavior to return correct type
Swiddis Aug 23, 2023
8af7f4a
Add tests for getSchemas
Swiddis Aug 23, 2023
a31963d
Add tests for asset and sample data backend methods
Swiddis Aug 23, 2023
feeaef5
Break flyout validation methods out of constructing method
Swiddis Aug 24, 2023
b644d11
Add tests for extracted flyout methods
Swiddis Aug 24, 2023
614613a
Switch validation method to use ValidationResult
Swiddis Aug 24, 2023
d97b931
Merge branch 'prototype-network-catalog' into feature/integrations-setup
Swiddis Sep 6, 2023
c8b5eff
Merge branch 'refactor-integ-frontend' into feature/integrations-setup
Swiddis Sep 6, 2023
af78c27
Merge branch 'main' into prototype-network-catalog
Swiddis Sep 6, 2023
ab30ed0
Swap out flyout for hello-world setup page
Swiddis Sep 6, 2023
85361f8
Add basic step incrementing
Swiddis Sep 7, 2023
af21831
Add basic field skeleton for each step
Swiddis Sep 7, 2023
42dbb54
Add a cancel button
Swiddis Sep 7, 2023
bfa9d1a
Merge branch 'main' into prototype-network-catalog
Swiddis Sep 11, 2023
533e6c7
Add config type to developing form
Swiddis Sep 11, 2023
467488c
Flatten integration config
Swiddis Sep 11, 2023
670d229
Add sample data table modal
Swiddis Sep 12, 2023
392640d
Merge branch 'main' into feature/integrations-setup
Swiddis Sep 13, 2023
0a997a4
Add toggle for standard and advanced asset config
Swiddis Sep 13, 2023
8c7b0f2
Simplify imports
Swiddis Sep 13, 2023
616d44e
Merge remote-tracking branch 'upstream/main' into feature/integration…
Swiddis Sep 13, 2023
339fb09
Refactor major class names
Swiddis Sep 13, 2023
fae680f
(WIP) begin refactoring functionality into adaptor
Swiddis Sep 13, 2023
3096626
Finish migrating functionality to data adaptor
Swiddis Sep 13, 2023
5498724
Rename integration types for more clarity
Swiddis Sep 14, 2023
ee33a51
Merge branch 'main' into prototype-network-catalog
Swiddis Sep 14, 2023
9f97f5a
Merge branch 'prototype-network-catalog' into feature/integrations-setup
Swiddis Sep 15, 2023
b465b9f
Refactor component usage
Swiddis Sep 15, 2023
2af52f5
Connect forms to config state
Swiddis Sep 15, 2023
1687fbc
Fix filetype selector
Swiddis Sep 15, 2023
cb8dfd3
Merge branch 'main' into feature/integrations-setup
Swiddis Sep 18, 2023
1ee1b75
Merge remote-tracking branch 'upstream/main' into feature/integration…
Swiddis Sep 18, 2023
1049b13
Remove hardcoded name in path
Swiddis Sep 18, 2023
edd6bbe
Write one snapshot test
Swiddis Sep 18, 2023
1c95d41
Add more tests
Swiddis Sep 18, 2023
1760038
Fix test naming
Swiddis Sep 18, 2023
d4b27fe
Update obsolete snapshots
Swiddis Sep 18, 2023
4293290
Move integration creation helpers to own file
Swiddis Sep 18, 2023
36d65c6
Break out integration creation methods
Swiddis Sep 19, 2023
fe16007
Isolate more create_integration helpers
Swiddis Sep 22, 2023
dfadcb5
Simplify setup form
Swiddis Sep 22, 2023
af1471d
Add data source picker items
Swiddis Sep 22, 2023
9c9d800
Add better selector logic
Swiddis Sep 22, 2023
69e4978
Add queries for data sources
Swiddis Sep 25, 2023
038eefb
Switch from selector to combobox
Swiddis Sep 25, 2023
16745d1
Update snapshots
Swiddis Sep 25, 2023
1892b8f
Connect validation button to data source validation method
Swiddis Sep 26, 2023
ebdb008
Reimplement add integration button
Swiddis Sep 26, 2023
6230a65
Temporarily remove validate button
Swiddis Sep 27, 2023
b2fdb69
Merge remote-tracking branch 'upstream/main' into feature/integration…
Swiddis Sep 28, 2023
be45f19
Merge branch 'main' into feature/integrations-setup
Swiddis Sep 29, 2023
764d028
Simplify dynamic table term selection
Swiddis Sep 29, 2023
28d761c
Remove unused validate code
Swiddis Sep 29, 2023
aaeddef
Undo wildcard import
Swiddis Sep 29, 2023
3ae20be
Switch from proxy to dataconnections endpoint
Swiddis Sep 29, 2023
0158cc3
Remove unused table fields
Swiddis Sep 29, 2023
3cb6c57
Switch dataconnections base to const
Swiddis Sep 29, 2023
dd1d763
Add console proxy to route constants
Swiddis Sep 29, 2023
3f2a79a
Update snapshots
Swiddis Sep 29, 2023
9a6dc3a
Move color to constants
Swiddis Oct 3, 2023
1e267c6
Move index name validation to constants and improve matching
Swiddis Oct 3, 2023
7d4854e
Move test constants to test constants
Swiddis Oct 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions common/constants/integrations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,7 @@

export const OPENSEARCH_DOCUMENTATION_URL = 'https://opensearch.org/docs/latest/integrations/index';
export const ASSET_FILTER_OPTIONS = ['index-pattern', 'search', 'visualization', 'dashboard'];
export const VALID_INDEX_NAME = /^[a-z\d\.][a-z\d\._\-\*]*$/;

// Upstream doesn't export this, so we need to redeclare it for our use.
export type Color = 'success' | 'primary' | 'warning' | 'danger' | undefined;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. Can add more descriptive name here TOAST_NOTIFICATION_COLOR

1 change: 1 addition & 0 deletions common/constants/shared.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export const EVENT_ANALYTICS = '/event_analytics';
export const SAVED_OBJECTS = '/saved_objects';
export const SAVED_QUERY = '/query';
export const SAVED_VISUALIZATION = '/vis';
export const CONSOLE_PROXY = '/api/console/proxy';

// Server route
export const PPL_ENDPOINT = '/_plugins/_ppl';
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,7 @@
import { configure, mount } from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';
import { waitFor } from '@testing-library/react';
import {
AddIntegrationFlyout,
checkDataSourceName,
doTypeValidation,
doNestedPropertyValidation,
doPropertyValidation,
fetchDataSourceMappings,
fetchIntegrationMappings,
doExistingDataSourceValidation,
} from '../add_integration_flyout';
import * as add_integration_flyout from '../add_integration_flyout';
import { AddIntegrationFlyout } from '../add_integration_flyout';
import React from 'react';
import { HttpSetup } from '../../../../../../../src/core/public';

Expand Down Expand Up @@ -44,320 +34,3 @@ describe('Add Integration Flyout Test', () => {
});
});
});

describe('doTypeValidation', () => {
it('should return true if required type is not specified', () => {
const toCheck = { type: 'string' };
const required = {};

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return true if types match', () => {
const toCheck = { type: 'string' };
const required = { type: 'string' };

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return true if object has properties', () => {
const toCheck = { properties: { prop1: { type: 'string' } } };
const required = { type: 'object' };

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return false if types do not match', () => {
const toCheck = { type: 'string' };
const required = { type: 'number' };

const result = doTypeValidation(toCheck, required);

expect(result.ok).toBe(false);
});
});

describe('doNestedPropertyValidation', () => {
it('should return true if type validation passes and no properties are required', () => {
const toCheck = { type: 'string' };
const required = { type: 'string' };

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(true);
});

it('should return false if type validation fails', () => {
const toCheck = { type: 'string' };
const required = { type: 'number' };

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(false);
});

it('should return false if a required property is missing', () => {
const toCheck = { type: 'object', properties: { prop1: { type: 'string' } } };
const required = {
type: 'object',
properties: { prop1: { type: 'string' }, prop2: { type: 'number' } },
};

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(false);
});

it('should return true if all required properties pass validation', () => {
const toCheck = {
type: 'object',
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
};
const required = {
type: 'object',
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
};

const result = doNestedPropertyValidation(toCheck, required);

expect(result.ok).toBe(true);
});
});

describe('doPropertyValidation', () => {
it('should return true if all properties pass validation', () => {
const rootType = 'root';
const dataSourceProps = {
prop1: { type: 'string' },
prop2: { type: 'number' },
};
const requiredMappings = {
root: {
template: {
mappings: {
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
},
},
},
};

const result = doPropertyValidation(rootType, dataSourceProps as any, requiredMappings);

expect(result.ok).toBe(true);
});

it('should return false if a property fails validation', () => {
const rootType = 'root';
const dataSourceProps = {
prop1: { type: 'string' },
prop2: { type: 'number' },
};
const requiredMappings = {
root: {
template: {
mappings: {
properties: {
prop1: { type: 'string' },
prop2: { type: 'boolean' },
},
},
},
},
};

const result = doPropertyValidation(rootType, dataSourceProps as any, requiredMappings);

expect(result.ok).toBe(false);
});

it('should return false if a required nested property is missing', () => {
const rootType = 'root';
const dataSourceProps = {
prop1: { type: 'string' },
};
const requiredMappings = {
root: {
template: {
mappings: {
properties: {
prop1: { type: 'string' },
prop2: { type: 'number' },
},
},
},
},
};

const result = doPropertyValidation(rootType, dataSourceProps as any, requiredMappings);

expect(result.ok).toBe(false);
});
});

describe('checkDataSourceName', () => {
it('Filters out invalid index names', () => {
const result = checkDataSourceName('ss4o_logs-no-exclams!', 'logs');

expect(result.ok).toBe(false);
});

it('Filters out incorrectly typed indices', () => {
const result = checkDataSourceName('ss4o_metrics-test-test', 'logs');

expect(result.ok).toBe(false);
});

it('Accepts correct indices', () => {
const result = checkDataSourceName('ss4o_logs-test-test', 'logs');

expect(result.ok).toBe(true);
});
});

describe('fetchDataSourceMappings', () => {
it('Retrieves mappings', async () => {
const mockHttp = {
post: jest.fn().mockResolvedValue({
source1: { mappings: { properties: { test: true } } },
source2: { mappings: { properties: { test: true } } },
}),
} as Partial<HttpSetup>;

const result = fetchDataSourceMappings('sample', mockHttp as HttpSetup);

await expect(result).resolves.toMatchObject({
source1: { properties: { test: true } },
source2: { properties: { test: true } },
});
});

it('Catches errors', async () => {
const mockHttp = {
post: jest.fn().mockRejectedValue(new Error('Mock error')),
} as Partial<HttpSetup>;

const result = fetchDataSourceMappings('sample', mockHttp as HttpSetup);

await expect(result).resolves.toBeNull();
});
});

describe('fetchIntegrationMappings', () => {
it('Returns schema mappings', async () => {
const mockHttp = {
get: jest.fn().mockResolvedValue({ data: { mappings: { test: true } }, statusCode: 200 }),
} as Partial<HttpSetup>;

const result = fetchIntegrationMappings('target', mockHttp as HttpSetup);

await expect(result).resolves.toStrictEqual({ test: true });
});

it('Returns null if response fails', async () => {
const mockHttp = {
get: jest.fn().mockResolvedValue({ statusCode: 404 }),
} as Partial<HttpSetup>;

const result = fetchIntegrationMappings('target', mockHttp as HttpSetup);

await expect(result).resolves.toBeNull();
});

it('Catches request error', async () => {
const mockHttp = {
get: jest.fn().mockRejectedValue(new Error('mock error')),
} as Partial<HttpSetup>;

const result = fetchIntegrationMappings('target', mockHttp as HttpSetup);

await expect(result).resolves.toBeNull();
});
});

describe('doExistingDataSourceValidation', () => {
it('Catches and returns checkDataSourceName errors', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest
.spyOn(add_integration_flyout, 'checkDataSourceName')
.mockReturnValue({ ok: false, errors: ['mock'] });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Catches data stream fetch errors', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest.spyOn(add_integration_flyout, 'fetchDataSourceMappings').mockResolvedValue(null);
jest
.spyOn(add_integration_flyout, 'fetchIntegrationMappings')
.mockResolvedValue({ test: { template: { mappings: {} } } });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Catches integration fetch errors', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest
.spyOn(add_integration_flyout, 'fetchDataSourceMappings')
.mockResolvedValue({ test: { properties: {} } });
jest.spyOn(add_integration_flyout, 'fetchIntegrationMappings').mockResolvedValue(null);

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Catches type validation issues', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest
.spyOn(add_integration_flyout, 'fetchDataSourceMappings')
.mockResolvedValue({ test: { properties: {} } });
jest
.spyOn(add_integration_flyout, 'fetchIntegrationMappings')
.mockResolvedValue({ test: { template: { mappings: {} } } });
jest
.spyOn(add_integration_flyout, 'doPropertyValidation')
.mockReturnValue({ ok: false, errors: ['mock'] });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', false);
});

it('Returns no errors if everything passes', async () => {
const mockHttp = {} as Partial<HttpSetup>;
jest.spyOn(add_integration_flyout, 'checkDataSourceName').mockReturnValue({ ok: true });
jest
.spyOn(add_integration_flyout, 'fetchDataSourceMappings')
.mockResolvedValue({ test: { properties: {} } });
jest
.spyOn(add_integration_flyout, 'fetchIntegrationMappings')
.mockResolvedValue({ test: { template: { mappings: {} } } });
jest.spyOn(add_integration_flyout, 'doPropertyValidation').mockReturnValue({ ok: true });

const result = doExistingDataSourceValidation('target', 'name', 'type', mockHttp as HttpSetup);

await expect(result).resolves.toHaveProperty('ok', true);
});
});
Loading
Loading