Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature ingest #58177

Merged
merged 214 commits into from
Feb 24, 2020
Merged

Feature ingest #58177

merged 214 commits into from
Feb 24, 2020

Conversation

jfsiii
Copy link
Contributor

@jfsiii jfsiii commented Feb 20, 2020

Summary

Merges the base Ingest Manager NP plugin from master / #56567 into feature-ingest

Differences from feature-ingest (consolidated diff of all changes)

  • From the PR
    • All flags are disabled by default (see below for how to enable)
    • Fleet and EPM routes are only added when enabled (see output below or check out server/integration_tests/router.test.ts)
    • some services were replaced by a BehaviorSubject and moved into the AppContext service
  • While merging the two
    • *Schema is reserved for @kbn/config-schema values
    • @kbn/config-schema only used in server (based on a version of joi which can't run in the client)
    • Default to TS types in common. Many HTTP responses are in common as *Response types

UI

Start ES with

--license trial -E xpack.security.authc.api_key.enabled=true

Start Kibana with

--xpack.ingestManager.enabled=true --xpack.ingestManager.epm.enabled=true --xpack.ingestManager.fleet.enabled=true

or set those values in config/kibana.yml, etc

Jest Integration tests

> node scripts/jest --testPathPattern plugins/ingest_manager/server/integration_tests/*.test.ts

 PASS  plugins/ingest_manager/server/integration_tests/router.test.ts (17.894s)
  ingestManager
    default. manager, EPM, and Fleet all disabled
      ✓ does not have agent config api (18ms)
      ✓ does not have datasources api (5ms)
      ✓ does not have EPM api (3ms)
      ✓ does not have Fleet api (3ms)
    manager only (no EPM, no Fleet)
      ✓ has agent config api (6ms)
      ✓ has datasources api (2ms)
      ✓ does not have EPM api (3ms)
      ✓ does not have Fleet api (2ms)
    manager and EPM; no Fleet
      ✓ has agent config api (3ms)
      ✓ has datasources api (3ms)
      ✓ does have EPM api (1162ms)
      ✓ does not have Fleet api (3ms)
    manager and Fleet; no EPM)
      ✓ has agent config api (3ms)
      ✓ has datasources api (2ms)
      ✓ does not have EPM api (3ms)
      ✓ does have Fleet api (3ms)
    all flags enabled: manager, EPM, and Fleet)
      ✓ has agent config api (4ms)
      ✓ has datasources api (2ms)
      ✓ does have EPM api (84ms)
      ✓ does have Fleet api (3ms)

Test Suites: 1 passed, 1 total
Tests:       20 passed, 20 total
Snapshots:   0 total
Time:        20.002s

alvarezmelissa87 and others added 30 commits February 11, 2020 10:21
* Create observability CODEOWNERS reference

* Update elastic/epm to elastic/ingest per Jen's request :)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* fixes events viewer tests

* fixes 'toggle column' tests

* fixes 'url state' tests

* fixes type check issues
* [ML] [NP] Removing ui imports

* replacing timefilter and ui context

* replacing ui/i18n and ui/metadata

* removing ui/system_api

* removing ui/notify

* removing most ui/new_platform

* fix explorer date format loading

* removing ui/chrome

* fixing timebuckets test

* fixing jest tests

* adding http

* testing odd CI type failure

* revrting type test changes

* fixing odd test type error

* refactoring dependencies

* removing injectI18n and using withKibana for context

* updating components to use kibana context

* re-enabling some tests

* fixing translation strings

* adding comments

* removing commented out code

* missing i18n

* fixing rebase conflicts

* removing unused ui contexts

* changes based on review

* adding text to errors

* fixing management plugin

* changes based on review

* refeactor after rebase

* fixing test
…lastic#57312)

* [ML] fix overall stats fetch with a saved search

* [ML] refactor
… new platform router (elastic#57206)

* [ML] modules with NP router

* [ML] use extendedRouteInitializationDeps

* [ML] add apidoc annotations

* [ML] address PR comments

* [ML] convert data_recognizer test to jest

* [ML] optional indexPatternName
…6931)

* advanced settings component registry to new platform
This log is very noisy when running in verbose and does not appear to
have much value in debugging.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…lastic#56428)

* [Add panel flyout] Moving create new to the top of SavedObjectFinder

* [Add panel flyout] Moving create new to the top of SavedObjectFinder

* Fixing failing unit test

* Readd missing test

* [NP] Move saved object modal into new platform (elastic#56383)

* Move saved object modal into new platform

* Fix TS

* Revert "Fix TS"

This reverts commit f2f9f5e.

* Revert "Move saved object modal into new platform"

This reverts commit d0f0ea6.

# Conflicts:
#	src/legacy/core_plugins/kibana/public/discover/np_ready/angular/discover.js

* Move save_object_save_modal

* Move show_saved_object_save_modal.tsx

* Move save_object_finder.tsx

* Remove unused export

* Pass I18nContext to showSaveModal

* Update i18n ids

* Fix map save

* Refactoring

* Load styles

* Revert importing styles

* Update snapshot

* Update snapshot

* Structural refactoring

* Fix path

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* Applying PR comments

Removing faulty rebase imports

Fixing unresolved conflict

Removing faulty merge files

Removing faulty import

Readd accidentally added file

* Removing unnecessary eslint-ignore

Co-authored-by: Maryia Lapata <mary.lopato@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…atform router (elastic#57067)

* update annotation routes to use NP router

* create ts file for feature check

* update schema to allow null value for earliest/latestMs

* update annotations model test

* use NP security plugin to access user info
* feat: 🎸 add UiComponent interface

* feat: 🎸 add adapter for react => ui components and back

* refactor: 💡 move registries to shared /common folder

* feat: 🎸 create expressions service state contaienr

* chore: 🤖 export some symbols

* feat: 🎸 add Executor class

* test: 💍 add simple integration test

* feat: 🎸 move registries into executor

* feat: 🎸 add initial implementation of Execution

* feat: 🎸 move Executor's state container into a signle file

* refactor: 💡 move createError() to /common folder

* feat: 🎸 use Executor in plugin definition

* refactor: 💡 rename handlers to FunctionHandlers

* feat: 🎸 improve function typings

* feat: 🎸 move types and func in sep folder, improve Execution

* refactor: 💡 cleanup expression_types folder

* refactor: 💡 improve typing names of expression types

* refactor: 💡 remove lodash from ExpressionType and improve types

* test: 💍 add ExpressionType tests

* refactor: 💡 remove function wrappers around types

* refactor: 💡 move functions to /common

* test: 💍 improve expression function tests

* feat: 🎸 create /parser folder

* refactor: 💡 move function types into /expression_functions dir

* refactor: 💡 improve parser setup

* refactor: 💡 fix export structure and move args into expr_func

* test: 💍 add ExpressionFunctionParameter tests

* fix: 🐛 fix executor types and imports

* refactor: 💡 move getByAlias to plugin, fix Execution types

* feat: 🎸 add function for argument parsing

* test: 💍 add Executor type tests

* test: 💍 add executor function and context tests

* test: 💍 check that Executor returns Execution

* test: 💍 add basic tests for Execution

* test: 💍 add basic test for execution of a chain of functions

* test: 💍 add "mult" function tot tests

* feat: 🎸 create separate expression_renderer folder

* feat: 🎸 use new executor in public plugin

* feat: 🎸 remove renderers from executor, add result to execution

* fix: 🐛 fix Kibana TypeScript errors

* test: 💍 add file to write integration tests for expr plugin

* refactor: 💡 move state_containers to /common

* refactor: 💡 move /parser to /ast and inline format() function

* refactor: 💡 remove remaining @kbn/interpreter imports

* feat: 🎸 better handling and typing for Executor context

* feat: 🎸 use Executor.run function in plugin

* fix: 🐛 fix TypeScript type errors

* test: 💍 move integration tests into one file

* feat: 🎸 create ExpressionsService

* chore: 🤖 clean up legacy code

* feat: 🎸 use ExpressionsService in /public

* refactor: 💡 move inspector adapters to /common

* feat: 🎸 improve execution

* feat: 🎸 add state to execution state and don't clone AST

* test: 💍 add tests for Execution object

* test: 💍 improve expression test helpers

* test: 💍 add Execution tests

* refactor: 💡 improve required argument checking

* fix: 🐛 fix Kibana TypeScript errors

* test: 💍 add ExpressionsService unit tests

* fix: 🐛 fix Expression plugin TypeScript types

* refactor: 💡 prefix React component with React*

* fix: 🐛 fix X-Pack TypeScript errors

* fix: 🐛 fix test TypeScript errors

* fix: 🐛 fix issues preventing loading

* feat: 🎸 remove getInitialContext() handler

* fix: 🐛 fix TypeScript errors

* chore: 🤖 remove uicomponent interface

* chore: 🤖 remove missing import

* fix: 🐛 correctly handle .query in "kibana" expression function

* refactor: 💡 call first arg in expression functions "input"

* fix: 🐛 do not free Execution state container

* test: 💍 fix tests after refactor

* test: 💍 fix more tests after refactor

* fix: 🐛 remove redundant export

* test: 💍 update intepreter_functional test shapshots

* fix: 🐛 relax "kibana" function throwing on missin gsearch ctx

* refactor: 💡 don't use ExpressionAST interface in Canvas

* docs: ✏️ improve ExpressionRenderer JSDocs

* refactor: 💡 rename context.types to inputTypes in internal fn

* refactor: 💡 replace context.types by unknown in ExprFuncDef

* refactor: 💡 improve expression function definitions in OSS

* fix: 🐛 correctly set name on metric_vis_fn

* refactor: 💡 improve Lens definitions of expression functions

* refactor: 💡 improve Canvas expression function definitions

* test: 💍 add createMockExecutionContext() helper

* refactor: 💡 add some type to events$ observable in expr handler

* feat: 🎸 add types to observables in data handler

* refactor: 💡 use inputTypes in canvas

* fix: 🐛 fix interpreter grammer generation script

* feat: 🎸 allow array in getByAlias

* test: 💍 simplify test function specs

* test: 💍 fix autocomplete tests

* fix: 🐛 use correct expression types and NP getFunctions() API

* refactor: 💡 use NP expressions to get renderer

* fix: 🐛 use context.types on server-side Canvas function defs

* refactor: 💡 use NP API to register Canvas renderers

* feat: 🎸 use NP API to get types

* style: 💄 minor formatting changes

* feat: 🎸 use NP API to get expression functions

* fix: 🐛 fix Canvas workpads

* test: 💍 add missing mock functions

* refactor: 💡 improve Lens func definition argument types

* fix: 🐛 fix Lens type error

* feat: 🎸 make lens datatable work again

* feat: 🎸 bootstrap ExpressionsService on server-side

* feat: 🎸 expose more registry related functions in contract

* feat: 🎸 add environment: server to server-side expressions

* docs: ✏️ add documentation

* test: 💍 add missing Jest mocks

* fix: 🐛 correct TypeScript type

* docs: ✏️ improve documentation

* fix: 🐛 make FunctionHelpDict type contain only Canvas functions

* fix: 🐛 fix merge conflict

* test: 💍 fix expression mocks
* Updating to kind-of@6.0.2

* Rebuilding @kbn\pm
* added test for pages

* fixed types

* update types

* update snap

* PR feedback

* PR feedback
## Summary

* Adds end to end integration tests
* Fixes a bug with import where on imports it was forcing all rules that were being imported to be set to be "enabled: false" instead of honoring what the original export has set for its enabled.
* Adds a few "to be safe" await block so that the front end does not get a race condition within the bulk deletes and other parts of the code.
* Fixes `statusCode` to be `status_code` and removes most of the Hapi Boomer errors
* Changes PUT to be PATCH for partial updates
* Adds true updates with PUT
* Put some TODO blocks around existing bugs found in the API in the e2e tests that we might have time to get to or might not. This will let others maintaining the tests know that once they fix the bug they should update the end to end test to change the behavior.  

Testing this:

Go to the latest CI logs and look for any particular lines from the test executing such as:

```ts
should set the response content types to be expected
```

Also run this manually on your machine through this command:

```ts
node scripts/functional_tests --config x-pack/test/detection_engine_api_integration/security_and_spaces/config.ts
```

Change a test manually and re-run the above command to watch something fail.


Screen shot of what you should see on the CI machine when these are running:
<img width="1825" alt="Screen Shot 2020-02-08 at 10 15 21 AM" src="https://user-images.githubusercontent.com/1151048/74089355-ae9a8e80-4a5d-11ea-9050-86e68d7e3bba.png">


### Checklist

Delete any items that are not applicable to this PR.

~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~

~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

~~- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)~~

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
* Remove get_doc_count function in favor of index status function.

* Rename adapter function.

* Remove obsolete import.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* adding accessibility statement to docs
Adds a list of Alert Instances to the Alert Details page based off of the current state of the Alert.
Adds some apm transaction boundaries for parts of task manager, so that
they will show up in APM as new types of transactions.  Should provide
some visibility into the ES calls made by task manager for alerting and
actions, especially under stress testing scenarios.
elastic#56564)

* Move sample data and feature registration to canvas np plugin

* Prepare to move i18n to new platform and adjust paths

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…ied Host (elastic#57388)

## Summary

Fixes an issue where the `Events Table` on the `Host Details` page would show all events instead of those for the specific host. Also adds tests to make sure this doesn't happen again 🙂 

Note: This will not interfere with any user-added queries/filters as those are added separately. 

##### Before / After (See Event count at bottom of image)
<img width="300" alt="Screen_Shot_2020-02-11_at_15_35_12" src="https://user-images.githubusercontent.com/2946766/74286988-2f6bbb80-4ce6-11ea-9ca2-35d4ae9ff3f2.png"> <img width="300" alt="Screen_Shot_2020-02-11_at_15_46_23" src="https://user-images.githubusercontent.com/2946766/74287150-840f3680-4ce6-11ea-9267-365753a52fc2.png">



### Checklist

Delete any items that are not applicable to this PR.

- [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~
- [ ] ~[Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~
- [ ] ~This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)~
- [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~

### For maintainers

- [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~
* Fix paths for kibana packages in prepare_project_dependencies(elastic#40858).

* Dist/index.js after yarn build.

* chore(NA): correctly include kbm pm new dist file

* Commit after cherry-pick 985c85c from recreation branch.

Co-authored-by: Tiago Costa <tiagoffcc@hotmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Explicit index pattern namespace

* typing of index patterns

* fix ts error on master

* export FieldList

* Move isFilterable into indexPatterns namespace

* Server index pattern cleanup

* comment

* isDefault

* karma mock fix + discover fix

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…6982)

* log meta in pattern layout

* address pgayvallet comment

* add conversion patterns for timestamp

* use comparison instead of inline snapshots

* logs. use elasticsearch and LP compatible timezone format in json layout

* use regexp groups and dot separator for ms. as in default iso format

* use America/New_York timezone as default as it set in guess anyway

* fix APM tests. they need to reset timezone locally and restore it after

* fix logging tests
Kerry350 and others added 11 commits February 19, 2020 09:02
…tic#57691)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [APM] Remove PaginationContainer

Removing the old pagination from the trace summary title.

* [APM] Removing PaginationContainer component ref

* [APM] Change the vertical alignment to center

* [APM] Leaving the EuiPagination component in place

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…d from APM indices (elastic#57311)

* get indices privileges from has_privileges api

* changing to ES privileges api

* changing missing permission page

* always show dimiss button

* always show dimiss button

* changing message and unit test

* fixing react warning message
`yarn start --no-base-path --xpack.ingestManager.enabled=true --xpack.ingestManager.epm.enabled=true --xpack.ingestManager.fleet.enabled=true`

builds but type check fails:

```
± ~/work/kibana [feature-ingest @ ac05485] ✗ ‣ node scripts/type_check.js --project x-pack/tsconfig.json
 ✖ x-pack

ERROR x-pack failed
      x-pack/plugins/ingest_manager/server/routes/agent/handlers.ts:36:7 - error TS2322: Type '{ status: AgentStatus; type: AgentTypeSchema; active: boolean; enrolled_at: string; user_provided_metadata: Record<string, string>; local_metadata: Record<...>; ... 9 more ...; access_api_key?: string | undefined; }' is not assignable to type 'Readonly<{ id: string; user_provided_metadata: Record<string, string>; local_metadata: Record<string, string>; access_api_key: string | undefined; status: string | undefined; ... 10 more ...; current_error_events: Readonly<...>[]; }>'.
        Property 'access_api_key' is optional in type '{ status: AgentStatus; type: AgentTypeSchema; active: boolean; enrolled_at: string; user_provided_metadata: Record<string, string>; local_metadata: Record<...>; ... 9 more ...; access_api_key?: string | undefined; }' but required in type 'Readonly<{ id: string; user_provided_metadata: Record<string, string>; local_metadata: Record<string, string>; access_api_key: string | undefined; status: string | undefined; ... 10 more ...; current_error_events: Readonly<...>[]; }>'.

      36       item: {
               ~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent.ts:34:3
          34   item: Agent;
               ~~~~
          The expected type comes from property 'item' which is declared here on type 'GetOneAgentResponse'

      x-pack/plugins/ingest_manager/server/routes/agent/handlers.ts:72:7 - error TS2322: Type 'AgentEventBase[]' is not assignable to type 'Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>[]'.
        Type 'AgentEventBase' is not assignable to type 'Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>'.
          Property 'payload' is optional in type 'AgentEventBase' but required in type 'Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>'.

      72       list: items,
               ~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent.ts:101:3
          101   list: AgentEvent[];
                ~~~~
          The expected type comes from property 'list' which is declared here on type 'GetOneAgentEventsResponse'

      x-pack/plugins/ingest_manager/server/routes/agent/handlers.ts:310:7 - error TS2322: Type '{ status: AgentStatus; type: AgentTypeSchema; active: boolean; enrolled_at: string; user_provided_metadata: Record<string, string>; local_metadata: Record<...>; ... 9 more ...; access_api_key?: string | undefined; }[]' is not assignable to type 'Readonly<{ id: string; user_provided_metadata: Record<string, string>; local_metadata: Record<string, string>; access_api_key: string | undefined; status: string | undefined; ... 10 more ...; current_error_events: Readonly<...>[]; }>[]'.
        Type '{ status: AgentStatus; type: AgentTypeSchema; active: boolean; enrolled_at: string; user_provided_metadata: Record<string, string>; local_metadata: Record<...>; ... 9 more ...; access_api_key?: string | undefined; }' is not assignable to type 'Readonly<{ id: string; user_provided_metadata: Record<string, string>; local_metadata: Record<string, string>; access_api_key: string | undefined; status: string | undefined; ... 10 more ...; current_error_events: Readonly<...>[]; }>'.
          Property 'access_api_key' is optional in type '{ status: AgentStatus; type: AgentTypeSchema; active: boolean; enrolled_at: string; user_provided_metadata: Record<string, string>; local_metadata: Record<...>; ... 9 more ...; access_api_key?: string | undefined; }' but required in type 'Readonly<{ id: string; user_provided_metadata: Record<string, string>; local_metadata: Record<string, string>; access_api_key: string | undefined; status: string | undefined; ... 10 more ...; current_error_events: Readonly<...>[]; }>'.

      310       list: agents.map(agent => ({
                ~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent.ts:20:3
          20   list: Agent[];
               ~~~~
          The expected type comes from property 'list' which is declared here on type 'GetAgentsResponse'

      x-pack/plugins/ingest_manager/server/routes/agent_config/handlers.ts:30:7 - error TS2322: Type 'AgentConfigSchema[]' is not assignable to type 'Readonly<{ id: string; status: AgentConfigStatus; datasources: string[] | Readonly<{ id: string; name: string; namespace: string | undefined; read_alias: string | undefined; agent_config_id: string; package: Readonly<...> | undefined; streams: Readonly<...>[]; }>[]; ... 4 more ...; description: string | undefined; }...'.
        Type 'AgentConfigSchema' is not assignable to type 'Readonly<{ id: string; status: AgentConfigStatus; datasources: string[] | Readonly<{ id: string; name: string; namespace: string | undefined; read_alias: string | undefined; agent_config_id: string; package: Readonly<...> | undefined; streams: Readonly<...>[]; }>[]; ... 4 more ...; description: string | undefined; }>'.
          Types of property 'datasources' are incompatible.
            Type '(string | DatasourceSchema)[]' is not assignable to type 'string[] | Readonly<{ id: string; name: string; namespace: string | undefined; read_alias: string | undefined; agent_config_id: string; package: Readonly<{ assets: Readonly<{ id: string; type: string; }>[]; description: string; name: string; title: string; version: string; }> | undefined; streams: Readonly<...>[]; }...'.
              Type '(string | DatasourceSchema)[]' is not assignable to type 'string[]'.
                Type 'string | DatasourceSchema' is not assignable to type 'string'.
                  Type 'DatasourceSchema' is not assignable to type 'string'.

      30       items,
               ~~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent_config.ts:15:3
          15   items: AgentConfig[];
               ~~~~~
          The expected type comes from property 'items' which is declared here on type 'GetAgentConfigsResponse'

      x-pack/plugins/ingest_manager/server/routes/agent_config/handlers.ts:53:9 - error TS2322: Type 'AgentConfigSchema' is not assignable to type 'Readonly<{ id: string; status: AgentConfigStatus; datasources: string[] | Readonly<{ id: string; name: string; namespace: string | undefined; read_alias: string | undefined; agent_config_id: string; package: Readonly<...> | undefined; streams: Readonly<...>[]; }>[]; ... 4 more ...; description: string | undefined; }>'.

      53         item: agentConfig,
                 ~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent_config.ts:29:3
          29   item: AgentConfig;
               ~~~~
          The expected type comes from property 'item' which is declared here on type 'GetOneAgentConfigResponse'

      x-pack/plugins/ingest_manager/server/routes/agent_config/handlers.ts:84:47 - error TS2322: Type 'AgentConfigSchema' is not assignable to type 'Readonly<{ id: string; status: AgentConfigStatus; datasources: string[] | Readonly<{ id: string; name: string; namespace: string | undefined; read_alias: string | undefined; agent_config_id: string; package: Readonly<...> | undefined; streams: Readonly<...>[]; }>[]; ... 4 more ...; description: string | undefined; }>'.

      84     const body: CreateAgentConfigResponse = { item: agentConfig, success: true };
                                                       ~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent_config.ts:38:3
          38   item: AgentConfig;
               ~~~~
          The expected type comes from property 'item' which is declared here on type 'CreateAgentConfigResponse'

      x-pack/plugins/ingest_manager/server/routes/agent_config/handlers.ts:112:47 - error TS2322: Type 'AgentConfigSchema' is not assignable to type 'Readonly<{ id: string; status: AgentConfigStatus; datasources: string[] | Readonly<{ id: string; name: string; namespace: string | undefined; read_alias: string | undefined; agent_config_id: string; package: Readonly<...> | undefined; streams: Readonly<...>[]; }>[]; ... 4 more ...; description: string | undefined; }>'.

      112     const body: UpdateAgentConfigResponse = { item: agentConfig, success: true };
                                                        ~~~~

        x-pack/plugins/ingest_manager/server/types/rest_spec/agent_config.ts:48:3
          48   item: AgentConfig;
               ~~~~
          The expected type comes from property 'item' which is declared here on type 'UpdateAgentConfigResponse'

      x-pack/plugins/ingest_manager/server/services/agents/acks.ts:26:5 - error TS2322: Type 'AgentActionBaseSchema[]' is not assignable to type 'Readonly<{ type: "POLICY_CHANGE" | "DATA_DUMP" | "RESUME" | "PAUSE"; id: string; created_at: string; data: string | undefined; sent_at: string | undefined; }>[]'.
        Type 'AgentActionBaseSchema' is not assignable to type 'Readonly<{ type: "POLICY_CHANGE" | "DATA_DUMP" | "RESUME" | "PAUSE"; id: string; created_at: string; data: string | undefined; sent_at: string | undefined; }>'.
          Property 'data' is optional in type 'AgentActionBaseSchema' but required in type 'Readonly<{ type: "POLICY_CHANGE" | "DATA_DUMP" | "RESUME" | "PAUSE"; id: string; created_at: string; data: string | undefined; sent_at: string | undefined; }>'.

      26     actions: updatedActions,
             ~~~~~~~

        x-pack/plugins/ingest_manager/server/types/models/agent.ts:92:3
          92   actions: schema.arrayOf(AgentActionSchema),
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          The expected type comes from property 'actions' which is declared here on type 'Partial<Readonly<{ user_provided_metadata: string; local_metadata: string; current_error_events: string | undefined; type: AgentTypeSchema; active: boolean; enrolled_at: string; ... 6 more ...; actions: Readonly<...>[]; }>>'

      x-pack/plugins/ingest_manager/server/services/agents/checkin.ts:75:79 - error TS2345: Argument of type '{ last_checkin: string; default_api_key?: string | undefined; actions?: AgentActionBaseSchema[] | undefined; local_metadata?: string | undefined; current_error_events?: string | undefined; }' is not assignable to parameter of type 'Partial<Readonly<{ user_provided_metadata: string; local_metadata: string; current_error_events: string | undefined; type: AgentTypeSchema; active: boolean; enrolled_at: string; ... 6 more ...; actions: Readonly<...>[]; }>>'.
        Types of property 'actions' are incompatible.
          Type 'AgentActionBaseSchema[] | undefined' is not assignable to type 'Readonly<{ type: "POLICY_CHANGE" | "DATA_DUMP" | "RESUME" | "PAUSE"; id: string; created_at: string; data: string | undefined; sent_at: string | undefined; }>[] | undefined'.
            Type 'AgentActionBaseSchema[]' is not assignable to type 'Readonly<{ type: "POLICY_CHANGE" | "DATA_DUMP" | "RESUME" | "PAUSE"; id: string; created_at: string; data: string | undefined; sent_at: string | undefined; }>[]'.

      75   await soClient.update<AgentSOAttributes>(AGENT_SAVED_OBJECT_TYPE, agent.id, updateData);
                                                                                       ~~~~~~~~~~

      x-pack/plugins/ingest_manager/server/services/agents/checkin.ts:122:9 - error TS2322: Type '{ attributes: { payload: string | undefined; type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; agent_id: string; }; type: string; }[]' is not assignable to type 'SavedObjectsBulkCreateObject<Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>>[]'.
        Type '{ attributes: { payload: string | undefined; type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; agent_id: string; }; type: string; }' is not assignable to type 'SavedObjectsBulkCreateObject<Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>>'.
          Types of property 'attributes' are incompatible.
            Type '{ payload: string | undefined; type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; agent_id: string; }' is not assignable to type 'Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>'.
              Property 'data' is optional in type '{ payload: string | undefined; type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; agent_id: string; }' but required in type 'Readonly<{ type: "STATE" | "ERROR" | "ACTION_RESULT" | "ACTION"; subtype: "RUNNING" | "STARTING" | "IN_PROGRESS" | "CONFIG" | "FAILED" | "STOPPED" | "DATA_DUMP" | "STOPPING" | "ACKNOWLEDGED" | "UNKNOWN"; ... 6 more ...; stream_id: string | undefined; }>'.

      122   const objects: Array<SavedObjectsBulkCreateObject<AgentEventSOAttributes>> = events.map(
                  ~~~~~~~

      x-pack/plugins/ingest_manager/server/services/api_keys/security.ts:31:43 - error TS2339: Property 'getClusterClient' does not exist on type 'AppContextService'.

      31   const clusterClient = appContextService.getClusterClient();
                                                   ~~~~~~~~~~~~~~~~

      x-pack/plugins/ingest_manager/server/services/epm/registry/extract.ts:7:17 - error TS7016: Could not find a declaration file for module 'tar'. '/Users/jfsiii/work/kibana/x-pack/node_modules/tar/index.js' implicitly has an 'any' type.
        Try `npm install @types/tar` if it exists or add a new declaration (.d.ts) file containing `declare module 'tar';`

      7 import tar from 'tar';
                        ~~~~~

      Found 12 errors.
```
@jfsiii
Copy link
Contributor Author

jfsiii commented Feb 20, 2020

@elasticmachine merge upstream

John Schulz added 8 commits February 21, 2020 12:39
The plugin is disabled by default and not currently running the FTR tests. I believe we'll add this back when we restore the EPM integration tests.
EPM routes currently return a 500 for these tests. For now they `.expect(500)` when enabled and `.expect(404)` when disabled. We can look into the issue and get them to `.expect(200)` in later tests.
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

current_error_events: schema.arrayOf(AgentEventSchema),
};

const AgentSchema = schema.object({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These Agent*Schema values were deleted because, afaict, they were only use to build the TS types. They are exported.

LMK if I should restore them.

@jfsiii jfsiii merged commit 79411a9 into elastic:feature-ingest Feb 24, 2020
@jen-huang jen-huang added the Team:Fleet Team label for Observability Data Collection Fleet team label Mar 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Development

Successfully merging this pull request may close these issues.