Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Index template] Refactor index template wizard #69037

Conversation

sebelga
Copy link
Contributor

@sebelga sebelga commented Jun 12, 2020

This PR adds 2 new pieces of reusable functionality to the "es_ui_shared/public/forms" folder.

  • Form wizard
  • Multi content

I then updated the current <TemplateForm /> component used to create and edit legacy index templates to make use of them.

I also added some small changes to the mappings editor, mainly around getting its validity fixed (see issue #55051) and removed unnecessary re-renders.

The end result of this PR should be the exact same behavior from the index template creation/editing flow that we have in master... but using the reusable components.

MultiContent

The problem

Building resource creations/edition flows in the UI, that have multiple contents that need to be merged together at the end of the flow and at the same time keeping a reference of each content state, is not trivial. Indeed, when we switch tab or we go to the next step, the old step data needs to be saved somewhere.

The first thing that comes to mind is: "Ok, I'll lift the state up" and make each step "content" a controlled component (when its value changes, it sends it to the global state and then it receives it back as prop). This works well up to a certain point. What happens if the internal state that the step content works with, is not the same as the outputted state?

Something like this:

// StepOne internal state, flat map of fields
const internalState: {
  fields: {
    ate426jn: { name: 'hello', value: 'world', parent: 'rwtsdg3' },
    rwtsdg3: { name: 'myObject', type: 'object' },
  }
}

// Outputed data

const output = {
  stepOne: {
    myObject: {
      hello: 'world'
    }
  }
}

We need some sort of serializer to go from the internal state to the output object. If we lift the state up this means that the global state needs to be aware of the intrinsic of the content, leaking implementation details.
This also means that the content can't be a reusable component as it depends on an external state to do part of its work (think: the mappings editor).

This is where MultiContent comes into play. It lets us declare content objects and automatically saves a snapshot of their content when the component unmounts (which occurs when switching a tab for example). If we navigate back to the tab, the tab content gets its defaultValue from that cache state.

Let see it through a concrete example

// Always good to have an interface for our contents
interface MyMultiContent {
  contentOne: { myField: string };
  contentTwo: { anotherField: string };
  contentThree: { yetAnotherField: boolean };
}

// Each content data will be a slice of the multi-content defaultValue
const defaultValue: MyMultiContent = {
  contentOne: {
    myField: 'value',
  },
  contentTwo: {
    anotherField: 'value',
  },
  contentThree: {
    yetAnotherField: true,
  },
};
// my_comp.tsx

/**
 * We wrap our component with the HOC that will provide the <MultiContentProvider /> and let us use the "useMultiContentContext()" hook
 * 
 * MyComponent connects to the multi-content context and renders each step
 * content without worrying about their internal state.
 */
const MyComponent = WithMultiContent(() => {
  const { validation, getData, validate } = useMultiContentContext<MyMultiContent>();

  const totalSteps = 3;
  const [currentStep, setCurrentStep] = useState(0);

  const renderContent = () => {
    switch (currentStep) {
      case 0:
        return <ContentOneContainer />;
      case 1:
        return <ContentTwoContainer />;
      case 2:
        return <ContentThreeContainer />;
    }
  };

  const onNext = () => {
    // Validate the multi content
    const isValid = await validate();

    if (!isValid) {
      return;
    }

    if (currentStep < totalSteps - 1) {
      // Navigate to the next content
      setCurrentStep((curentStep += 1));
    } else {
      // On last step we need to save so we read the multi-content data
      console.log('About to save:', getData());
    }
  };

  return (
    <>
      {renderContent()}

      {/* Each content validity is accessible from the `validation.contents` object */}
      <EuiButton onClick={onNext} disabled={validation.contents[currentStep] === false}>
        Next
      </EuiButton>
    </>
  );
});
// content_one_container.tsx

// From the good old days of Redux, it is a good practice to separate the connection to the multi-content
// from the UI that is rendered.
const ContentOneContainer = () => {

  // Declare a new content and get its default Value + a handler to update the content in the multi-content
  // This will update the "contentOne" slice of the multi-content.
  const { defaultValue, updateContent } = useContent<MyMultiContent>('contentOne');

  return <ContentOne defaultValue={defaultValue} onChange={updateContent} />
};
// content_one.tsx

const ContentOne = ({ defaultValue, onChange }) => {
  // Use the defaultValue as a starting point for the internal state
  const [internalStateValue, setInternalStateValue] = useState(defaultValue.myField);

  useEffect(() => {
    // Update the multi content state for this content
    onChange({
      isValid: true, // because in this example it is always valid
      validate: async () => true,
      getData: () => ({
        myField: internalStateValue,
      }),
    });
  }, [internalStateValue]);

  return (
    <input value={internalStateValue} onChange={(e) => setInternalStateValue(e.target.value)} />
  );
}

And just like that, <ContentOne /> is a reusable component that gets a defaultValue object and an onChange handler to communicate any internal state changes. He is responsible to provide a getData() handler as part of the onChange that will do any necessary serialization and sanitization, and the outside world does not need to know about it.

FormWizard

The <FormWizard /> and <FormWizardStep /> components lets us declare form wizard in a declarative way. It works hand in hand with the MultiContent explained above to make building form wizards a breeze. 😊

It takes care of enabling, disabling the <EuiStepsHorizontal /> steps as well as the "Back" and "Next" button.

Let's see it through an example

const MyForm = () => {
  return (
    <FormWizard<MyMultiContent>
      defaultValue={wizardDefaultValue} // The MultiContent default value as explained above
      onSave={onSaveTemplate} // A handler that will receive the multi-content data
      isEditing={isEditing} // A boolean that will indicate if all steps are already "completed" and thus valid or if we need to complete them in order
      isSaving={isSaving} // A boolean to show a "Saving..." text on the button on the last step
      apiError={apiError} // Any API error to display on top of wizard
      texts={i18nTexts} // i18n translations for the nav button.
    >
      <FormWizarStep id="contentOne" lable="Label for the step">
        <div>
          Here you can put anything... but you probably want to put a Container from the
          MultiContent example above.
        </div>
      </FormWizarStep>

      <FormWizarStep id="contentTwo" lable="Label for the step" isRequired>
        <div>
          Here you can put anything... but you probably want to put a Container from the
          MultiContent example above.
        </div>
      </FormWizarStep>

      <FormWizarStep id="contentThree" lable="Label for the step">
        <div>
          Here you can put anything... but you probably want to put a Container from the
          MultiContent example above.
        </div>
      </FormWizarStep>
    </FormWizard>
  );
};

That's all we need to build a multi-step form wizard, making sure the data is cached when switching steps.

Note to reviewer: In the current PR, try adding isRequired to some of the steps to see how we can easily let a user skip optional steps and at the same time not let him pass over a required step.

Items left TODO

In a separate PR I will

  • Add README for both the <FormWizard />
  • Add tests to the <FormWizard />
  • Add README for both the MultiContent
  • Add tests to the MultiContent

Fixes #55051

@sebelga sebelga force-pushed the index-template/reusable-settings-mappings-aliases-configurator branch from b5920c9 to 337b872 Compare June 17, 2020 13:15
@sebelga sebelga requested a review from alisonelizabeth June 17, 2020 14:52
@sebelga sebelga marked this pull request as ready for review June 17, 2020 14:53
@sebelga sebelga requested a review from a team as a code owner June 17, 2020 14:53
@sebelga sebelga added Feature:Index Management Index and index templates UI release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.9.0 v8.0.0 labels Jun 17, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

Copy link
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great @sebelga! I'm excited to try it out with the component templates form wizard.

I had two questions that I commented on in the code. Everything else is minor typo fixes 😄. Tested locally and did not find any regressions.

Some other general comments:

  • I think it would be preferred if the mappings editor changes were completed in a separate PR. It wasn't clear to me if they were necessary to get the existing flow working with the shared components, or it was just something you were addressing in parallel. I won't block on this though.

  • I noticed now we have static/forms (form lib) and public/forms (wizard/multi-content) in the es_ui_shared directory. I think this is a little confusing. WDYT?

  • Were you planning on moving the mappings/aliases/settings steps into es_ui_shared in a separate PR? I can also work on this if you'd like.

  • The PR description you wrote is awesome! I know you mentioned adding a readme in a separate PR, but if it's not something you plan to address soon, I wonder if it makes sense to repurpose the description as the readme in the interim?

* Get the step index from a step id.
*/
const getStepIndex = useCallback(
(stepId: number | string) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the scenario where a string id would be provided instead of a number (or vice versa)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to leave it to the consumer to decide if he wants to navigate to a step through index (position) or id. I'd always recommend to navigate by Id so steps could be reordered, but I preferred to leave the decision to the consumer.

(_content as Content).validate = async () => isValid;
updatedValidation[id] = isValid;

// Only update if if its true, don't override a previous "false" or "undefined"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Only update if if its true, don't override a previous "false" or "undefined"
// Only update if it's true, don't override a previous "false" or "undefined"

if (stepIndex > activeStepIndex + 1) {
/**
* Rule explained:
* - all the previsous steps are always enabled (we can go back anytime)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you be able to click "Back" in this scenario? The button is enabled, but it does not take you to the previous step.

back_enabled

Copy link
Contributor Author

@sebelga sebelga Jun 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was a bug, the "Back" button should be disabled when the step is invalid. I fixed it. 👍

sebelga and others added 9 commits June 18, 2020 15:05
…usable-settings-mappings-aliases-configurator
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
…ent_context.tsx

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
…content.ts

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
…context.tsx

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
…gurator' of github.com:sebelga/kibana into index-template/reusable-settings-mappings-aliases-configurator
@sebelga
Copy link
Contributor Author

sebelga commented Jun 18, 2020

Thanks for the review @alisonelizabeth !

I think it would be preferred if the mappings editor changes were completed in a separate PR. It wasn't clear to me if they were necessary to get the existing flow working with the shared components, or it was just something you were addressing in parallel. I won't block on this though.

Most of the changes were required to make the whole flow work. For example the eslint comment for the form.subscribe() are especially important to avoid a infinite render loop. This is my top most priority to fix on the form lib. Then I fixed the validity issue not being sent to the consumer and a small glitch issue.

I noticed now we have static/forms (form lib) and public/forms (wizard/multi-content) in the es_ui_shared directory. I think this is a little confusing. WDYT?

Yes, the form lib will be moved to the "public/forms" folder, it is also a TODO on my list but this will have to be a separate PR as it will touch multiple plugins.

Were you planning on moving the mappings/aliases/settings steps into es_ui_shared in a separate PR? I can also work on this if you'd like.

I was not planning in moving them to the "es_ui_shared" folder but to the "shared" folder that you created. Also as a separate work, but you can go ahead and do it, I will rebase my branch as you will merge to master before me for sure 😊

The PR description you wrote is awesome! I know you mentioned adding a readme in a separate PR, but if it's not something you plan to address soon, I wonder if it makes sense to repurpose the description as the readme in the interim?

Great point. I went ahead and added 2 README.md for FormWizard and MultiContent. 👍

@alisonelizabeth
Copy link
Contributor

Thanks @sebelga!

Most of the changes were required to make the whole flow work. For example the eslint comment for the form.subscribe() are especially important to avoid a infinite render loop. This is my top most priority to fix on the form lib. Then I fixed the validity issue not being sent to the consumer and a small glitch issue.

👍

Yes, the form lib will be moved to the "public/forms" folder, it is also a TODO on my list but this will have to be a separate PR as it will touch multiple plugins.

👍

I was not planning in moving them to the "es_ui_shared" folder but to the "shared" folder that you created. Also as a separate work, but you can go ahead and do it, I will rebase my branch as you will merge to master before me for sure

Ah, right. I forgot about that - makes sense and I can plan on taking care of it.

@sebelga sebelga merged commit 5e8d824 into elastic:master Jun 18, 2020
@sebelga sebelga deleted the index-template/reusable-settings-mappings-aliases-configurator branch June 18, 2020 16:28
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

id value diff baseline
indexManagement 143 +1 142

page load asset size

beta
id value diff baseline
/bundles/plugin/esUiShared/esUiShared.plugin.js 924.0KB +23.5KB 900.6KB
/bundles/plugin/indexManagement/indexManagement.plugin.js 200.8KB +140.0B 200.7KB
total - +23.6KB -

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

sebelga added a commit that referenced this pull request Jun 18, 2020
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
mistic added a commit to mistic/kibana that referenced this pull request Jun 24, 2020
chore(NA): merge changes on renovate config file

chore(NA): move everything under alias lodash4

chore(NA): missing lodash to lodash4 alias

chore(NA): migrate packages to lodash4

chore(NA): update renovate cfg file

chore(NA): skip failing test

chore(NA): first remove of alias lodash4

chore(NA): remove lodash4 alias

chore(NA): remove skip from test

[ES UI] Error handling (#68809)

[Console] Scrub the lead and trailing brackets from ipv6 host names (#68991)

* scrub the lead and trailing brackets from ipv6 host names

* Update comment

* refactor: scrub -> sanitize

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

fix spec color highlighting not working on vega vis (#68995)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[SECURITY] rename server plugin to siem (#69044)

* rename server plugin to siem to avoid privilege issue

* review from alerting

* missing change with rename

* fix tests

* missing api integration test

* fix api integration spaces

Type safe browser.executeAsync (#69018)

* make browserAsync type safe

* adopt tests

* prefer unknown over any

* simplify signature

fixes 'Configures a new connector' flaky test (#69011)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Embed dashboard by value example & some embeddable clean up (#67783)

Added example for using dashboard container by value
1.1 Refactored embeddable explorer e2e test to use new example, removed not needed kbn_tp_embeddable_explorer plugin.
For embeddable explorer examples went away from using getFactoryById() to improve type checks
There is new component a replacement for EmbeddableFactoryRenderer with slightly more flexible api: EmbeddableRenderer.
3.1 We can improve it going forward to support more use case

[Endpoint] Adding alerts route (#68183)

* Adding alerts route

* Adding related alerts generator changes, tests, and script updates

* Fixing missed parameter

* Aligning the AlertEvent and ResolverEvent definition

* Fixing type errors

* Fixing import error

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Monitoring] Ensure we use existing Elasticsearch config (#68389)

* Ensure we use existing Elasticsearch config

* Use separate type for this to ensure custom properties work

* PR suggestions

* PR feedback

* PR feedback

* Fix type issues

* PR feedback

[APM] Changes to duration formatting (#69039)

* [APM] Changes to duration formatting

* Change the threshold for showing microseconds to 1 millisecond instead of 10. This means you now get "900 µs/1.3 ms/20.0 ms" instead of "900 µs/1300 µs/20 ms."
* Change milliseconds formatted with `asDuration` to be `asDecimal` instead of `asInteger`. That means you get "0.0 ms/2.5 ms/3.0 ms" instead of "0 ms/2 ms/3 ms."
* Tables were all using their own module-scoped functions that called `asDuration` to format things as decimal milliseconds. Extract this to an `asMillisecondDuration` function exported from the duration helpers and use it in all the tables.
* Change `getResponseTimeseries` in chart selectors to use `asDuration` to make all chart timeseries units consistent.
* Don't export `convertTo` from the duration helpers as it's now not used anywhere. Always use a more specific exported function for more consistency.
* Change ">=" to "≥" in the ML flyout text.

* Update e2e snapshot

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[SECURITY SOLUTION] Task/endpoint details to fleet (#68710)

[SECURITYSOLUTION][INGEST] Fixes endpoint link, adds link from endpoint details to fleet

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Set wrap to the errorLink so it doesn't go outside of box (#67797)

* set wrap to the errorLink so it doesnt go outside of box

* updating

* set wrap to the errorLink so it doesnt go outside of box

* updating

* Show full message on hover

Co-authored-by: Nathan L Smith <smith@nlsmith.com>

[Metrics UI] Add preview feature for metric threshold alerts (#67684)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[skip-ci] Rename @elastic/pulse to @elastic/kibana-telemetry (#69181)

try to fix integration test by making template more specific (#69138)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Alerting] Update Index Threshold to use columns in EuiExpression (#69051)

Add enroll agent action to config action menu (#68840)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Don't use filter to get default config & output (#69088)

* Don't use filter to get default config & output

the KQL parsing takes up a lot of CPU time. Avoid it with a search string

Also add a `refresh: false` as suggested by @bkobel

* Does restoring await fix CI?

* Remove 'refresh: false' from agent enroll

Does this this fix the failing test? It does keep the PR more focused

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[SIEM][Detections] In progress alert state (#68569)

[Endpoint] Remove dependency on ingest for the index patterns (#69058)

* Remove dependency on ingest for the index patterns

* Fixing the tests

* Fixing test

* Use variable instead of class

Updates the Release Notes content in CONTRIBUTING (#69032)

* Updates the Release Notes content in CONTRIBUTING

* Release Notes guidelines

* Fixes bulleted list indentation

Fix plugin lifecycle log to only include server plugins (#68686)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Ingest Manager] Add ability to copy an agent config (#68737)

* Add copy agent config endpoint, schema types, and request hook

* Add copy config to context menu and modal

* Change data sources to be copied via bulk instead of singles in parallel

* Add copy endpoint to openapi spec file

* Add api integration tests

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[ML] Allow editing of model_snapshot_retention_days (#69003)

* [ML] Allow editing of model_snapshot_retention_days

* removing commented code

* flooring number input

* adding daily_model_snapshot_retention_after_days

* updating default values

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Remove obsolete custom types from super date picker handlers. (#69038)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Using correct index patterns (#69208)

[Endpoint] [ES Archiver] Allowing create option to be passed through the cli for es archiver (#69191)

* Allowing create option to be passed through the cli

* Using kebab casing

[Security] siem to securitySolution SO migration (#68776)

chore(NA): include hidden files when creating package with plugin helpers (#68247)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Reporting/Server] register plugin routes synchronously (#68976)

* register routes synchronously

* back out some refactoring

* comment fix

* fix tests

* register route handler context provider

* Add function level comments in core methods

* fix tests

* revert editor help

* route context is the ReportingStart contract

* Fix reporting job route tests

* Fix generation tests

Co-authored-by: Joel Griffith <joel.griffith@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[release-notes] add script to generate release notes from PRs (#68816)

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Sort filter fields by items that start with typed characters. (#68585)

[kbn/optimizer] share all plugin bundles (#68986)

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[CI] GitHub check details link to test reports and errorlevel (#69157)

[Alerting] Adds API for executing actions immediately (#68897)

This PR adds an API to the AlertsClient which allows users to execute actions immediately, rather than via TaskManager.

This also moves the TaskManager based execution to a new api on ActionsClient along side the immediate execution.

stabilize flaky embeddables/adding_children take 2 (#68873)

* wait for loading indicator to be deleted

* improve with retry

* Update test/examples/embeddables/adding_children.ts

Co-authored-by: Spencer <email@spalger.com>

Co-authored-by: Spencer <email@spalger.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Update vega version (#68639)

* update vega version

* a11y skip test

* a11y skip test attempt 2

* adding back all a11y tests

* fix failed karma test

* remove extra  0BSD license

* coalesce yarn.lock versions a little

* update kbn/pm dist

* fix CI

* fix Vega View for ML

Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[APM] Fix confusing request/minute viz (#69143)

[Ingest pipelines] Fix create pipeline button copy (#69118)

[ILM] Update doc link (#68923)

[ML] Model snapshot management (#68182)

* [ML] Model snapshot management

* more updates

* adding calendar range

* updating layout

* multiple calendars

* moving calendar creator

* fixing chart issues

* fixing chart issues

* improving calendar rendering

* adding capabilities checks

* code clean up

* fixing end time argument type

* fix translations

* code clean up

* comments based on review

* changes based on review

* fixing include

* adding useMemo to theme function

[Maps] Security layer wizards (#68290)

* [Maps] security layer wizard

* index pattern select

* use IndexPattern

* destination layer

* source layer

* cache security index patterns

* line layer

* use top hits

* tslint

* unit test

* fix loading speed and i18n

* handle case where ui setting siem:defaultIndex does not exist

* tslint

* review feedback

* extract lazy load bundle from map embeddables

* expose createSecurityLayers via plugin start

* tslint

* export MapsPluginStart

* fix path move

* review feedback

* fix import

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Ingest Node Pipelines] Pipeline Processors Editor (#66021)

* initial plugin setup

* add smoke test

* fix license check

* refactor plugin setup

* Server-side create/update ingest pipelines (#62744)

* List pipelines (#62785)

* First iteration of ingest table

* Add action placeholders

* Refactor list and address PR feedback

Refactored the list into smaller pieces and assemble in main.tsx

Also addressed feedback on copy, removed unused notifications dep

* WiP on flyout

Showing name in title

* Add reload button

* Finish first version of flyout

* Slight update to copy

* `delete` -> `edit`

* Address PR feedback

Copy and a11y updates

* Add on failure JSON to flyout if it is available

* Add details json block file and remove ununsed import

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* [Ingest pipelines] Create pipeline UI (#63017)

* First vertical slice of pipeline editor component

* Made a space for common parameters

* [Ingest pipelines] Edit pipeline page (#63522)

* First iteration of CRUD functionality working

* WiP on moving the pipeline editor to pipeline processor editor

* Finish refactor to work with passing state out

* Refactor and fix tests

* [Ingest pipelines] Polish details panel and empty list (#63926)

* Address some early feedback and use FormDataProvider

FormDataProvider gives a more declarative way to listen to form
state updates.

Also refactored the state reader mechanism.

* [Ingest pipelines] Delete pipeline (#63635)

* [Ingest Node Pipelines] Clone Pipeline (#64049)

* First iteration of clone functionality

Wired up for both the list table and the details flyout in the
list section.

* satisfy eslint

* Turn on sorting for the list table

* Clean up const declarations

* Address PR feedback

Sentence-casify and update some other copy.

* Mark edit and delete as primary actions in list table

* Handle URI encoded chars in pipeline name when cloning

* Update to using the more flexible controlled component pattern

To make this component and mappings editor more consistent, both
will expose a more traditional controlled component interface to
consumers.

In the current implementation this requires that consumers use
`useCallback` for `onUpdate` handlers to avoid an infinite
rendering cycle and to avoid staleness bugs in their `onUpdate`
handlers they need to think about what might make the handler
stale.

This approach comes with the benefits and flexibility of
controlled components at the cost of slightly more complex
consumption of the components.

In future, we can explore adding the uncontrolled component
interface too so that consumers have the option to more simply
render and pull data out only when needed

* Handle sub-form validity

The pipelines processor editor not emits overall validity to
consumers

* Fix Jest test

* Refactor some names

prepareDataOut -> serialize
prepareDataIn -> deserialize
EditorProcessor -> ProcessorInternal

* Mark as private

* Major WiP

Started working on the drag-and-drop-tree and updated some
typings stuff

* [Ingest node pipelines] Privileges (#63850)

* Create privileges check for ingest pipelines app

Also moved the public side logic for checking and rendering
privilege related messages to es_ui_shared/public following the
new __packages_do_not_import__ convention.

* Add ,

* Fix import paths

* Address PR feedback

Fix i18n strings (remove reference to snapshot and restore) and
fix copy referencing snapshot and restore - all copy-pasta errors.

Also remove unused field from missing privileges object.

* Fix issue from resolving merge conflicts

* Add missing app privilege

* Use non-deprecated privilige name

* First iteration of drag and drop tree on feature parity

* First steps toward add on failure handler

Updated reducer logic to create the next copy value using immer.

* First iteration of nested tree with add on failure working

* Refactor and some UI layout updates

- Remove the "id" field on processors for now
- Implement the nested remove and update functionality again

* Remove immer (not call stack safe)

Refac to remove immer and reimplemented the immutable set and
get functions.

* [Ingest Node Pipelines] More lenient treatment of on-failure value (#64411)

* Move file to components folder

* [Ingest pipelines] Simulate pipeline (#64223)

* Updated tree rendering

- turn off dropzones for children
- fixed up some padding and margins
- fixed integration with pipeline_form.tsx

The current implementation still has a lot of jank stemming from
the UX with DnD. Unfortunately the nesting has opened a number
of issues here.

* [Ingest Node Pipelines] Show flyout after editing or creating a pipeline (#64409)

* Show flyout after editing or creating a pipeline

* JSX comment

* Show not found flyout

Copied from CCR

* update not found flyout and fix behavior when viewing details from table

* Reset pipeline name in URI when closing flyout

* Remove encodeURI

Already using encodingURIComponent for unsafe string.

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>

* Clarification of terms

- addProcessor -> addTopLevelProcessor (same in editor modes)
- Expanded comment on ProcessorSelector

* Implement move between lists functionality

Added tests to the reducer for moving in and out of a recusrsive
structure.

* fix TS

* Prevent nesting a parent inside of its own child

* Add comment

* [Ingest pipelines] Cleanup (#64794)

* address review feedback

* remove unused import

* Big refactor to tree rendering structure

DnD tree now converts the nested tree to a flat one and only
consists of _1_ droppable area with a flat array of draggables
that can be combined.

Using the existing logic in the reducer combined with translating
the flat structure changes to a format the nested reducer can
understand looks like a really promising avenue.

There still seems to be a bug with a longer list where items do
not interact properly.

* Remove unused component

* A number of NB changes

- Fixed a subtle serialisation bug with on_failure in options
- Fixed a performance bug by using useRef in pipeline form
- Memoized the drag and drop tree
- Removed use of "isValid" from form lib... I think this should
be removed entirely.

* fix bad conflict resolution

* Implemented a slightly better destination resolution algo.

Also added tests.

* Fix subtle staleness bug, whitelist keys for setValue

* NB styling fix!!

Due to a parent's setting of overflow: hidden the drag and drop
tree had a dead zone that would equal the page overflow limit,
unsetting on the component itself (overflow: auto) relaxes this
limitation.

* Fix stale delete bug too

* Update naming of editor modes and update comments

* Use field types combo box

* Add delete confirmation modal

* Refactor delete modal component file name

* Better visual integration with existing form

* Update layout and styling of form

- added some padding around the new processors editor field
- Updated use of flex box in the pipeline form fields

* Move pipeline processor copy into pipeline processor component

The test button is also now inside of the pipeline processor
editor. Eventually all of this functionality should be moved
into the pipeline processor editor.

* First step of refactor to moving between trees

* First iteration of x-tree drag and drop

* Remove unused import

* Fix jest test types

* Fix up minor i18n issues and fix up layout of on failure

* Remove unnecessary prop

* Update spacing above add processor button to make it more center

* Fix destination resolution algo

* Update dragging resolver unit tests and add a lot more comments

* Use one sorting algo (removed use of euiReorder for now)

* Add placeholder tests and update comments

* Quite a big refactor

- Remove DraggableLocation entirely, only use ProcessorSelector
this simplifies mapping back to reducer instructions quite a lot
- Add tests for special case dragging behaviour

* Fix off by one bug in tests and implementation 🤦🏼‍♂

- Also move processor reducer to it's own folder

* Update behaviour for dragging up across trees and add tests

* Fix combine instruction

* Fix test and i18n issues

* Remove background color

* Fix selector after selector refactor

* A major performance

- Do not re-render the entire tree when editing settings

* Fix component smoke test

* Fix reading value from processor state using processor selector

* [Ingest pipelines] Custom processor form (#66022)

* Re add background color and refactor name of processor item

* Fix file naming and refactor 🚜 dnd tree rendering

- deserialze and serialize were backwards, fixed
- the dnd tree was rendering a flat tree which needed to be
mapped back to the nested tree for updates. This is still the
case but we do use recursive rendering. This enables tree nodes
to better hold local state like whether they are collapsed or not.

* Fix getting of initial on failure processors value

* Update padding styles for containers

* A lot of styling updates to get closer to look of mockup

* A WiP version of th click-tree an alternative to dnd

As a response to really long pipelines we may pivot away from
dnd.

* Remove dnd tree

* clean up reamining dnd tree references

* Clean up and refactor of tree component

To simplify the logic of the tree component processor id's
were added. Also had to update the jest spec to support this

* Added the ability to duplicate a processor

* Fix types in test

* Added duplicate functionality to ui

* Memoize tree components

* address es lint issues

* remove unused import

* Fix editing of custom json

* Address form performance issues

* Add all known missing processors

* Add ability to cancel move

* Fix staleness in test and view request flyouts

* fix type issue

* Remove unused translations and skip funcitonal test for now

* add todo comment

* Fix type issues

* remove poc styles

* disable other move buttons if we have a selected processor

* Refactor drop zone pin to button and add some styling

* Refactor processor editor item

* Update styling and use icon for cancel move action too

* fix nasty integration bug

* some minor optimizations

* prevent parent from being placed in own on failure handler

* Re-add cancel button

* Re-introduce failure processors toggle

* Fix typo

* Add Handler types for processor editor item

* Fix staleness bug for type, refactor classname and fix duplicate
bug not immutably copying values

* Experimenting with padels (revert this to undo if no further
changes have been made)

* Add description and unique ids

* Share links via component-wide context rather than props

* Virtualized list and back to outline dropzones

* Refactor id getter to a service and make it an incrementing number

* Temporary fix for double rendering issue.

This means the pipeline processors component is not controllable
but fixes the double rendering even when processors have not
changed. This is a problem coming from the ingest pipelines
plugin form system outside of this component

* add todo comment

* remove euicode element

* properly handle duplicate flow

* attempt to fix i18n

* split private_tree into it's own component and add comments

* refactor 🚜. rename Tree to ProcessorsTree and move things around

* do not delete the top level arrays for processors and onfailures

* fix typescript error

* Move duplicate, addOnFailure and delete actions into ctx menu

* remove unused import

* add support for pressing esc key to cancel move

* Add outside click listener

* always prompt before deleting a processor

* refactor remove distinction between adding top level and on fail

* add processor button to tree

* Hide the add on failure context menu item for processors with
failure handlers

* Reinstated x-tree moving and highlight and disable for buttons
on move and on edit

* removing ids step 1: remove idGenerator

* remove ids step 2: added inline text input

Also refactored a lot of the tree actions away. Now using
context and the processors editor tree item component for
dispatching actions. The tree item has access to the dispatch
and to the selector which makes it a well positioned component.

Also saves on some props drilling.

* Slight improvement to styling of text input (border)

* Re-implement missing failure toggle test

* address type todo

* Address many type issues and fix yarn.lock

* re-enable create pipeline functional test

* prevent multiple flyouts from opening

* change flyout title when editing an on-failure processor

* absolutely position the failure handlers label

when we render a dropzone on the label, then we move the label
up without affecting overall component height

* fix description behaviour not removing tag if empty

* some minor clean up

* add onflyoutopen cb to tests

* refactor processors editor item to multiple files

also refactored i18n into it's own file. would be good to come
up with a pttaern for doing this more broadly.

* fix add on-failure handler in context menu after refactor

* tag -> new description field

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Alison Goryachev <alison.goryachev@elastic.co>

[ML] DF Analytics Results: do not hide query bar and table on query syntax error (#69196)

* do not hide query bar and table if query syntax error

* check for query error in analytics exploration

[Ingest Manager] Use search param vs KQL for events count (#69209)

* Replace filter with search in events count

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Ingest Manager] Use search param vs KQL to find default agent (#69211)

* Remove another instance of filter for is_default

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[SIEM][Detection Engine] Fixes a stale view/query within the signals table when re-opening/closing signals

Fixes a one liner bug where we were not using waitFor and could have stale views of data from the timeline. This removes that by adding a `waitFor` when setting the signals to be closed, open, in-progress, etc...

This also adds a new `waitFor` for the end to end tests and fixes other tests to use that rather than doing a wait for 5 seconds. This should keep the end to end backend tests fast.

Before this you could sometimes try to re-open 3 signals like below and it would not change the signals to being open but rather re-query the stale view and show the same signals as being closed when they are not closed:
![crop_bug](https://user-images.githubusercontent.com/1151048/84713572-8b5e7f00-af28-11ea-80b6-2ad67f16b7df.gif)

Now with this PR, those signals will show as being re-opened and the closed signals will update correctly within the same UI on the same tab.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

[Observability] Update landing page copy and content (#69247)

* removing try it section

* removing try it section

feat: instrument navigation changes using RUM agent (#67633)

* feat: instrument navigation changes using RUM agent

* chore: rebase and change application contract

* chore: fix type tests

* docs: update public.md doc

* chore: remove internal application export

[Uptime] Monitor availability reporting (#67790)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[DOCS] Adds classification to data frame analytics overview (#69068)

Bump react-redux (#69182)

[kbn/es] only make one attempt in tests to avoid timeout (#69197)

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Resolve security cloud test failures (#68935)

Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

SIEM] Moves validation up to the common section

Moves validation up to the common section so it can be used by others in common for API boundary validation.

[QA] Code coverage: fix flaky tests (#69272)

* skip test

* [page_objects/common_page] update closing toast

use navigateToUrl instead of window location (#69167)

[kbn/optimizer] fix windows compatibility (#69304)

Co-authored-by: spalger <spalger@users.noreply.github.com>

[Search service] Refactor the way server-side search strategies are registered (#68452)

* [search] Refactor the way search strategies are registered/retrieved on the server

* Fix types and tests and update docs

* Fix failing test

* Fix build of example plugin

* Fix functional test

* Make server strategies sync

Co-authored-by: Liza K <liza.katz@elastic.co>

[ci/getCheckoutInfo] retry fetching upstream changes to calculate mergeBase (#69320)

Co-authored-by: spalger <spalger@users.noreply.github.com>

[QA] [Code Coverage] Doc update (#69204)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[CI] Fix packer cache git branch reference value (#69207)

[kbn/pm] only count cached project (#69113)

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Reporting] Prepare export type definitions for Task Manager (#65213)

[APM] Fix service maps not loading when there are no APM ML jobs (#69240)

* Closes #69238 by handling 404 thrown error from the query for APM ML jobs.

* Improved coded readability

* moved anomaly job fetch to new function getApmAnomalyDetectionJobs for
improved readability and only handle a 404 status code response else throw

[BundleRefPlugin] resolve imports to files too (#69241)

Co-authored-by: spalger <spalger@users.noreply.github.com>

[DOCS] Fixes license management links (#69347)

[Component template] Details flyout (#68732)

[SIEM] Adds example unit test to convert KQL using a nested query

Adds example unit test to convert KQL using a nested query

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

Explore underlying data (#68496)

* feat: 🎸 stub discover_enhanced plugin

* feat: 🎸 improve view in discover action

* feat: 🎸 add URL generator to "View in Discover" action

* feat: 🎸 implement navigation and getHref in view raw logs actio

* fix: 🐛 disable action in "edit" mode

* refactor: 💡 renamce context menu view in discover action

* feat: 🎸 rename action to "explore data"

* fix: 🐛 correctly generate action path

* feat: 🎸 add internationalization to "explore action"

* fix: 🐛 correctly parse generated Discover URL path

* test: 💍 setup basic functional tests

* refactor: 💡 modularize url generation logic

* feat: 🎸 export CommonlyUsed type

* test: 💍 add test subjects to panel custom time range modal

* test: 💍 add index patterna and time range functional tests

* refactor: 💡 rename action file

* refactor: 💡 use URL generator from Discover plugin's contract

* test: 💍 add "Explore raw data" action unit tests

* fix: 🐛 import share plugin to check if it is enabled

* Update x-pack/plugins/discover_enhanced/public/actions/view_in_discover/explore_data_context_menu_action.ts

Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>

* chore: 🤖 add discover_enhanced to KibanaApp codeowners

* test: 💍 improve "Explore underlying data" functional tests

* test: 💍 improve <a> link assertion

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>

[APM] Only add decimals for numbers below 10 (#69334)

Replaces the Custom Color Picker on TSVB with the EuiColorPicker (#68888)

* Replace the Custom Color Picker on TSVB with the EuiColorPicker

* Remove the custom picker sass

* Remove private modules of eui and the custom color swatches

* Clear the color

* changes in test implementation

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[SIEM][Timeline] Persist timeline to localStorage (#67156)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Xavier Mouligneau <189600+XavierM@users.noreply.github.com>

feat: 🎸 don't show drilldown action in "edit" mode (#69371)

[ML] Functional tests - add more recognize and setup module API tests (#69251)

This PR adds a couple recognize module and setup module API integration tests.

[ML] Functional tests - Reduce DFA job model memory (#69295)

With the improvements in the model memory estimation for DFA jobs, the required memory limit could be reduced, which allows execution in a low-memory environment (like the 1GB ML node in a cloud trial).

Update dependency @elastic/charts to v19.5.2 (#69126)

Update @elastic/charts to 19.5.2

Co-authored-by: nickofthyme <nick.ryan.partridge@gmail.com>

[SIEM][Timeline] Minor timeline improvement (#69386)

[DOCS] Updates titles in Maps docs (#68703)

* [DOCS] Updates titles in Maps docs

* [DOCS] Changes Elastic Maps to Maps to match the UI

* [DOCS] Fixes titles

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Ingest Manager][Endpoint] Add Endpoint Create Policy flow with Ingest (#68955)

* Ingest: add data-test-subj prop support to Ingest components
* Ingest: Add Context Provider to bridge History from Kibana to Hash router
* Ingest: Added support for route state in Create Datasource page
* Endpoint: Add Create button to Polices List header
* Endpoint: Added support for passing of state from endpoint to ingest on policy
* Endpoint: additional functional test cases

[APM] Replace ML index queries with searching via mlAnomalySearch API (#69099)

* Closes #69092 by replacing direct queries on ml indices with seaching
via the `mlAnomalySearch` client API + job_id filters. Also removes
`getMlIndex` since it is no longer relevant.

* Use the mlCapabilities API to ensure the required license is active for ml queries

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Drilldown docs 2 (#69375)

* docs: ✏️ add drilldown user docs link to REAMDE

* feat: 🎸 wire drilldown docs link into <HelloBar> component

* feat: 🎸 set new page attributes on drilldown docs link

* feat: 🎸 add external prop to link to show icon

* fix: 🐛 remove rel= attribute

* fix: 🐛 add TypeScript type for drilldown docLink

[Endpoint] add policy data to Host list UI (#69202)

[Ingest Pipelines] Add doc links to processor types (#69279)

* added doc links to processor types

* Bring doc links in line with mappings editor

Also refactor the processors type map

* remove helpText prop from Custom field

* fix i18n

* rename doc button and refactor type map const name

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Removing flaky axe rule (#69418)

[Ingest Manager] Replace `datasources` with `inputs` when generating agent config (#69226)

* Adjust agent config generation, change `datasources` to `inputs`

* Add dataset.type

* Remove dead code

* Revert "Add dataset.type"

This reverts commit fbcf50cbe2b6a93536386c82d754a6a7c7f2b8e9.

* Update endpoint policy test assertion

[ML] Add Anomaly Swimlane Embeddable to the dashboard from the Anomaly Explorer page (#68784)

* [ML] WIP attach swimlane embeddable to dashboard from the explorer page

* [ML] fix deps

* [ML] getDefaultPanelTitle

* [ML] fix TS issue

* [ML] DashboardService

* [ML] unit tests

* [ML] redirect to the dashboard

* [ML] swimlane_panel

* [ML] Anomaly Timeline panel

* [ML] swimlane container

* [ML] fix ts

* [ML] Add multiple swimlanes

* [ML] fix SwimlaneType usage

* [ML] disable edit button on update

* [ML] fix i18n translation key

* [ML] use ViewMode enum

* [ML] use navigateToUrl

* [ML] TODO for edit dashboard

* [ML] check kibana dashboard capabilities

* [ML] mlApiServicesProvider

* [ML] mlResultsServiceProvider

* [ML] fix alignment

* [ML] labels and tooltips

* [ML] fix ts issue for proxyHttpStart

* [ML] canEditDashboards check

* [ML] fix TS

* [ML] update add_to_dashboard_control.tsx

* [ML] add form label, disable control on empty swimlanes selection

* [ML] resolve PR review comments

* [ML] e2e test

* [ML] increase panel padding

* [ML] position in row

* [ML] update e2e

* [ML] add data-test-subj for search box

* [ML] PR remarks

[docker] add spaces settings (#69019)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Maps] layer wizard select re-design (#69313)

* [Maps] layer wizard select re-design

* review feedback

* tslint

* add unit test

* use smaller gutters

* review feedback

skip flaky suite (#52854)

[APM] License feature tracking for service maps (#69455)

Use the license feature API to register the service maps feature and track its usage when the API endpoint is accessed.

Fixes #64850.

fixes screenshots upload (#69392)

Remove unused deps (#69243)

* remove unused deps

* move types in devDeps

[Observability] Creates "Add data" links in all Observability app headers (#69016)

* Add data options to all Observability app headers

* Updated failing snapshot

* [Uptime] Update snapshot

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Beats Management: Pre-migration cleanup (#69155)

* adapt KibanaDatabaseAdapter to use core ES service

* remove unused `exposeStaticDir` method

* create empty KP plugin

* remove unused and broken tests

* delete unused wallaby config file

* delete unused utils

* delete unused d.ts and move formsy definition to new plugin

* fix findNonExistentItems

* remove dead code and useless exports from common package

* delete non migratable test suite + remove unused test adapter

* remove yet another unused adapter

* restore awaits in KibanaDatabaseAdapter

Add section about marble testing to `TESTING.md` (#68749)

* Add section about marble testing

* improve `callServerAPI` example

* review comments

* add comment on abort observable anti-pattern

savedObjects: add `score` to repository.find results (#68894)

* add `score` to repository.find results

* update generated doc

* fix FTR result set

* remove score from exports

* fix FTR for find API

* fix label

* fix tsdoc

Add functional test for Kibana embedded in iframe (#68544)

* convert kbn test config into TS

* add test  for Kibana embedded in iframe

* run embedded tests in functional suite

* ignore tls errors in functional tests by default

* switch test to https

* remove env vars mutation

* allow to pass ssl config to Kibana

* pass ssl config to axios

* adopt KbnClient interfaces

* adopt KibanaServer

* use KbnRequester in security service

* set sameSiteCookies:None in test

* acceptInsecureCerts in chrome

* remove leftovers

* fix type error

* remove unnecessary field

* address comments

* refactor plugin

* refactor test

* make acceptInsecureCerts configurable

* run firefox tests on ci

* up TS version

* fix firefox.sh script

* fix path

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[APM] Add error rate chart to Errors overview and detail views (#67327)

* creating error rate chart

* adding error line chart

* creating error rate chart

* using date_histogram

* reapplying prettier style

* changing to theme color

* dont sync tooltips

* adding avg on error charts

* addressing pr comments

* adding possibility to disable legend toggle

* removing x-axis ticks from histogram

* return no percent when transaction count doesn return hits

* addressing PR comments

* addressing PR comments

* returning null when there is no transaction count

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[APM] Change the unsaved changes text color (#69493)

Fixes dark mode issue where the text would render dark on a dark background.

[refactor] remove drilldown plugin, move components to uiActionsEnhanced (#69403)

* refactor: remove drilldown plugin by moving components to uiActionsEnhanced

* fix

* fix mistake in test

* fix i18n

HttpService: duplicate some APIs from `setup` to `start` and clean mocks. (#69021)

* add apis to http start contract and clean mocks

* add apis to http start contract and clean mocks

* add `getStartContract` method

* fix new calls

* remove isTlsEnabled

* deprecates HttpServiceSetup.auth

[SIEM] Fixes REST formatter bugs from io-ts migration

Fixes io-ts formatter bugs for REST and validation by:

* First trying to get the correct key from the io-ts context. If no keys are found, then it will fall back on trying to get the first name from the context.
* If the key is a value and an object then this will do a `JSON.stringify()` on the value object
* This fixes a few places where `formatError` was not being used within the code base resulting in `[object Object]` within the validations to show up.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

[RUM Dashboard] Initial Version (#68778)

Co-authored-by: Casper Hübertz <casper@formgeist.com>

Fix home page loading if telemetry plugin disabled (#69394)

* Fix home page loading

* Fix jest test, update telemetry mocks

[Maps] Remove bounds param (#69396)

[IngestManager] fix endpoint setup in api integration tests (#69503)

[APM] Service map download in debug mode (#69350)

* [APM] Service map download in debug mode

Add a download button when debug mode is enabled that downloads JSON of
the map.

Add an upload button to the Storybook.

Embed documentation on input and output state (#69443)

* Embed documentation on input and output state

* json -> js

* Add section on how id is used by custom time range badge action to determine isCompatible

Revert "[RUM Dashboard] Initial Version (#68778)"

This reverts commit 72111702e95214594d5d56bc856e1d5c78e6d802.

[Ingest Manager] Fleet require encrypted saved object encryption key … (#69408)

[rfc][skip-ci][reporting] Rendering API RFC (#64372)

* WIP working on reporting rfc round 2

* First draft is complete

[Index template] Refactor index template wizard (#69037)

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>

Amend export used for default (#69158)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[SIEM][Exceptions] - Updates exception structure and corresponding UI types (#69120)

This PR is meant to update the `ExceptionListItemSchema.entries` structure to align with the most recent conversations regarding the need for a more explicit depiction of `nested` fields. To summarize:

- Adds schema validation for requests and responses within `lists/public/exceptions/api.ts`. It was super helpful in catching existing bugs. Anyone that uses the api will run through this validation. If the client tries to send up a malformed request, the request will not be made and an error returned. If the request is successful, but somehow the response is malformed, an error is returned. There may be some UX things to figure out about how to best communicate these errors to the user, or if surfacing the raw error is fine.
- Updates `entries` structure in lists plugin api
- Updates hooks and tests within `lists/public` that make reference to new structure
- Updates and adds unit tests for updated schemas
- Removes unused temporary types in `security_solution/public/common/components/exceptions/` to now reference updated schema
- Updates UI tests
- Updates `lists/server/scripts`

[Ingest Pipelines] Add test coverage for ingest pipelines editor component (#69283)

* first iteration of CIT tests

* address pr feedback

- use dot notation where we can
- use string literals instead of + concatentation

[Ingest Pipelines] Encode URI component pipeline names (#69489)

* Properly encode URI component pipeline names

* safely URI decode to handle % case

[Console] Added license headers to worker files (#69387)

* added license headers to worker files

* add missing @notice

* update notice.txt

* fix notice

* remove start-end license block markers

* Added license pre-amble

don't include group fields with no child fields in index pattern (#69457)

Update endpoint event and alert types (#69292)

* start redoing types

* finish redoing types

* fix bad test

* rework tests

* fix more types

* fix test

* Fix endpoints test and render error

* add deletePolicyStream to alerts api tests

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Paul Tavares <paul.tavares@elastic.co>

[Security Solution] Add cypress tests for global search bar (#68535)

[RUM Dashboard] Initial version resubmit (#69531)

Co-authored-by: Casper Hübertz <casper@formgeist.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Ingest Manager] Use long polling for agent checkin (#68922)

Fixes home page welcome link (#69539)

[Maps] Migrate maps client router to react (#65079)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[CCR] Fix follower indices table not updating after pausing (#69228)

Document authentication settings. (#69284)

Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>

[SIEM][Detection Engine] Fixes 7.8 and 7.9 upgrade issue within rules where you can get the error "params invalid: [lists]: definition for this key is missing"

* https://github.com/elastic/kibana/issues/69463
* See here for manual backport to 7.8: https://github.com/elastic/kibana/pull/69434

This fixes a bug where if you import rules and set your overwrite to `true` multiple times in a row within 7.7 you can end up with a lists array. When upgrading to 7.8, we change the name of `lists` to `exceptions_lists` and suddenly when you enable/disable a rule you can get the following error below:

![image](https://user-images.githubusercontent.com/1151048/84945824-fa60e280-b0a4-11ea-8e05-bffdec2e4765.png)

The fix is to allow the lists array still if it is present within saved objects to avoid seeing this error screen and being tolerant. We also fix the area of code that is causing the data bug so it cannot happen again with `exceptions_list` which is what the name of lists was renamed to causing this problem.

Note that this has unit tests and I also manually tested this by intentionally injecting a `lists` and `exceptions_lists` and using the UI to verify there wasn't another validation spot that needed to be relaxed to allow for the data.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

[DOCS] Adds kibana-pull attribute for release docs (#69554)

skip tests using hostDetailsPolicyResponseActionBadge

Merge/restyle nodes table (#69098)

Adds panel views and drilldowns to Resolver

[ftr] add support for docker servers (#68173)

Co-authored-by: spalger <spalger@users.noreply.github.com>

Remove endpoint alert fields from signal mapping (#68934)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[DOCS] Add related link to the ingest management docs (#69467)

* [DOCS] Add related link to the ingest management docs

* Add link to ingest manager topic in Kibana

* Remove link to ingest manager topic in kibana

Resolving conflicts (#69597)

[ML] Data Grid Histograms (#68359)

Adds support for histogram charts to data grid columns.
- Adds a toggle button to the data grid's header to enabled/disable column charts.
- When enabled, the charts get rendered as part of the data grid header.
- Histogram charts will get rendered for fields based on date, number, string and boolean.

[APM] Add support for dark mode (#69362)

* Use theme colors for stacktrace

* [APM] Use theme for all components

* Switch HoC withTheme to useTheme and convert classes to hooks

* Change hardcoded white to euiColorEmptyShade for ServiceMap

* Snapshots and Legends fix

* Switch to context and add test helper

* Fix tests and update snaps

* New snaps + new anomaly detection

* Remove shallow from testHelpers

* Remove commented tests

* Fix prettier

* Pass correct theme to cytoscape

* Fix ServiceMap

* fixes some rendering issues in service maps

* removed the old anomaly detection logic from service map popover contents, since it's been moved to a new component.

* Fix eslint, tsc lint issues and unit tests

* Remove types for styled-components default theme

* Update x-pack/plugins/apm/public/components/shared/KueryBar/Typeahead/Suggestions.js

Co-authored-by: Casper Hübertz <casper@formgeist.com>

* fix OuterTheme

* Ise function declaration instead of expression

Co-authored-by: Balthazar Gronon <git@balthazar.dev>
Co-authored-by: Balthazar Gronon <bgronon@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Oliver Gupte <olivergupte@gmail.com>
Co-authored-by: Casper Hübertz <casper@formgeist.com>

[Endpoint] add policy empty state (#69449)

Fixing resolver alert generation (#69587)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Security_Solution][Endpoint] Resolver leverage ancestry array for queries  (#69264)

* Adding alerts route

* Adding related alerts generator changes, tests, and script updates

* Fixing missed parameter

* Aligning the AlertEvent and ResolverEvent definition

* Fixing type errors

* Fixing import error

* Adding ancestry functionality in generator

* Creating some tests for ancestry field

* Making progress on the ancestry

* Fixing the ancestry verification

* Fixing existing tests

* Removing unused code and fixing test

* Adding more comments

* Fixing endgame queries

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

skip failing suite (#69595)

[Endpoint] Fix flaky endpoints list unit test (#69591)

* Fix flaky endpoints list unit test
* un-skip test

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

disable pageLoadMetrics job, it's gotten really flaky

[SECURITY] Introduce kibana nav (#68862)

* Change the bootstrap of the app

* rename SiemPageName to SecurityPageName

* modify alerts routes

* modify cases routes

* modify hosts routes

* modify network routes

* modify overview routes

* modify timelines routes

* wip change management route

* change route for common

* some fixing from the first commit

* modify route for management

* update url format hook to use history

* bug when you click on external alerts from host or network

* improvement from josh feedback

* redirect siem to security solution

* a little clean up

* Fix types

* fix breadcrumbs

* fix unit test

* Update index.tsx

* Fix cypress

* bug remove timeline when you are in case configure

* Fix functionel test for management

* Fix redirect siem + ml

* fixes some cypress tests

* adds 'URL compatibility' test

* bring ml back to alerts

* review I

* Fix memory leak in timelines page

* fix storage bug for timeline search bar

* fix endpoint merge + functional test

* avoid timeline flyout toggle

* Fix link to ml score

* Fix breadcrumb

* Fix management url

* fix unit test

* fixes typecheck issue

* fixes remaining url cypress tests

* fixes timeline scenario

* fix link to details rule from timeline

* review remove absolute path for consistency

* Fixing resolver alert generation (#69587)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* [Security_Solution][Endpoint] Resolver leverage ancestry array for queries  (#69264)

* Adding alerts route

* Adding related alerts generator changes, tests, and script updates

* Fixing missed parameter

* Aligning the AlertEvent and ResolverEvent definition

* Fixing type errors

* Fixing import error

* Adding ancestry functionality in generator

* Creating some tests for ancestry field

* Making progress on the ancestry

* Fixing the ancestry verification

* Fixing existing tests

* Removing unused code and fixing test

* Adding more comments

* Fixing endgame queries

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* fix cypress test

* skip failing suite (#69595)

* [Endpoint] Fix flaky endpoints list unit test (#69591)

* Fix flaky endpoints list unit test
* un-skip test

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* remove flaky test

Co-authored-by: patrykkopycinski <contact@patrykkopycinski.com>
Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>

Bump jest related packages (#58095)

[IM] Move template wizard steps to shared directory + update legacy details panel (#69559)

Migrate dashboard mode (#69305)

Closes: #67469

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Observability]Adding specific Return APIs for each plugin (#69257)

* Adding specific apis for each plugin

* adding metric hosts stat

* addressing PR comment

* addressing PR comments

* changing series to key/value

* exporting interfaces

* adding label to stat

* refactoring types

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[ML] fix ml api services (#69681)

[Discover] Unskip context navigation functional test (#68771)

[APM] Use asPercent to format breakdown chart (#69384)

Co-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Updates swim lane UI text (#69454)

Fix Advanced Settings Panel number editing in Graph (#69672)

Remove App communication from URL (#67064)

Removed all inter-app communication via url in favour of a new service in the embeddable start contract called the state transfer service.

[Security_Solution] Split up indices (#69589)

* Fixing resolver alert generation

* Splitting indices up

* Removing tests that could randomly fail because of the generation code

* Adding support for multiple indices

* Updating archives with the new index names

* Removing alerts data stream

* Switching to process instead of fake

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

completes top-level navigation tests (#69694)

[CI] Record Github commit statuses outside of PRs (#69432)

Document workaround for using HashRouter (#69140)

* add section on hashrouter

* review comments

Disabled multiple select for preconfigured connectors to avoid requesting bulk delete on them (#69459)

[chore] TS 3.9: convert ts-ignore to ts-expect-error (#69541)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Monitoring] Love for APM (#69052)

* Fix broken colors for APM

* Use default derivative

* Fix UI issues with APM

* Add new charts

* Fix tests

* Use EUI color palette

* Remove old translations

* PR feedback

* Fix tests

* Fix up overview page

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[ML] Transform: Table enhancements (#69307)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[ML] Transform: Enable force delete if one of the transforms failed (#69472)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

fix link to analytics results from management (#69550)

adds kibana navigation tests (#69733)

Add plugin API for customizing the logging configuration (#68704)

[Maps] Remove extra layer of telemetry nesting under "attributes" (#66137)

* Return attributes when telemetry created instead of whole saved object. Update integration test

* Change 'maps-telemetry' to 'maps'

* No need to create a saved object anymore. This is leftover from task manager telemetry mgmt

* Add test confirming attrs undefined. Change tests to check for 'maps' iso 'maps-telemetry'

* Add two more tests confirming expected telemetry shape

* Review feedback. Use TELEMETRY_TYPE constant and set to APP_ID

[Index Management] Fix API Integration Test and use of `timestamp_field` (#69666)

* fix types and functional api integration test

* access timestamp field name in object

* temporarily skip the API integration test and fix ts issue

[SECURITY] Add endpoint alerts url (#69707)

* Add back endpoint alerts url

* hack to move on

* fix type

* fix test

remove scroll in drag & drop context (#69710)

[Metrics UI] Add inventory alert preview (#68909)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Obs] Update Observability landing page text (#69727)

Support deep links inside of `RelayState` for SAML IdP initiated login. (#69401)

[ML] Fixes anomaly chart and validation for one week bucket span (#69671)

* [ML] Fixes anomaly chart and validation for one week bucket span

* [ML] Fix interval Jest tests

Migrate legacy import/export endpoints (#69474)

* migrate legacy export routes to `legacy_export` plugin

* adapt unit tests

* remove already dead (already moved) libs

Fixes #69344: Don't allow empty string for server.basePath config (#69377)

* Fixes #69344: Don't allow empty string for server.basePath config

* Remove unused basepath group

Redirect to Logged Out UI on SAML Logout Response. Prefer Login Selector UI to Logged Out UI whenever possible. (#69676)

Properly redirect legacy URLs (#68284)

expose DocLinks API from start only (#68745)

* exose DocLinks API from start only

* update docs

* fix type errors

* update docs

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

remove top-nav. it is not rendered in KP (#69488)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[APM] Storybook theme fixes (#69730)

* [APM] Storybook theme fixes

The changes adding theme support in #69362 broke some of the Storybook stories.

Add decorators to wrap some of the stories in the theme context.

This should be done in a global decorator, but our current storybook setup doesn't support this. It also would be nice to be able to switch between light/dark mode, but that's something we can add in the future.

* Remove unused import

* Adds missing decorator to cytoscape examples + adds a new real-world example

Co-authored-by: Oliver Gupte <olivergupte@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

adding Stats interface with type (#69784)

[data.search.aggs]: Add AggConfig.toSerializedFieldFormat (#69114)

[Ingest Manager] Do not await in start. Return a Promise (#69505)

  1. Do not `await` in the public `start` lifecycle method. Fixes https://github.com/elastic/kibana/issues/66125
PR based on https://github.com/elastic/kibana/issues/66125#issuecomment-640790837 & https://github.com/elastic/kibana/issues/66125#issuecomment-642218799
  2. Change `success` to be Promise

[pre-req] Convert Palettes and Components (#69065)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Lens] Stabilize filter popover (#69519)

* stabilize filter popovwer

* remove text exclusion

Add lists plugin to optimized security_solution TS config (#69705)

As security_solution continues to integrate with lists, the absents of
these types will lead to lots of implicit anys and false positives.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[data.search.aggs] Move agg-specific field formats to search service (#69586)

[Security][Network] Exclude glob-only (*) Index Pattern from map layers (#69736)

* Exclude glob-only (*) index pattern from map layers

This pattern is a special case that our map should ignore, as including
it causes all indexes to be queried.

* Ignore CCS glob pattern in our embedded map

Users may have this pattern for cross-cluster search, and it should
similarly be excluded when matching Security indexes.

[IM] Move common step containers to shared (#69713)

[Maps] New mappings: maps-telemetry -> maps (#69816)

Don't set a min-length on encryption key for reportin (#69827)

[ML] DF Analytics Creation: add progress indicator (#69583)

* add progress indicator to creation wizard page

* only show progress bar if job is started immediately

* add title and switch to timeout

* fix progress check

* clean up interval on unmount

* fix types

* clear interval if stats undefined. show progress if job created

[Uptime] Fix charts dark theme (#69748)

[Ingest Manager] Support registration of server side callbacks for Create Datasource API (#69428)

* Ingest: Expose `registerExternalCallback()` method out of Ingest server `start` lifecycle
* Ingest: Add support for External Callbacks on REST `createDatasourceHandler()`
* Ingest: expose DatasourceServices to Plugin start interface
* Endpoint: Added Endpoint Ingest handler for Create Datasources
  - Also moved the temporary logic from the middleware
    to the handler (still temporary)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[IngestManager] Expose agent authentication using access key (#69650)

* [IngestManager] Expose agent authentication using access key

* Add unit tests to authenticateAgentWithAccessToken service

[APM] Pulls legacy ML code from service maps and integrations (#69779)

* Pulls out existing ML integration from the service maps

* - removes ML job creation flyout in integrations menu on the service details UI
- removes ML searches and transforms in the transaction charts API
- removes unused shared functions and types related to the legacy ML integration

* removes unused translations for APM anomaly detection

* Adds tags to TODOs for easy searching later

Convert Positionable, RenderToDom and RenderWithFn to functional/hooks/no recompose. (#68202)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Index Management Index and index templates UI release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Mappings editor] Template wizard steps not disabled with invalid dynamic template data
4 participants