[Lens] Allow a single operation to create multiple user-facing operations

[wylieconlon]

This is an issue describing a current limitation of the Lens architecture and outlining some approaches to fixing it.

Example use case

There are some types of functions that create key/value pairs, or that produce multiple columns of output instead of multiple rows. For example:

• Percentiles aggregation
• Proposed cumulative distribution function
• Proposed scatterplot function

As a user, I would expect that I could follow the following user flow:

1. On the X axis of my chart, I can choose the "Percentiles" function on a field like bytes. This will set the X axis to the key, for example 99%.
2. The Y axis is updated automatically with the value, which in this case is the value of the bytes field.
3. The X and Y configurations are linked: if I change one, they both change. If I delete one, they are both deleted.

The end result of this configuration is that I only created the X axis manually, and the Y axis was assigned automatically.

Why is this currently not supported?

Because datasources can't unilaterally generate new columns and assign them to visual dimensions. The closest behavior that we have is when the user drops a field into the Lens editor, and then a suggestion is generated and applied.

Datasources can unilaterally generate columns, but because the IDs aren't known by the visualization, it won't be editable by the user. So the only requirements here are:

1. The datasource must be able to generate IDs in a way that visualizations can use internally
2. The datasource must be able to remove IDs that the visualization was previously using

Implementation plan

Because the closest behavior is happening with suggestions, I propose that we create a new function that allows the Lens datasource to "push" suggestions, as opposed to having suggestions "pulled" from the datasource. Suggestions will be allowed to be pushed from the Dimension Editor, which is the component in the UI that manages operations. In this case, the expected flow is:

1. The user starts configuring a more complicated function that has multiple output columns
2. The editor component will save the state of the datasource
3. The editor component will push the suggestion

Essentially, this changes the interface of the dimension editor by adding a new pushSuggestion function:

export type DatasourceDimensionEditorProps<T = unknown> = DatasourceDimensionProps<T> & {
  setState: StateSetter<T>;
  pushSuggestion: (suggestion: DatasourceSuggestion<T>) => boolean;
  core: Pick<CoreSetup, 'http' | 'notifications' | 'uiSettings'>;
  dateRange: DateRange;
}

The suggestion will be taken by the editor frame, and gets applied to the state.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[markov00]

In order to provide better transparency of priorities, issues that will not be prioritized within the next 24 months are being closed.

Tracking request in Lens general improvements ice box #184459