[ML] Data Frame Analytics: Enhancements for results view

[alvarezmelissa87]

Enhancements for DFA outlier detection results view:

Scatterplot matrix

• The scatterplot matrix charts should always show the full data sample and, when the user has added a filter/query in the query bar, the portion of the data reflecting the filter should be highlighted so it can be differentiated from the background data

  • It would be very useful if the points drawn comprised a background search and a foreground search. The foreground search should retrieve all points which satisfy the search condition and also the outlier score range filter and the background search, which is for showing typical data, and should not apply any filters. As it stands, there is no guarantee that outliers are plotted at all if they are not sampled and also filters distort the background set so present a misleading picture about what is outlying.
  • the consistency will be that whatever is unselected (either not part of the brush-selection or filter) turns gray. If there's both a filter and a brush-selection just the intersection of both will have colors.
  • [ML] Data Frame Analytics: Highlight filtered data in scatterplot charts  #144871 (8.6)

• Selecting/highlighting an area in one scatterplot chart should highlight the corresponding area in the rest of the charts. For reference, a Vega scatterplot matrix example can be found here. Note that the example is v5 and the scatterplot matrix we create is v4 so we need to find out if we can apply that approach of brushing to our version. The code that generates the Vega scatterplot code uses getScatterplotMatrixVegaLiteSpec() and lives here.

  • [ML] Data Frame Analytics: Add brushing to link docs in scatterplot charts #144354 (8.6)

• For scatterplot selection, show an option to apply selection filter to entire page (can add to the url and then it will be reflected in the table as well) - this will ensure it's reflected in the table as well.

• Add ability to drill down from a value in a column of the table (typically the field identifying the entity) to a visualization which is filtered to that value.

  • Add an Actions column to the table which consists of drilldown actions
    • View in discover filtering for particular column field/value
    • View in dashboard filtering for column field/value

• Add ability to add scatterplot matrix charts to a dashboard. Here is a python utility that creates the JSON structure for a raw Kibana saved object that includes the Vega chart as a Kibana visualization. Note however that it has been suggested to use a higher level API within Kibana to create a visualization (https://www.elastic.co/guide/en/kibana/8.2/saved-objects-api-create.html#saved-objects-api-create-example)

• Add ability to link to custom visualizations (vega) page and view scatterplot charts there ***

  • [ML] Data Frame Analytics results view: add link to custom visualizations for viewing scatterplot charts #149647

• Add control to change colors (accessibility issue)

• There is no cross probing between the small multiples and the table: if you select a point in the chart or table you want to see it highlighted in the table and the charts.

Histogram charts

• Link the rows in the results table to the distributions shown in the histogram charts above the table. So clicking / selecting a row in the table should add some sort of indicator / highlight to the histograms to show where the values for the row fall in the distributions.

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[alvarezmelissa87]

Closing this in lieu of individual issues opened for each item for 8.8