[Lens] Enable dataView field formatters in Metric visualization

[dej611]

Summary

Fixes #154306

This PR fully integrates the field formatter into the new metric visualisation in Lens.
The logic implemented here to integrate all the formatters is the following:

• user's custom formats are preserved as they are most of the time
  • there's the exception of the currency format which is always manipulated to be more "compact"
  • duration is enriched with humanPrecision parameters, but without override
• in case of default numeric formats for regular numbers or percentages, a compact version is used instead

In case of transitioning from a visualization with custom parameters on the Lens formatting options to the new Metric, decimals and suffix parameters won't be carried on (functional test added).

Here's a summary table of a rendering of the value 10000 with the new system with various configurations:

Scenario	Lens config	Field format config	Advanced setting pattern	Rendering	Notes
No Formatting applied	Default	Default	Default		⚠️ K => k
Number configured in Lens	Number	Default	Default		⚠️ K => k
Percent configured in Lens	Percent	Default	Default		⚠️ M => m - see notes below
Bytes configured in Lens	Bytes	Default	Default		⚠️ X KB => XKB - space removed between value and unit
Bits configured in Lens	Bits	Default	Default		⚠️ X kbit => Xkbit - space removed between value and unit
Bytes configured in Field formats	Default	Bytes with explicit pattern	Default		⚠️ X KB => XKB - space removed between value and unit
Currency configured with custom format in Field formats	Default	Currency	($0,0.[000]a)		✅
Duration configured in Field formats	Default	Duration	Default		✅
Number with currency format in Field formats	Default	Number with format ($0,0.[000])	Default		✅ (before it was ignored)
Number with custom format in Field formats	Default	Number with format 0,0.[00]	Default		✅
Percent configured in Field formats	Default	Percent	Default		✅
Percent configured with custom format in Field formats	Default	Percent with custom format	Default		✅
Currency configured in Field formats	Default	Currency	Default		✅
Currency configured with different locale in Field formats	Default	Currency	Locale set to it-IT		✅
Currency configured with custom format and locale in Field formats	Default	Currency	($0,0.[000]a) and locale to it-IT		✅

Important notes for reviewers
Now that all the formatting has been migrated to the embedded numeral.js system, there are few important changes on the final rendering of the values:

• decimals in number representations are not always reported as most of the time the default format contains them as optionals
• The kilo suffix is always in lowercase, with the exception of the Bytes format.
  • Previously the K was used also for regular numbers - hence the change in some functional tests
  • I did some more research in this area beyond the previous PR and found out that the SI defines all units beyond 10^3 to be capitalized, so k is lowercase but m should be capitalized as M(French document, Tableau 7 page 33)
  • There's an issue in numeral.js about the millions and beyond cases, but apparently this is a language issue. The kibana fork can be adapted for this, but this has a completely different impact than this PR.

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/kibana-visualizations @elastic/kibana-visualizations-external (Team:Visualizations)

[stratoula]

@dej611 thanx a lot for working on this! I have some questions:

• I see that when I have everything in default we still round the numbers, there are cases though that I want the non-rounded value. In my mind the default should always work as the rest visualizations and if the user applies a Lens formatter then do the rounding etc

• With the same logic I was also expecting to see the suffix and decimals to the Lens number format

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: ec32abb

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
expressionMetricVis	62	61	-1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
lens	516	517	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
expressionMetricVis	34.9KB	34.4KB	-528.0B
lens	1.2MB	1.2MB	+310.0B
total			-218.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
lens	52	53	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
expressionMetricVis	14.1KB	14.0KB	-58.0B

Unknown metric groups

API count

id	before	after	diff
lens	612	614	+2

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	19	21	+2
expressionMetricVis	1	0	-1
securitySolution	399	403	+4
total			+5

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	20	22	+2
expressionMetricVis	1	0	-1
securitySolution	479	483	+4
total			+5

History

• 💚 Build #129487 succeeded a3dd0f8
• 💔 Build #129348 failed 19a6255
• 💔 Build #129334 failed ab9a9fe
• 💔 Build #129285 failed a3f9c47
• 💔 Build #129212 failed 9aeb0c2
• 💔 Build #128894 failed bb93c97

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[dej611]

I see that when I have everything in default we still round the numbers, there are cases though that I want the non-rounded value. In my mind the default should always work as the rest visualizations and if the user applies a Lens formatter then do the rounding etc

With the same logic I was also expecting to see the suffix and decimals to the Lens number format

I see what you mean. Right now this PR only adds the support for field formatters.
I can produce a followup PR with the additional controls, but unfortunately it's not possible to provide a unique "custom" Lens compact formatter as planned as the "compact"-ness should be applied to many formatters.

Maybe we could consider an additional Compact switch for all Lens custom formatters?

This would apply the compact format used in the new Metric also to other visualizations. cc @MichaelMarcialis

[stratoula]

I am personally ok with the switch, not sure if it makes sense for all visualizations but I dont think it hurts. The most important thing is that each option on the Value format has the same behavior in each visualization. Otherwise it will be very confusing for the users (it already is)

[MichaelMarcialis]

I see that when I have everything in default we still round the numbers, there are cases though that I want the non-rounded value. In my mind the default should always work as the rest visualizations and if the user applies a Lens formatter then do the rounding etc

With the same logic I was also expecting to see the suffix and decimals to the Lens number format

I see what you mean. Right now this PR only adds the support for field formatters. I can produce a followup PR with the additional controls, but unfortunately it's not possible to provide a unique "custom" Lens compact formatter as planned as the "compact"-ness should be applied to many formatters.

Maybe we could consider an additional Compact switch for all Lens custom formatters?

Just to make sure I'm understanding, am I correct with the following summary?

• The default formatter for the new metric visualization will show the full, unrounded value.
• To render an abbreviated version of that value (i.e. how the new metric visualization renders values today), we want to add a new "Compact format" switch.
• The reason we want to use this switch is because it's not possible to provide a custom formatter.

If that's correct, would you mind elaborating on why the custom formatter route isn't possible? The reason I ask is because I can imagine it will be asked by our users at some point. For example, what if I don't want Lens to automatically decide when to use kilobytes versus megabytes? What if I know I always want my value to be rendered as a megabyte? It would be cool to have a new "Custom" option in the "Value format" selector that exposed an input for numeral.js formatting. Apologies if I'm oversimplifying.

[dej611]

Just to make sure I'm understanding, am I correct with the following summary?

• The default formatter for the new metric visualization will show the full, unrounded value.
• To render an abbreviated version of that value (i.e. how the new metric visualization renders values today), we want to add a new "Compact format" switch.
• The reason we want to use this switch is because it's not possible to provide a custom formatter.

If that's correct, would you mind elaborating on why the custom formatter route isn't possible? The reason I ask is because I can imagine it will be asked by our users at some point. For example, what if I don't want Lens to automatically decide when to use kilobytes versus megabytes? What if I know I always want my value to be rendered as a megabyte? It would be cool to have a new "Custom" option in the "Value format" selector that exposed an input for numeral.js formatting. Apologies if I'm oversimplifying.

I think the unique custom format is a great idea but should require some additional work to provide some more context than just a numeral.js pattern field. Lens resisted so far without exposing an actual pattern-text like format waiting for something better than that.

What is possible right now would be to create a limited experience, like the one within the Number field formatter:

This can probably take us a bit further than the current situation, but not too much.
Things like currency or duration would still be limited. For instance, take the currency format where it's not just about a pattern, but also the locale context to render the final value string to be complete.
Let me build a PoC for that and we can discuss again.

[dej611]

Super-seeded by #158468