[Logs UI] Return 403s rather than 500s for ML privilege errors

[Kerry350]

Summary

This ticket fixes #70414. The changes here ensure that our routes that integrate with ML return a 403 status code, rather than 500, when there is a recognised ML privilege / capabilities based error.

Testing

Overview

⚠️ Please note that the changes here are based on the changes in #70069, as noted in the description there is a caveat in that: Note, due to limitations in alerting, the functions called by SIEM (jobsSummary and mlAnomalySearch) currently have their capabilities checks disabled., some of our ML integrated routes only call mlAnomalySearch, so those routes currently have these checks disabled. However, I have added the changes to those routes regardless, so they're ready when the alerting changes take effect.

Prerequisites

• You'll need a user who doesn't have ML app privileges.

Testing changes

There are two ways you could test this, I have used the latter method.

• You could disable the upfront privilege checks we do in the UI, and ensure the requests from that point fail as expected.

• Testing the API directly. Below are all of the routes that are affected, and some curl requests that should help. Where I've included not working it's because of the caveat noted in the summary. These routes should still be checked though, as they have changes.

(The UI should, of course, also be tested to ensure it still works as expected - with a user who does and does not have ML privileges).

/api/infra/log_analysis/results/log_entry_anomalies_datasets

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_anomalies_datasets" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default"
  }
}
'

/api/infra/log_analysis/results/log_entry_anomalies

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_anomalies" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default"
  }
}
'

/api/infra/log_analysis/results/log_entry_categories (not working yet)

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_categories" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "categoryCount": 5,
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default",
    "histograms":[{"bucketCount":10,"id":"history","timeRange":{"endTime":1596712908010,"startTime":1594293708008}},    {"bucketCount":1,"id":"reference","timeRange":{"endTime":1595503308009,"startTime":1594293708008}}]
  }
}
'

/api/infra/log_analysis/results/log_entry_category_datasets (not working yet)

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_category_datasets" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default"
  }
}
'

/api/infra/log_analysis/results/log_entry_category_examples

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_category_examples" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default",
    "categoryId": 1,
    "exampleCount": 5
  }
}
'

/api/infra/log_analysis/results/log_entry_examples

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_examples" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default",
    "dataset": "nginx.error",
    "exampleCount": 5
  }
}
'

/api/infra/log_analysis/results/log_entry_rate (not working)

curl -X POST "<KIBANA_URL>/api/infra/log_analysis/results/log_entry_rate" --user user_without_ml:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: professionally-crafted-string-of-text' -d'
{
  "data": {
    "timeRange":{"startTime":1595425114104,"endTime":1596634714104},
    "sourceId": "default",
    "bucketDuration": 15000
  }
}
'

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[Kerry350]

@elasticmachine merge upstream

[weltenwort]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 00929bd

Build metrics

✅ unchanged

History

• 💚 Build #66882 succeeded a09aa3b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[weltenwort]

👀 Reviewing on behalf of @elastic/logs-metrics-ui...

[weltenwort]

Seems to work as described 👍

Testing via the UI proved difficult because the jobs_summary requests fail earlier such that the routes changed here are never called.

[weltenwort]

Even though it's internally equivalent, did you consider using response.forbidden for the sake of discoverability?

Suggested change

	return response.customError({
	statusCode: 403,
	body: {
	message: error.message,
	},
	});
	return response.forbidden({
	body: {
	message: error.message,
	},
	});

[weltenwort]

And thank you for the detailed commentary and helpful testing instructions!

[Kerry350]

@weltenwort

Testing via the UI proved difficult because the jobs_summary requests fail earlier such that the routes changed here are never called.

Yeah, UI testing was a bit gnarly 😬

Thanks for the review. I'm going to merge this now as there's a green build; I will do a mini-followup with the response.forbidden changes as that would be nice for discoverability like you say.