Deprecating kibana_user and kibana_dashboard_only_user roles

This change adds a new `kibana_admin` role, and deprecates
the old `kibana_user` and`kibana_dashboard_only_user`roles.

The deprecation is implemented via a new reserved metadata
attribute, which can be consumed from the API and also triggers
deprecation logging when used (by a user authenticating to
Elasticsearch).

Some docs have been updated to avoid references to these
deprecated roles.

Backport of: elastic#46456

Co-authored-by: Tim Vernum <tim@adjective.org>
Co-authored-by: Larry Gregory <legrego@users.noreply.github.com>

client/rest-high-level/src/test/java/org/elasticsearch/client/documentation/SecurityDocumentationIT.java

@@ -693,8 +693,8 @@ public void testGetRoles() throws Exception {
 
             List<Role> roles = response.getRoles();
             assertNotNull(response);
-            // 28 system roles plus the three we created
-            assertThat(roles.size(), equalTo(28 + 3));
+            // 29 system roles plus the three we created
+            assertThat(roles.size(), equalTo(29 + 3));
         }
 
         {

docs/reference/monitoring/configuring-filebeat.asciidoc

@@ -117,7 +117,7 @@ If {security-features} are enabled, you must provide a valid user ID and
 password so that {filebeat} can connect to {kib}: 
 
 .. Create a user on the monitoring cluster that has the 
-<<built-in-roles,`kibana_user` built-in role>> or equivalent
+<<built-in-roles,`kibana_admin` built-in role>> or equivalent
 privileges.
 
 .. Add the `username` and `password` settings to the {es} output information in 

x-pack/docs/en/security/authentication/oidc-guide.asciidoc

@@ -418,21 +418,25 @@ through either the
 NOTE: You cannot use <<mapping-roles-file,role mapping files>>
 to grant roles to users authenticating via OpenID Connect.
 
-This is an example of a simple role mapping that grants the `kibana_user` role
+This is an example of a simple role mapping that grants the `example_role` role
 to any user who authenticates against the `oidc1` OpenID Connect realm:
 
 [source,console]
 --------------------------------------------------
-PUT /_security/role_mapping/oidc-kibana
+PUT /_security/role_mapping/oidc-example
 {
-  "roles": [ "kibana_user" ],
+  "roles": [ "example_role" ], <1>
   "enabled": true,
   "rules": {
     "field": { "realm.name": "oidc1" }
   }
 }
 --------------------------------------------------
 
+<1> The `example_role` role is *not* a builtin Elasticsearch role.
+This example assumes that you have created a custom role of your own, with
+appropriate access to your <<roles-indices-priv,indices>> and
+{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features].
 
 The user properties that are mapped via the realm configuration are used to process
 role mapping rules, and these rules determine which roles a user is granted.

x-pack/docs/en/security/authentication/saml-guide.asciidoc

@@ -631,21 +631,25 @@ through either the
 NOTE: You cannot use <<mapping-roles-file,role mapping files>>
 to grant roles to users authenticating via SAML.
 
-This is an example of a simple role mapping that grants the `kibana_user` role
+This is an example of a simple role mapping that grants the `example_role` role
 to any user who authenticates against the `saml1` realm:
 
 [source,console]
 --------------------------------------------------
-PUT /_security/role_mapping/saml-kibana
+PUT /_security/role_mapping/saml-example
 {
-  "roles": [ "kibana_user" ],
+  "roles": [ "example_role" ], <1>
   "enabled": true,
   "rules": {
     "field": { "realm.name": "saml1" }
   }
 }
 --------------------------------------------------
 
+<1> The `example_role` role is *not* a builtin Elasticsearch role.
+This example assumes that you have created a custom role of your own, with
+appropriate access to your <<roles-indices-priv,indices>> and
+{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features].
 
 The attributes that are mapped via the realm configuration are used to process
 role mapping rules, and these rules determine which roles a user is granted.

x-pack/docs/en/security/authorization/built-in-roles.asciidoc

@@ -72,10 +72,12 @@ NOTE: This role does *not* provide the ability to create indices; those privileg
 must be defined in a separate role.
 
 [[built-in-roles-kibana-dashboard]] `kibana_dashboard_only_user` ::
-Grants access to the {kib} Dashboard and read-only permissions to Kibana.
-This role does not have access to editing tools in {kib}. For more
-information, see
-{kibana-ref}/xpack-dashboard-only-mode.html[{kib} Dashboard Only Mode].
+(This role is deprecated, please use 
+{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[{kib} feature privileges]
+instead).
+Grants read-only access to the {kib} Dashboard in every 
+{kibana-ref}/xpack-spaces.html[space in {kib}].
+This role does not have access to editing tools in {kib}. 
 
 [[built-in-roles-kibana-system]] `kibana_system` ::
 Grants access necessary for the {kib} system user to read from and write to the
@@ -87,9 +89,15 @@ see {kibana-ref}/using-kibana-with-security.html[Configuring Security in {kib}].
 NOTE: This role should not be assigned to users as the granted permissions may
 change between releases.
 
+[[built-in-roles-kibana-admin]] `kibana_admin`::
+Grants access to all features in {kib}. For more information on {kib} authorization,
+see {kibana-ref}/xpack-security-authorization.html[Kibana authorization].
+
 [[built-in-roles-kibana-user]] `kibana_user`::
-Grants access to all features in {kib}. For more information on Kibana authorization,
-see {kibana-ref}/xpack-security-authorization.html[Kibana Authorization].
+(This role is deprecated, please use the
+<<built-in-roles-kibana-admin,`kibana_admin`>> role instead.)
+Grants access to all features in {kib}. For more information on {kib} authorization,
+see {kibana-ref}/xpack-security-authorization.html[Kibana authorization].
 
 [[built-in-roles-logstash-admin]] `logstash_admin` ::
 Grants access to the `.logstash*` indices for managing configurations.
@@ -127,7 +135,8 @@ Grants the minimum privileges required for any user of {monitoring} other than t
 required to use {kib}. This role grants access to the monitoring indices and grants
 privileges necessary for reading basic cluster information. This role also includes
 all {kibana-ref}/kibana-privileges.html[Kibana privileges] for the {stack-monitor-features}.
-Monitoring users should also be assigned the `kibana_user` role.
+Monitoring users should also be assigned the `kibana_admin` role, or another role
+with {kibana-ref}/xpack-security-authorization.html[access to the {kib} instance].
 
 [[built-in-roles-remote-monitoring-agent]] `remote_monitoring_agent`::
 Grants the minimum privileges required to write data into the monitoring indices
@@ -140,9 +149,10 @@ Grants the minimum privileges required to collect monitoring data for the {stack
 [[built-in-roles-reporting-user]] `reporting_user`::
 Grants the specific privileges required for users of {reporting} other than those
 required to use {kib}. This role grants access to the reporting indices; each 
-user has access to only their own reports. Reporting users should also be 
-assigned the `kibana_user` role and a role that grants them access to the data 
-that will be used to generate reports.
+user has access to only their own reports.
+Reporting users should also be assigned additional roles that grant 
+{kibana-ref}/xpack-security-authorization.html[access to {kib}] as well as read
+access to the <<roles-indices-priv,indices>> that will be used to generate reports.
 
 [[built-in-roles-snapshot-user]] `snapshot_user`::
 Grants the necessary privileges to create snapshots of **all** the indices and

x-pack/docs/en/security/ccs-clients-integrations/cross-cluster-kibana.asciidoc

@@ -31,8 +31,9 @@ NOTE: If you configure the local cluster as another remote in {es}, the
 `logstash_reader` role on your local cluster also needs to grant the
 `read_cross_cluster` privilege.
 
-. Assign your {kib} users the `kibana_user` role and your `logstash_reader`
-role.
+. Assign your {kib} users a role that grants
+{kibana-ref}/xpack-security-authorization.html[access to {kib}]
+as well as your `logstash_reader` role.
 
 . On the remote cluster, create a `logstash_reader` role that grants the
 `read_cross_cluster` privilege and `read` and `view_index_metadata` privileges

x-pack/docs/en/security/get-started-security.asciidoc

@@ -167,15 +167,16 @@ Select a role to see more information about its privileges. For example, select
 the `kibana_system` role to see its list of cluster and index privileges. To
 learn more, see <<privileges-list-indices>>. 
 
-Let's assign the `kibana_user` role to your user. Go back to the 
-*Management / Security / Users* page and select your user. Add the `kibana_user` 
+Let's assign the `kibana_admin` role to your user. Go back to the 
+*Management / Security / Users* page and select your user. Add the `kibana_admin` 
 role and save the change. For example:
 
 [role="screenshot"]
 image::security/images/assign-role.jpg["Assigning a role to a user in Kibana"]
 
-This user now has access to all features in {kib}. For more information about granting
-access to Kibana see {kibana-ref}/xpack-security-authorization.html[Kibana Authorization].
+This user now has administrative access to all features in {kib}.
+For more information about granting access to Kibana see
+{kibana-ref}/xpack-security-authorization.html[Kibana authorization].
 
 If you completed all of the steps in 
 {stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], you should 

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStore.java

@@ -52,11 +52,9 @@ private static Map<String, RoleDescriptor> initializeReservedRoles() {
                 .put("superuser", SUPERUSER_ROLE_DESCRIPTOR)
                 .put("transport_client", new RoleDescriptor("transport_client", new String[] { "transport_client" }, null, null,
                         MetadataUtils.DEFAULT_RESERVED_METADATA))
-                .put("kibana_user", new RoleDescriptor("kibana_user", null, null, new RoleDescriptor.ApplicationResourcePrivileges[] {
-                        RoleDescriptor.ApplicationResourcePrivileges.builder()
-                            .application("kibana-.kibana").resources("*").privileges("all").build() },
-                        null, null,
-                        MetadataUtils.DEFAULT_RESERVED_METADATA, null))
+                .put("kibana_admin", kibanaAdminUser("kibana_admin", MetadataUtils.DEFAULT_RESERVED_METADATA))
+                .put("kibana_user", kibanaAdminUser("kibana_user",
+                    MetadataUtils.getDeprecatedReservedMetadata("Please use the [kibana_admin] role instead")))
                 .put("monitoring_user", new RoleDescriptor("monitoring_user",
                         new String[] { "cluster:monitor/main", "cluster:monitor/xpack/info", RemoteInfoAction.NAME },
                         new RoleDescriptor.IndicesPrivileges[] {
@@ -110,7 +108,7 @@ private static Map<String, RoleDescriptor> initializeReservedRoles() {
                             RoleDescriptor.ApplicationResourcePrivileges.builder()
                             .application("kibana-.kibana").resources("*").privileges("read").build() },
                         null, null,
-                        MetadataUtils.DEFAULT_RESERVED_METADATA,
+                        MetadataUtils.getDeprecatedReservedMetadata("Please use Kibana feature privileges instead"),
                         null))
                 .put(KibanaUser.ROLE_NAME, new RoleDescriptor(KibanaUser.ROLE_NAME,
                         new String[] {
@@ -266,6 +264,16 @@ private static Map<String, RoleDescriptor> initializeReservedRoles() {
                 .immutableMap();
     }
 
+    private static RoleDescriptor kibanaAdminUser(String name, Map<String, Object> metadata) {
+        return new RoleDescriptor(name, null, null,
+            new RoleDescriptor.ApplicationResourcePrivileges[] {
+                RoleDescriptor.ApplicationResourcePrivileges.builder()
+                    .application("kibana-.kibana")
+                    .resources("*").privileges("all")
+                    .build() },
+            null, null, metadata, null);
+    }
+
     public static boolean isReserved(String role) {
         return RESERVED_ROLES.containsKey(role) || UsernamesField.SYSTEM_ROLE.equals(role) || UsernamesField.XPACK_ROLE.equals(role);
     }

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/support/MetadataUtils.java

@@ -5,13 +5,17 @@
  */
 package org.elasticsearch.xpack.core.security.support;
 
+import org.elasticsearch.common.collect.MapBuilder;
+
 import java.util.Collections;
 import java.util.Map;
 
 public class MetadataUtils {
 
     public static final String RESERVED_PREFIX = "_";
     public static final String RESERVED_METADATA_KEY = RESERVED_PREFIX + "reserved";
+    public static final String DEPRECATED_METADATA_KEY = RESERVED_PREFIX + "deprecated";
+    public static final String DEPRECATED_REASON_METADATA_KEY = RESERVED_PREFIX + "deprecated_reason";
     public static final Map<String, Object> DEFAULT_RESERVED_METADATA = Collections.singletonMap(RESERVED_METADATA_KEY, true);
 
     private MetadataUtils() {
@@ -25,4 +29,12 @@ public static boolean containsReservedMetadata(Map<String, Object> metadata) {
         }
         return false;
     }
+
+    public static Map<String, Object> getDeprecatedReservedMetadata(String reason) {
+        return MapBuilder.<String, Object>newMapBuilder()
+            .put(RESERVED_METADATA_KEY, true)
+            .put(DEPRECATED_METADATA_KEY, true)
+            .put(DEPRECATED_REASON_METADATA_KEY, reason)
+            .immutableMap();
+    }
 }

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStoreTests.java

@@ -169,6 +169,7 @@
 
 import static org.hamcrest.Matchers.hasEntry;
 import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.not;
 import static org.mockito.Mockito.mock;
 
 /**
@@ -184,6 +185,7 @@ public void testIsReserved() {
         assertThat(ReservedRolesStore.isReserved("foobar"), is(false));
         assertThat(ReservedRolesStore.isReserved(SystemUser.ROLE_NAME), is(true));
         assertThat(ReservedRolesStore.isReserved("transport_client"), is(true));
+        assertThat(ReservedRolesStore.isReserved("kibana_admin"), is(true));
         assertThat(ReservedRolesStore.isReserved("kibana_user"), is(true));
         assertThat(ReservedRolesStore.isReserved("ingest_admin"), is(true));
         assertThat(ReservedRolesStore.isReserved("monitoring_user"), is(true));
@@ -409,13 +411,62 @@ public void testKibanaSystemRole() {
         assertNoAccessAllowed(kibanaRole, RestrictedIndicesNames.ASYNC_SEARCH_PREFIX + randomAlphaOfLengthBetween(0, 2));
     }
 
+    public void testKibanaAdminRole() {
+        final TransportRequest request = mock(TransportRequest.class);
+        final Authentication authentication = mock(Authentication.class);
+
+        RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("kibana_admin");
+        assertNotNull(roleDescriptor);
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+        assertThat(roleDescriptor.getMetadata(), not(hasEntry("_deprecated", true)));
+
+        Role kibanaAdminRole = Role.builder(roleDescriptor, null).build();
+        assertThat(kibanaAdminRole.cluster().check(ClusterHealthAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterStateAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterStatsAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(PutIndexTemplateAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterRerouteAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterUpdateSettingsAction.NAME, request, authentication),
+                is(false));
+        assertThat(kibanaAdminRole.cluster().check(MonitoringBulkAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(DelegatePkiAuthenticationAction.NAME, request, authentication),
+                is(false));
+
+        assertThat(kibanaAdminRole.runAs().check(randomAlphaOfLengthBetween(1, 12)), is(false));
+
+        assertThat(kibanaAdminRole.indices().allowedIndicesMatcher(IndexAction.NAME).test("foo"), is(false));
+        assertThat(kibanaAdminRole.indices().allowedIndicesMatcher(IndexAction.NAME).test(".reporting"), is(false));
+        assertThat(
+                kibanaAdminRole.indices().allowedIndicesMatcher("indices:foo").test(randomAlphaOfLengthBetween(8, 24)),
+                is(false));
+
+        final String randomApplication = "kibana-" + randomAlphaOfLengthBetween(8, 24);
+        assertThat(kibanaAdminRole.application().grants(new ApplicationPrivilege(randomApplication, "app-random", "all"),
+                "*"), is(false));
+
+        final String application = "kibana-.kibana";
+        assertThat(kibanaAdminRole.application().grants(new ApplicationPrivilege(application, "app-foo", "foo"), "*"),
+                is(false));
+        assertThat(kibanaAdminRole.application().grants(new ApplicationPrivilege(application, "app-all", "all"), "*"),
+                is(true));
+
+        final String applicationWithRandomIndex = "kibana-.kibana_" + randomAlphaOfLengthBetween(8, 24);
+        assertThat(
+                kibanaAdminRole.application()
+                        .grants(new ApplicationPrivilege(applicationWithRandomIndex, "app-random-index", "all"), "*"),
+                is(false));
+
+        assertNoAccessAllowed(kibanaAdminRole, RestrictedIndicesNames.RESTRICTED_NAMES);
+    }
+
     public void testKibanaUserRole() {
         final TransportRequest request = mock(TransportRequest.class);
         final Authentication authentication = mock(Authentication.class);
 
         RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("kibana_user");
         assertNotNull(roleDescriptor);
         assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_deprecated", true));
 
         Role kibanaUserRole = Role.builder(roleDescriptor, null).build();
         assertThat(kibanaUserRole.cluster().check(ClusterHealthAction.NAME, request, authentication), is(false));
@@ -745,6 +796,7 @@ public void testKibanaDashboardOnlyUserRole() {
         RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("kibana_dashboard_only_user");
         assertNotNull(roleDescriptor);
         assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_deprecated", true));
 
         Role dashboardsOnlyUserRole = Role.builder(roleDescriptor, null).build();
         assertThat(dashboardsOnlyUserRole.cluster().check(ClusterHealthAction.NAME, request, authentication), is(false));