Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove KMS requiring metadata files (closes #4375) #4700

Merged
merged 19 commits into from
Jan 2, 2024
Merged

Remove KMS requiring metadata files (closes #4375) #4700

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a6957bb
Remove KMS requiring metadata files (closes #4375)
keeganwitt Nov 28, 2023
58085c4
PR feedback changes
keeganwitt Dec 4, 2023
834212d
Add Azure key validation
keeganwitt Dec 8, 2023
a0bc1e5
Ensure server ID is always logged
keeganwitt Dec 12, 2023
023e717
Replace backticks with single quotes
keeganwitt Dec 14, 2023
0b8c4f3
Update doc/plugin_server_keymanager_aws_kms.md
keeganwitt Dec 15, 2023
0b79dd1
Update doc/plugin_server_keymanager_aws_kms.md
keeganwitt Dec 15, 2023
9be5ede
Update doc/plugin_server_keymanager_aws_kms.md
keeganwitt Dec 15, 2023
45b2cb1
Remove character check for Azure
keeganwitt Dec 15, 2023
fb061c0
Add GCP validation
keeganwitt Dec 16, 2023
e078c06
GCP validation PR suggestions
keeganwitt Dec 18, 2023
3ab2e5a
Fix unit test
keeganwitt Dec 19, 2023
d6f09c6
Fix unit test
keeganwitt Dec 19, 2023
e357dd9
Merge branch 'main' into remove_kms_metadata_file_dependencies
amartinezfayo Dec 26, 2023
7dffcfa
Replace deprecated key_metadata_file with key_identifier_file
keeganwitt Jan 1, 2024
8141cbf
Mark key_metadata_file as not required
keeganwitt Jan 1, 2024
09d54e4
Indicate that one of key_identifier_file and key_identifier_file are…
keeganwitt Jan 1, 2024
c500465
Log that key_metadata_file is deprecated
keeganwitt Jan 2, 2024
25e0c37
Merge branch 'main' into remove_kms_metadata_file_dependencies
amartinezfayo Jan 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions doc/plugin_server_keymanager_aws_kms.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,15 @@ The `aws_kms` key manager plugin leverages the AWS Key Management Service (KMS)

The plugin accepts the following configuration options:

| Key | Type | Required | Description | Default |
|-------------------|--------|---------------------------------------|-------------------------------------------------------------------------------|---------------------------------------------------------|
| access_key_id | string | see [AWS KMS Access](#aws-kms-access) | The Access Key Id used to authenticate to KMS | Value of the AWS_ACCESS_KEY_ID environment variable |
| secret_access_key | string | see [AWS KMS Access](#aws-kms-access) | The Secret Access Key used to authenticate to KMS | Value of the AWS_SECRET_ACCESS_KEY environment variable |
| region | string | yes | The region where the keys will be stored | |
| key_metadata_file | string | yes | A file path location where information about generated keys will be persisted | |
| key_policy_file | string | no | A file path location to a custom key policy in JSON format | "" |
| Key | Type | Required | Description | Default |
|----------------------|--------|---------------------------------------|--------------------------------------------------------------------------------------------|---------------------------------------------------------|
| access_key_id | string | see [AWS KMS Access](#aws-kms-access) | The Access Key Id used to authenticate to KMS | Value of the AWS_ACCESS_KEY_ID environment variable |
| secret_access_key | string | see [AWS KMS Access](#aws-kms-access) | The Secret Access Key used to authenticate to KMS | Value of the AWS_SECRET_ACCESS_KEY environment variable |
| region | string | yes | The region where the keys will be stored | |
| key_metadata_file | string | yes | A file path location where information about generated keys will be persisted (deprecated) | |
keeganwitt marked this conversation as resolved.
Show resolved Hide resolved
| key_identifier_file | string | yes | A file path location where information about generated keys will be persisted | |
keeganwitt marked this conversation as resolved.
Show resolved Hide resolved
| key_identifier_value | string | yes | A static identifier for the SPIRE server instance (used instead of `key_metadata_file`) | |
keeganwitt marked this conversation as resolved.
Show resolved Hide resolved
| key_policy_file | string | no | A file path location to a custom key policy in JSON format | "" |

### Alias and Key Management

Expand Down
22 changes: 12 additions & 10 deletions doc/plugin_server_keymanager_azure_key_vault.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,17 @@ SPIRE.

The plugin accepts the following configuration options:

| Key | Type | Required | Description | Default |
|-------------------|---------|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------|
| key_metadata_file | string | yes | A file path location where key metadata used by the plugin will be persisted. See "[Management of keys](#management-of-keys)" for more information. | "" |
| key_vault_uri | string | Yes | The Key Vault URI where the keys managed by this plugin reside. | "" |
| use_msi | boolean | [Deprecated](#authenticating-to-azure) | Whether or not to use MSI to authenticate to Azure Key Vault. | false |
| subscription_id | string | [Optional](#authenticating-to-azure) | The subscription id. | "" |
| app_id | string | [Optional](#authenticating-to-azure) | The application id. | "" |
| app_secret | string | [Optional](#authenticating-to-azure) | The application secret. | "" |
| tenant_id | string | [Optional](#authenticating-to-azure) | The tenant id. | "" |
| Key | Type | Required | Description | Default |
|----------------------|---------|----------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
| key_metadata_file | string | yes | A file path location where key metadata used by the plugin will be persisted (deprecated). See "[Management of keys](#management-of-keys)" for more information. | "" |
| key_identifier_file | string | yes | A file path location where key metadata used by the plugin will be persisted. See "[Management of keys](#management-of-keys)" for more information. | "" |
| key_identifier_value | string | yes | A static identifier for the SPIRE server instance (used instead of `key_metadata_file`) | "" |
| key_vault_uri | string | Yes | The Key Vault URI where the keys managed by this plugin reside. | "" |
| use_msi | boolean | [Deprecated](#authenticating-to-azure) | Whether or not to use MSI to authenticate to Azure Key Vault. | false |
| subscription_id | string | [Optional](#authenticating-to-azure) | The subscription id. | "" |
| app_id | string | [Optional](#authenticating-to-azure) | The application id. | "" |
| app_secret | string | [Optional](#authenticating-to-azure) | The application secret. | "" |
| tenant_id | string | [Optional](#authenticating-to-azure) | The tenant id. | "" |

### Authenticating to Azure

Expand Down Expand Up @@ -52,7 +54,7 @@ following table is provided for informational purposes only:

| Label | Description |
|-----------------|----------------------------------------------------------------------------------------------------------------------------------------|
| spire-server-td | A string representing the trust domain name of the server. |
| spire-server-td | A string representing the trust domain name of the server. |
| spire-server-id | Auto-generated ID that is unique to the server and is persisted in the _Key Metadata File_ (see the `key_metadata_file` configurable). |

If the _Key Metadata File_ is not found during server startup, the file is
Expand Down
26 changes: 14 additions & 12 deletions doc/plugin_server_keymanager_gcp_kms.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@ SPIRE.

The plugin accepts the following configuration options:

| Key | Type | Required | Description | Default |
| --- | ---- | -------- | ----------- | ------- |
| key_policy_file | string | no | A file path location to a custom [IAM Policy (v3)](https://cloud.google.com/pubsub/docs/reference/rpc/google.iam.v1#google.iam.v1.Policy) in JSON format to be attached to created CryptoKeys. | "" |
| key_metadata_file | string | yes | A file path location where key metadata used by the plugin will be persisted. See "[Management of keys](#management-of-keys)" for more information. | "" |
| key_ring | string | yes | Resource ID of the key ring where the keys managed by this plugin reside, in the format projects/\*/locations/\*/keyRings/\* | "" |
| service_account_file | string | no | Path to the service account file used to authenticate with the Cloud KMS API. | Value of `GOOGLE_APPLICATION_CREDENTIALS` environment variable. |
| Key | Type | Required | Description | Default |
|----------------------|--------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------|
| key_policy_file | string | no | A file path location to a custom [IAM Policy (v3)](https://cloud.google.com/pubsub/docs/reference/rpc/google.iam.v1#google.iam.v1.Policy) in JSON format to be attached to created CryptoKeys. | "" |
| key_metadata_file | string | yes | A file path location where key metadata used by the plugin will be persisted (deprecated). See "[Management of keys](#management-of-keys)" for more information. | "" |
| key_identifier_file | string | yes | A file path location where key metadata used by the plugin will be persisted. See "[Management of keys](#management-of-keys)" for more information. | "" |
| key_identifier_value | string | yes | A static identifier for the SPIRE server instance (used instead of `key_metadata_file`) | "" |
| key_ring | string | yes | Resource ID of the key ring where the keys managed by this plugin reside, in the format projects/\*/locations/\*/keyRings/\* | "" |
| service_account_file | string | no | Path to the service account file used to authenticate with the Cloud KMS API. | Value of `GOOGLE_APPLICATION_CREDENTIALS` environment variable. |

### Authenticating with the Cloud KMS API

Expand Down Expand Up @@ -49,12 +51,12 @@ the service. All the labels are named with the `spire-` prefix.
Users don't need to interact with the labels managed by the plugin. The
following table is provided for informational purposes only:

| Label | Description |
| ----- | ----------- |
| spire-server-td | SHA-1 checksum of the trust domain name of the server. |
| spire-server-id | Auto-generated ID that is unique to the server and is persisted in the _Key Metadata File_ (see the `key_metadata_file` configurable). |
| spire-last-update | Unix time of the last time that the plugin updated the CryptoKey to keep it active. |
| spire-active | Indicates if the CryptoKey is still in use by the plugin. |
| Label | Description |
|-------------------|----------------------------------------------------------------------------------------------------------------------------------------|
| spire-server-td | SHA-1 checksum of the trust domain name of the server. |
| spire-server-id | Auto-generated ID that is unique to the server and is persisted in the _Key Metadata File_ (see the `key_metadata_file` configurable). |
| spire-last-update | Unix time of the last time that the plugin updated the CryptoKey to keep it active. |
| spire-active | Indicates if the CryptoKey is still in use by the plugin. |

If the _Key Metadata File_ is not found during server startup, the file is
recreated, with a new auto-generated server ID. Consequently, if the file is
Expand Down
59 changes: 45 additions & 14 deletions pkg/server/plugin/keymanager/awskms/awskms.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"fmt"
"os"
"path"
"regexp"
"strings"
"sync"
"time"
Expand Down Expand Up @@ -92,11 +93,13 @@ type Plugin struct {

// Config provides configuration context for the plugin
type Config struct {
AccessKeyID string `hcl:"access_key_id" json:"access_key_id"`
SecretAccessKey string `hcl:"secret_access_key" json:"secret_access_key"`
Region string `hcl:"region" json:"region"`
KeyMetadataFile string `hcl:"key_metadata_file" json:"key_metadata_file"`
KeyPolicyFile string `hcl:"key_policy_file" json:"key_policy_file"`
AccessKeyID string `hcl:"access_key_id" json:"access_key_id"`
SecretAccessKey string `hcl:"secret_access_key" json:"secret_access_key"`
Region string `hcl:"region" json:"region"`
KeyMetadataFile string `hcl:"key_metadata_file" json:"key_metadata_file"`
KeyIdentifierFile string `hcl:"key_identifier_file" json:"key_identifier_file"`
KeyIdentifierValue string `hcl:"key_identifier_value" json:"key_identifier_value"`
KeyPolicyFile string `hcl:"key_policy_file" json:"key_policy_file"`
}

// New returns an instantiated plugin
Expand Down Expand Up @@ -131,12 +134,6 @@ func (p *Plugin) Configure(ctx context.Context, req *configv1.ConfigureRequest)
return nil, err
}

serverID, err := loadServerID(config.KeyMetadataFile)
if err != nil {
return nil, err
}
p.log.Debug("Loaded server id", "server_id", serverID)

if config.KeyPolicyFile != "" {
policyBytes, err := os.ReadFile(config.KeyPolicyFile)
if err != nil {
Expand All @@ -146,6 +143,22 @@ func (p *Plugin) Configure(ctx context.Context, req *configv1.ConfigureRequest)
p.keyPolicy = &policyStr
}

var serverID = config.KeyIdentifierValue
if serverID == "" && config.KeyMetadataFile != "" {
p.log.Warn("'key_metadata_file' is deprecated in favor of 'key_identifier_file' and will be removed in a future version")
serverID, err = getOrCreateServerID(config.KeyMetadataFile)
if err != nil {
return nil, err
}
}
if serverID == "" && config.KeyIdentifierFile != "" {
serverID, err = getOrCreateServerID(config.KeyIdentifierFile)
if err != nil {
return nil, err
}
}
p.log.Debug("Loaded server id", "server_id", serverID)

awsCfg, err := newAWSConfig(ctx, config)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to create client configuration: %v", err)
Expand Down Expand Up @@ -832,8 +845,26 @@ func parseAndValidateConfig(c string) (*Config, error) {
return nil, status.Error(codes.InvalidArgument, "configuration is missing a region")
}

if config.KeyMetadataFile == "" {
return nil, status.Error(codes.InvalidArgument, "configuration is missing server id file path")
if config.KeyIdentifierValue != "" {
re := regexp.MustCompile(".*[^A-z0-9/_-].*")
if re.MatchString(config.KeyIdentifierValue) {
return nil, status.Error(codes.InvalidArgument, "Key identifier must contain only alphanumeric characters, forward slashes (/), underscores (_), and dashes (-)")
}
if strings.HasPrefix(config.KeyIdentifierValue, "alias/aws/") {
return nil, status.Error(codes.InvalidArgument, "Key identifier must not start with alias/aws/")
}
if len(config.KeyIdentifierValue) > 256 {
return nil, status.Error(codes.InvalidArgument, "Key identifier must not be longer than 256 characters")
}
}
if config.KeyMetadataFile == "" && config.KeyIdentifierFile == "" && config.KeyIdentifierValue == "" {
return nil, status.Error(codes.InvalidArgument, "configuration requires server id or server id file path")
}
if (config.KeyMetadataFile != "" || config.KeyIdentifierFile != "") && config.KeyIdentifierValue != "" {
return nil, status.Error(codes.InvalidArgument, "configuration must not contain both server id and server id file path")
}
if config.KeyMetadataFile != "" && config.KeyIdentifierFile != "" {
return nil, status.Error(codes.InvalidArgument, "configuration must not contain both 'key_identifier_file' and deprecated 'key_metadata_file'")
}

return config, nil
Expand Down Expand Up @@ -923,7 +954,7 @@ func min(x, y time.Duration) time.Duration {
return y
}

func loadServerID(idPath string) (string, error) {
func getOrCreateServerID(idPath string) (string, error) {
// get id from path
data, err := os.ReadFile(idPath)
switch {
Expand Down
Loading
Loading