Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing Cryptography building block in .NET #1217

Merged
merged 43 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
8897cd0
Added method to DaprClient and GRPC implementation to call cryptograp…
WhitWaldo Dec 20, 2023
b98926e
First pass at implementing all exposed Cryptography methods on Go int…
WhitWaldo Dec 22, 2023
e391735
Added examples for Cryptography block
WhitWaldo Dec 22, 2023
43dd902
Added missing copyright statements
WhitWaldo Dec 22, 2023
9c5ef09
Updated to properly support Crypto API this time
WhitWaldo Dec 22, 2023
14f23df
Added copyright statements
WhitWaldo Dec 22, 2023
dee30d1
Removed deprecated examples as the subtle APIs are presently disabled
WhitWaldo Dec 22, 2023
37640bb
Updated example to reflect new API shape
WhitWaldo Dec 22, 2023
c398780
Updated example and readme
WhitWaldo Dec 23, 2023
6e93dd5
Added overloads for encrypting/decrypting streams instead of just fix…
WhitWaldo Dec 23, 2023
2913fcf
Added some unit tests to pair with the implementation
WhitWaldo Dec 23, 2023
89dfda3
Added null check for the stream argument
WhitWaldo Jan 2, 2024
3f5a489
Changed case of the arguments as they should read "plaintext" and not…
WhitWaldo Jan 2, 2024
f706227
Reduced number of encryption implementations by just wrapping byte ar…
WhitWaldo Jan 2, 2024
735b079
Constrainted returned member types per review suggestion
WhitWaldo Jan 2, 2024
38b9e04
Updated methods to use ReadOnlyMemory<byte> instead of byte[] - updat…
WhitWaldo Jan 3, 2024
8a80a86
Updated to use encryption/decryption options instead of lots of metho…
WhitWaldo Jan 3, 2024
337961e
Updated tests
WhitWaldo Jan 3, 2024
a75603f
Removed unused reference
WhitWaldo Jan 3, 2024
86315b4
Updated examples to reflect new method shapes. Downgraded package to …
WhitWaldo Jan 3, 2024
c2e59bc
Updated to reflect non-aliased values per review suggestion
WhitWaldo Jan 3, 2024
bdc7ffd
Update to ensure that both send/receive streams run at the same time …
WhitWaldo Jan 3, 2024
705cf77
Updated to support streamed results in addition to fixed byte arrays.…
WhitWaldo Jan 3, 2024
6b6d709
Updated example to fix compile issue
WhitWaldo Jan 3, 2024
3c0bcdc
Removed encrypt/decrypt methods that accepted streams and returned Re…
WhitWaldo Jan 3, 2024
ba61812
Added missing Obsolete attributes on Encrypt/Decrypt methods. Added o…
WhitWaldo Jan 3, 2024
1d0764b
Updated encrypt/decrypt options so the streaming block size no longer…
WhitWaldo Jan 3, 2024
6e827e1
Updated how validation works in the options to accommodate lack of th…
WhitWaldo Jan 3, 2024
ea3992a
Updated names of encrypt/decrypt streaming methods so everything uses…
WhitWaldo Jan 3, 2024
50bf981
Fixed regression that would have prevented data from being sent entir…
WhitWaldo Jan 3, 2024
0c25a56
Updated examples to reflect changed API
WhitWaldo Jan 3, 2024
5bb55fe
Updated so IAsyncEnumerable methods (encrypt and decrypt) return IAsy…
WhitWaldo Jan 3, 2024
4ed3562
Updated example to reflect change from IAsyncEnumerable<byte> to IAsy…
WhitWaldo Jan 3, 2024
878b8e7
Avoiding allocation by using MemoryMarshal instead of .ToArray() to c…
WhitWaldo Jan 3, 2024
7c83888
Performance updates to minimize unnecessary byte array copies and eli…
WhitWaldo Jan 4, 2024
17bf79f
Removed unnecessary return from SendPlaintextStreamAsync and SendCiph…
WhitWaldo Jan 4, 2024
770b94a
Updated exception text to be more specific as to what's wrong with th…
WhitWaldo Jan 4, 2024
d5bcebb
Minor tweak to prefer using using a Memory
WhitWaldo Jan 4, 2024
095405e
Deduplicated some of the Decrypt methods, simplifying the implementation
WhitWaldo Jan 4, 2024
27366a4
Eliminated duplicate encryption method, simplifying implementation
WhitWaldo Jan 4, 2024
ff670ee
Updated to eliminate an unnecessary `await` and `async foreach`.
WhitWaldo Jan 4, 2024
05ebc8e
Updated stream example to reflect the changes to the API shape
WhitWaldo Jan 4, 2024
48bd541
Added notes about operations with stream-based data
WhitWaldo Jan 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions all.sln
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "BulkPublishEventExample", "
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "WorkflowUnitTest", "examples\Workflow\WorkflowUnitTest\WorkflowUnitTest.csproj", "{8CA09061-2BEF-4506-A763-07062D2BD6AC}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Cryptography", "examples\Client\Cryptography\Cryptography.csproj", "{C74FBA78-13E8-407F-A173-4555AEE41FF3}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -248,6 +250,14 @@ Global
{DDC41278-FB60-403A-B969-2AEBD7C2D83C}.Release|Any CPU.Build.0 = Release|Any CPU
{8CA09061-2BEF-4506-A763-07062D2BD6AC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8CA09061-2BEF-4506-A763-07062D2BD6AC}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C74FBA78-13E8-407F-A173-4555AEE41FF3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C74FBA78-13E8-407F-A173-4555AEE41FF3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C74FBA78-13E8-407F-A173-4555AEE41FF3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C74FBA78-13E8-407F-A173-4555AEE41FF3}.Release|Any CPU.Build.0 = Release|Any CPU
{D1786E2B-CAA0-4B2D-A974-9845EB9E420F}.Debug|Any CPU.ActiveCfg = Debug
{D1786E2B-CAA0-4B2D-A974-9845EB9E420F}.Debug|Any CPU.Build.0 = Debug
{D1786E2B-CAA0-4B2D-A974-9845EB9E420F}.Release|Any CPU.ActiveCfg = Release
{D1786E2B-CAA0-4B2D-A974-9845EB9E420F}.Release|Any CPU.Build.0 = Release
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -293,6 +303,7 @@ Global
{4A175C27-EAFE-47E7-90F6-873B37863656} = {0EF6EA64-D7C3-420D-9890-EAE8D54A57E6}
{DDC41278-FB60-403A-B969-2AEBD7C2D83C} = {0EF6EA64-D7C3-420D-9890-EAE8D54A57E6}
{8CA09061-2BEF-4506-A763-07062D2BD6AC} = {BF3ED6BF-ADF3-4D25-8E89-02FB8D945CA9}
{C74FBA78-13E8-407F-A173-4555AEE41FF3} = {A7F41094-8648-446B-AECD-DCC2CC871F73}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {65220BF2-EAE1-4CB2-AA58-EBE80768CB40}
Expand Down
25 changes: 25 additions & 0 deletions examples/Client/Cryptography/Components/azurekeyvault.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: azurekeyvault
spec:
type: crypto.azure.keyvault
metadata:
- name: vaultName
value: "<changeMe>"
- name: azureEnvironment
value: AZUREPUBLICCLOUD
- name: azureTenantId
secretKeyRef:
name: read_azure_tenant_id
key: read_azure_tenant_id
- name: azureClientId
secretKeyRef:
name: read_azure_client_id
key: read_azure_client_id
- name: azureClientSecret
secretKeyRef:
name: read_azure_client_secret
key: read_azure_client_secret
auth:
secureStore: envvar-secret-store
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: envvar-secret-store
spec:
type: secretstores.local.env
version: v1
25 changes: 25 additions & 0 deletions examples/Client/Cryptography/Cryptography.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<LangVersion>latest</LangVersion>
</PropertyGroup>

<ItemGroup>
<Compile Remove="C:\Users\whit_\source\repos\dapr-dotnet-sdk\properties\\IsExternalInit.cs" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\..\..\src\Dapr.Client\Dapr.Client.csproj" />
</ItemGroup>

<ItemGroup>
<None Update="file.txt">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
</ItemGroup>

</Project>
22 changes: 22 additions & 0 deletions examples/Client/Cryptography/Example.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
// ------------------------------------------------------------------------
// Copyright 2023 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------

namespace Cryptography
{
internal abstract class Example
{
public abstract string DisplayName { get; }

public abstract Task RunAsync(CancellationToken cancellationToken);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
// ------------------------------------------------------------------------
// Copyright 2023 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------

using System.Buffers;
using Dapr.Client;
#pragma warning disable CS0618 // Type or member is obsolete

namespace Cryptography.Examples
{
internal class EncryptDecryptFileStreamExample : Example
{
public override string DisplayName => "Use Cryptography to encrypt and decrypt a file";
public override async Task RunAsync(CancellationToken cancellationToken)
{
using var client = new DaprClientBuilder().Build();

const string componentName = "azurekeyvault"; // Change this to match the name of the component containing your vault
const string keyName = "myKey";

// The name of the file we're using as an example
const string fileName = "file.txt";

Console.WriteLine("Original file contents:");
foreach (var line in await File.ReadAllLinesAsync(fileName, cancellationToken))
{
Console.WriteLine(line);
}
Console.WriteLine();

//Encrypt from a file stream and buffer the resulting bytes to an in-memory buffer
await using var encryptFs = new FileStream(fileName, FileMode.Open);

var bufferedEncryptedBytes = new ArrayBufferWriter<byte>();
await foreach (var bytes in (await client.EncryptAsync(componentName, encryptFs, keyName,
new EncryptionOptions(KeyWrapAlgorithm.Rsa), cancellationToken))
.WithCancellation(cancellationToken))
{
bufferedEncryptedBytes.Write(bytes.Span);
}

Console.WriteLine($"Encrypted bytes: {Convert.ToBase64String(bufferedEncryptedBytes.GetSpan())}");
Console.WriteLine();

//We'll write to a temporary file via a FileStream
var tempDecryptedFile = Path.GetTempFileName();
await using var decryptFs = new FileStream(tempDecryptedFile, FileMode.Create);

//We'll stream the decrypted bytes from a MemoryStream into the above temporary file
await using var encryptedMs = new MemoryStream(bufferedEncryptedBytes.WrittenMemory.ToArray());
await foreach (var result in (await client.DecryptAsync(componentName, encryptedMs, keyName,
cancellationToken)).WithCancellation(cancellationToken))
{
decryptFs.Write(result.Span);
}

decryptFs.Close();

//Let's confirm the value as written to the file
var decryptedValue = await File.ReadAllTextAsync(tempDecryptedFile, cancellationToken);
Console.WriteLine($"Decrypted value: ");
Console.WriteLine(decryptedValue);

//And some cleanup to delete our temp file
File.Delete(tempDecryptedFile);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// ------------------------------------------------------------------------
// Copyright 2023 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------

using System.Text;
using Dapr.Client;
#pragma warning disable CS0618 // Type or member is obsolete

namespace Cryptography.Examples
{
internal class EncryptDecryptStringExample : Example
{
public override string DisplayName => "Using Cryptography to encrypt and decrypt a string";

public override async Task RunAsync(CancellationToken cancellationToken)
{
using var client = new DaprClientBuilder().Build();

const string componentName = "azurekeyvault"; //Change this to match the name of the component containing your vault
const string keyName = "myKey"; //Change this to match the name of the key in your Vault


const string plaintextStr = "This is the value we're going to encrypt today";
Console.WriteLine($"Original string value: '{plaintextStr}'");

//Encrypt the string
var plaintextBytes = Encoding.UTF8.GetBytes(plaintextStr);
var encryptedBytesResult = await client.EncryptAsync(componentName, plaintextBytes, keyName, new EncryptionOptions(KeyWrapAlgorithm.Rsa),
cancellationToken);

Console.WriteLine($"Encrypted bytes: '{Convert.ToBase64String(encryptedBytesResult.Span)}'");

//Decrypt the string
var decryptedBytes = await client.DecryptAsync(componentName, encryptedBytesResult, keyName, new DecryptionOptions(), cancellationToken);
Console.WriteLine($"Decrypted string: '{Encoding.UTF8.GetString(decryptedBytes.ToArray())}'");
}
}
}
47 changes: 47 additions & 0 deletions examples/Client/Cryptography/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// ------------------------------------------------------------------------
// Copyright 2023 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------

using Cryptography;
using Cryptography.Examples;

namespace Samples.Client
{
class Program
{
private static readonly Example[] Examples = new Example[]
{
new EncryptDecryptStringExample(),
new EncryptDecryptFileStreamExample()
};

static async Task<int> Main(string[] args)
{
if (args.Length > 0 && int.TryParse(args[0], out var index) && index >= 0 && index < Examples.Length)
{
var cts = new CancellationTokenSource();
Console.CancelKeyPress += (object? sender, ConsoleCancelEventArgs e) => cts.Cancel();

await Examples[index].RunAsync(cts.Token);
return 0;
}

Console.WriteLine("Hello, please choose a sample to run:");
for (var i = 0; i < Examples.Length; i++)
{
Console.WriteLine($"{i}: {Examples[i].DisplayName}");
}
Console.WriteLine();
return 1;
}
}
}
92 changes: 92 additions & 0 deletions examples/Client/Cryptography/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Dapr .NET SDK Cryptography example

## Prerequisites

- [.NET 8+](https://dotnet.microsoft.com/download) installed
- [Dapr CLI](https://docs.dapr.io/getting-started/install-dapr-cli)
- [Initialized Dapr environment](https://docs.dapr.io/getting-started/installation)
- [Dapr .NET SDK](https://docs.dapr.io/developing-applications/sdks/dotnet/)
- [Azure Key Vault instance](https://learn.microsoft.com/en-us/azure/key-vault/general/quick-create-portal)
- [Entra Service Principal](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app)

### Service Principal/Environment Variables Setup
In your Azure portal, open Microsoft Entra ID and click `App Registrations`. Click the button at the top to create a new registration. Select a name for your service principal
and click register, noting this name for later.

Once the registration is completed, open it from the list and select Certificates & Secrets from the left navigation. Select "Client secrets" from the page body (middle column)
and click the button to add a new client secret giving it an optional description and changing the expiry date as you desire. Click Add to create the secret. Record the secret
value it shows you - it will not be shown to you again without creating another client secret.

Click Overview from the left navigation and record the "Application (client) ID" and the "Directory (tenant) ID" values.

On your computer (assuming Windows), open your start menu and type "Environment Variables". An option should appear named "Edit the system environment variables". Select this
and your System Properties window will open. Click the "Environment Variables" button in the bottom and said window will appear. Click the "New..." button under System variables
to add the requisite service principal values to your environment variables. You can change these names as to want by updating the `./Components/azurekeyvault.yaml` names, but for now
configure as follows:

| Variable Name | Value |
|--|--|
| read_azure_client_id | Paste the value from your app registration overview for "Application (client) ID" |
| read_azure_client_secret | Paste the value of the client secret you generated for your app registration |
| read_azure_tenant_id | Paste the valeu from your app registration overview for "Directory (tenant) ID" |

Click OK to save your environment variables and to close your System Properties window. You may need to close restart your command line tool for it to recognize the new values.

### Azure Key Vault Setup

This example is implemented using the Azure Key Vault and will not work without it. Assuming you have a Key Vault instance configured, ensure that
you have the `Key Vault Crypto Officer` role assigned to yourself as you'll need to in order to generate a new key in the instance. After selecting Keys
under the Objects header, click the `Generate/Import` button at the top of the instance panel.

Under options, select `Generate` and name your key. This example is pre-configured to assume a key name of 'myKey', but feel free to change this (but also update the name in the example
you wish to run). The other default options are fine for our purposes, so click Create at the bottom and if you've got the appropriate roles, it will show up in the list of Keys.

Update your `./Components/azurekeyvault.yaml` file with the name of your Key Vault under `vaultName` where it currently reads "changeMe". This sample assumes authentication
via a service principal, so you might also need to set this up.

Back in the Azure Portal, assign at least the `Key Vault Crypto User` role to the service principal you previously created in the last step. Do this by clicking
`Access Control (IAM)` from the left navigation, clicking "Add" from the top and clicking "Add Role Assignment". Select `Key Vault Crypto User` from the list and click the Next
button. Ensuring that the "User, group or service principal" option is selected, click the "Select members" link and search for the name of the app registration you created. Click
Add to add this service principal to the list of members for the new role assignment and click Review + Assign twice to assign the role. This will take effect within a few seconds
or minutes. This step ensures that while Dapr can authenticate as your service principal, that it also has permission to access and use the key in your Key Vault.

## Running the example

To run the sample locally, run this command in the DaprClient directory:

```sh
dapr run --resources-path ./Components --app-id DaprClient -- dotnet run <zero-indexed sample number>
```

Running the following command will output a list of the samples included:

```sh
dapr run --resources-path ./Components --app-id DaprClient -- dotnet run
```

Press Ctrl+C to exit, and then run the command again and provide a sample number to run the samples.

For example, run this command to run the first sample from the list produced earlier (the 0th example):

```sh
dapr run --resources-path ./Components --app-id DaprClient -- dotnet run 0
```

## Encryption/Decryption with strings
See [EncryptDecryptStringExample.cs](./EncryptDecryptStringExample.cs) for an example of using `DaprClient` for basic
string-based encryption and decryption operations as performed against UTF-8 encoded byte arrays.

## Encryption/Decryption with streams
See [EncryptDecryptFileStreamExample.cs](./EncryptDecryptFileStreamExample.cs) for an example of using `DaprClient`
to perform an encrypt and decrypt operation against a stream of data. In the example, we stream a local file to the
sidecar to encrypt and write the result (as it's streamed back) to an in-memory buffer. Once the operation fully
completes, we perform the decrypt operation against this in-memory buffer and write the decrypted result back out to a
temporary file.

In either operation, rather than load the entire stream into memory and send all at once to the
sidecar as we do in the other string-based example (as this might cause you to run out of memory either on the
node the app is running on or do the same to the sidecar itself), this example instead breaks the input stream into
more manageable 4KB chunks (a value you can override via the `EncryptionOptions` or `DecryptionOptions` parameters
respectively up to 64KB. Further, rather than waiting for the entire stream to send to the sidecar before the
encryption operation proceeds, it immediately works to process the sidecar response, continuing to minimize resource
usage.
Loading
Loading