Skip to content

Commit

Permalink
[Event Hubs Client] Initial Migration to Central Repository (#5862)
Browse files Browse the repository at this point in the history 
* [Event Hubs Client] Code Drop

  Initial drop of the Event Hubs client library code from its stand-alone repository,
  based on commit d8c1e3ef321f3873b311e3b795c5e19d78e1de29, by Serkant Karaca on
  March 28, 2019


* [Event Hubs Client] Initial Migration

Project Structure
  - Solution and projects adapated to central engineering system structure,
    format, and conventions

  - Target platforms updated to conform to the central engineering system.  For
    this initial effort, the Xamarin.iOS10 target has been converted to
    netstandard1.4, which is compatible with the previously configured iOS verison.

  - Target platforms have been updated to only include uap10.0 on Windows and when a flag is passed
    to indicate that MsBuild is being used, as UWP currently cannot be build from .NET core.  In the
    current configuration for the engineering system, this means that the uap10.0 target will not be
    built; discussions for adding support have begun.

        NOTE: This is subject to change pending the outcome of discussions with the engineering team
        on the best way to enable support within the build system.

  - Updated conditional compilation to recognize netstandard1.4 and uap10.0 as the target platforms
    for consumers which the Identity Model for Active Directory package does not support
    client certificate use; the constructs previously hidden for iOS and UWP are now
    hidden for netstandard1.4 and UWP.

  - Forced a target platform of x64 for Service Fabric Processor and Test projects,
    due to the restrictions imposed by the Service Fabric library references.

  - Configured overrides to the engineering system default assembly signing key, to
    allow for use of the existing Azure Messaging key while the client library is in Track 1,
    to avoid breaking changes.

  - Normalized and updated package references to known-good versions used by the centralized
    SDKs and attempted to remove fragmentation between target platform references.

Azure Resource Templates
  - Converted values which are assumed by the test infrastructure from parameters to variables
    with the expected names.  This ensures that deployment is compatible with the test suite without
    a user needing to have awareness of the expected names.

ReadMe
  - Restructured and updated content to snap to the Azure SDK repository ReadMe template, as closely
    as possible for a Track 1 migration.

Event Hubs Tests
  - Designated tests that interact with Azure services as "Live" tests, allowing them to be filtered when
    Azure environments are not available or when it is desirable to run only fast, self-contained tests, such
    as build verification for pull requests.

  - Moved hardcoded values tied to Azure resources into the TestConstants file, so that well-known
    and assumed values are centrally located.

  - Updated the approach for reading environment variables to a lazy approach instead of in a static
    constructor, allowing tests that do not use Azure resoruces to make use of TestUtility functionality
    without the need for a set of "Live" environment variables.

  - Moved test utility and infrastructure classes to a dedcated folder for better organization and in
    anticipation of additional test infrastructure additions during the upcoming test stabilization.

  - Added configuration for the XUnit test runner to enable diagnostics output and in anticipation of
    parallelization of the tests during stabilization work.

  - Added access modifiers to tests in order to satisfy code analysis rules for unused private members.

  - Small fixes to async patterns in some tests, to await rather than returning the task directly, per
    XUnit recommended practices.

  - Small additions to failure messages, to grant additional context and highlight actual vs expected values.

  - Temporarily set some unstable tests in the the ServiceFabricProcessor area to be skipped
    until for the initial migration; to be investiated and stablized as part of issue #5451.
  • Loading branch information
@jsquire
jsquire authored Apr 24, 2019
1 parent 5978dd5 commit 64a0ee6
Show file tree
Hide file tree
Showing 159 changed files with 21,225 additions and 0 deletions.
32 changes: 32 additions & 0 deletions src/SDKs/EventHub/data-plane/Directory.Build.props
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory).., Directory.Build.props))\Directory.Build.props" />

<PropertyGroup>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<GenerateAssemblyConfigurationAttribute>false</GenerateAssemblyConfigurationAttribute>
<GenerateAssemblyCompanyAttribute>false</GenerateAssemblyCompanyAttribute>
<GenerateAssemblyProductAttribute>false</GenerateAssemblyProductAttribute>

<!-- This is a workaorund until https://github.com/Azure/azure-sdk-for-net/issues/5214 is addressed -->
<RequiredTargetFrameworks>net461;netstandard2.0</RequiredTargetFrameworks>

<!-- Do not inherit implicit dependencies from the engineering system during build or packaging -->
<ImportDefaultReferences>false</ImportDefaultReferences>

<!--
If the assembly is to be signed, emit a constant so that any constructs which rely on the assembly
names, such as declaration of friend assemblies for testing, can be adjusted as necessary
-->
<DefineConstants Condition="'$(SignAssembly)' == 'true'">$(DefineConstants);CODESIGN</DefineConstants>
</PropertyGroup>

<!--
For Track 1, override the engineering system signing key and use the existing key for
Azure messaging client libraries.
-->
<PropertyGroup Condition="'$(SignAssembly)' == 'true'">
<AssemblyOriginatorKeyFile>$(MSBuildThisFileDirectory)assets\azure-messaging.snk</AssemblyOriginatorKeyFile>
<DelaySign>false</DelaySign>
<PublicSign>false</PublicSign>
</PropertyGroup>
</Project>
61 changes: 61 additions & 0 deletions src/SDKs/EventHub/data-plane/Microsoft.Azure.EventHubs.sln
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.28729.10
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Azure.EventHubs", "src\Microsoft.Azure.EventHubs\Microsoft.Azure.EventHubs.csproj", "{126D946D-CE0F-4F14-9F13-8FD7098B81D8}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Azure.EventHubs.Tests", "tests\Microsoft.Azure.EventHubs.Tests\Microsoft.Azure.EventHubs.Tests.csproj", "{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Azure.EventHubs.Processor", "src\Microsoft.Azure.EventHubs.Processor\Microsoft.Azure.EventHubs.Processor.csproj", "{8C89967A-4E1F-46B0-8458-81B545C822B2}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Azure.EventHubs.ServiceFabricProcessor", "src\Microsoft.Azure.EventHubs.ServiceFabricProcessor\Microsoft.Azure.EventHubs.ServiceFabricProcessor.csproj", "{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Debug|x64 = Debug|x64
Release|Any CPU = Release|Any CPU
Release|x64 = Release|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Debug|Any CPU.Build.0 = Debug|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Debug|x64.ActiveCfg = Debug|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Debug|x64.Build.0 = Debug|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Release|Any CPU.ActiveCfg = Release|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Release|Any CPU.Build.0 = Release|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Release|x64.ActiveCfg = Release|Any CPU
{126D946D-CE0F-4F14-9F13-8FD7098B81D8}.Release|x64.Build.0 = Release|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Debug|Any CPU.Build.0 = Debug|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Debug|x64.ActiveCfg = Debug|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Debug|x64.Build.0 = Debug|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Release|Any CPU.ActiveCfg = Release|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Release|Any CPU.Build.0 = Release|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Release|x64.ActiveCfg = Release|Any CPU
{154F7B4C-B998-4FA0-933F-F34DB0CA9B88}.Release|x64.Build.0 = Release|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Debug|x64.ActiveCfg = Debug|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Debug|x64.Build.0 = Debug|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Release|Any CPU.Build.0 = Release|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Release|x64.ActiveCfg = Release|Any CPU
{8C89967A-4E1F-46B0-8458-81B545C822B2}.Release|x64.Build.0 = Release|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Debug|x64.ActiveCfg = Debug|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Debug|x64.Build.0 = Debug|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Release|Any CPU.Build.0 = Release|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Release|x64.ActiveCfg = Release|Any CPU
{D96BCC8F-D5EE-464A-9C15-EF59613F9F82}.Release|x64.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {4BD75992-031F-413D-A801-BCB181DCFFD1}
EndGlobalSection
EndGlobal
92 changes: 92 additions & 0 deletions src/SDKs/EventHub/data-plane/ReadMe.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Azure Event Hubs client library for .NET

Azure Event Hubs is a highly scalable publish-subscribe service that can ingest millions of events per second and stream them into multiple applications. This lets you process and analyze the massive amounts of data produced by your connected devices and applications. Once Event Hubs has collected the data, you can retrieve, transform and store it by using any real-time analytics provider or with batching/storage adapters.

The Azure Events Hubs client library for .NET allows for both sending and receiving of events. Most common scenarios call for an application to act as either an event publisher or an event consumer, but rarely both.

An **event publisher** is a source of telemetry data, diagnostics information, usage logs, or other log data, as
part of an embedded device solution, a mobile device application, a game title running on a console or other device,
some client or server based business solution, or a web site.

An **event consumer** picks up such information from the Event Hub and processes it. Processing may involve aggregation, complex
computation and filtering. Processing may also involve distribution or storage of the information in a raw or transformed fashion.
Event Hub consumers are often robust and high-scale platform infrastructure parts with built-in analytics capabilities, like Azure
Stream Analytics, Apache Spark, or Apache Storm.

This directory contains the open source subset of the .NET SDK. For documentation of the complete Azure SDK, please see the [Microsoft Azure .NET Developer Center](http://azure.microsoft.com/en-us/develop/net/).

Use the client library for Event Hubs to:

- Emit telemetry about your application for business intelligence and diagnostic purposes.

- Publish facts about the state of your application which interested parties may observe and use as a trigger for taking action.

- Observe interesting operations and interactions happening within your business or other ecosystem, allowing loosely coupled systems to interact without the need to bind them together.

- Receive events from one or more publishers, transform them to better meet the needs of your ecosystem, then publish the transformed events to a new stream for consumers to observe.

[Source code](https://github.com/Azure/azure-sdk-for-net/tree/master/src/SDKs/EventHub/data-plane) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.EventHubs/) | [API reference documentation](https://docs.microsoft.com/en-us/dotnet/api/overview/azure/event-hubs?view=azure-dotnet) | [Product documentation](https://docs.microsoft.com/en-us/azure/event-hubs/)

## Getting started

The complete Microsoft Azure SDK can be downloaded from the [Microsoft Azure Downloads Page](http://azure.microsoft.com/en-us/downloads/?sdk=net) and ships with support for building deployment packages, integrating with tooling, rich command line tooling, and more.

If you are not already familiar with Azure Event Hubs, please review: [What is Event Hubs?](https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about).

For the best development experience, developers should use the official Microsoft NuGet packages for libraries. NuGet packages are regularly updated with new functionality and hotfixes.

## Prerequisites

- Microsoft Azure Subscription: To call Microsoft Azure services, including Azure Event Hubs, you need to first [create an account](https://account.windowsazure.com/Home/Index). If you do not have an existing Azure account, you may sign up for a free trial or use your MSDN subscriber benefits.

- The Azure Event Hubs client library shares the same [Prerequisites](https://github.com/azure/azure-sdk-for-net#prerequisites) as the Microsoft Azure SDK for .NET.

## Samples

Code samples for the Azure Event Hubs client library that detail how to get started and how to implement common scenarios can be found in the following locations:

- [Azure Code Samples](https://azure.microsoft.com/en-us/resources/samples/?sort=0&service=event-hubs&platform=dotnet)
- [Azure Event Hubs Documentation](https://docs.microsoft.com/en-us/azure/event-hubs/)
- [Azure Event Hubs Sample Repository](https://github.com/Azure/azure-event-hubs/tree/master/samples)
- [Azure Event Hubs Notification Sample](event-hubs-dotnet-user-notifications)
- [Azure Event Hubs Publishing Sample](https://github.com/Azure-Samples/event-hubs-dotnet-ingest)

## To build

For information on building the Azure Event Hubs client library, please see [Building the Microsoft Azure SDK for .NET](https://github.com/azure/azure-sdk-for-net#to-build)

## Running tests

1. Deploy the Azure Resource Manager template located at [/assets/azure-deploy-test-dependencies.json](https://github.com/Azure/azure-sdk-for-net/blob/master/src/SDKs/EventHub/data-plane/assets/azure-deploy-test-dependencies.json) by clicking the following button:

<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-sdk-for-net%2Fmaster%2Fsrc%2FSDKs%2FEventHub%2Fdata-plane%2Fassets%2Fazure-deploy-test-dependencies.json" target="_blank">
<img src="http://azuredeploy.net/deploybutton.png"/>
</a>

*Running the above template will provision a standard Event Hubs namespace along with the required entities to successfully run the unit tests.*

1. Add an Environment Variable named `AZ_EVENT_HUBS_CONNECTION` and set the value as the connection string of the newly created namespace. **Please note that if you are using Visual Studio, you must restart Visual Studio in order to use new Environment Variables.**

1. Add an Environment Variable named `AZ_EVENT_HUBS_STORAGE_CONNECTION` and set the value as the connection string of the newly created storage account. **Please note that if you are using Visual Studio, you must restart Visual Studio in order to use new Environment Variables.**

Once you have completed the above, you can run `dotnet test` from the `/src/SDKs/EventHub/data-plane/tests/Microsoft.Azure.EventHubs.Tests` directory.

## Versioning information

The Azure Event Hubs client library uses [the semantic versioning scheme.](http://semver.org/)

## Target frameworks

For information about the target frameworks of the Azure Event Hubs client library, please refer to the [Target Frameworks](https://github.com/azure/azure-sdk-for-net#target-frameworks) of the Microsoft Azure SDK for .NET.

## Contributing

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

## Additional documentation

- [Azure Event Hubs General Documentation](https://docs.microsoft.com/en-us/azure/event-hubs/)
- [Azure Event Hubs REST API Reference](https://docs.microsoft.com/en-us/rest/api/eventhub/)
- [Azure Event Hubs SDK for .NET Documentation](https://docs.microsoft.com/en-us/dotnet/api/overview/azure/event-hubs?view=azure-dotnet)

![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsrc%2FSDKs%2FEventHub%2Fdata-plane%2FREADME.png)
98 changes: 98 additions & 0 deletions src/SDKs/EventHub/data-plane/assets/azure-deploy-test-dependencies.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
{
"$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"namespaceName": {
"type": "string",
"metadata": {
"description": "Name of the EventHub namespace"
}
},
"storageAccountName": {
"type": "string",
"metadata": {
"description": "Name of the storage account that is used by the Event Processor Host"
}
},
"storageAccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_GRS",
"Standard_ZRS",
"Premium_LRS"
],
"metadata": {
"description": "Storage Account type"
}
}
},
"variables": {
"eventHubName": "eventhubs-sdk-test-hub",
"consumerGroupName": "sdk-test-consumer",
"location": "[resourceGroup().location]",
"apiVersion": "2015-08-01",
"defaultSASKeyName": "RootManageSharedAccessKey",
"authRuleResourceId": "[resourceId('Microsoft.EventHub/namespaces/authorizationRules', parameters('namespaceName'), variables('defaultSASKeyName'))]",
"storageAccountResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('namespaceName')]",
"type": "Microsoft.EventHub/Namespaces",
"location": "[variables('location')]",
"sku": {
"name": "Standard",
"tier": "Standard"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[variables('eventHubName')]",
"type": "EventHubs",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', parameters('namespaceName'))]"
],
"properties": {
"path": "[variables('eventHubName')]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[variables('consumerGroupName')]",
"type": "ConsumerGroups",
"dependsOn": [
"[variables('eventHubName')]"
],
"properties": {}
}
]
}
]
},
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('storageAccountName')]",
"apiVersion": "2016-01-01",
"location": "[variables('location')]",
"sku": {
"name": "[parameters('storageAccountType')]"
},
"kind": "Storage",
"properties": {
}
}
],
"outputs": {
"NamespaceConnectionString": {
"type": "string",
"value": "[listkeys(variables('authRuleResourceId'), variables('apiVersion')).primaryConnectionString]"
},
"StorageAccountConnectionString": {
"type": "string",
"value": "[Concat('DefaultEndpointsProtocol=https;AccountName=',parameters('storageAccountName'),';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value)]"
}
}
}
Binary file added src/SDKs/EventHub/data-plane/assets/azure-messaging.snk
View file
Binary file not shown.
58 changes: 58 additions & 0 deletions src/SDKs/EventHub/data-plane/src/Microsoft.Azure.EventHubs.Processor/AzureBlobLease.cs
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

namespace Microsoft.Azure.EventHubs.Processor
{
using System.Threading.Tasks;
using Newtonsoft.Json;
using WindowsAzure.Storage.Blob;

class AzureBlobLease : Lease
{
readonly bool isOwned;

// ctor needed for deserialization
internal AzureBlobLease()
{
}

internal AzureBlobLease(string partitionId, CloudBlockBlob blob) : base(partitionId)
{
this.Blob = blob;
this.isOwned = blob.Properties.LeaseState == LeaseState.Leased;
}

internal AzureBlobLease(string partitionId, string owner, CloudBlockBlob blob) : base(partitionId)
{
this.Blob = blob;
this.Owner = owner;
this.isOwned = blob.Properties.LeaseState == LeaseState.Leased;
}

internal AzureBlobLease(AzureBlobLease source)
: base(source)
{
this.Offset = source.Offset;
this.SequenceNumber = source.SequenceNumber;
this.Blob = source.Blob;
this.isOwned = source.isOwned;
}

internal AzureBlobLease(AzureBlobLease source, CloudBlockBlob blob) : base(source)
{
this.Offset = source.Offset;
this.SequenceNumber = source.SequenceNumber;
this.Blob = blob;
this.isOwned = blob.Properties.LeaseState == LeaseState.Leased;
}

// do not serialize
[JsonIgnore]
public CloudBlockBlob Blob { get; }

public override Task<bool> IsExpired()
{
return Task.FromResult(!this.isOwned);
}
}
}
Loading

0 comments on commit 64a0ee6

Please sign in to comment.